Vectors/norlm/norbert - Revision history

Andreku: Updated links to NorBERT

2023-03-21T10:47:57Z

Updated links to NorBERT

Andreku: /* NorBERT 1 */

2022-11-14T00:39:14Z

‎NorBERT 1

Andreku: /* Evaluation */

2022-11-12T20:29:35Z

‎Evaluation

Andreku: /* Evaluation */

2022-07-26T15:22:30Z

‎Evaluation

Andreku: /* Evaluation */

2022-07-26T15:22:02Z

‎Evaluation

Andreku: /* NorBERT: Bidirectional Encoder Representations from Transformers */

2022-02-16T19:06:08Z

‎NorBERT: Bidirectional Encoder Representations from Transformers

Andreku: /* Evaluation */

2022-02-15T17:14:08Z

‎Evaluation

Andreku: /* Evaluation */

2022-02-11T15:14:51Z

‎Evaluation

Andreku: /* Evaluation */

2022-02-11T15:14:07Z

‎Evaluation

Andreku: /* Evaluation */

2022-02-11T11:57:29Z

‎Evaluation

@@ Line 15: / Line 15: @@
 - '''[http://vectors.nlpl.eu/repository/20/216.zip Download from the NLPL Vector Repository]'''
-- '''[https://huggingface.co/ltgoslo/norbert Use with the Huggingface Transformers library]'''
+- '''[https://huggingface.co/ltg/norbert Use with the Huggingface Transformers library]'''
 - Available locally on Saga: ''/cluster/shared/nlpl/data/vectors/latest/216/''
@@ Line 24: / Line 24: @@
 - '''[http://vectors.nlpl.eu/repository/20/221.zip Download from the NLPL Vector Repository]'''
-- '''[https://huggingface.co/ltgoslo/norbert2 Use with the Huggingface Transformers library]'''
+- '''[https://huggingface.co/ltg/norbert2 Use with the Huggingface Transformers library]'''
 - Available locally on Saga: ''/cluster/shared/nlpl/data/vectors/latest/221/''

@@ Line 19: / Line 19: @@
 - Available locally on Saga: ''/cluster/shared/nlpl/data/vectors/latest/216/''
-'''NorBERT 1''' features a custom 30 000 WordPiece.
+'''NorBERT 1''' features a custom 30 000 WordPiece vocabulary.
 ==NorBERT 2==

@@ Line 89: / Line 89: @@
 | Binary sentiment analysis (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/sentiment_finetuning.py code] || 67.7 || 71.8 || 77.1 || '''84.2''' || 83.9
 |-
-| Named entity recognition Bokmål (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 78.8  || || 85.5  || 88.2 ||  '''90.2'''
+| Named entity recognition Bokmål (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 78.8  || 84.5 || 85.5  || 88.2 ||  '''90.2'''
 |-
-| Named entity recognition Nynorsk (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 81.7 || ||  82.8 ||  84.5 || '''88.6'''
+| Named entity recognition Nynorsk (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 81.7 || 86.6 ||  82.8 ||  84.5 || '''88.6'''
 |-
 |}

@@ Line 83: / Line 83: @@
 | Part-of-Speech tagging Bokmål (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 98.0 || 97.5 || 98.5 || 98.3 || '''98.7'''
 |-
-| Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || 97.5 || 98.0 || 97.7 || '''98.3'''
+| Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || 97.3 || 98.0 || 97.7 || '''98.3'''
 |-
 | Fine-grained sentiment analysis (Mean Targeted F1 across 5 runs) [https://github.com/jerbarnes/sentiment_graphs code] || 34.9 || 33.9 || '''35.0''' || 33.8  || 34.4

@@ Line 81: / Line 81: @@
 ! Model/task !! mBERT !! XLM-R !! NorBERT !! NorBERT 2 !! NB-BERT-Base
 |-
-| Part-of-Speech tagging Bokmål (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 98.0 || || 98.5 || 98.3 || '''98.7'''
+| Part-of-Speech tagging Bokmål (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 98.0 || 97.5 || 98.5 || 98.3 || '''98.7'''
 |-
-| Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || || 98.0 || 97.7 || '''98.3'''
+| Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || 97.5 || 98.0 || 97.7 || '''98.3'''
 |-
 | Fine-grained sentiment analysis (Mean Targeted F1 across 5 runs) [https://github.com/jerbarnes/sentiment_graphs code] || 34.9 || 33.9 || '''35.0''' || 33.8  || 34.4
 |-
-| Binary sentiment analysis (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/sentiment_finetuning.py code] || 67.7 || || 77.1 || '''84.2''' || 83.9
+| Binary sentiment analysis (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/sentiment_finetuning.py code] || 67.7 || 71.8 || 77.1 || '''84.2''' || 83.9
 |-
 | Named entity recognition Bokmål (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 78.8  || || 85.5  || 88.2 ||  '''90.2'''

@@ Line 79: / Line 79: @@
 {| class="wikitable"
 |-
-! Model/task !! mBERT !! NorBERT !! NorBERT 2 !! NB-BERT-Base
+! Model/task !! mBERT !! XLM-R !! NorBERT !! NorBERT 2 !! NB-BERT-Base
 |-
-| Part-of-Speech tagging Bokmål (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 98.0 || 98.5 || 98.3 || '''98.7'''
+| Part-of-Speech tagging Bokmål (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 98.0 || || 98.5 || 98.3 || '''98.7'''
 |-
-| Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || 98.0 || 97.7 || '''98.3'''
+| Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || || 98.0 || 97.7 || '''98.3'''
 |-
-| Fine-grained sentiment analysis (Mean Targeted F1 across 5 runs) [https://github.com/jerbarnes/sentiment_graphs code] || 34.9 || '''35.0''' || 33.8  || 34.4
+| Fine-grained sentiment analysis (Mean Targeted F1 across 5 runs) [https://github.com/jerbarnes/sentiment_graphs code] || 34.9 || || '''35.0''' || 33.8  || 34.4
 |-
-| Binary sentiment analysis (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/sentiment_finetuning.py code] || 67.7 || 77.1 || '''84.2''' || 83.9
+| Binary sentiment analysis (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/sentiment_finetuning.py code] || 67.7 || || 77.1 || '''84.2''' || 83.9
 |-
-| Named entity recognition Bokmål (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 78.8  || 85.5  || 88.2 ||  '''90.2'''
+| Named entity recognition Bokmål (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 78.8  || || 85.5  || 88.2 ||  '''90.2'''
 |-
-| Named entity recognition Nynorsk (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 81.7 ||  82.8 ||  84.5 || '''88.6'''
+| Named entity recognition Nynorsk (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/bert_ner.py code] || 81.7 || ||  82.8 ||  84.5 || '''88.6'''
 |-
 |}

@@ Line 85: / Line 85: @@
 | Part-of-Speech tagging Nynorsk (accuracy) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/pos_finetuning.py code] || 97.9 || 98.0 || 97.7 || '''98.3'''
 |-
-| Fine-grained sentiment analysis (Targeted F1) [https://github.com/jerbarnes/sentiment_graphs code] || 34.9 || '''35.3''' || 33.7  || 34.4
+| Fine-grained sentiment analysis (Targeted F1) [https://github.com/jerbarnes/sentiment_graphs code] || 34.9 || '''35.0''' || 33.8  || 34.4
 |-
 | Binary sentiment analysis (F1 score) [https://github.com/ltgoslo/NorBERT/blob/main/benchmarking/experiments/sentiment_finetuning.py code] || 67.7 || 77.1 || '''84.2''' || 83.9