Difference between revisions of "Lumi/pilot"
(→Model Architectures) |
(→Model Architectures) |
||
| Line 16: | Line 16: | ||
* [[Eosc/pretraining#BERT|BERT]] | * [[Eosc/pretraining#BERT|BERT]] | ||
* [[Eosc/pretraining#RoBERTa|RoBERTa]] | * [[Eosc/pretraining#RoBERTa|RoBERTa]] | ||
| − | * [[ | + | * [[Eosc/pretraining#ELECTRA|ELECTRA]] |
* [https://openai.com/blog/gpt-2-1-5b-release/ GPT] | * [https://openai.com/blog/gpt-2-1-5b-release/ GPT] | ||
* [https://arxiv.org/pdf/1910.10683.pdf T5] | * [https://arxiv.org/pdf/1910.10683.pdf T5] | ||
Revision as of 23:29, 25 March 2021
LUMI-G Pilot
In late 2021, the shared LUMI supercomputer will (likely) open for trial usage of its vast gpu partition. NLPL partners in Finland (Turku and Helsinki) and Norway (Oslo) are coordinating their efforts towards the creation of very large-scale (neural) language models for multiple Nordic languages. This work is part of the Nordic Language Modeling (NorLM) initiative.
