Difference between revisions of "Lumi/pilot"
(→Model Architectures) |
(→Software Support) |
||
| Line 21: | Line 21: | ||
= Software Support = | = Software Support = | ||
| + | See the links above for particular model's requirements. | ||
| + | In general, we rely on Python (>=3.7) and its [https://www.scipy.org/ SciPy] stack. | ||
| + | |||
| + | We definitely will require fully functional GPU-enabled installations of PyTorch (1.8.1) and TensorFlow (preferably, both 1.15.5 and 2.4.1). | ||
| + | |||
| + | Multi-GPU and multi-node training must be possible. In the NVIDIA world, [https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/overview.html NCCL] and [https://github.com/horovod/horovod Horovod] are used for this. | ||
| + | In the AMD world? No idea. | ||
= Data: Norwegian = | = Data: Norwegian = | ||
Revision as of 23:45, 25 March 2021
LUMI-G Pilot
In late 2021, the shared LUMI supercomputer will (likely) open for trial usage of its vast gpu partition. NLPL partners in Finland (Turku and Helsinki) and Norway (Oslo) are coordinating their efforts towards the creation of very large-scale (neural) language models for multiple Nordic languages. This work is part of the Nordic Language Modeling (NorLM) initiative.
Model Architectures
Software Support
See the links above for particular model's requirements.
In general, we rely on Python (>=3.7) and its SciPy stack.
We definitely will require fully functional GPU-enabled installations of PyTorch (1.8.1) and TensorFlow (preferably, both 1.15.5 and 2.4.1).
Multi-GPU and multi-node training must be possible. In the NVIDIA world, NCCL and Horovod are used for this. In the AMD world? No idea.
