Difference between revisions of "Eosc/easybuild/modules"
(→List of modules) |
(→List of modules) |
||
| Line 14: | Line 14: | ||
Intel Math Kernel Library, making them significantly faster in CPU tasks | Intel Math Kernel Library, making them significantly faster in CPU tasks | ||
with Intel processors. | with Intel processors. | ||
| + | |||
| + | === "Bundle" modules === | ||
| + | These are the modules with the most cryptic names. Each of them contains a bunch of software pieces (Python packages, as a rule). | ||
| + | Here are the details: | ||
| + | |||
| + | * nlpl-python-candy/2021.01-gomkl-2019b-Python-3.7.4 | ||
| + | * nlpl-nlptools/2021.01-gomkl-2019b-Python-3.7.4 | ||
| + | * nlpl-scikit-bundle/0.22.2.post1-gomkl-2019b-Python-3.7.4 | ||
| + | * nlpl-scipy-ecosystem/2021.01-gomkl-2019b-Python-3.7.4 | ||
| + | |||
| + | === "Regular" modules === | ||
| + | These are more obvious modules, each one gives you one software piece: | ||
* nlpl-cython/0.29.21-gomkl-2019b-Python-3.7.4 | * nlpl-cython/0.29.21-gomkl-2019b-Python-3.7.4 | ||
| Line 19: | Line 31: | ||
* nlpl-gensim/3.8.3-gomkl-2019b-Python-3.7.4 | * nlpl-gensim/3.8.3-gomkl-2019b-Python-3.7.4 | ||
* nlpl-horovod/0.20.3-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4 | * nlpl-horovod/0.20.3-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4 | ||
| − | |||
* nlpl-nltk/3.5-gomkl-2019b-Python-3.7.4 | * nlpl-nltk/3.5-gomkl-2019b-Python-3.7.4 | ||
* nlpl-numpy/1.18.1-gomkl-2019b-Python-3.7.4 | * nlpl-numpy/1.18.1-gomkl-2019b-Python-3.7.4 | ||
* nlpl-nvidia-bert/20.06.8-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4 | * nlpl-nvidia-bert/20.06.8-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4 | ||
| − | |||
* nlpl-pytorch/1.6.0-gomkl-2019b-cuda-10.1.243-Python-3.7.4 | * nlpl-pytorch/1.6.0-gomkl-2019b-cuda-10.1.243-Python-3.7.4 | ||
| − | |||
| − | |||
* nlpl-simple_elmo/0.6.0-gomkl-2019b-Python-3.7.4 | * nlpl-simple_elmo/0.6.0-gomkl-2019b-Python-3.7.4 | ||
* nlpl-stanza/1.1.1-gomkl-2019b-Python-3.7.4 | * nlpl-stanza/1.1.1-gomkl-2019b-Python-3.7.4 | ||
Revision as of 13:27, 5 November 2021
Contents
NLPL virtual laboratory
The laboratory is a reproducible custom-built set of NLP software. It is currently installed on Saga and Puhti HPC clusters.
To use on Saga: run the following command on (can be put in the ~/.bashrc file to be run automatically at login):
module use -a /cluster/shared/nlpl/software/eb/etc/all/
After that, the "nlpl"-branded modules will be available via module avail, module load, etc.
List of modules
Note that the modules which have "gomkl" in their names are built using Intel Math Kernel Library, making them significantly faster in CPU tasks with Intel processors.
"Bundle" modules
These are the modules with the most cryptic names. Each of them contains a bunch of software pieces (Python packages, as a rule). Here are the details:
- nlpl-python-candy/2021.01-gomkl-2019b-Python-3.7.4
- nlpl-nlptools/2021.01-gomkl-2019b-Python-3.7.4
- nlpl-scikit-bundle/0.22.2.post1-gomkl-2019b-Python-3.7.4
- nlpl-scipy-ecosystem/2021.01-gomkl-2019b-Python-3.7.4
"Regular" modules
These are more obvious modules, each one gives you one software piece:
- nlpl-cython/0.29.21-gomkl-2019b-Python-3.7.4
- nlpl-dllogger/0.1.0-gomkl-2019b-Python-3.7.4
- nlpl-gensim/3.8.3-gomkl-2019b-Python-3.7.4
- nlpl-horovod/0.20.3-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4
- nlpl-nltk/3.5-gomkl-2019b-Python-3.7.4
- nlpl-numpy/1.18.1-gomkl-2019b-Python-3.7.4
- nlpl-nvidia-bert/20.06.8-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4
- nlpl-pytorch/1.6.0-gomkl-2019b-cuda-10.1.243-Python-3.7.4
- nlpl-simple_elmo/0.6.0-gomkl-2019b-Python-3.7.4
- nlpl-stanza/1.1.1-gomkl-2019b-Python-3.7.4
- nlpl-tensorflow/1.15.2-gomkl-2019b-cuda-10.1.243-Python-3.7.4
- nlpl-tensorflow/2.3.2-gomkl-2019b-cuda-10.1.243-Python-3.7.4
- nlpl-tokenizers/0.10.2-gomkl-2019b-Python-3.7.4
- nlpl-transformers/4.5.1-gomkl-2019b-Python-3.7.4
- nlpl-wandb/0.12.6-gomkl-2019b-Python-3.7.4
- sentencepiece/0.1.94-gomkl-2019b-Python-3.7.4
Source
Currently, the virtual laboratory is generated using EasyBuild, all the code and easyconfigs available here.
