Difference between revisions of "Eosc/easybuild/modules"
(→List of modules) |
(→"Bundle" modules) |
||
| Line 17: | Line 17: | ||
=== "Bundle" modules === | === "Bundle" modules === | ||
These are the modules with the most cryptic names. Each of them contains a bunch of software pieces (Python packages, as a rule). | These are the modules with the most cryptic names. Each of them contains a bunch of software pieces (Python packages, as a rule). | ||
| + | Many of these modules will be loaded automatically as dependencies of the main modules, but sometimes they can be useful themselves. | ||
Here are the details: | Here are the details: | ||
| − | * nlpl-python-candy/2021.01-gomkl-2019b-Python-3.7.4 | + | * '''nlpl-python-candy/2021.01-gomkl-2019b-Python-3.7.4''': various utility packages not directly related to NLP |
| − | * nlpl-nlptools/2021.01-gomkl-2019b-Python-3.7.4 | + | ** tqdm 4.62.3 |
| − | * nlpl-scikit-bundle/0.22.2.post1-gomkl-2019b-Python-3.7.4 | + | ** pydot 1.4.2 |
| − | * nlpl-scipy-ecosystem/2021.01-gomkl-2019b-Python-3.7.4 | + | ** smart_open 5.2.1 |
| + | ** cached-property 1.5.2 | ||
| + | ** filelock 3.0.12 | ||
| + | ** regex 2021.10.23 | ||
| + | ** sacremoses 0.0.46 | ||
| + | ** mpi4py 3.1.1 | ||
| + | ** jsonlines 2.0.0 | ||
| + | * '''nlpl-nlptools/2021.01-gomkl-2019b-Python-3.7.4''' | ||
| + | * '''nlpl-scikit-bundle/0.22.2.post1-gomkl-2019b-Python-3.7.4''' | ||
| + | * '''nlpl-scipy-ecosystem/2021.01-gomkl-2019b-Python-3.7.4''' | ||
=== "Regular" modules === | === "Regular" modules === | ||
Revision as of 15:49, 5 November 2021
Contents
NLPL virtual laboratory
The laboratory is a reproducible custom-built set of NLP software. It is currently installed on Saga and Puhti HPC clusters.
To use on Saga: run the following command on (can be put in the ~/.bashrc file to be run automatically at login):
module use -a /cluster/shared/nlpl/software/eb/etc/all/
After that, the "nlpl"-branded modules will be available via module avail, module load, etc.
List of modules
Note that the modules which have "gomkl" in their names are built using Intel Math Kernel Library, making them significantly faster in CPU tasks with Intel processors.
"Bundle" modules
These are the modules with the most cryptic names. Each of them contains a bunch of software pieces (Python packages, as a rule). Many of these modules will be loaded automatically as dependencies of the main modules, but sometimes they can be useful themselves. Here are the details:
- nlpl-python-candy/2021.01-gomkl-2019b-Python-3.7.4: various utility packages not directly related to NLP
- tqdm 4.62.3
- pydot 1.4.2
- smart_open 5.2.1
- cached-property 1.5.2
- filelock 3.0.12
- regex 2021.10.23
- sacremoses 0.0.46
- mpi4py 3.1.1
- jsonlines 2.0.0
- nlpl-nlptools/2021.01-gomkl-2019b-Python-3.7.4
- nlpl-scikit-bundle/0.22.2.post1-gomkl-2019b-Python-3.7.4
- nlpl-scipy-ecosystem/2021.01-gomkl-2019b-Python-3.7.4
"Regular" modules
These are more obvious modules, each one gives you one software piece:
- nlpl-cython/0.29.21-gomkl-2019b-Python-3.7.4
- nlpl-dllogger/0.1.0-gomkl-2019b-Python-3.7.4
- nlpl-gensim/3.8.3-gomkl-2019b-Python-3.7.4
- nlpl-horovod/0.20.3-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4
- nlpl-nltk/3.5-gomkl-2019b-Python-3.7.4
- nlpl-numpy/1.18.1-gomkl-2019b-Python-3.7.4
- nlpl-nvidia-bert/20.06.8-gomkl-2019b-tensorflow-1.15.2-Python-3.7.4
- nlpl-pytorch/1.6.0-gomkl-2019b-cuda-10.1.243-Python-3.7.4
- nlpl-simple_elmo/0.6.0-gomkl-2019b-Python-3.7.4
- nlpl-stanza/1.1.1-gomkl-2019b-Python-3.7.4
- nlpl-tensorflow/1.15.2-gomkl-2019b-cuda-10.1.243-Python-3.7.4
- nlpl-tensorflow/2.3.2-gomkl-2019b-cuda-10.1.243-Python-3.7.4
- nlpl-tokenizers/0.10.2-gomkl-2019b-Python-3.7.4
- nlpl-transformers/4.5.1-gomkl-2019b-Python-3.7.4
- nlpl-wandb/0.12.6-gomkl-2019b-Python-3.7.4
- sentencepiece/0.1.94-gomkl-2019b-Python-3.7.4
Source
Currently, the virtual laboratory is generated using EasyBuild, all the code and easyconfigs available here.
