Eosc/easybuild/modules - Revision history

Andreku: /* "Bundle" modules */

2025-11-05T13:14:13Z

‎"Bundle" modules

Andreku: /* "Regular" modules */

2025-10-27T11:22:13Z

‎"Regular" modules

Andreku: /* List of modules */

2025-10-21T23:24:54Z

‎List of modules

Andreku: 2024a

2025-10-21T23:23:19Z

2024a

Andreku: /* Others */

2025-03-24T18:21:26Z

‎Others

Andreku: /* Others */

2024-09-30T12:43:45Z

‎Others

Andreku: /* Others */

2024-08-05T17:16:16Z

‎Others

Andreku: /* "Bundle" modules */

2024-06-14T10:16:28Z

‎"Bundle" modules

Andreku at 10:14, 14 June 2024

2024-06-14T10:14:57Z

Andreku: /* "Bundle" modules */

2024-01-29T13:14:20Z

‎"Bundle" modules

@@ Line 80: / Line 80: @@
 ** ufal.chu-liu-edmonds
 ** gensim
 * '''nlpl-scipy-ecosystem/VERS-ARCH''': everything that constitutes the [https://scipy.org/ SciPy ecosystem]. Too many packages to enumerate them all, but the most important are:
 ** scipy

@@ Line 150: / Line 150: @@
 * '''nlpl-torch-geometric/VERS-ARCH''': [https://pyg.org/ PyTorch Geometric]
 * '''nlpl-torchmetrics/VERS-ARCH''': [https://pypi.org/project/torchmetrics/ TorchMetrics]
 * '''nlpl-trl/VERS-ARCH''': [https://huggingface.co/docs/trl/index HuggingFace Transformer Reinforcement Learning]
 * '''nlpl-wandb/VERS-ARCH''': [https://pypi.org/project/wandb/ Weights and Biases (wandb)]
 * '''nlpl-warc2text/VERS-ARCH''': [https://github.com/bitextor/warc2text warc2text]
 * '''nlpl-dllogger/0.1.0-ARCH-2019b-Python-3.7.4''': [https://github.com/NVIDIA/dllogger DLLogger] 0.1.0 ''(status unclear)''
 * '''nlpl-nvidia-bert/20.06.8-ARCH-2019b-tensorflow-1.15.2-Python-3.7.4''': [https://github.com/NVIDIA/DeepLearningExamples/tree/master/TensorFlow/LanguageModeling/BERT NVIDIA's BERT implementation] for TensorFlow 1 ''(status unclear)''

@@ Line 101: / Line 101: @@
 ** mistral_common ([https://pypi.org/project/mistral_common/ Mistral-common]: common utilities for Mistral AI)
 ** ...
-* nlpl-torch-audio-vision/VERS-ARCH: multimodal extensions for PyTorch
+* '''nlpl-torch-audio-vision/VERS-ARCH''': multimodal extensions for PyTorch
 ** torch-vision ([https://github.com/pytorch/vision torchvision]: image and video datasets and models for PyTorch deep learning)
 ** torch-audio ([https://github.com/pytorch/audio torchaudio]: an audio library for PyTorch)

@@ Line 2: / Line 2: @@
 The laboratory is a reproducible custom-built set of NLP software.
-It is currently installed on ''Saga'', ''Fox'', and ''Puhti'' HPC clusters.
+It is currently installed on ''Saga'' and ''Fox'' HPC clusters.
 - To use on ''Saga'': run the following command (can be put in the ''~/.bashrc'' file to be run automatically at login):
@@ Line 12: / Line 12: @@
 ''module use -a /fp/projects01/ec30/software/easybuild/modules/all/''
-After that, the "nlpl"-branded modules will be available via ''module avail'', ''module load'', etc.
+After that, the "nlpl"-branded modules will be available via ''module avail'', ''module load'', etc. See all the NLPL modules with the ''module avail nlpl'' command.
 It is highly recommended to use them, instead of installing a copy in one's own home directory.
@@ Line 24: / Line 24: @@
 with Intel processors (for example, on ''Saga'', '''except ''a100'' partition''' which uses AMD CPUs).
-Those with "foss" in their names are CPU-agnostic and will run on machines with all kinds of CPUs (for example, on ''Fox'', but also on all ''Saga'' partitions).
+Those with "foss" in their names are CPU-agnostic and will run on machines with all kinds of CPUs (for example, on ''Fox'', but also on all ''Saga'' partitions). These are the default ones.
-The next element in the module name after "foss", "gomkl" or "intel" is the virtual laboratory (stack) version: for example, "2019b", "2021a" or "2022b".
+The next element in the module name after "foss", "gomkl" or "intel" is the virtual laboratory (stack) version: for example, "2021a", "2022b" or "2024a".
-Modules from different stack versions are incompatible  with each other: you cannot load a module from "foss-2019b" and a module from "foss-2022b" simultaneously.
+Modules from different stack versions are incompatible  with each other: you cannot load a module from "foss-2021a" and a module from "foss-2024a" simultaneously.
-'''Currently, "2022b" version is the recommended one, with the most up-to-date software built with the most up-to-date compilers.'''
+'''Currently, "2024a" version is the recommended one, with the most up-to-date software built with the most up-to-date compilers.'''
-Further on, we just use the placeholder '''ARCH''', replace it with "gomkl", "intel" or "foss" and "2021a" or "2022b", depending on which machine you are working on and what stack version you want to use.
+Further on, we just use the placeholder '''ARCH''', replace it with "gomkl", "intel" or "foss" and "2022b" or "2024a", depending on which machine you are working on and what stack version you want to use.
-Some modules also have the Python version specified in their names (for example, "''nlpl-numpy-1.24.4-foss-2022b-Python-3.10.8''").
+Some modules also have the Python version specified in their names (for example, "''nlpl-transformers/4.55.4-foss-2024a-Python-3.12.3''").
-For stack version "2021a" it is usually Python 3.9.5, for stack version "2022b" it is usually Python 3.10.8.
+For stack version "2022b" it is usually Python 3.10.8, for stack version "2024a" it is usually Python 3.12.3.
-Check the output of the '''module avail''' command for the exact module names.
+Check the output of the '''module avail nlpl''' command for the exact module names.
 === "Bundle" modules ===
@@ Line 66: / Line 66: @@
 ** plotly
 ** toolz
 ** ...
 * '''nlpl-nlptools/VERS-ARCH''': various utility packages related to NLP
@@ Line 72: / Line 73: @@
 ** seqeval
 ** langdetect
-** leven
+** levenshtein
 ** rouge_score
 ** sacrebleu
 ** udapi
 ** word2number
 * '''nlpl-scipy-ecosystem/VERS-ARCH''': everything that constitutes the [https://scipy.org/ SciPy ecosystem]. Too many packages to enumerate them all, but the most important are:
 ** scipy
@@ Line 90: / Line 91: @@
 ** beautifulsoup4
 ** numexpr
 * '''nlpl-llmtools/VERS-ARCH''': various utility packages for working with large language models (LLMs)
 ** peft ([https://pypi.org/project/peft/ HuggingFace PEFT]: State-of-the-art Parameter-Efficient Fine-Tuning)
@@ Line 95: / Line 98: @@
 ** lm-evaluation-harness ([https://github.com/EleutherAI/lm-evaluation-harness EleutherAI Language Model Evaluation Harness])
 ** bert_score: [https://pypi.org/project/bert-score/ BERTScore] to evaluate NLG tasks
 ** ...
 === "Regular" modules ===
@@ Line 106: / Line 114: @@
 * '''nlpl-pytorch/1.11.0-ARCH-cuda-11.3.1-Python-3.9.5''': [https://pytorch.org/ PyTorch] 1.11.0 (for CUDA 11)
 * '''nlpl-pytorch/2.1.2-ARCH-cuda-12.0.0-Python-3.10.8''': [https://pytorch.org/ PyTorch] 2.1.2 (for CUDA 12)
 * '''nlpl-tensorflow/1.15.2-ARCH-cuda-10.1.243-Python-3.7.4''': [https://www.tensorflow.org/ TensorFlow] 1.15.2 (for CUDA 10)
@@ Line 112: / Line 121: @@
 * '''nlpl-tensorflow/2.6.5-ARCH-cuda-11.3.1-Python-3.9.5''': [https://www.tensorflow.org/ TensorFlow] 2.6.5 (for CUDA 11)
 * '''nlpl-tensorflow/2.15.0-ARCH-cuda-12.0.0-Python-3.10.8''': [https://www.tensorflow.org/ TensorFlow] 2.15.0 (for CUDA 12)
 * '''nlpl-accelerate/0.13.2-ARCH-Python-3.9.5''': [https://pypi.org/project/accelerate/ Accelerate] 0.13.2
 * '''nlpl-accelerate/0.27.2-ARCH-Python-3.10.8''': [https://pypi.org/project/accelerate/ Accelerate] 0.27.2
 ==== Others ====
 * '''nlpl-bitsandbytes/VERS-ARCH''': [https://pypi.org/project/bitsandbytes/ BitsAndBytes]
 * '''nlpl-cython/VERS-ARCH''': [http://cython.org/ Cython]
 * '''nlpl-datasets/VERS-ARCH''': [https://github.com/huggingface/datasets HuggingFace Datasets]
 * '''nlpl-gensim/VERS-ARCH''': [https://github.com/RaRe-Technologies/gensim Gensim]
 * '''nlpl-horovod/VERS-ARCH''': [https://github.com/horovod/horovod Horovod]
-* '''nlpl-huggingface-hub/VERS-ARCH-2019b-Python-3.7.4''': [https://pypi.org/project/huggingface-hub/ HuggingFace Hub]
+* '''nlpl-huggingface-hub/VERS-ARCH''': [https://pypi.org/project/huggingface-hub/ HuggingFace Hub]
 * '''nlpl-nltk/VERS-ARCH''': [https://www.nltk.org/ NLTK], together with '''all''' the corpora and datasets (no need to download them separately!)
 * '''nlpl-numpy/VERS-ARCH''': [https://numpy.org/ NumPy]
-* '''nlpl-pytorch-lightning/VERS-ARCH-cuda-11.3.1''': [https://www.pytorchlightning.ai/ PyTorch Lightning]
+* '''nlpl-pytorch-lightning/VERS-ARCH''': [https://www.pytorchlightning.ai/ PyTorch Lightning]
 * '''nlpl-scikit-bundle/VERS-ARCH''': [https://scikit-learn.org/ Scikit-Learn]
 * '''nlpl-sentencepiece/VERS-ARCH''': [https://github.com/google/sentencepiece SentencePiece]
@@ Line 136: / Line 151: @@
 * '''nlpl-torchmetrics/VERS-ARCH''': [https://pypi.org/project/torchmetrics/ TorchMetrics]
 * '''nlpl-torchtext/VERS-ARCH''': [https://pypi.org/project/torchtext/ TorchText]
 * '''nlpl-trl/VERS-ARCH''': [https://huggingface.co/docs/trl/index HuggingFace Transformer Reinforcement Learning]
 * '''nlpl-wandb/VERS-ARCH''': [https://pypi.org/project/wandb/ Weights and Biases (wandb)]

@@ Line 137: / Line 137: @@
 * '''nlpl-torchtext/VERS-ARCH''': [https://pypi.org/project/torchtext/ TorchText]
 * '''nlpl-transformers/VERS-ARCH''': [https://huggingface.co/transformers/ HuggingFace Transformers]
 * '''nlpl-wandb/VERS-ARCH''': [https://pypi.org/project/wandb/ Weights and Biases (wandb)]
 * '''nlpl-warc2text/VERS-ARCH''': [https://github.com/bitextor/warc2text warc2text]

@@ Line 128: / Line 128: @@
 * '''nlpl-scikit-bundle/VERS-ARCH''': [https://scikit-learn.org/ Scikit-Learn]
 * '''nlpl-sentencepiece/VERS-ARCH''': [https://github.com/google/sentencepiece SentencePiece]
 * '''nlpl-simple_elmo/VERS-ARCH''': [https://pypi.org/project/simple-elmo/ Simple_elmo]
 * '''nlpl-stanza/VERS-ARCH''': [https://stanfordnlp.github.io/stanza/ Stanza]

@@ Line 91: / Line 91: @@
 ** numexpr
 * '''nlpl-llmtools/VERS-ARCH''': various utility packages for working with large language models (LLMs)
-** peft ([HuggingFace PEFT](https://pypi.org/project/peft/): State-of-the-art Parameter-Efficient Fine-Tuning)
+** peft ([https://pypi.org/project/peft/ HuggingFace PEFT]: State-of-the-art Parameter-Efficient Fine-Tuning)
-** promptsource ([Toolkit for creating, sharing and using natural language prompts](https://pypi.org/project/promptsource/))
+** promptsource ([https://pypi.org/project/promptsource/ Toolkit for creating, sharing and using natural language prompts])
-** lm-evaluation-harness ([EleutherAI Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness))
+** lm-evaluation-harness ([https://github.com/EleutherAI/lm-evaluation-harness EleutherAI Language Model Evaluation Harness])
-** bert_score: [BERTScore](https://pypi.org/project/bert-score/) to evaluate NLG tasks
+** bert_score: [https://pypi.org/project/bert-score/ BERTScore] to evaluate NLG tasks
 ** ...

@@ Line 88: / Line 88: @@
 ** peft (HuggingFace PEFT: State-of-the-art Parameter-Efficient Fine-Tuning)
 ** lm-evaluation-harness (EleutherAI Language Model Evaluation Harness)
 ** ...