From Nordic Language Processing Laboratory
Revision as of 11:49, 26 September 2018 by Oe (talk | contribs) (Usage on Abel)
Jump to: navigation, search


In mid-2018 at least, PyTorch is one of the widely used programming frameworks for Deep Learning (with corporate backing by facebook); in and of itself PyTorch is not NLP-specific, but it is a critical dependency for several of the NLPL strands of activity.

Usage on Abel

The module nlpl-pytorch provides a PyTorch installation in a Python 3.5 virtual environment. Besides PyTorch and its dependencies (e.g. NumPy), the virtual environment includes a selection of popular add-on packages, e.g. SciKit-Learn, the Python Data Analysis Library (Pandas), GenSim, and Keras. This installation should support both cpu and gpu nodes on Abel.

module purge
module use -a /projects/nlpl/software/modulefiles
module load nlpl-pytorch

There is a short sample program that test availability of cpu vs. gpu computing devices.

python /projects/nlpl/software/pytorch/0.4.1/

Available Versions

On Abel, as of September 2018, PyTorch 0.4.1 is available.

Installation on Abel

module purge
module load cuda/8.0
module load python3/3.5.0
cd /projects/nlpl/software
mkdir pytorch
virtualenv pytorch/0.4.1

Next, we need to create a module definition, in this case /projects/nlpl/software/modulefiles/nlpl-pytorch/0.4.1.

module load nlpl-pytorch/0.4.1
pip install --upgrade pip
pip install --upgrade numpy pillow six
pip install torch torchvision
wget -O /tmp/requirements.txt \
pip install -r /tmp/requirements.txt