Infrastructure/software/pytorch: Difference between revisions
| Line 65: | Line 65: | ||
https://raw.githubusercontent.com/OpenNMT/OpenNMT-py/master/requirements.txt | https://raw.githubusercontent.com/OpenNMT/OpenNMT-py/master/requirements.txt | ||
pip install -r /tmp/requirements.txt | pip install -r /tmp/requirements.txt | ||
</pre> | |||
= Installation on Taito = | |||
<pre> | |||
module purge | |||
module load cuda/9.0 | |||
module load python-env/3.5.3 | |||
</pre> | |||
<pre> | |||
cd /proj/nlpl/software | |||
svn co http://svn.nlpl.eu/software/pytorch | |||
virtualenv pytorch/0.4.1 | |||
</pre> | |||
Next, we need to create a module definition, in this case | |||
<tt>/proj/nlpl/software/modulefiles/nlpl-pytorch/0.4.1.lua</tt>. | |||
<pre> | |||
module load nlpl-pytorch/0.4.1 | |||
pip install --upgrade pip | |||
pip install --upgrade -r /proj/nlpl/software/pytorch/0.4.1/modules.txt | |||
</pre> | </pre> | ||
Revision as of 19:21, 26 September 2018
Background
In mid-2018 at least, PyTorch is one of the widely used programming frameworks for Deep Learning (with corporate backing by facebook); in and of itself PyTorch is not NLP-specific, but it is a critical dependency for several of the NLPL strands of activity.
Usage on Abel
The module nlpl-pytorch provides a PyTorch installation in a Python 3.5 virtual environment. Besides PyTorch and its dependencies (e.g. NumPy), the virtual environment includes a selection of popular add-on packages, e.g. SciKit-Learn, the Python Data Analysis Library (Pandas), GenSim, and Keras. This installation should support both cpu and gpu nodes on Abel.
module purge module use -a /projects/nlpl/software/modulefiles module load nlpl-pytorch
There is a short sample program that test availability of cpu vs. gpu computing devices.
python /projects/nlpl/software/pytorch/0.4.1/test.py
Available Versions
On Abel, as of September 2018, PyTorch 0.4.1 is available.
Installation on Abel
module purge module load cuda/8.0 module load python3/3.5.0
cd /projects/nlpl/software mkdir pytorch virtualenv pytorch/0.4.1
Next, we need to create a module definition, in this case /projects/nlpl/software/modulefiles/nlpl-pytorch/0.4.1.
module load nlpl-pytorch/0.4.1 pip install --upgrade pip pip install --upgrade numpy pillow six pip install torch torchvision
wget -O /tmp/requirements.txt \ https://raw.githubusercontent.com/OpenNMT/OpenNMT-py/master/requirements.txt pip install -r /tmp/requirements.txt
Installation on Taito
module purge module load cuda/9.0 module load python-env/3.5.3
cd /proj/nlpl/software svn co http://svn.nlpl.eu/software/pytorch virtualenv pytorch/0.4.1
Next, we need to create a module definition, in this case /proj/nlpl/software/modulefiles/nlpl-pytorch/0.4.1.lua.
module load nlpl-pytorch/0.4.1 pip install --upgrade pip pip install --upgrade -r /proj/nlpl/software/pytorch/0.4.1/modules.txt