Infrastructure/software/pytorch

From Nordic Language Processing Laboratory
(Difference between revisions)
Jump to: navigation, search
m (Oe moved page Infrastructure/pytorch to Infrastructure/software/pytorch without leaving a redirect)
(Installation on Abel)
Line 6: Line 6:
 
of activity.
 
of activity.
  
= Installation on Abel =
+
= Usage on Abel =
  
 
The module <tt>nlpl-pytorch</tt> provides a PyTorch installation
 
The module <tt>nlpl-pytorch</tt> provides a PyTorch installation
 
in a Python 3.5 virtual environment.
 
in a Python 3.5 virtual environment.
 +
Besides PyTorch and its dependencies (e.g. NumPy), the virtual
 +
environment includes a selection of popular add-on packages, e.g.
 +
[http://scikit-learn.org/stable/documentation.html SciKit-Learn],
 +
the [https://pandas.pydata.org/ Python Data Analysis Library] (Pandas),
 +
[https://radimrehurek.com/gensim/ GenSim],
 +
and [https://keras.io/ Keras].
 +
This installation should support both cpu and gpu nodes on Abel.
 +
 +
= Installation on Abel =
 +
  
 
<pre>
 
<pre>

Revision as of 16:12, 20 September 2018

Background

PyTorch is one of the widely used programming frameworks for Deep Learning; in and of itself it is not NLP-specific, but it is a critical dependency for several of the NLPL strands of activity.

Usage on Abel

The module nlpl-pytorch provides a PyTorch installation in a Python 3.5 virtual environment. Besides PyTorch and its dependencies (e.g. NumPy), the virtual environment includes a selection of popular add-on packages, e.g. SciKit-Learn, the Python Data Analysis Library (Pandas), GenSim, and Keras. This installation should support both cpu and gpu nodes on Abel.

Installation on Abel

module purge
module load cuda/8.0
module load python3/3.5.0
cd /projects/nlpl/software
mkdir pytorch
virtualenv pytorch/0.4.1

Next, we need to create a module definition, in this case /projects/nlpl/software/modulefiles/nlpl-pytorch/0.4.1.

module load nlpl-pytorch/0.4.1
pip install --upgrade pip
pip install --upgrade numpy pillow six
pip install torch torchvision
wget -O /tmp/requirements.txt \
  https://raw.githubusercontent.com/OpenNMT/OpenNMT-py/master/requirements.txt
pip install -r /tmp/requirements.txt
Personal tools
Namespaces

Variants
Actions
Navigation
Tools