Infrastructure/software/tensorflow

From Nordic Language Processing Laboratory
(Difference between revisions)
Jump to: navigation, search
m (Oe moved page Infrastructure/frameworks/tensorflow to Infrastructure/software/tensorflow without leaving a redirect)
(Installation on Abel)
Line 5: Line 5:
  
 
= Installation on Abel =
 
= Installation on Abel =
 +
The module <tt>nlpl-pytorch</tt> provides a PyTorch installation
 +
in a Python 3.5 virtual environment.
 +
 +
<pre>
 +
module purge
 +
module load gcc/4.9.2 cuda/8.0
 +
module load python3/3.5.0
 +
</pre>
 +
 +
<pre>
 +
cd /projects/nlpl/software
 +
mkdir tensorflow
 +
virtualenv tensorflow/1.11
 +
</pre>
 +
 +
First things first: Enable use of our custom (more modern) GNU C Library
 +
installation, by wrapping the basic <tt>python</tt> binary:
 +
<pre>
 +
mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5
 +
sed 's@pytorch/0.4.1@tensorflow/1.11@' \
 +
  /projects/nlpl/software/pytorch/0.4.1/bin/python3.5 \
 +
  > /projects/nlpl/software/tensorflow/1.11/bin/python3.5
 +
</pre>
 +
 +
Next, create a module definition, in this case
 +
<tt>/projects/nlpl/software/modulefiles/nlpl-tensorflow/1.11</tt>.
 +
 +
<pre>
 +
module load nlpl-pytorch/0.4.1
 +
pip install --upgrade pip
 +
pip install --upgrade numpy pillow six
 +
pip install torch torchvision
 +
</pre>
 +
 +
<pre>
 +
wget -O /tmp/requirements.txt \
 +
  https://raw.githubusercontent.com/OpenNMT/OpenNMT-py/master/requirements.txt
 +
pip install -r /tmp/requirements.txt
 +
</pre>

Revision as of 16:35, 19 September 2018

Background

TensorFlow is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.


Installation on Abel

The module nlpl-pytorch provides a PyTorch installation in a Python 3.5 virtual environment.

module purge
module load gcc/4.9.2 cuda/8.0
module load python3/3.5.0
cd /projects/nlpl/software
mkdir tensorflow
virtualenv tensorflow/1.11

First things first: Enable use of our custom (more modern) GNU C Library installation, by wrapping the basic python binary:

mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5
sed 's@pytorch/0.4.1@tensorflow/1.11@' \
  /projects/nlpl/software/pytorch/0.4.1/bin/python3.5 \
  > /projects/nlpl/software/tensorflow/1.11/bin/python3.5

Next, create a module definition, in this case /projects/nlpl/software/modulefiles/nlpl-tensorflow/1.11.

module load nlpl-pytorch/0.4.1
pip install --upgrade pip
pip install --upgrade numpy pillow six
pip install torch torchvision
wget -O /tmp/requirements.txt \
  https://raw.githubusercontent.com/OpenNMT/OpenNMT-py/master/requirements.txt
pip install -r /tmp/requirements.txt
Personal tools
Namespaces

Variants
Actions
Navigation
Tools