Difference between revisions of "Infrastructure/software/tensorflow"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(Installation on Abel)
(Installation on Abel)
Line 2: Line 2:
  
 
TensorFlow is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.
 
TensorFlow is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.
 +
 +
 +
= Usage on Abel =
 +
 +
The module <tt>nlpl-pytorch</tt> provides a TensorFlow installation
 +
in a Python 3.5 virtual environment.
 +
 +
<pre>
 +
module use -a /projects/nlpl/software/modulesfiles
 +
module load nlpl-tensorflow
 +
</pre>
  
  
 
= Installation on Abel =
 
= Installation on Abel =
The module <tt>nlpl-pytorch</tt> provides a PyTorch installation
+
 
in a Python 3.5 virtual environment.
 
  
 
<pre>
 
<pre>

Revision as of 19:02, 19 September 2018

Background

TensorFlow is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.


Usage on Abel

The module nlpl-pytorch provides a TensorFlow installation in a Python 3.5 virtual environment.

module use -a /projects/nlpl/software/modulesfiles
module load nlpl-tensorflow


Installation on Abel

module purge
module load gcc/4.9.2 cuda/9.0
module load python3/3.5.0
cd /projects/nlpl/software
mkdir tensorflow
virtualenv tensorflow/1.11

First things first: Enable use of our custom (more modern) GNU C Library installation, by wrapping the basic python binary:

mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5
sed 's@pytorch/0.4.1@tensorflow/1.11@' \
  /projects/nlpl/software/pytorch/0.4.1/bin/python3.5 \
  > /projects/nlpl/software/tensorflow/1.11/bin/python3.5
chmod 755 /projects/nlpl/software/tensorflow/1.11/bin/python3.5

Next, create a module definition, in this case /projects/nlpl/software/modulefiles/nlpl-tensorflow/1.11.

module load nlpl-tensorflow/1.11
pip install --upgrade pip
pip install --upgrade $(pip list | tail -n +3 | awk '{print $1}')
qlogin --account=nn9106k --time=6:00:00 --nodes=1 --ntasks-per-node=1 --partition=accel --gres=gpu:1
module purge
module use -a /projects/nlpl/software/modulefiles
module load nlpl-tensorflow
pip install --upgrade tensorflow-gpu