Infrastructure/software/tensorflow - Revision history

Oe: /* Installation on Abel */

2018-10-24T21:22:19Z

‎Installation on Abel

2018-10-04T16:05:04Z

‎Installation on Abel

2018-10-02T14:22:41Z

‎Usage on Abel

2018-10-02T14:22:24Z

‎Usage on Abel

2018-09-29T23:13:54Z

‎Available Versions

2018-09-29T22:24:24Z

‎Installation on Taito

2018-09-29T22:22:03Z

‎Installation on Taito

2018-09-29T21:54:01Z

‎Installation on Abel

2018-09-26T11:46:09Z

‎Usage on Abel

2018-09-26T11:40:22Z

‎Background

@@ Line 80: / Line 80: @@
 <pre>
-qlogin --account=nn9447k --time=1:00:00 --partition=accel --gres=gpu:1
+qlogin --account=nn9447k --time=00:30:00 \
 cp -av /usr/lib64/libcuda.so* /usr/lib64/libnvidia* \
    /projects/nlpl/software/tensorflow/1.11/lib

@@ Line 64: / Line 64: @@
 <pre>
 mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5
-sed 's@pytorch/0.4.1@tensorflow/1.11@' \
+cp /projects/nlpl/software/glibc/2.18/wrapper \
-  /projects/nlpl/software/pytorch/0.4.1/bin/python3.5 \
+   /projects/nlpl/software/tensorflow/1.11/bin/python3.5
 </pre>
 Next, create a module definition, in this case
-<tt>/projects/nlpl/software/modulefiles/nlpl-tensorflow/1.11</tt>.
+<tt>/projects/nlpl/software/modulefiles/nlpl-tensorflow/1.11</tt>,
 <pre>

@@ Line 3: / Line 3: @@
 [https://www.tensorflow.org/ TensorFlow] is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.
-= Usage on Abel =
+= Usage =
 The module <tt>nlpl-tensorflow</tt> provides a TensorFlow installation
@@ Line 15: / Line 15: @@
 the [https://pandas.pydata.org/ Python Data Analysis Library] (Pandas),
 and [https://keras.io/ Keras].
-This installation should support both cpu and gpu nodes on Abel.
+This installation should support both cpu and gpu nodes on Abel and Taito.
 <pre>

@@ Line 7: / Line 7: @@
 The module <tt>nlpl-tensorflow</tt> provides a TensorFlow installation
 in a Python 3.5 virtual environment.
 Besides TensorFlow and its dependencies (e.g. NumPy), the virtual
 environment includes a selection of popular add-on packages, e.g.
 [http://scikit-learn.org/stable/documentation.html SciKit-Learn],
 the [https://pandas.pydata.org/ Python Data Analysis Library] (Pandas),
 and [https://keras.io/ Keras].
 This installation should support both cpu and gpu nodes on Abel.
@@ Line 17: / Line 19: @@
 <pre>
 module purge
-module use -a /projects/nlpl/software/modulefiles
+module use -a /proj*/nlpl/software/modulefiles
 module load nlpl-tensorflow
 </pre>
@@ Line 25: / Line 27: @@
 <pre>
-python /projects/nlpl/software/tensorflow/1.11/test.py
+python /proj*/nlpl/software/tensorflow/1.11/test.py
 </pre>

@@ Line 37: / Line 37: @@
 As of September 2018, TensorFlow 1.11 is available (and, thus, the default version for this module).
-An older installation of
+On Abel, an older installation of
-[https://www.uio.no/english/services/it/research/hpc/abel/help/software/tensorflow.html TensorFlow 1.0.1] is provided by USIT (the operators of Abel);
+[https://www.uio.no/english/services/it/research/hpc/abel/help/software/tensorflow.html TensorFlow 1.0.1] is provided by USIT (the system operators);
 this installation is ‘containerized’, however, i.e. is not easily interoperable with other software modules, and it does not work transparently on both cpu and gpu nodes.
 = Installation on Abel =

@@ Line 115: / Line 115: @@
 pip install --upgrade pip
 pip install --upgrade $(pip list | tail -n +3 | awk '{print $1}')
-pip install --upgrade -r /projects/nlpl/software/tensorflow/1.11/modules.txt
+pip install --upgrade -r /proj/nlpl/software/tensorflow/1.11/modules.txt
 </pre>
@@ Line 126: / Line 126: @@
 module load nlpl-tensorflow
 pip install --upgrade tensorflow-gpu
 </pre>

@@ Line 95: / Line 95: @@
 <pre>
-mkdir /proj/nlpl/software/tensorflow
+cd /proj/nlpl/software
 virtualenv /proj/nlpl/software/tensorflow/1.11
 </pre>
@@ Line 118: / Line 119: @@
 <pre>
-qlogin --account=nn9447k --time=1:00:00 --partition=accel --gres=gpu:1
+ssh taito-gpu
 cp -av /usr/lib64/libcuda.so* /usr/lib64/libnvidia* \
-   /projects/nlpl/software/tensorflow/1.11/lib
+   /proj/nlpl/software/tensorflow/1.11/lib
 module purge
 module use -a /projects/nlpl/software/modulefiles

@@ Line 1: / Line 1: @@
 = Background =
-TensorFlow is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.
+[https://www.tensorflow.org/ TensorFlow] is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.
 = Usage on Abel =