Difference between revisions of "Infrastructure/software/tensorflow"
(→Installation on Abel) |
(→Installation on Abel) |
||
| Line 21: | Line 21: | ||
First things first: Enable use of our custom (more modern) GNU C Library | First things first: Enable use of our custom (more modern) GNU C Library | ||
| − | installation, by wrapping the basic <tt>python</tt> binary: | + | installation, by [http://wiki.nlpl.eu/index.php/Infrastructure/software/glibc wrapping the basic <tt>python</tt> binary]: |
<pre> | <pre> | ||
mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5 | mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5 | ||
| Line 44: | Line 44: | ||
module use -a /projects/nlpl/software/modulefiles | module use -a /projects/nlpl/software/modulefiles | ||
module load nlpl-tensorflow | module load nlpl-tensorflow | ||
| + | pip install --upgrade tensorflow-gpu | ||
</pre> | </pre> | ||
Revision as of 15:07, 19 September 2018
Background
TensorFlow is one of the most widely used Deep Learning frameworks in NLP (in mid-2018, at least), with corporate support from Google.
Installation on Abel
The module nlpl-pytorch provides a PyTorch installation in a Python 3.5 virtual environment.
module purge module load gcc/4.9.2 cuda/8.0 module load python3/3.5.0
cd /projects/nlpl/software mkdir tensorflow virtualenv tensorflow/1.11
First things first: Enable use of our custom (more modern) GNU C Library installation, by wrapping the basic python binary:
mv /projects/nlpl/software/tensorflow/1.11/bin/{,.}python3.5
sed 's@pytorch/0.4.1@tensorflow/1.11@' \
/projects/nlpl/software/pytorch/0.4.1/bin/python3.5 \
> /projects/nlpl/software/tensorflow/1.11/bin/python3.5
chmod 755 /projects/nlpl/software/tensorflow/1.11/bin/python3.5
Next, create a module definition, in this case /projects/nlpl/software/modulefiles/nlpl-tensorflow/1.11.
module load nlpl-tensorflow/1.11
pip install --upgrade pip
pip install --upgrade $(pip list | tail -n +3 | awk '{print $1}')
qlogin --account=nn9106k --time=6:00:00 --nodes=1 --ntasks-per-node=1 --partition=accel --gres=gpu:1 module purge module use -a /projects/nlpl/software/modulefiles module load nlpl-tensorflow pip install --upgrade tensorflow-gpu
