Difference between revisions of "Eosc/easybuild/andreku"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(Remaining issues)
(Status)
Line 21: Line 21:
  
 
21/11/2020: successfully built everything (including TensorFlow 1.15.2) with the '''gomkl''' toolchain.
 
21/11/2020: successfully built everything (including TensorFlow 1.15.2) with the '''gomkl''' toolchain.
;
+
 
 +
22/11/2020: built Horovod and made sure the TensorFlow+Horovod combination is able to train a Bert model.
  
 
= To use: =
 
= To use: =

Revision as of 20:03, 23 November 2020

Important stuff to remember

export EB_PYTHON=python3

module load EasyBuild/4.3.0

Playground on Saga: /cluster/shared/nlpl/software/easybuild_ak

export EASYBUILD_ROBOT_PATHS=/cluster/software/EasyBuild/4.3.0/easybuild/easyconfigs:/cluster/shared/nlpl/software/easybuild_ak

(or just source PATH.local)

Repository: https://source.coderefinery.org/nlpl/easybuild/-/tree/ak-dev

Status

03/11/2020: successfully built cython-0.29.21-foss-2019b-Python-3.7.4, numpy-1.18.1-foss-2019b-Python-3.7.4, SciPy-bundle-2020.03-foss-2019b-Python-3.7.4, Bazel-0.26.1-foss-2019b, h5py-2.10.0-foss-2019b-Python-3.7.4.

04/11/2020: TensorFlow 1.15.2 successfully built and installed, using CUDA 10.1.243

19/11/2020: gomkl toolchain built with Intel MKL 2019.1.144

21/11/2020: successfully built everything (including TensorFlow 1.15.2) with the gomkl toolchain.

22/11/2020: built Horovod and made sure the TensorFlow+Horovod combination is able to train a Bert model.

To use:

module use -a /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/modules/all/

module load NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4

Remaining issues