Vectors/elmo/tutorial

From Nordic Language Processing Laboratory
(Difference between revisions)
Jump to: navigation, search
(Created page with "= Background =")
 
Line 1: Line 1:
 
= Background =
 
= Background =
 +
 +
[https://allennlp.org/elmo ELMo] is a family of contextualized word embeddings first introduced in [Peter et al. 2018].
 +
 +
= Training ELMo on Saga =
 +
 +
Example SLURM file:
 +
 +
#!/bin/bash
 +
#SBATCH --job-name=elmo
 +
#SBATCH --mail-type=FAIL
 +
#SBATCH --account=nn9447k  # Use your project number
 +
#SBATCH --partition=accel    # To use the accelerator nodes
 +
#SBATCH --gres=gpu:2        # To specify how many GPUs to use
 +
#SBATCH --time=10:00:00      # Max walltime is 14 days.
 +
#SBATCH --mem-per-cpu=6G
 +
#SBATCH --ntasks=8
 +
set -o errexit  # Recommended for easier debugging
 +
module purge  # Recommended for reproducibility
 +
module load Anaconda3/2019.03
 +
# >>> conda initialize >>>
 +
# !! Contents within this block are managed by 'conda init' !!
 +
__conda_setup="$('/cluster/software/Anaconda3/2019.03/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
 +
if [ $? -eq 0 ]; then
 +
    eval "$__conda_setup"
 +
else
 +
    if [ -f "/cluster/software/Anaconda3/2019.03/etc/profile.d/conda.sh" ]; then
 +
                . "/cluster/software/Anaconda3/2019.03/etc/profile.d/conda.sh"
 +
    else
 +
        export PATH="/cluster/software/Anaconda3/2019.03/bin:$PATH"
 +
    fi
 +
fi
 +
unset __conda_setup
 +
# <<< conda initialize <<<
 +
conda activate python3.6
 +
python3 bin/train_elmo.py --train_prefix $DATA  --vocab_file $VOCAB --save_dir ${3} --size $SIZE

Revision as of 20:08, 29 September 2019

Background

ELMo is a family of contextualized word embeddings first introduced in [Peter et al. 2018].

Training ELMo on Saga

Example SLURM file:

#!/bin/bash
#SBATCH --job-name=elmo
#SBATCH --mail-type=FAIL
#SBATCH --account=nn9447k  # Use your project number
#SBATCH --partition=accel    # To use the accelerator nodes
#SBATCH --gres=gpu:2         # To specify how many GPUs to use
#SBATCH --time=10:00:00      # Max walltime is 14 days.
#SBATCH --mem-per-cpu=6G
#SBATCH --ntasks=8
set -o errexit  # Recommended for easier debugging
module purge   # Recommended for reproducibility
module load Anaconda3/2019.03
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/cluster/software/Anaconda3/2019.03/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
   eval "$__conda_setup"
else
   if [ -f "/cluster/software/Anaconda3/2019.03/etc/profile.d/conda.sh" ]; then
               . "/cluster/software/Anaconda3/2019.03/etc/profile.d/conda.sh"
   else
       export PATH="/cluster/software/Anaconda3/2019.03/bin:$PATH"
   fi
fi
unset __conda_setup
# <<< conda initialize <<<
conda activate python3.6
python3 bin/train_elmo.py --train_prefix $DATA  --vocab_file $VOCAB --save_dir ${3} --size $SIZE
Personal tools
Namespaces

Variants
Actions
Navigation
Tools