Difference between revisions of "Translation/home"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(Background)
(Using the Moses module)
Line 18: Line 18:
 
= Using the Moses module =
 
= Using the Moses module =
  
* Log into Taito
+
* Log into Taito or Abel
 
* Activate the NLPL module repository:
 
* Activate the NLPL module repository:
  module use -a /proj/nlpl/software/modulefiles/
+
  module use -a /proj/nlpl/software/modulefiles/       # Taito
 +
module use -a /projects/nlpl/software/modulefiles/  # Abel
 
* Load the most recent version of the Moses module:
 
* Load the most recent version of the Moses module:
 
  module load moses
 
  module load moses
 
* Start using Moses, e.g. using the tutorial at http://statmt.org/moses/ <br />
 
* Start using Moses, e.g. using the tutorial at http://statmt.org/moses/ <br />
 
For word alignment, you can use GIZA++, Mgiza and fast_align. The word alignment tools efmaral and eflomal are part of a separate module.
 
For word alignment, you can use GIZA++, Mgiza and fast_align. The word alignment tools efmaral and eflomal are part of a separate module.
 +
* The module contains the standard installation as described at http://www.statmt.org/moses/?n=Development.GetStarted :
 +
** cmph, irstlm, xmlprc
 +
** with-mm
 +
** max-kenlm-order 10
 +
** max-factors 7
 +
** SALM + filter-pt
 
* If you need to specify absolute paths in your scripts, you can find them on the help page of the module:
 
* If you need to specify absolute paths in your scripts, you can find them on the help page of the module:
 
  module help moses
 
  module help moses

Revision as of 11:06, 24 November 2017

Background

An experimentation environment for Statistical and Neural Machine Translations (SMT and NMT) is maintained for NLPL under the coordination of the University of Helsinki (UoH). Initially, the software and data are commissioned on the Finnish Taito supercluster.

Current status (11/2017):

  • moses module: SMT pipeline (Moses + various word alignment tools) installed on Taito and Abel (Moses release 4.0)
  • efmaral module: efmaral and eflomal word alignment tools installed on Taito and Abel
  • Older versions of moses and efmaral modules (installed 7/2017) are still available on Taito

Coming up (Goal: 12/2017):

  • NMT toolkits
  • Datasets

Using the Moses module

  • Log into Taito or Abel
  • Activate the NLPL module repository:
module use -a /proj/nlpl/software/modulefiles/       # Taito
module use -a /projects/nlpl/software/modulefiles/   # Abel
  • Load the most recent version of the Moses module:
module load moses

For word alignment, you can use GIZA++, Mgiza and fast_align. The word alignment tools efmaral and eflomal are part of a separate module.

  • The module contains the standard installation as described at http://www.statmt.org/moses/?n=Development.GetStarted :
    • cmph, irstlm, xmlprc
    • with-mm
    • max-kenlm-order 10
    • max-factors 7
    • SALM + filter-pt
  • If you need to specify absolute paths in your scripts, you can find them on the help page of the module:
module help moses

Contact: Yves Scherrer, University of Helsinki, firstname.lastname@helsinki.fi