Infrastructure/installation/guide - Revision history

Oe: /* Open Questions */

2019-09-17T12:05:55Z

‎Open Questions

Oe: /* Version History */

2019-09-17T12:05:27Z

‎Version History

Oe: /* The Modules System */

2019-09-17T11:54:00Z

‎The Modules System

Oe: /* Software */

2019-09-17T11:53:37Z

‎Software

Oe: /* The Modules System */

2019-09-17T09:38:41Z

‎The Modules System

Oe: /* Data */

2019-09-17T09:37:49Z

‎Data

Oe: /* Software */

2019-09-17T09:37:20Z

‎Software

Oe: /* Software */

2019-09-17T09:36:19Z

‎Software

Oe: /* The Modules System */

2017-12-04T21:53:33Z

‎The Modules System

Oe: /* The Modules System */

2017-12-04T21:53:01Z

‎The Modules System

@@ Line 111: / Line 111: @@
 * How to test/validate a software that it actually works (e.g. by someone else than the one who has installed it ... based on the provided documentation)
 * How to test software to ensure functional identity across systems
 * Pick VCS to maintain installation scripts and notes.
 = Version History =

@@ Line 127: / Line 127: @@
 |-
 | 0.4 || 2017-11-06 || oe || updates from the infrastructure task force
 |}

@@ Line 102: / Line 102: @@
 Integrating modules in the system requires the directory containing these files be included in the search path of the module system (see <tt>module use -a</tt> above). For example, include the command
-   module use -a $NLPLCODE/etc
+   module use -a $NLPLCODE/modules/etc
 in your <tt>.bashrc</tt> on Abel and Taito with the correct, system-specific value for <tt>$NLPLCODE</tt> (see above).

@@ Line 46: / Line 46: @@
 The user environment on Abel and Taito is configured using the so-called ''modules'' system which abstracts actual installation paths and sets certain environment variables in a standardized form.
-For each installed software component, there should be a module definition file under <tt>$NLPLCODE/etc/nlpl-''__name__''/''__version__''</tt>
+For each installed software component, there should be a module definition file under <tt>$NLPLCODE/modules/etc/nlpl-''__name__''/''__version__''</tt>
 See the separate section on the modules system below for further details on the definition files.

@@ Line 95: / Line 95: @@
 For modules specific to the laboratory, module names shall be prefixed with <tt>nlpl-</tt> and module definition files shall be stored in the abovementioned directory layout (see the sections on Software and Data Sets).
 The Moses software environment, for example, should be installed into <tt>$NLPL_SOFTWARE/moses/</tt>, and its module definition
-below the <tt>$NLPL_SOFTWARE/nlpl-moses/</tt> directory.
+below the <tt>$NLPLCODE/nlpl-moses/</tt> directory.
 Users will activate it (in its ‘default’ version, in case multiple versions are available) using the
 command
@@ Line 102: / Line 102: @@
 Integrating modules in the system requires the directory containing these files be included in the search path of the module system (see <tt>module use -a</tt> above). For example, include the command
- module use -a $NLPL_SOFTWARE/modulefiles
+  module use -a $NLPLCODE/etc
-in your <tt>.bashrc</tt> on Abel and Taito with the correct, system-specific value for <tt>$NLPL_SOFTWARE</tt> (see above).
+in your <tt>.bashrc</tt> on Abel and Taito with the correct, system-specific value for <tt>$NLPLCODE</tt> (see above).
 = Open Questions =

@@ Line 56: / Line 56: @@
 The data sets provided by NLPL shall be presented as uniformly as possible from a user point perspective. The data set environment is predominantly comprised of freely available data. Restricted data sets are not considered at the current stage (see the section on Open Questions below).
-Data sets shall be installed under <tt>$NLPL_DATA</tt> which points to <tt>/proj/nlpl/data/</tt> and <tt>/projects/nlpl/data/</tt> for Taito and Abel, respectively.
+Data sets shall be installed under <tt>$NLPLDATA</tt> which points to <tt>/proj/nlpl/data/</tt> and <tt>/projects/nlpl/data/</tt> for Taito and Abel, respectively.
 The top-level data directory is sub-divided by NLPL activities, with sub-directories <tt>translation</tt>, <tt>parsing</tt>, <tt>corpora</tt>, <tt>vectors</tt>, and <tt>opus</tt>.
 In principle, each activity is free to decide on the directory layout within their sub-directory, where it may or may not be

@@ Line 36: / Line 36: @@
 Under <pre>$NLPLCODE</pre> on each system, software modules (e.g. binaries and and libraries) are stored with the following directory layout:
-: $NLPLCODE/modules/''__NAME__''/''__VERSION__''
+: $NLPLCODE/modules/''__name__''/''__version__''
-where ''__NAME__'' contains only lower case symbols (with some exceptions such as R) and ''__VERSION__'' contains the version of the software (as provided by its vendor).
+where ''__name__'' contains only lower case symbols (with some exceptions such as R) and ''__version__'' contains the version of the software (as provided by its vendor).
 For each installation, there should be a separate page on the NLPL wiki (or minimally a <tt>README</tt> file) documenting the exact versions used (and how to obtain them) and the steps used to build the software.
@@ Line 46: / Line 46: @@
 The user environment on Abel and Taito is configured using the so-called ''modules'' system which abstracts actual installation paths and sets certain environment variables in a standardized form.
-For each installed software component, there should be a module definition file under <tt>$NLPLCODE/etc/nlpl-''__NAME__''/''__VERSION__''</tt>
+For each installed software component, there should be a module definition file under <tt>$NLPLCODE/etc/nlpl-''__name__''/''__version__''</tt>
 See the separate section on the modules system below for further details on the definition files.

@@ Line 31: / Line 31: @@
 <tt>infrastructure@nlpl.eu</tt> task force.
-Laboratory-specific software is stored in a location referred to as <tt>$NLPL_SOFTWARE</tt> which points to <tt>/proj/nlpl/software/</tt>
+Laboratory-specific software is stored in a location referred to as <pre>$NLPLCODE</pre> which points to <pre>/proj/nlpl/software/</pre>
-and <tt>/projects/nlpl/software/</tt> for Taito and Abel, respectively.
+and <pre>/projects/nlpl/software/</pre> for Taito and Abel, respectively.
-Under <tt>$NLPL_SOFTWARE</tt> on each system, applications and libraries are stored with the following directory layout:
+Under <pre>$NLPLCODE</pre> on each system, software modules (e.g. binaries and and libraries) are stored with the following directory layout:
-: $NLPL_SOFTWARE/__APPNAME__/__APPVERSION__
+: $NLPLCODE/modules/''__NAME__''/''__VERSION__''
-where ''__APPNAME__'' contains only lower case symbols (with some exceptions such as R) and ''__APPVERSION__'' contains the version of the software (as provided by its vendor).
+where ''__NAME__'' contains only lower case symbols (with some exceptions such as R) and ''__VERSION__'' contains the version of the software (as provided by its vendor).
-For each installation, there should be a <tt>README</tt> file (or page on the wiki) documenting the exact versions used (and how to obtain them) and the steps used to build the software.
+For each installation, there should be a separate page on the NLPL wiki (or minimally a <tt>README</tt> file) documenting the exact versions used (and how to obtain them) and the steps used to build the software.
 The infrastructure task force will soon make a proposal for how to manage installation scripts and notes in an NLPL-associated version control system.
-Optionally, for each software that is installed the sources and build scripts (if adjusted to the NLPL environment) can be stored under <tt>$NLPL_SOFTWARE/sources/</tt>.  This directory should be ‘cleaned up’ after the installation and possibly compressed.  If possible, it may be useful to create a log file of the complete installation session and preserve that file (in compressed form).
+Optionally, for each software that is installed the sources and build scripts (if adjusted to the NLPL environment) can be stored under <tt>$NLPLCODE/build/</tt>.  This directory should be ‘cleaned up’ after the installation and possibly compressed.  If possible, it may be useful to create a log file of the complete installation session and preserve that file (in compressed form).
 The user environment on Abel and Taito is configured using the so-called ''modules'' system which abstracts actual installation paths and sets certain environment variables in a standardized form.
-For each installed software component, there should be a module definition file under <tt>$NLPL_SOFTWARE/modulefiles/__APPNAME__/__APPVERSION__</tt>
+For each installed software component, there should be a module definition file under <tt>$NLPLCODE/etc/nlpl-''__NAME__''/''__VERSION__''</tt>
 See the separate section on the modules system below for further details on the definition files.

@@ Line 77: / Line 77: @@
 ; module list
 : Shows which modules have been loaded.
-; module load name/version
+; module load name ''or'' module load name/version
 : Loads module with name name and version version.
 ; module rm name/version