Parsing/uuparser - Revision history

Sara at 20:41, 14 January 2020

2020-01-14T20:41:08Z

2020-01-14T20:16:50Z

‎Using the Uppsala Parser on Saga

2020-01-14T20:09:25Z

2020-01-14T20:08:55Z

2020-01-14T19:58:17Z

‎Segmentation

2020-01-14T19:52:32Z

‎Training a multitreebank parsing model

2020-01-14T19:48:26Z

‎Predicting with a pre-trained parsing model

2020-01-14T19:48:00Z

2020-01-14T19:46:45Z

‎Predicting with a pre-trained parsing model

2020-01-14T13:07:04Z

‎Training a parsing model

@@ Line 86: / Line 86: @@
   uuparser --testfile TESTFILE --outdir ~/experiments --modeldir MODEL_DIR/scandinavian --predict --multiling --forced-tbank-emb no_nynorsk
 == Segmentation ==
 In the above examples, we assume pre-segmented input data already in the [http://universaldependencies.org/format.html CONLL-U] format. If your input is raw text, we recommend using UDPipe to segment and convert the format first. The UDPipe module can be loaded using <code>module load nlpl-udpipe</code> and then run by typing <code>udpipe</code> at the command line, see [http://wiki.nlpl.eu/index.php/Parsing/udpipe UDPipe].

@@ Line 12: / Line 12: @@
   module use -a /cluster/shared/nlpl/software/modules/etc
 * Load the most recent version of the uuparser module:
-  module load nlpl-uuparser/2.3.0
+  module load nlpl-uuparser/2.3.1
 == Training a parsing model ==

@@ Line 72: / Line 72: @@
 This commands runs the parser on TESTFILE (in CoNLLU format), using a model found in MODELDIR, and prints the resulting parse in a file in ~/experiments. In the example sv_talbanken is used as the proxy treebank.
-== Pre-trained models
+== Pre-trained models ==
 There are three multilingual pre-trained models available, covering English and most of the Nordic languages. The models can be found in /cluster/shared/nlpl/software/modules/uuparser/2.3.1/models ($MODEL_DIR)

@@ Line 71: / Line 71: @@
 This commands runs the parser on TESTFILE (in CoNLLU format), using a model found in MODELDIR, and prints the resulting parse in a file in ~/experiments. In the example sv_talbanken is used as the proxy treebank.
 == Segmentation ==
 In the above examples, we assume pre-segmented input data already in the [http://universaldependencies.org/format.html CONLL-U] format. If your input is raw text, we recommend using UDPipe to segment and convert the format first. The UDPipe module can be loaded using <code>module load nlpl-udpipe</code> and then run by typing <code>udpipe</code> at the command line, see [http://wiki.nlpl.eu/index.php/Parsing/udpipe UDPipe].

← Older revision		Revision as of 19:58, 14 January 2020
Line 74:		Line 74:
	== Segmentation ==		== Segmentation ==

−	In the above examples, we assume pre-segmented input data already in the [http://universaldependencies.org/format.html CONLL-U] format. If your input is raw text, we recommend using UDPipe to segment first. The UDPipe module can be loaded using <code>module load nlpl-udpipe</code> and then run by typing <code>udpipe</code> at the command line, see [http://wiki.nlpl.eu/index.php/Parsing/udpipe UDPipe].	+	In the above examples, we assume pre-segmented input data already in the [http://universaldependencies.org/format.html CONLL-U] format. If your input is raw text, we recommend using UDPipe to segment and convert the format first. The UDPipe module can be loaded using <code>module load nlpl-udpipe</code> and then run by typing <code>udpipe</code> at the command line, see [http://wiki.nlpl.eu/index.php/Parsing/udpipe UDPipe].

@@ Line 52: / Line 52: @@
   uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments --multiling --dynet-mem 5000 --datadir /cluster/shared/nlpl/data/parsing/ud/ud-treebanks-v2.5
-In this case, instead of creating three separate models in the language-specific subdirectories within ~/experiments, a single model will be created directly in this folder. Predicting on test data is then as easy as:
+In this case, instead of creating three separate models in the language-specific subdirectories within ~/experiments, a single model will be created directly in this folder. Predicting on UD test data is then:
   uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments --multiling --dynet-mem 5000 --predict --datadir /cluster/shared/nlpl/data/parsing/ud/ud-treebanks-v2.5
@@ Line 65: / Line 65: @@
 Where in both cases, sv_pud will be parsed using sv_talbanken as a proxy (i.e. parsing sv_pud as if it is is sv_talbanken). With the first option, multiple treebanks and proxies can be defined separated by space.
 == Segmentation ==
 In the above examples, we assume pre-segmented input data already in the [http://universaldependencies.org/format.html CONLL-U] format. If your input is raw text, we recommend using UDPipe to segment first. The UDPipe module can be loaded using <code>module load nlpl-udpipe</code> and then run by typing <code>udpipe</code> at the command line, see [http://wiki.nlpl.eu/index.php/Parsing/udpipe UDPipe].

@@ Line 32: / Line 32: @@
   uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments --predict --datadir /cluster/shared/nlpl/data/parsing/ud/ud-treebanks-v2.5
-To predict on other texts, prepare the data into the CoNLLU format (see below) and run the following command, assuming the model trained above:
+To predict on other texts, prepare the data into the CoNLLU format (see below) and run the following command, assuming the model trained above, for the file TESTFILE:
   uuparser --testfile TESTFILE --outdir ~/experiments --modeldir ~/experiments/sv_talbanken --predict

@@ Line 30: / Line 30: @@
 To predict on UD test data with the models trained above:
-  uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments --predict --datadir UD_DATA_DIR
+  uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments --predict --datadir /cluster/shared/nlpl/data/parsing/ud/ud-treebanks-v2.5
 To predict on other texts, prepare the data into the CoNLLU format (see below) and run the following command, assuming the model trained above:

@@ Line 18: / Line 18: @@
 To train a set of parsing models on treebanks from Universal Dependencies (v2.2 or later):
-  uuparser --include [languages to include denoted by their treebank id] --outdir my-output-directory
+  uuparser --include [languages to include denoted by their treebank id] --outdir my-output-directory  --datadir ud-treebank-dir
 For example:
-  uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments
+  uuparser --include "sv_talbanken en_partut ru_syntagrus" --outdir ~/experiments  --datadir /cluster/shared/nlpl/data/parsing/ud/ud-treebanks-v2.5
 will train separate models on UD Swedish-Talbanken, UD English-ParTUT and UD Russian-SynTagRus and store the results in the <code>experiments</code> folder in your home directory. Model selection is included in the training process by default; that is, at each epoch the current model is evaluated on the UD dev data, and at the end of training the best performing model for each language is selected.