Difference between revisions of "Corpora/home"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(Corpus Catalogue)
Line 4: Line 4:
 
NLPL creates and makes available various very large collections of textual data, for example drawing on Wikipedia and the Common Crawl.
 
NLPL creates and makes available various very large collections of textual data, for example drawing on Wikipedia and the Common Crawl.
  
They are available from the connected infrastructure. Please, check the individual pages of each resource.
+
These data resources are available from the connected infrastructure, below the project directory
 
+
<tt>/projects/nlpl/data/corpora/</tt> or <tt>/proj/nlpl/data/corpora/</tt> on Abel and Taito, respectively.
* [[Corpora/OPUS|OPUS - The Collection of Open Parallel Corpora]]
+
In early 2018, the NLPL corpora resources are not yet replicated across the two systems; please see
 +
the summary table below to determine which resource is available on which system.
  
 
= Corpus Catalogue =
 
= Corpus Catalogue =
Line 16: Line 17:
 
| conll17 || || Taito || November 2017 || Jenna Kaverna
 
| conll17 || || Taito || November 2017 || Jenna Kaverna
 
|}
 
|}
 +
 +
 +
= Many Languages: The Collection of Open Parallel Corpora (Helsinki) =
 +
 +
 +
[[Corpora/OPUS|OPUS]] provides parallel texts in many languages extracted from
 +
a broad range of freely available sources; there is a
 +
[[Corpora/OPUS|separate wiki page]] with additional information on OPUS.
  
  

Revision as of 13:12, 25 January 2018

Background

NLPL creates and makes available various very large collections of textual data, for example drawing on Wikipedia and the Common Crawl.

These data resources are available from the connected infrastructure, below the project directory /projects/nlpl/data/corpora/ or /proj/nlpl/data/corpora/ on Abel and Taito, respectively. In early 2018, the NLPL corpora resources are not yet replicated across the two systems; please see the summary table below to determine which resource is available on which system.

Corpus Catalogue

Directory Description System Install Date Maintainer
conll17 Taito November 2017 Jenna Kaverna


Many Languages: The Collection of Open Parallel Corpora (Helsinki)

OPUS provides parallel texts in many languages extracted from a broad range of freely available sources; there is a separate wiki page with additional information on OPUS.


Many Languages: The CoNLL 2017 Text Collection (Turku)

English: 130 Billion Words Extracted from the Common Crawl (Oslo)

English: Two Variants of Text Extraction from Wikipedia (Oslo)