Difference between revisions of "Community/training/2019"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(→‎Logistics and Registration)
m (Oe moved page Community/training to Community/training/2019 without leaving a redirect)
 
(75 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
<br/>
 +
[[File:neic.png|center]]
 +
<br/>
 +
<br/>
 +
 
= Background =
 
= Background =
  
Line 5: Line 10:
 
in NLP and advancing a shared level of knowledge and experience in using
 
in NLP and advancing a shared level of knowledge and experience in using
 
national e-Infrastructures for large-scale NLP research.
 
national e-Infrastructures for large-scale NLP research.
−
Towards these goals, the project will organize some training and outreach events.
+
Towards these goals, the project organizes an annual three-day winter school.
 +
For additional background, please see the archival pages from the
 +
[http://wiki.nlpl.eu/index.php/Community/training/2018 2018 NLPL Winter School].
 +
 
 +
For early 2019, NLPL will hold its winter school from Monday, February 4, to
 +
Wednesday, February 6, 2019, at a
 +
[https://www.thonhotels.com/our-hotels/norway/skeikampen/ mountain-side hotel]
 +
(with skiing opportunities) about two hours north of Oslo.
 +
The project will organize group bus transfer from and to the Oslo
 +
airport ''Gardermoen'', leaving the airport at 9:30 on Monday morning
 +
and returning there around 17:30 on Wednesday afternoon.
 +
 
 +
The main instructors in 2019 will be
 +
[http://www.cs.cmu.edu/~cdyer/ Chris Dyer]
 +
[http://www.phontron.com/ and Graham Neubig],
 +
core developers of the
 +
[http://dynet.io/ DyNet framework], for a combination
 +
of lectures and hands-on exercises in applications of ‘dynamic’ deep
 +
neural networks to NLP.
 +
The winter school programme will be complemented with a ‘walk-through’
 +
of available software, data, and service resources from NLPL, as well
 +
as with a ‘research bazar’ involving all participants.
 +
 
 +
The winter school is subsidized by the project (with financial support
 +
from NeIC).
 +
There will be a limited number of fully funded ‘slots’ available to
 +
members of the core NLPL consortium, which each partner is free to
 +
allocate among their staff and student as they see fit.
 +
Additionally, the winter school will welcome partly self-funded
 +
participants from project partners and other Nordic NLP research
 +
centers, who will have to cover their own travel expenses to and
 +
from Oslo airport, as well as their cost of accommodation at the
 +
winter school hotel (the latter is expected to come to
 +
at most NOK 2,500).
 +
In total, we anticipate 30–35 participants in the 2019 Winter School.
 +
 
 +
= Logistics =
 +
NLPL Winter School participants  will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf.  Be at the meeting point at 09:00 (CET) 4. February, 2019.  We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).
 +
 
 +
The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).
 +
 
 +
= Registration =
  
−
For early 2018, NLPL will hold a winter school in conjunction with
+
We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe)
−
the [http://ahm17.neic.nordforsk.org/ NeIC All-Hands Meeting] (AHM),
+
to register their interest in participation through the  
−
which some NLPL team members will attend.  
+
[https://neicdellingr.typeform.com/to/kmwhTy on-line registration form]
−
The NeIC meeting will be held from January 29 to February 1, 2018,
+
The deadline for requesting involvement was Friday, December 7, 2019;
−
in the Norwegian mountain resort at [https://www.thonhotels.no/skeikampen Skeikampen].
+
registration is now closed.
 +
Registration through this form does not automatically qualify for
 +
participation in the winter school, however.
  
−
Starting already after lunch on Monday, January 29, NLPL will kick off
+
Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots,
−
its winter school in ''E-Infrastructure and Scientific Computing
+
which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen
−
for Nordic Natural Language Processing Research''.
+
University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at
−
The winter school will have a duration of two days, i.e. end with
+
Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter
−
lunch on Wednesday, January 31.
+
at the University of Turku, and Joakim Nivre at Uppsala University).
−
We anticipate participation of around 20–25 people, where the majority will
 
−
come from the NLPL partner sites.
 
  
−
NLPL team members, associates at partner sites (e.g. doctoral and post-doctoral fellows,
+
In addition to these slots, five to ten additional participants will be
−
possibly also some MSc students), and other prospective users of the infrastructure will be invited to attend;
+
welcome to participate partly at their own expense, i.e. covering the
−
NLPL partners(=project team members) can have their cost of participation covered by the project.
+
cost of accomodation at Skeikampen (and, of course, travel to and from
 +
Oslo airport) themselves.
 +
The NLPL outreach task force will review registrations in mid-December
 +
2019 and then finally confirm participation by non–project members,
 +
based on a combination of the first come–first served principle and a
 +
goal of institutional and regional diversity.
  
 
= Scientific Programme =
 
= Scientific Programme =
  
−
= Logistics and Registration =
+
 
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Monday, February 4, 2019
 +
|-
 +
| 13:30 || 14:30 || '''Session 1''' Introduction (Graham Neubig)<ul><li>Introduction to Neural Networks<li>What Neural Nets can Do To Help<li>Computation Graphs and Training<li>Tutorial Outline</ul>
 +
|-
 +
| 14:30 || 14:50 || '''Coffee Break'''
 +
|-
 +
| 14:50 || 16:10 || '''Session 2''' Language Modeling and Neural Net Training (Chris Dyer)<ul><li>Feed-Forward Language Modeling<li>SGD and Friends<li>Generalization, Evaluation, Regularization</ul>
 +
|-
 +
| 16:10 || 16:30 || '''Coffee Break'''
 +
|-
 +
| 16:30 || 17:50 || '''Session 3''' Distributional Semantics and Word Vectors (Graham Neubig)<ul><li>Describing a Word by the Company that it Keeps<li>Counting and Predicting<li>Skip-Grams and CBOW<li>Evaluating and Visualizing Word Vectors<li>Advanced Methods for Word Vectors</ul>
 +
|-
 +
| 18:00 || 19:00 || '''Session 4''' NLPL Walk-Through (Project Team)
 +
|-
 +
| 19:30 ||  || '''Dinner'''
 +
|}
 +
 
 +
 
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Tuesday, February 5, 2019
 +
|-
 +
|colspan=3 | Breakfast is served from 07:30
 +
|-
 +
| 08:30 || 10:00 || '''Session 5''' Computational Efficiency (Graham Neubig)<ul><li>Batching<li>CPU vs. GPU Issues</ul>
 +
|-
 +
|colspan=3| Lunch is served from 13:00-14:30
 +
|-
 +
| 15:00 || 16:30 || '''Session 6''' RNNs (Chris Dyer)<ul><li>Unconditional LMs<li>Sentence Representation, Representation of Longer Sequences</ul>
 +
|-
 +
| 16:30 || 16:50 || Coffee Break
 +
|-
 +
| 16:50 || 17:50 || '''Session 7''' Debugging (Graham Neubig)<ul><li>Identifying Problems<li>Debugging Training Time Problems<li>Debugging Test Time Problems</ul>
 +
|-
 +
| 17:50 || 19:20 || '''Session 8''' Sequence Prediction (Chris Dyer)<ul><li>Tagging with Bi-LSTMs<li>Conditional LMs and Applications</ul>
 +
 
 +
|-
 +
| 19:30 ||  || '''Dinner'''
 +
|}
 +
 
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Wednesday, February 6, 2019
 +
|-
 +
|colspan=3| Breakfast is served from 07:30
 +
|-
 +
| 09:00 || 10:20 || '''Session 9''' Structured Prediction (Chris Dyer)<ul><li>RNNGs<li>Parsing or CRFs</ul>
 +
|-
 +
| 10:20 || 10:40 || Coffee Break
 +
|-
 +
| 10:40 || 12:00 || '''Session 10''' Current Research Questions (Chris Dyer)<ul><li>Transfer Learning<li>Structure Induction<li>Emergent Communication<li>Language Grounding</ul>
 +
|-
 +
| 12:30 || 13:30 || '''Lunch'''
 +
|}
 +
 
 +
slides: https://phontron.com/class/nn4nlp2019/schedule.html
  
 
= Software Environment =
 
= Software Environment =
  
−
The deep learning sessions will include hands-on laboratory sessions following the
+
Graham uses code from  https://github.com/neubig/nn4nlp-code .
−
[https://github.com/LxMLS/lxmls_guide LxMLS tutorial], although switching over
+
 
−
to PyTorch (instead Theano).
+
There is an installation of DyNet (so far without GPU support) available on Abel:
−
Stephan Oepen has prepared a Python virtual environment will all LxLMS dependencies
+
 
−
installed, including PyTorch with GPU support:
+
<pre>
 +
module use -a /projects/nlpl/software/modulesfiles
 +
module load nlpl-dynet
 +
</pre>
  
−
module use -a /projects/nlpl/software/modulefiles/
+
= Subsized Participants =
−
module add nlpl-lxmls
 
  
−
There is a one-time preparaptory step to obtain the ‘student’ starting package from LXMLS:
+
In early December 2018, there has been great interest in participation
 +
already (and registration technically remains open), and we will regrettably
 +
have to decline numerous prospective participants.
 +
In a first round of allocations, we are currently confirming participation
 +
from members of the core consortium (including associate partners), who
 +
will have their cost of staying at the conference site covered directly
 +
by NeIC.
 +
Everyone else, please expect to hear more from us by mid-December.
  
−
git clone https://github.com/LxMLS/lxmls-toolkit.git
+
# Mostafa Abdou (University of Copenhagen)
−
cd lxmls-toolkit
+
# Rahul Aralikatte (University Copenhagen)
−
git checkout student
+
# Victor Petrén Bach Hansen (University of Copenhagen)
 +
# Eivind Alexander Bergem (University of Oslo)
 +
# Leon Derczynski (Steering Group)
 +
# Chris Dyer (Lecturer)
 +
# Filip Ginter (Steering Group)
 +
# Suwisa Kaewphan (University of Turku)
 +
# Jenna Kanerva (University of Turku)
 +
# Sigrid Klerke (IT University Copenhagen)
 +
# Artur Kulmizev (Uppsala University)
 +
# Andrey Kutuzov (University of Oslo)
 +
# Sharid Loáiciga (Gothenburg University)
 +
# Akseli Leino (University of Turku)
 +
# Bjørn Lindi (Project Manager)
 +
# Miryam de Lhoneux (Uppsala University)
 +
# Graham Neubig (Lecturer)
 +
# Joakim Nivre (Steering Group)
 +
# Stephan Oepen (Steering Group)
 +
# Vinit Ravishankar (University of Oslo)
 +
# Yves Scherrer (University of Helsinki)
 +
# Umut Sulubacak (University of Helsinki)
 +
# Gongbo Tang (Uppsala University)
 +
# Andre Tättar (University of Tartu)
 +
# Jörg Tiedemann (Steering Group)
 +
# Raul Vazquez (University of Helsinki)
  
−
Just now at least, there is a minor incompatibility with the standard version of SciPy
+
= Self-Funded Participants =
−
installed on Abel and a newer version that the LXMLS toolkit assumes.
 
−
To accomodate for this mismatch, all occurence of <tt>scipy.special</tt>
 
−
must be changed to <tt>scipy.misc</tt>, including in the LXMLS library file
 
−
<tt>lxmls/deep_learning/numpy_models/mlp.py</tt>.
 
  
−
Finally, the LXMLS code assumes that its top-level directory is visible for Python <tt>import</tt> commands;
+
Our public call for participation was met with great interest by
−
when inside the <tt>lxmls-toolkit</tt> directory, the following shell command should have
+
NLP researchers from all over Northern Europe (and beyond).
−
the right effect:
+
A total of 54 registrations were received by the deadline, of which
 +
six have been declined for various reasons.
 +
Everyone else (who as of December 10 has yet to hear from us)
 +
will be invited to participate on a ''self-funding basis'',
 +
meaning that these participants will have to pay their own
 +
accomodation and meals while at Skeikampen (but are of course
 +
not expected to contribute to the cost of transfer between the
 +
Oslo airport and the hotel or otherwise pay a participation fee).
 +
Bjørn Lindi, the NLPL project manager, will email pre-registered
 +
participants by December 15, 2018, with more information on the
 +
cost for self-funded participation and the process for confirming
 +
your involvement.
  
−
export PYTHONPATH=$PYTHONPATH:.
+
# Chiara Argese (University of Tromsø)
 +
# Jeremy Barnes (University of Oslo)
 +
# Sidsel Boldsen (University of Copenhagen)
 +
# Amaru Cuba Gyllensten (KTH Stockholm)
 +
# Andrew Dyer (University of Uppsala)
 +
# Mark Fishel (University of Tartu)
 +
# Fredrik Jørgensen (University of Oslo)
 +
# Robin Kurtz (University of Linköping)
 +
# Jan Tore Lønning (University of Oslo)
 +
# Ailsa Meechan (University of Uppsala)
 +
# Farrokh Mehryary (University of Turku)
 +
# Luis Nieto Piña (Gothenburg University)
 +
# Asad Sayeed (Gothenburg University)
 +
# Felipe Soares (Barcelona Supercomputing Center)
 +
# Steinþór Steingrímsson (University of Reykjavik)
 +
# Trond Trosterud (University of Tromsø)
 +
# Erik Velldal (University of Oslo)
 +
# Antti Virtanen (University of Turku)
 +
# Lilja Øvrelid (University of Oslo)
  
−
= Confirmed Participants =
+
[[File:skeikampen.2019.png|center]]

Latest revision as of 11:58, 4 December 2019


Neic.png



Background

A desirable side-effect of the NLPL cooperation is community formation, i.e. strengthening interaction and collaboration among Nordic research teams in NLP and advancing a shared level of knowledge and experience in using national e-Infrastructures for large-scale NLP research. Towards these goals, the project organizes an annual three-day winter school. For additional background, please see the archival pages from the 2018 NLPL Winter School.

For early 2019, NLPL will hold its winter school from Monday, February 4, to Wednesday, February 6, 2019, at a mountain-side hotel (with skiing opportunities) about two hours north of Oslo. The project will organize group bus transfer from and to the Oslo airport Gardermoen, leaving the airport at 9:30 on Monday morning and returning there around 17:30 on Wednesday afternoon.

The main instructors in 2019 will be Chris Dyer and Graham Neubig, core developers of the DyNet framework, for a combination of lectures and hands-on exercises in applications of ‘dynamic’ deep neural networks to NLP. The winter school programme will be complemented with a ‘walk-through’ of available software, data, and service resources from NLPL, as well as with a ‘research bazar’ involving all participants.

The winter school is subsidized by the project (with financial support from NeIC). There will be a limited number of fully funded ‘slots’ available to members of the core NLPL consortium, which each partner is free to allocate among their staff and student as they see fit. Additionally, the winter school will welcome partly self-funded participants from project partners and other Nordic NLP research centers, who will have to cover their own travel expenses to and from Oslo airport, as well as their cost of accommodation at the winter school hotel (the latter is expected to come to at most NOK 2,500). In total, we anticipate 30–35 participants in the 2019 Winter School.

Logistics

NLPL Winter School participants will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf. Be at the meeting point at 09:00 (CET) 4. February, 2019. We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).

The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).

Registration

We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe) to register their interest in participation through the on-line registration form The deadline for requesting involvement was Friday, December 7, 2019; registration is now closed. Registration through this form does not automatically qualify for participation in the winter school, however.

Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots, which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter at the University of Turku, and Joakim Nivre at Uppsala University).

In addition to these slots, five to ten additional participants will be welcome to participate partly at their own expense, i.e. covering the cost of accomodation at Skeikampen (and, of course, travel to and from Oslo airport) themselves. The NLPL outreach task force will review registrations in mid-December 2019 and then finally confirm participation by non–project members, based on a combination of the first come–first served principle and a goal of institutional and regional diversity.

Scientific Programme

Monday, February 4, 2019
13:30 14:30 Session 1 Introduction (Graham Neubig)
  • Introduction to Neural Networks
  • What Neural Nets can Do To Help
  • Computation Graphs and Training
  • Tutorial Outline
14:30 14:50 Coffee Break
14:50 16:10 Session 2 Language Modeling and Neural Net Training (Chris Dyer)
  • Feed-Forward Language Modeling
  • SGD and Friends
  • Generalization, Evaluation, Regularization
16:10 16:30 Coffee Break
16:30 17:50 Session 3 Distributional Semantics and Word Vectors (Graham Neubig)
  • Describing a Word by the Company that it Keeps
  • Counting and Predicting
  • Skip-Grams and CBOW
  • Evaluating and Visualizing Word Vectors
  • Advanced Methods for Word Vectors
18:00 19:00 Session 4 NLPL Walk-Through (Project Team)
19:30 Dinner


Tuesday, February 5, 2019
Breakfast is served from 07:30
08:30 10:00 Session 5 Computational Efficiency (Graham Neubig)
  • Batching
  • CPU vs. GPU Issues
Lunch is served from 13:00-14:30
15:00 16:30 Session 6 RNNs (Chris Dyer)
  • Unconditional LMs
  • Sentence Representation, Representation of Longer Sequences
16:30 16:50 Coffee Break
16:50 17:50 Session 7 Debugging (Graham Neubig)
  • Identifying Problems
  • Debugging Training Time Problems
  • Debugging Test Time Problems
17:50 19:20 Session 8 Sequence Prediction (Chris Dyer)
  • Tagging with Bi-LSTMs
  • Conditional LMs and Applications
19:30 Dinner
Wednesday, February 6, 2019
Breakfast is served from 07:30
09:00 10:20 Session 9 Structured Prediction (Chris Dyer)
  • RNNGs
  • Parsing or CRFs
10:20 10:40 Coffee Break
10:40 12:00 Session 10 Current Research Questions (Chris Dyer)
  • Transfer Learning
  • Structure Induction
  • Emergent Communication
  • Language Grounding
12:30 13:30 Lunch

slides: https://phontron.com/class/nn4nlp2019/schedule.html

Software Environment

Graham uses code from https://github.com/neubig/nn4nlp-code .

There is an installation of DyNet (so far without GPU support) available on Abel: 

module use -a /projects/nlpl/software/modulesfiles
module load nlpl-dynet

Subsized Participants

In early December 2018, there has been great interest in participation already (and registration technically remains open), and we will regrettably have to decline numerous prospective participants. In a first round of allocations, we are currently confirming participation from members of the core consortium (including associate partners), who will have their cost of staying at the conference site covered directly by NeIC. Everyone else, please expect to hear more from us by mid-December.

  1. Mostafa Abdou (University of Copenhagen)
  2. Rahul Aralikatte (University Copenhagen)
  3. Victor Petrén Bach Hansen (University of Copenhagen)
  4. Eivind Alexander Bergem (University of Oslo)
  5. Leon Derczynski (Steering Group)
  6. Chris Dyer (Lecturer)
  7. Filip Ginter (Steering Group)
  8. Suwisa Kaewphan (University of Turku)
  9. Jenna Kanerva (University of Turku)
  10. Sigrid Klerke (IT University Copenhagen)
  11. Artur Kulmizev (Uppsala University)
  12. Andrey Kutuzov (University of Oslo)
  13. Sharid Loáiciga (Gothenburg University)
  14. Akseli Leino (University of Turku)
  15. Bjørn Lindi (Project Manager)
  16. Miryam de Lhoneux (Uppsala University)
  17. Graham Neubig (Lecturer)
  18. Joakim Nivre (Steering Group)
  19. Stephan Oepen (Steering Group)
  20. Vinit Ravishankar (University of Oslo)
  21. Yves Scherrer (University of Helsinki)
  22. Umut Sulubacak (University of Helsinki)
  23. Gongbo Tang (Uppsala University)
  24. Andre Tättar (University of Tartu)
  25. Jörg Tiedemann (Steering Group)
  26. Raul Vazquez (University of Helsinki)

Self-Funded Participants

Our public call for participation was met with great interest by NLP researchers from all over Northern Europe (and beyond). A total of 54 registrations were received by the deadline, of which six have been declined for various reasons. Everyone else (who as of December 10 has yet to hear from us) will be invited to participate on a self-funding basis, meaning that these participants will have to pay their own accomodation and meals while at Skeikampen (but are of course not expected to contribute to the cost of transfer between the Oslo airport and the hotel or otherwise pay a participation fee). Bjørn Lindi, the NLPL project manager, will email pre-registered participants by December 15, 2018, with more information on the cost for self-funded participation and the process for confirming your involvement.

  1. Chiara Argese (University of Tromsø)
  2. Jeremy Barnes (University of Oslo)
  3. Sidsel Boldsen (University of Copenhagen)
  4. Amaru Cuba Gyllensten (KTH Stockholm)
  5. Andrew Dyer (University of Uppsala)
  6. Mark Fishel (University of Tartu)
  7. Fredrik Jørgensen (University of Oslo)
  8. Robin Kurtz (University of Linköping)
  9. Jan Tore Lønning (University of Oslo)
  10. Ailsa Meechan (University of Uppsala)
  11. Farrokh Mehryary (University of Turku)
  12. Luis Nieto Piña (Gothenburg University)
  13. Asad Sayeed (Gothenburg University)
  14. Felipe Soares (Barcelona Supercomputing Center)
  15. Steinþór Steingrímsson (University of Reykjavik)
  16. Trond Trosterud (University of Tromsø)
  17. Erik Velldal (University of Oslo)
  18. Antti Virtanen (University of Turku)
  19. Lilja Øvrelid (University of Oslo)
Skeikampen.2019.png
Retrieved from "http://wiki.nlpl.eu/index.php?title=Community/training/2019&oldid=806"