Difference between revisions of "Community/training/2019"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(→‎Background)
m (Oe moved page Community/training to Community/training/2019 without leaving a redirect)
 
(113 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
<br/>
 +
[[File:neic.png|center]]
 +
<br/>
 +
<br/>
 +
 
= Background =
 
= Background =
  
Line 5: Line 10:
 
in NLP and advancing a shared level of knowledge and experience in using
 
in NLP and advancing a shared level of knowledge and experience in using
 
national e-Infrastructures for large-scale NLP research.
 
national e-Infrastructures for large-scale NLP research.
−
Towards these goals, the project will organize some training and outreach events.
+
Towards these goals, the project organizes an annual three-day winter school.
 +
For additional background, please see the archival pages from the
 +
[http://wiki.nlpl.eu/index.php/Community/training/2018 2018 NLPL Winter School].
 +
 
 +
For early 2019, NLPL will hold its winter school from Monday, February 4, to
 +
Wednesday, February 6, 2019, at a
 +
[https://www.thonhotels.com/our-hotels/norway/skeikampen/ mountain-side hotel]
 +
(with skiing opportunities) about two hours north of Oslo.
 +
The project will organize group bus transfer from and to the Oslo
 +
airport ''Gardermoen'', leaving the airport at 9:30 on Monday morning
 +
and returning there around 17:30 on Wednesday afternoon.
 +
 
 +
The main instructors in 2019 will be
 +
[http://www.cs.cmu.edu/~cdyer/ Chris Dyer]
 +
[http://www.phontron.com/ and Graham Neubig],
 +
core developers of the
 +
[http://dynet.io/ DyNet framework], for a combination
 +
of lectures and hands-on exercises in applications of ‘dynamic’ deep
 +
neural networks to NLP.
 +
The winter school programme will be complemented with a ‘walk-through’
 +
of available software, data, and service resources from NLPL, as well
 +
as with a ‘research bazar’ involving all participants.
 +
 
 +
The winter school is subsidized by the project (with financial support
 +
from NeIC).
 +
There will be a limited number of fully funded ‘slots’ available to
 +
members of the core NLPL consortium, which each partner is free to
 +
allocate among their staff and student as they see fit.
 +
Additionally, the winter school will welcome partly self-funded
 +
participants from project partners and other Nordic NLP research
 +
centers, who will have to cover their own travel expenses to and
 +
from Oslo airport, as well as their cost of accommodation at the
 +
winter school hotel (the latter is expected to come to
 +
at most NOK 2,500).
 +
In total, we anticipate 30–35 participants in the 2019 Winter School.
 +
 
 +
= Logistics =
 +
NLPL Winter School participants  will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf.  Be at the meeting point at 09:00 (CET) 4. February, 2019.  We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).
 +
 
 +
The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).
  
−
For early 2018, NLPL will hold a winter school in conjunction with
+
= Registration =
−
the [http://ahm17.neic.nordforsk.org/ NeIC All-Hands Meeting] (AHM),
 
−
which some NLPL team members will attend.
 
−
The NeIC meeting will be held from January 29 to February 1, 2018,
 
−
in the Norwegian mountain resort at [https://www.thonhotels.no/skeikampen Skeikampen].
 
  
−
Starting already after lunch on Monday, January 29, NLPL will kick off
+
We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe)
−
its winter school in ''E-Infrastructure and Scientific Computing
+
to register their interest in participation through the
−
for Nordic Natural Language Processing Research''.
+
[https://neicdellingr.typeform.com/to/kmwhTy on-line registration form]
−
The winter school will have a duration of two days, i.e. end with
+
The deadline for requesting involvement was Friday, December 7, 2019;
−
lunch on Wednesday, January 31.
+
registration is now closed.
−
We anticipate participation of around 20–25 people, where the majority will
+
Registration through this form does not automatically qualify for
−
come from the NLPL partner sites.
+
participation in the winter school, however.
  
−
NLPL team members, associates at partner sites (e.g. doctoral and post-doctoral fellows,
+
Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots,
−
possibly also some MSc students), and other prospective users of the infrastructure will be invited to attend;
+
which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen
−
NLPL partners can have their cost of participation covered by the project.
+
University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at
 +
Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter
 +
at the University of Turku, and Joakim Nivre at Uppsala University).
  
−
= Programme =
+
In addition to these slots, five to ten additional participants will be
 +
welcome to participate partly at their own expense, i.e. covering the
 +
cost of accomodation at Skeikampen (and, of course, travel to and from
 +
Oslo airport) themselves.
 +
The NLPL outreach task force will review registrations in mid-December
 +
2019 and then finally confirm participation by non–project members,
 +
based on a combination of the first come–first served principle and a
 +
goal of institutional and regional diversity.
  
−
The winter school will comprise different types of activities,
+
= Scientific Programme =
−
including (a) ''overview talks'', for example on GPU programming
+
 
−
(with emphasis on Taito and Abel), ‘deep’ learning paradigms and toolkits,
+
 
−
or other scientific programming and HPC techniques; (b) ''in-depth tutorials''
+
{| class="wikitable"
−
on parts of the NLPL infrastructure (e.g. translation, parsing, and
+
|-
−
extrinsic evaluation software, as well as corpora and embeddings) and
+
!colspan=3|Monday, February 4, 2019
−
other topics of relevance to the project (e.g. ‘containerization’);
+
|-
−
and (c) ''hands-on hackathons'', i.e. collective programming and
+
| 13:30 || 14:30 || '''Session 1''' Introduction (Graham Neubig)<ul><li>Introduction to Neural Networks<li>What Neural Nets can Do To Help<li>Computation Graphs and Training<li>Tutorial Outline</ul>
−
experimentation in specific toolkits (e.g. environments like DyNet,
+
|-
−
MPI, or CUDA).
+
| 14:30 || 14:50 || '''Coffee Break'''
−
Participation in the hackathons will require preparation ''prior'' to
+
|-
−
the winter school, which NLPL partner sites will organize locally.
+
| 14:50 || 16:10 || '''Session 2''' Language Modeling and Neural Net Training (Chris Dyer)<ul><li>Feed-Forward Language Modeling<li>SGD and Friends<li>Generalization, Evaluation, Regularization</ul>
 +
|-
 +
| 16:10 || 16:30 || '''Coffee Break'''
 +
|-
 +
| 16:30 || 17:50 || '''Session 3''' Distributional Semantics and Word Vectors (Graham Neubig)<ul><li>Describing a Word by the Company that it Keeps<li>Counting and Predicting<li>Skip-Grams and CBOW<li>Evaluating and Visualizing Word Vectors<li>Advanced Methods for Word Vectors</ul>
 +
|-
 +
| 18:00 || 19:00 || '''Session 4''' NLPL Walk-Through (Project Team)
 +
|-
 +
| 19:30 ||  || '''Dinner'''
 +
|}
  
−
Two of the co-organizers of the
 
−
[http://lxmls.it.pt Lisbon Machine Learning School] (LxML),
 
−
[https://andre-martins.github.io/ André Martins] and
 
−
[https://www.l2f.inesc-id.pt/wiki/index.php/Ramon_Fernandez_Astudillo Ramon Fernandez Astudillo]
 
−
will work through selections of the LxML programme, with a special focus
 
−
on deep learning approaches to NLP.
 
−
Additionally, NLPL team members will provide a walk-through of the
 
−
emerging infrastructure, there will be a high-level tutorial on the
 
−
use of Singularity containers, and representatives of the Finnish
 
−
and Norwegian national providers will provide a survey of available
 
−
and upcoming computing infrastructures, with emphasis on GPU resources.
 
  
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
−
!colspan=3|Monday, January 29, 2018
+
!colspan=3|Tuesday, February 5, 2019
 +
|-
 +
|colspan=3 | Breakfast is served from 07:30
 +
|-
 +
| 08:30 || 10:00 || '''Session 5''' Computational Efficiency (Graham Neubig)<ul><li>Batching<li>CPU vs. GPU Issues</ul>
 
|-
 
|-
−
| 13:30 || 15:00 || Session 1: Deep Learning for NLP (LxMLS)
+
|colspan=3| Lunch is served from 13:00-14:30
 
|-
 
|-
−
| 15:00 || 15:30 || Coffee Break
+
| 15:00 || 16:30 || '''Session 6''' RNNs (Chris Dyer)<ul><li>Unconditional LMs<li>Sentence Representation, Representation of Longer Sequences</ul>
 
|-
 
|-
−
| 15:30 || 17:00 || Session 2: Deep Learning for NLP (LxMLS)
+
| 16:30 || 16:50 || Coffee Break
 
|-
 
|-
−
| 17:00 || 17:30 || Coffee Break
+
| 16:50 || 17:50 || '''Session 7''' Debugging (Graham Neubig)<ul><li>Identifying Problems<li>Debugging Training Time Problems<li>Debugging Test Time Problems</ul>
 
|-
 
|-
−
| 17:30 || 19:00 || Session 3: Walk-Through of NLPL Infrastructure
+
| 17:50 || 19:20 || '''Session 8''' Sequence Prediction (Chris Dyer)<ul><li>Tagging with Bi-LSTMs<li>Conditional LMs and Applications</ul>
  
 +
|-
 +
| 19:30 ||  || '''Dinner'''
 +
|}
 +
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Wednesday, February 6, 2019
 +
|-
 +
|colspan=3| Breakfast is served from 07:30
 +
|-
 +
| 09:00 || 10:20 || '''Session 9''' Structured Prediction (Chris Dyer)<ul><li>RNNGs<li>Parsing or CRFs</ul>
 +
|-
 +
| 10:20 || 10:40 || Coffee Break
 +
|-
 +
| 10:40 || 12:00 || '''Session 10''' Current Research Questions (Chris Dyer)<ul><li>Transfer Learning<li>Structure Induction<li>Emergent Communication<li>Language Grounding</ul>
 +
|-
 +
| 12:30 || 13:30 || '''Lunch'''
 
|}
 
|}
  
−
= Logistics =
+
slides: https://phontron.com/class/nn4nlp2019/schedule.html
 +
 
 +
= Software Environment =
 +
 
 +
Graham uses code from  https://github.com/neubig/nn4nlp-code .
 +
 
 +
There is an installation of DyNet (so far without GPU support) available on Abel:
 +
 
 +
<pre>
 +
module use -a /projects/nlpl/software/modulesfiles
 +
module load nlpl-dynet
 +
</pre>
 +
 
 +
= Subsized Participants =
 +
 
 +
In early December 2018, there has been great interest in participation
 +
already (and registration technically remains open), and we will regrettably
 +
have to decline numerous prospective participants.
 +
In a first round of allocations, we are currently confirming participation
 +
from members of the core consortium (including associate partners), who
 +
will have their cost of staying at the conference site covered directly
 +
by NeIC.
 +
Everyone else, please expect to hear more from us by mid-December.
 +
 
 +
# Mostafa Abdou (University of Copenhagen)
 +
# Rahul Aralikatte (University Copenhagen)
 +
# Victor Petrén Bach Hansen (University of Copenhagen)
 +
# Eivind Alexander Bergem (University of Oslo)
 +
# Leon Derczynski (Steering Group)
 +
# Chris Dyer (Lecturer)
 +
# Filip Ginter (Steering Group)
 +
# Suwisa Kaewphan (University of Turku)
 +
# Jenna Kanerva (University of Turku)
 +
# Sigrid Klerke (IT University Copenhagen)
 +
# Artur Kulmizev (Uppsala University)
 +
# Andrey Kutuzov (University of Oslo)
 +
# Sharid Loáiciga (Gothenburg University)
 +
# Akseli Leino (University of Turku)
 +
# Bjørn Lindi (Project Manager)
 +
# Miryam de Lhoneux (Uppsala University)
 +
# Graham Neubig (Lecturer)
 +
# Joakim Nivre (Steering Group)
 +
# Stephan Oepen (Steering Group)
 +
# Vinit Ravishankar (University of Oslo)
 +
# Yves Scherrer (University of Helsinki)
 +
# Umut Sulubacak (University of Helsinki)
 +
# Gongbo Tang (Uppsala University)
 +
# Andre Tättar (University of Tartu)
 +
# Jörg Tiedemann (Steering Group)
 +
# Raul Vazquez (University of Helsinki)
 +
 
 +
= Self-Funded Participants =
 +
 
 +
Our public call for participation was met with great interest by
 +
NLP researchers from all over Northern Europe (and beyond).
 +
A total of 54 registrations were received by the deadline, of which
 +
six have been declined for various reasons.
 +
Everyone else (who as of December 10 has yet to hear from us)
 +
will be invited to participate on a ''self-funding basis'',
 +
meaning that these participants will have to pay their own
 +
accomodation and meals while at Skeikampen (but are of course
 +
not expected to contribute to the cost of transfer between the
 +
Oslo airport and the hotel or otherwise pay a participation fee).
 +
Bjørn Lindi, the NLPL project manager, will email pre-registered
 +
participants by December 15, 2018, with more information on the
 +
cost for self-funded participation and the process for confirming
 +
your involvement.
 +
 
 +
# Chiara Argese (University of Tromsø)
 +
# Jeremy Barnes (University of Oslo)
 +
# Sidsel Boldsen (University of Copenhagen)
 +
# Amaru Cuba Gyllensten (KTH Stockholm)
 +
# Andrew Dyer (University of Uppsala)
 +
# Mark Fishel (University of Tartu)
 +
# Fredrik Jørgensen (University of Oslo)
 +
# Robin Kurtz (University of Linköping)
 +
# Jan Tore Lønning (University of Oslo)
 +
# Ailsa Meechan (University of Uppsala)
 +
# Farrokh Mehryary (University of Turku)
 +
# Luis Nieto Piña (Gothenburg University)
 +
# Asad Sayeed (Gothenburg University)
 +
# Felipe Soares (Barcelona Supercomputing Center)
 +
# Steinþór Steingrímsson (University of Reykjavik)
 +
# Trond Trosterud (University of Tromsø)
 +
# Erik Velldal (University of Oslo)
 +
# Antti Virtanen (University of Turku)
 +
# Lilja Øvrelid (University of Oslo)
  
−
NLPL will provide bus transfer from and to Oslo Aiport Gardermoen (OSL),
+
[[File:skeikampen.2019.png|center]]
−
leaving at 9:30 on Monday morning and returning to OSL around 16:30 on
 
−
Wedensday.
 

Latest revision as of 11:58, 4 December 2019


Neic.png



Background

A desirable side-effect of the NLPL cooperation is community formation, i.e. strengthening interaction and collaboration among Nordic research teams in NLP and advancing a shared level of knowledge and experience in using national e-Infrastructures for large-scale NLP research. Towards these goals, the project organizes an annual three-day winter school. For additional background, please see the archival pages from the 2018 NLPL Winter School.

For early 2019, NLPL will hold its winter school from Monday, February 4, to Wednesday, February 6, 2019, at a mountain-side hotel (with skiing opportunities) about two hours north of Oslo. The project will organize group bus transfer from and to the Oslo airport Gardermoen, leaving the airport at 9:30 on Monday morning and returning there around 17:30 on Wednesday afternoon.

The main instructors in 2019 will be Chris Dyer and Graham Neubig, core developers of the DyNet framework, for a combination of lectures and hands-on exercises in applications of ‘dynamic’ deep neural networks to NLP. The winter school programme will be complemented with a ‘walk-through’ of available software, data, and service resources from NLPL, as well as with a ‘research bazar’ involving all participants.

The winter school is subsidized by the project (with financial support from NeIC). There will be a limited number of fully funded ‘slots’ available to members of the core NLPL consortium, which each partner is free to allocate among their staff and student as they see fit. Additionally, the winter school will welcome partly self-funded participants from project partners and other Nordic NLP research centers, who will have to cover their own travel expenses to and from Oslo airport, as well as their cost of accommodation at the winter school hotel (the latter is expected to come to at most NOK 2,500). In total, we anticipate 30–35 participants in the 2019 Winter School.

Logistics

NLPL Winter School participants will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf. Be at the meeting point at 09:00 (CET) 4. February, 2019. We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).

The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).

Registration

We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe) to register their interest in participation through the on-line registration form The deadline for requesting involvement was Friday, December 7, 2019; registration is now closed. Registration through this form does not automatically qualify for participation in the winter school, however.

Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots, which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter at the University of Turku, and Joakim Nivre at Uppsala University).

In addition to these slots, five to ten additional participants will be welcome to participate partly at their own expense, i.e. covering the cost of accomodation at Skeikampen (and, of course, travel to and from Oslo airport) themselves. The NLPL outreach task force will review registrations in mid-December 2019 and then finally confirm participation by non–project members, based on a combination of the first come–first served principle and a goal of institutional and regional diversity.

Scientific Programme

Monday, February 4, 2019
13:30 14:30 Session 1 Introduction (Graham Neubig)
  • Introduction to Neural Networks
  • What Neural Nets can Do To Help
  • Computation Graphs and Training
  • Tutorial Outline
14:30 14:50 Coffee Break
14:50 16:10 Session 2 Language Modeling and Neural Net Training (Chris Dyer)
  • Feed-Forward Language Modeling
  • SGD and Friends
  • Generalization, Evaluation, Regularization
16:10 16:30 Coffee Break
16:30 17:50 Session 3 Distributional Semantics and Word Vectors (Graham Neubig)
  • Describing a Word by the Company that it Keeps
  • Counting and Predicting
  • Skip-Grams and CBOW
  • Evaluating and Visualizing Word Vectors
  • Advanced Methods for Word Vectors
18:00 19:00 Session 4 NLPL Walk-Through (Project Team)
19:30 Dinner


Tuesday, February 5, 2019
Breakfast is served from 07:30
08:30 10:00 Session 5 Computational Efficiency (Graham Neubig)
  • Batching
  • CPU vs. GPU Issues
Lunch is served from 13:00-14:30
15:00 16:30 Session 6 RNNs (Chris Dyer)
  • Unconditional LMs
  • Sentence Representation, Representation of Longer Sequences
16:30 16:50 Coffee Break
16:50 17:50 Session 7 Debugging (Graham Neubig)
  • Identifying Problems
  • Debugging Training Time Problems
  • Debugging Test Time Problems
17:50 19:20 Session 8 Sequence Prediction (Chris Dyer)
  • Tagging with Bi-LSTMs
  • Conditional LMs and Applications
19:30 Dinner
Wednesday, February 6, 2019
Breakfast is served from 07:30
09:00 10:20 Session 9 Structured Prediction (Chris Dyer)
  • RNNGs
  • Parsing or CRFs
10:20 10:40 Coffee Break
10:40 12:00 Session 10 Current Research Questions (Chris Dyer)
  • Transfer Learning
  • Structure Induction
  • Emergent Communication
  • Language Grounding
12:30 13:30 Lunch

slides: https://phontron.com/class/nn4nlp2019/schedule.html

Software Environment

Graham uses code from https://github.com/neubig/nn4nlp-code .

There is an installation of DyNet (so far without GPU support) available on Abel: 

module use -a /projects/nlpl/software/modulesfiles
module load nlpl-dynet

Subsized Participants

In early December 2018, there has been great interest in participation already (and registration technically remains open), and we will regrettably have to decline numerous prospective participants. In a first round of allocations, we are currently confirming participation from members of the core consortium (including associate partners), who will have their cost of staying at the conference site covered directly by NeIC. Everyone else, please expect to hear more from us by mid-December.

  1. Mostafa Abdou (University of Copenhagen)
  2. Rahul Aralikatte (University Copenhagen)
  3. Victor Petrén Bach Hansen (University of Copenhagen)
  4. Eivind Alexander Bergem (University of Oslo)
  5. Leon Derczynski (Steering Group)
  6. Chris Dyer (Lecturer)
  7. Filip Ginter (Steering Group)
  8. Suwisa Kaewphan (University of Turku)
  9. Jenna Kanerva (University of Turku)
  10. Sigrid Klerke (IT University Copenhagen)
  11. Artur Kulmizev (Uppsala University)
  12. Andrey Kutuzov (University of Oslo)
  13. Sharid Loáiciga (Gothenburg University)
  14. Akseli Leino (University of Turku)
  15. Bjørn Lindi (Project Manager)
  16. Miryam de Lhoneux (Uppsala University)
  17. Graham Neubig (Lecturer)
  18. Joakim Nivre (Steering Group)
  19. Stephan Oepen (Steering Group)
  20. Vinit Ravishankar (University of Oslo)
  21. Yves Scherrer (University of Helsinki)
  22. Umut Sulubacak (University of Helsinki)
  23. Gongbo Tang (Uppsala University)
  24. Andre Tättar (University of Tartu)
  25. Jörg Tiedemann (Steering Group)
  26. Raul Vazquez (University of Helsinki)

Self-Funded Participants

Our public call for participation was met with great interest by NLP researchers from all over Northern Europe (and beyond). A total of 54 registrations were received by the deadline, of which six have been declined for various reasons. Everyone else (who as of December 10 has yet to hear from us) will be invited to participate on a self-funding basis, meaning that these participants will have to pay their own accomodation and meals while at Skeikampen (but are of course not expected to contribute to the cost of transfer between the Oslo airport and the hotel or otherwise pay a participation fee). Bjørn Lindi, the NLPL project manager, will email pre-registered participants by December 15, 2018, with more information on the cost for self-funded participation and the process for confirming your involvement.

  1. Chiara Argese (University of Tromsø)
  2. Jeremy Barnes (University of Oslo)
  3. Sidsel Boldsen (University of Copenhagen)
  4. Amaru Cuba Gyllensten (KTH Stockholm)
  5. Andrew Dyer (University of Uppsala)
  6. Mark Fishel (University of Tartu)
  7. Fredrik Jørgensen (University of Oslo)
  8. Robin Kurtz (University of Linköping)
  9. Jan Tore Lønning (University of Oslo)
  10. Ailsa Meechan (University of Uppsala)
  11. Farrokh Mehryary (University of Turku)
  12. Luis Nieto Piña (Gothenburg University)
  13. Asad Sayeed (Gothenburg University)
  14. Felipe Soares (Barcelona Supercomputing Center)
  15. Steinþór Steingrímsson (University of Reykjavik)
  16. Trond Trosterud (University of Tromsø)
  17. Erik Velldal (University of Oslo)
  18. Antti Virtanen (University of Turku)
  19. Lilja Øvrelid (University of Oslo)
Skeikampen.2019.png
Retrieved from "http://wiki.nlpl.eu/index.php?title=Community/training/2019&oldid=806"