Difference between revisions of "Community/training/2019"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(→‎Background)
m (Oe moved page Community/training to Community/training/2019 without leaving a redirect)
 
(118 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
<br/>
 +
[[File:neic.png|center]]
 +
<br/>
 +
<br/>
 +
 
= Background =
 
= Background =
  
Line 5: Line 10:
 
in NLP and advancing a shared level of knowledge and experience in using
 
in NLP and advancing a shared level of knowledge and experience in using
 
national e-Infrastructures for large-scale NLP research.
 
national e-Infrastructures for large-scale NLP research.
−
Towards these goals, the project will organize some training and outreach events.
+
Towards these goals, the project organizes an annual three-day winter school.
 +
For additional background, please see the archival pages from the
 +
[http://wiki.nlpl.eu/index.php/Community/training/2018 2018 NLPL Winter School].
 +
 
 +
For early 2019, NLPL will hold its winter school from Monday, February 4, to
 +
Wednesday, February 6, 2019, at a
 +
[https://www.thonhotels.com/our-hotels/norway/skeikampen/ mountain-side hotel]
 +
(with skiing opportunities) about two hours north of Oslo.
 +
The project will organize group bus transfer from and to the Oslo
 +
airport ''Gardermoen'', leaving the airport at 9:30 on Monday morning
 +
and returning there around 17:30 on Wednesday afternoon.
 +
 
 +
The main instructors in 2019 will be
 +
[http://www.cs.cmu.edu/~cdyer/ Chris Dyer]
 +
[http://www.phontron.com/ and Graham Neubig],
 +
core developers of the
 +
[http://dynet.io/ DyNet framework], for a combination
 +
of lectures and hands-on exercises in applications of ‘dynamic’ deep
 +
neural networks to NLP.
 +
The winter school programme will be complemented with a ‘walk-through’
 +
of available software, data, and service resources from NLPL, as well
 +
as with a ‘research bazar’ involving all participants.
 +
 
 +
The winter school is subsidized by the project (with financial support
 +
from NeIC).
 +
There will be a limited number of fully funded ‘slots’ available to
 +
members of the core NLPL consortium, which each partner is free to
 +
allocate among their staff and student as they see fit.
 +
Additionally, the winter school will welcome partly self-funded
 +
participants from project partners and other Nordic NLP research
 +
centers, who will have to cover their own travel expenses to and
 +
from Oslo airport, as well as their cost of accommodation at the
 +
winter school hotel (the latter is expected to come to
 +
at most NOK 2,500).
 +
In total, we anticipate 30–35 participants in the 2019 Winter School.
 +
 
 +
= Logistics =
 +
NLPL Winter School participants  will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf.  Be at the meeting point at 09:00 (CET) 4. February, 2019.  We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).
 +
 
 +
The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).
 +
 
 +
= Registration =
 +
 
 +
We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe)
 +
to register their interest in participation through the
 +
[https://neicdellingr.typeform.com/to/kmwhTy on-line registration form]
 +
The deadline for requesting involvement was Friday, December 7, 2019;
 +
registration is now closed.
 +
Registration through this form does not automatically qualify for
 +
participation in the winter school, however.
 +
 
 +
Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots,
 +
which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen
 +
University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at
 +
Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter
 +
at the University of Turku, and Joakim Nivre at Uppsala University).
 +
 
 +
In addition to these slots, five to ten additional participants will be
 +
welcome to participate partly at their own expense, i.e. covering the
 +
cost of accomodation at Skeikampen (and, of course, travel to and from
 +
Oslo airport) themselves.
 +
The NLPL outreach task force will review registrations in mid-December
 +
2019 and then finally confirm participation by non–project members,
 +
based on a combination of the first come–first served principle and a
 +
goal of institutional and regional diversity.
 +
 
 +
= Scientific Programme =
 +
 
 +
 
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Monday, February 4, 2019
 +
|-
 +
| 13:30 || 14:30 || '''Session 1''' Introduction (Graham Neubig)<ul><li>Introduction to Neural Networks<li>What Neural Nets can Do To Help<li>Computation Graphs and Training<li>Tutorial Outline</ul>
 +
|-
 +
| 14:30 || 14:50 || '''Coffee Break'''
 +
|-
 +
| 14:50 || 16:10 || '''Session 2''' Language Modeling and Neural Net Training (Chris Dyer)<ul><li>Feed-Forward Language Modeling<li>SGD and Friends<li>Generalization, Evaluation, Regularization</ul>
 +
|-
 +
| 16:10 || 16:30 || '''Coffee Break'''
 +
|-
 +
| 16:30 || 17:50 || '''Session 3''' Distributional Semantics and Word Vectors (Graham Neubig)<ul><li>Describing a Word by the Company that it Keeps<li>Counting and Predicting<li>Skip-Grams and CBOW<li>Evaluating and Visualizing Word Vectors<li>Advanced Methods for Word Vectors</ul>
 +
|-
 +
| 18:00 || 19:00 || '''Session 4''' NLPL Walk-Through (Project Team)
 +
|-
 +
| 19:30 ||  || '''Dinner'''
 +
|}
 +
 
 +
 
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Tuesday, February 5, 2019
 +
|-
 +
|colspan=3 | Breakfast is served from 07:30
 +
|-
 +
| 08:30 || 10:00 || '''Session 5''' Computational Efficiency (Graham Neubig)<ul><li>Batching<li>CPU vs. GPU Issues</ul>
 +
|-
 +
|colspan=3| Lunch is served from 13:00-14:30
 +
|-
 +
| 15:00 || 16:30 || '''Session 6''' RNNs (Chris Dyer)<ul><li>Unconditional LMs<li>Sentence Representation, Representation of Longer Sequences</ul>
 +
|-
 +
| 16:30 || 16:50 || Coffee Break
 +
|-
 +
| 16:50 || 17:50 || '''Session 7''' Debugging (Graham Neubig)<ul><li>Identifying Problems<li>Debugging Training Time Problems<li>Debugging Test Time Problems</ul>
 +
|-
 +
| 17:50 || 19:20 || '''Session 8''' Sequence Prediction (Chris Dyer)<ul><li>Tagging with Bi-LSTMs<li>Conditional LMs and Applications</ul>
 +
 
 +
|-
 +
| 19:30 ||  || '''Dinner'''
 +
|}
 +
 
 +
{| class="wikitable"
 +
|-
 +
!colspan=3|Wednesday, February 6, 2019
 +
|-
 +
|colspan=3| Breakfast is served from 07:30
 +
|-
 +
| 09:00 || 10:20 || '''Session 9''' Structured Prediction (Chris Dyer)<ul><li>RNNGs<li>Parsing or CRFs</ul>
 +
|-
 +
| 10:20 || 10:40 || Coffee Break
 +
|-
 +
| 10:40 || 12:00 || '''Session 10''' Current Research Questions (Chris Dyer)<ul><li>Transfer Learning<li>Structure Induction<li>Emergent Communication<li>Language Grounding</ul>
 +
|-
 +
| 12:30 || 13:30 || '''Lunch'''
 +
|}
 +
 
 +
slides: https://phontron.com/class/nn4nlp2019/schedule.html
 +
 
 +
= Software Environment =
 +
 
 +
Graham uses code from  https://github.com/neubig/nn4nlp-code .
 +
 
 +
There is an installation of DyNet (so far without GPU support) available on Abel:
 +
 
 +
<pre>
 +
module use -a /projects/nlpl/software/modulesfiles
 +
module load nlpl-dynet
 +
</pre>
 +
 
 +
= Subsized Participants =
  
−
For early 2018, NLPL will hold a winter school in conjunction with
+
In early December 2018, there has been great interest in participation
−
the [http://ahm17.neic.nordforsk.org/ NeIC All-Hands Meeting] (AHM),
+
already (and registration technically remains open), and we will regrettably
−
to which some NLPL team members will be invited.  
+
have to decline numerous prospective participants.
−
The NeIC meeting will be held from January 29 to February 1, 2018,
+
In a first round of allocations, we are currently confirming participation
−
in the Norwegian mountain resort at [https://www.thonhotels.no/skeikampen Skeikampen].
+
from members of the core consortium (including associate partners), who
 +
will have their cost of staying at the conference site covered directly
 +
by NeIC.
 +
Everyone else, please expect to hear more from us by mid-December.
  
−
Probably starting after lunch on Monday, January 29, NLPL will kick off
+
# Mostafa Abdou (University of Copenhagen)
−
its winter school in ''E-Infrastructure and Scientific Computing
+
# Rahul Aralikatte (University Copenhagen)
−
for Nordic Natural Language Processing Research''.
+
# Victor Petrén Bach Hansen (University of Copenhagen)
−
The winter school will have a duration of two days, i.e. end with
+
# Eivind Alexander Bergem (University of Oslo)
−
lunch on Wednesday, January 31.
+
# Leon Derczynski (Steering Group)
−
We anticipate participation of around 20 people, where the majority will
+
# Chris Dyer (Lecturer)
−
come from the NLPL partner sites.
+
# Filip Ginter (Steering Group)
 +
# Suwisa Kaewphan (University of Turku)
 +
# Jenna Kanerva (University of Turku)
 +
# Sigrid Klerke (IT University Copenhagen)
 +
# Artur Kulmizev (Uppsala University)
 +
# Andrey Kutuzov (University of Oslo)
 +
# Sharid Loáiciga (Gothenburg University)
 +
# Akseli Leino (University of Turku)
 +
# Bjørn Lindi (Project Manager)
 +
# Miryam de Lhoneux (Uppsala University)
 +
# Graham Neubig (Lecturer)
 +
# Joakim Nivre (Steering Group)
 +
# Stephan Oepen (Steering Group)
 +
# Vinit Ravishankar (University of Oslo)
 +
# Yves Scherrer (University of Helsinki)
 +
# Umut Sulubacak (University of Helsinki)
 +
# Gongbo Tang (Uppsala University)
 +
# Andre Tättar (University of Tartu)
 +
# Jörg Tiedemann (Steering Group)
 +
# Raul Vazquez (University of Helsinki)
  
−
NLPL team members, associates at partner sites (e.g. doctoral and post-doctoral fellows,
+
= Self-Funded Participants =
−
possibly also some MSc students), and other prospective users of the infrastructure will be invited to attend;
 
−
NLPL partners can have their cost of participation covered by the project.
 
  
−
= Programme =
+
Our public call for participation was met with great interest by
 +
NLP researchers from all over Northern Europe (and beyond).
 +
A total of 54 registrations were received by the deadline, of which
 +
six have been declined for various reasons.
 +
Everyone else (who as of December 10 has yet to hear from us)
 +
will be invited to participate on a ''self-funding basis'',
 +
meaning that these participants will have to pay their own
 +
accomodation and meals while at Skeikampen (but are of course
 +
not expected to contribute to the cost of transfer between the
 +
Oslo airport and the hotel or otherwise pay a participation fee).
 +
Bjørn Lindi, the NLPL project manager, will email pre-registered
 +
participants by December 15, 2018, with more information on the
 +
cost for self-funded participation and the process for confirming
 +
your involvement.
  
−
The winter school will comprise different types of activities,
+
# Chiara Argese (University of Tromsø)
−
including (a) ''overview talks'', for example on GPU programming
+
# Jeremy Barnes (University of Oslo)
−
(with emphasis on Taito and Abel), ‘deep’ learning paradigms and toolkits,
+
# Sidsel Boldsen (University of Copenhagen)
−
or other scientific programming and HPC techniques; (b) ''in-depth tutorials''
+
# Amaru Cuba Gyllensten (KTH Stockholm)
−
on parts of the NLPL infrastructure (e.g. translation, parsing, and
+
# Andrew Dyer (University of Uppsala)
−
extrinsic evaluation software, as well as corpora and embeddings) and
+
# Mark Fishel (University of Tartu)
−
other topics of relevance to the project (e.g. ‘containerization’);
+
# Fredrik Jørgensen (University of Oslo)
−
and (c) ''hands-on hackathons'', i.e. collective programming and
+
# Robin Kurtz (University of Linköping)
−
experimentation in specific toolkits (e.g. environments like DyNet,
+
# Jan Tore Lønning (University of Oslo)
−
MPI, or CUDA).
+
# Ailsa Meechan (University of Uppsala)
−
Participation in the hackathons will require preparation ''prior'' to
+
# Farrokh Mehryary (University of Turku)
−
the winter school, which NLPL partner sites will organize locally.
+
# Luis Nieto Piña (Gothenburg University)
 +
# Asad Sayeed (Gothenburg University)
 +
# Felipe Soares (Barcelona Supercomputing Center)
 +
# Steinþór Steingrímsson (University of Reykjavik)
 +
# Trond Trosterud (University of Tromsø)
 +
# Erik Velldal (University of Oslo)
 +
# Antti Virtanen (University of Turku)
 +
# Lilja Øvrelid (University of Oslo)
  
−
We are still working to confirm instructors at the winter school
+
[[File:skeikampen.2019.png|center]]
−
but expect to collaborate with the organizers of the
 
−
[http://lxmls.it.pt Lisbon Machine Learning School] (LxML) and
 
−
borrow some of their experience and course materials.
 

Latest revision as of 11:58, 4 December 2019


Neic.png



Background

A desirable side-effect of the NLPL cooperation is community formation, i.e. strengthening interaction and collaboration among Nordic research teams in NLP and advancing a shared level of knowledge and experience in using national e-Infrastructures for large-scale NLP research. Towards these goals, the project organizes an annual three-day winter school. For additional background, please see the archival pages from the 2018 NLPL Winter School.

For early 2019, NLPL will hold its winter school from Monday, February 4, to Wednesday, February 6, 2019, at a mountain-side hotel (with skiing opportunities) about two hours north of Oslo. The project will organize group bus transfer from and to the Oslo airport Gardermoen, leaving the airport at 9:30 on Monday morning and returning there around 17:30 on Wednesday afternoon.

The main instructors in 2019 will be Chris Dyer and Graham Neubig, core developers of the DyNet framework, for a combination of lectures and hands-on exercises in applications of ‘dynamic’ deep neural networks to NLP. The winter school programme will be complemented with a ‘walk-through’ of available software, data, and service resources from NLPL, as well as with a ‘research bazar’ involving all participants.

The winter school is subsidized by the project (with financial support from NeIC). There will be a limited number of fully funded ‘slots’ available to members of the core NLPL consortium, which each partner is free to allocate among their staff and student as they see fit. Additionally, the winter school will welcome partly self-funded participants from project partners and other Nordic NLP research centers, who will have to cover their own travel expenses to and from Oslo airport, as well as their cost of accommodation at the winter school hotel (the latter is expected to come to at most NOK 2,500). In total, we anticipate 30–35 participants in the 2019 Winter School.

Logistics

NLPL Winter School participants will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf. Be at the meeting point at 09:00 (CET) 4. February, 2019. We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).

The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).

Registration

We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe) to register their interest in participation through the on-line registration form The deadline for requesting involvement was Friday, December 7, 2019; registration is now closed. Registration through this form does not automatically qualify for participation in the winter school, however.

Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots, which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter at the University of Turku, and Joakim Nivre at Uppsala University).

In addition to these slots, five to ten additional participants will be welcome to participate partly at their own expense, i.e. covering the cost of accomodation at Skeikampen (and, of course, travel to and from Oslo airport) themselves. The NLPL outreach task force will review registrations in mid-December 2019 and then finally confirm participation by non–project members, based on a combination of the first come–first served principle and a goal of institutional and regional diversity.

Scientific Programme

Monday, February 4, 2019
13:30 14:30 Session 1 Introduction (Graham Neubig)
  • Introduction to Neural Networks
  • What Neural Nets can Do To Help
  • Computation Graphs and Training
  • Tutorial Outline
14:30 14:50 Coffee Break
14:50 16:10 Session 2 Language Modeling and Neural Net Training (Chris Dyer)
  • Feed-Forward Language Modeling
  • SGD and Friends
  • Generalization, Evaluation, Regularization
16:10 16:30 Coffee Break
16:30 17:50 Session 3 Distributional Semantics and Word Vectors (Graham Neubig)
  • Describing a Word by the Company that it Keeps
  • Counting and Predicting
  • Skip-Grams and CBOW
  • Evaluating and Visualizing Word Vectors
  • Advanced Methods for Word Vectors
18:00 19:00 Session 4 NLPL Walk-Through (Project Team)
19:30 Dinner


Tuesday, February 5, 2019
Breakfast is served from 07:30
08:30 10:00 Session 5 Computational Efficiency (Graham Neubig)
  • Batching
  • CPU vs. GPU Issues
Lunch is served from 13:00-14:30
15:00 16:30 Session 6 RNNs (Chris Dyer)
  • Unconditional LMs
  • Sentence Representation, Representation of Longer Sequences
16:30 16:50 Coffee Break
16:50 17:50 Session 7 Debugging (Graham Neubig)
  • Identifying Problems
  • Debugging Training Time Problems
  • Debugging Test Time Problems
17:50 19:20 Session 8 Sequence Prediction (Chris Dyer)
  • Tagging with Bi-LSTMs
  • Conditional LMs and Applications
19:30 Dinner
Wednesday, February 6, 2019
Breakfast is served from 07:30
09:00 10:20 Session 9 Structured Prediction (Chris Dyer)
  • RNNGs
  • Parsing or CRFs
10:20 10:40 Coffee Break
10:40 12:00 Session 10 Current Research Questions (Chris Dyer)
  • Transfer Learning
  • Structure Induction
  • Emergent Communication
  • Language Grounding
12:30 13:30 Lunch

slides: https://phontron.com/class/nn4nlp2019/schedule.html

Software Environment

Graham uses code from https://github.com/neubig/nn4nlp-code .

There is an installation of DyNet (so far without GPU support) available on Abel: 

module use -a /projects/nlpl/software/modulesfiles
module load nlpl-dynet

Subsized Participants

In early December 2018, there has been great interest in participation already (and registration technically remains open), and we will regrettably have to decline numerous prospective participants. In a first round of allocations, we are currently confirming participation from members of the core consortium (including associate partners), who will have their cost of staying at the conference site covered directly by NeIC. Everyone else, please expect to hear more from us by mid-December.

  1. Mostafa Abdou (University of Copenhagen)
  2. Rahul Aralikatte (University Copenhagen)
  3. Victor Petrén Bach Hansen (University of Copenhagen)
  4. Eivind Alexander Bergem (University of Oslo)
  5. Leon Derczynski (Steering Group)
  6. Chris Dyer (Lecturer)
  7. Filip Ginter (Steering Group)
  8. Suwisa Kaewphan (University of Turku)
  9. Jenna Kanerva (University of Turku)
  10. Sigrid Klerke (IT University Copenhagen)
  11. Artur Kulmizev (Uppsala University)
  12. Andrey Kutuzov (University of Oslo)
  13. Sharid Loáiciga (Gothenburg University)
  14. Akseli Leino (University of Turku)
  15. Bjørn Lindi (Project Manager)
  16. Miryam de Lhoneux (Uppsala University)
  17. Graham Neubig (Lecturer)
  18. Joakim Nivre (Steering Group)
  19. Stephan Oepen (Steering Group)
  20. Vinit Ravishankar (University of Oslo)
  21. Yves Scherrer (University of Helsinki)
  22. Umut Sulubacak (University of Helsinki)
  23. Gongbo Tang (Uppsala University)
  24. Andre Tättar (University of Tartu)
  25. Jörg Tiedemann (Steering Group)
  26. Raul Vazquez (University of Helsinki)

Self-Funded Participants

Our public call for participation was met with great interest by NLP researchers from all over Northern Europe (and beyond). A total of 54 registrations were received by the deadline, of which six have been declined for various reasons. Everyone else (who as of December 10 has yet to hear from us) will be invited to participate on a self-funding basis, meaning that these participants will have to pay their own accomodation and meals while at Skeikampen (but are of course not expected to contribute to the cost of transfer between the Oslo airport and the hotel or otherwise pay a participation fee). Bjørn Lindi, the NLPL project manager, will email pre-registered participants by December 15, 2018, with more information on the cost for self-funded participation and the process for confirming your involvement.

  1. Chiara Argese (University of Tromsø)
  2. Jeremy Barnes (University of Oslo)
  3. Sidsel Boldsen (University of Copenhagen)
  4. Amaru Cuba Gyllensten (KTH Stockholm)
  5. Andrew Dyer (University of Uppsala)
  6. Mark Fishel (University of Tartu)
  7. Fredrik Jørgensen (University of Oslo)
  8. Robin Kurtz (University of Linköping)
  9. Jan Tore Lønning (University of Oslo)
  10. Ailsa Meechan (University of Uppsala)
  11. Farrokh Mehryary (University of Turku)
  12. Luis Nieto Piña (Gothenburg University)
  13. Asad Sayeed (Gothenburg University)
  14. Felipe Soares (Barcelona Supercomputing Center)
  15. Steinþór Steingrímsson (University of Reykjavik)
  16. Trond Trosterud (University of Tromsø)
  17. Erik Velldal (University of Oslo)
  18. Antti Virtanen (University of Turku)
  19. Lilja Øvrelid (University of Oslo)
Skeikampen.2019.png
Retrieved from "http://wiki.nlpl.eu/index.php?title=Community/training/2019&oldid=806"