Difference between revisions of "Community/training/2019"

From Nordic Language Processing Laboratory
Jump to: navigation, search
(→‎Programme)
m (Oe moved page Community/training to Community/training/2019 without leaving a redirect)
 
(79 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
<br/>
 +
[[File:neic.png|center]]
 +
<br/>
 +
<br/>
 +
 
= Background =
 
= Background =
  
Line 5: Line 10:
 
in NLP and advancing a shared level of knowledge and experience in using
 
in NLP and advancing a shared level of knowledge and experience in using
 
national e-Infrastructures for large-scale NLP research.
 
national e-Infrastructures for large-scale NLP research.
−
Towards these goals, the project will organize some training and outreach events.
+
Towards these goals, the project organizes an annual three-day winter school.
−
 
+
For additional background, please see the archival pages from the
−
For early 2018, NLPL will hold a winter school in conjunction with
+
[http://wiki.nlpl.eu/index.php/Community/training/2018 2018 NLPL Winter School].
−
the [http://ahm17.neic.nordforsk.org/ NeIC All-Hands Meeting] (AHM),
 
−
which some NLPL team members will attend.
 
−
The NeIC meeting will be held from January 29 to February 1, 2018,
 
−
in the Norwegian mountain resort at [https://www.thonhotels.no/skeikampen Skeikampen].
 
−
 
 
−
Starting already after lunch on Monday, January 29, NLPL will kick off
 
−
its winter school in ''E-Infrastructure and Scientific Computing
 
−
for Nordic Natural Language Processing Research''.
 
−
The winter school will have a duration of two days, i.e. end with
 
−
lunch on Wednesday, January 31.
 
−
We anticipate participation of around 20–25 people, where the majority will
 
−
come from the NLPL partner sites.
 
−
 
 
−
NLPL team members, associates at partner sites (e.g. doctoral and post-doctoral fellows,
 
−
possibly also some MSc students), and other prospective users of the infrastructure will be invited to attend;
 
−
NLPL partners(=project team members) can have their cost of participation covered by the project.
 
−
 
 
−
= Programme =
 
−
 
 
−
The winter school will comprise different types of activities,
 
−
including (a) ''overview talks'', for example on GPU programming
 
−
(with emphasis on Taito and Abel), ‘deep’ learning paradigms and toolkits,
 
−
or other scientific programming and HPC techniques; (b) ''in-depth tutorials''
 
−
on parts of the NLPL infrastructure (e.g. translation, parsing, and
 
−
extrinsic evaluation software, as well as corpora and embeddings) and
 
−
other topics of relevance to the project (e.g. ‘containerization’);
 
−
and (c) ''hands-on hackathons'', i.e. collective programming and
 
−
experimentation in specific toolkits (e.g. environments like DyNet,
 
−
MPI, or CUDA).
 
−
Participation in the hackathons will require preparation ''prior'' to
 
−
the winter school, which NLPL partner sites will organize locally.
 
−
 
 
−
Two of the co-organizers of the
 
−
[http://lxmls.it.pt Lisbon Machine Learning School] (LxML),
 
−
[https://andre-martins.github.io/ André Martins] and
 
−
[https://www.l2f.inesc-id.pt/wiki/index.php/Ramon_Fernandez_Astudillo Ramon Fernandez Astudillo]
 
−
will work through selections of the LxML programme, with a special focus
 
−
on deep learning approaches to NLP.
 
−
Additionally, NLPL team members will provide a walk-through of the
 
−
emerging infrastructure, there will be a high-level tutorial on the
 
−
use of Singularity containers, and representatives of the Finnish
 
−
and Norwegian national providers will provide a survey of available
 
−
and upcoming computing infrastructures, with emphasis on GPU resources.
 
−
 
 
−
{| class="wikitable"
 
−
|-
 
−
!colspan=3|Monday, January 29, 2018  
 
−
!colspan=3|Tuesday, January 30, 2018  
 
−
!colspan=3|Wednesday, January 31, 2018
 
−
|-
 
−
|rowspan=2 | 09:00
 
−
|rowspan= 2 |12:00
 
−
| rowspan=2 | Travel to Skeikampen. Lunch at 12:30
 
−
| 09:00 || 10:30 || Session 4: Deep Learning for NLP (LxMLS) Room: Olav 2 || 09:00 || 10:00 
 
−
| Session 8: Deep Learning for NLP Wrap-Up (LxMLS) Room: Olav 2
 
−
|-
 
−
| 10:30 || 15:00
 
−
|  Break and lunch, which is served 13:00 - 14:30
 
−
| 10:30 || 12:00 || Session 9: Tutorial on Containers (Abdulrahman Azab, UiO/NeIC) Room: Olav 2
 
−
|-
 
−
| 13:30 || 15:00 || Session 1: Deep Learning for NLP (LxMLS)|| 15:00|| 16:30 || Session 5: Deep Learning for NLP (LxMLS) Room: Vinterhagen
 
−
| 12:00 || 13:00 || Lunch
 
−
|-
 
−
| 15:00 || 15:30 || Coffee Break || 16:30 || 17:00 || Coffee Break
 
−
| 13:45 || 14:00 || Get on the bus. The Bus leaves for OSL Airport at 14:00
 
−
|-
 
−
| 15:30 || 17:00 || Session 2: Deep Learning for NLP (LxMLS) || 17:00 || 18:30 || Session 6: Deep Learning for NLP(LxMLS) Room: Vinterhagen
 
−
|-
 
−
| 17:00 || 17:30 || Coffee Break || 18:30 || 18:45 || Coffee Break
 
−
|-
 
−
| 17:30 || 19:00 || Session 3: Walk-Through of NLPL Infrastructure || 18:45 || 19:30 || Session 7: Experience Exchange on DL and GPUs, Room: Vinterhagen
 
−
|}
 
  
−
'''Dinner 19:30 Monday and Tuesday'''
+
For early 2019, NLPL will hold its winter school from Monday, February 4, to
 +
Wednesday, February 6, 2019, at a
 +
[https://www.thonhotels.com/our-hotels/norway/skeikampen/ mountain-side hotel]
 +
(with skiing opportunities) about two hours north of Oslo.
 +
The project will organize group bus transfer from and to the Oslo
 +
airport ''Gardermoen'', leaving the airport at 9:30 on Monday morning
 +
and returning there around 17:30 on Wednesday afternoon.
  
−
= Logistics and Registration =
+
The main instructors in 2019 will be
 +
[http://www.cs.cmu.edu/~cdyer/ Chris Dyer]
 +
[http://www.phontron.com/ and Graham Neubig],
 +
core developers of the
 +
[http://dynet.io/ DyNet framework], for a combination
 +
of lectures and hands-on exercises in applications of ‘dynamic’ deep
 +
neural networks to NLP.
 +
The winter school programme will be complemented with a ‘walk-through’
 +
of available software, data, and service resources from NLPL, as well
 +
as with a ‘research bazar’ involving all participants.
  
−
The bus to Skeikampen will leave at 09:30 (CET). We will be informed about which pier on Monday morning. This is due to the traffic conditions at OSL.
+
The winter school is subsidized by the project (with financial support
−
Please meet at the OSL Arrival area Meeting Point at 09:00. [https://avinor.no/en/airport/oslo-airport/plan-your-trip/map/  Complete Map of OSL]
+
from NeIC).
 +
There will be a limited number of fully funded ‘slots’ available to
 +
members of the core NLPL consortium, which each partner is free to
 +
allocate among their staff and student as they see fit.
 +
Additionally, the winter school will welcome partly self-funded
 +
participants from project partners and other Nordic NLP research
 +
centers, who will have to cover their own travel expenses to and
 +
from Oslo airport, as well as their cost of accommodation at the
 +
winter school hotel (the latter is expected to come to
 +
at most NOK 2,500).
 +
In total, we anticipate 30–35 participants in the 2019 Winter School.
  
−
NLPL will provide bus transfer from and to Oslo Aiport Gardermoen (OSL),
+
= Logistics =
−
leaving at 9:30 on Monday morning and returning to OSL around 16:30 on
+
NLPL Winter School participants  will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf.  Be at the meeting point at 09:00 (CET) 4. February, 2019.  We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).
−
Wedensday.
 
−
The transfer and accomidation at Skeikampen will be directly covered
 
−
by NeIC.
 
−
Travel expenses to and from Oslo Airport will need to be submitted
 
−
to NeIC for reimbursement after completion of the winter school.
 
−
Note that reimbursement for the travel to OSL is for NLPL Project Team members only.
 
  
−
Please [https://neicdellingr.typeform.com/to/w0ZwLM record] your intent to participate in the winter school
+
The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).
−
through the [https://neicdellingr.typeform.com/to/w0ZwLM on-line registration form].
 
−
A limited number of hotel rooms and travel stipends are available.
 
−
For reasons of fairness, we will want to make sure that all NLPL partner sites can send
 
−
some participants (three per site, on average).
 
−
We will post confirmed registrations on this page by Monday, December 18;
 
−
please make your own travel arrangements once your participation has been
 
−
confirmed.
 
  
−
= Software Environment =
+
= Registration =
  
−
The deep learning sessions will include hands-on laboratory sessions following the
+
We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe)
−
[https://github.com/LxMLS/lxmls_guide LxMLS tutorial], although switching over
+
to register their interest in participation through the  
−
to PyTorch (instead Theano).
+
[https://neicdellingr.typeform.com/to/kmwhTy on-line registration form]
−
Stephan Oepen has prepared a Python virtual environment will all LxLMS dependencies
+
The deadline for requesting involvement was Friday, December 7, 2019;
−
installed, including PyTorch with GPU support:
+
registration is now closed.
 +
Registration through this form does not automatically qualify for
 +
participation in the winter school, however.
  
−
module use -a /projects/nlpl/software/modulefiles/
+
Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots,
−
module add nlpl-lxmls
+
which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen
 +
University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at
 +
Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter
 +
at the University of Turku, and Joakim Nivre at Uppsala University).
  
−
There is a one-time preparaptory step to obtain the ‘student’ starting package from LXMLS:
+
In addition to these slots, five to ten additional participants will be
 +
welcome to participate partly at their own expense, i.e. covering the
 +
cost of accomodation at Skeikampen (and, of course, travel to and from
 +
Oslo airport) themselves.
 +
The NLPL outreach task force will review registrations in mid-December
 +
2019 and then finally confirm participation by non–project members,
 +
based on a combination of the first come–first served principle and a
 +
goal of institutional and regional diversity.
  
−
git clone https://github.com/LxMLS/lxmls-toolkit.git
+
= Scientific Programme =
−
cd lxmls-toolkit
 
−
git checkout student
 
  
−
Just now at least, there is a minor incompatibility with the standard version of SciPy
 
−
installed on Abel and a newer version that the LXMLS toolkit assumes.
 
−
To accomodate for this mismatch, all occurence of <tt>scipy.special</tt>
 
−
must be changed to <tt>scipy.misc</tt>, including in the LXMLS library file
 
−
<tt>lxmls/deep_learning/numpy_models/mlp.py</tt>.
 
  
−
Finally, the LXMLS code assumes that its top-level directory is visible for Python <tt>import</tt> commands;
+
{| class="wikitable"
−
when inside the <tt>lxmls-toolkit</tt> directory, the following shell command should have
 
−
the right effect:
 
−
 
 
−
export PYTHONPATH=$PYTHONPATH:.
 
−
 
 
−
= Confirmed Participants =
 
−
 
 
−
{|
 
 
|-
 
|-
−
| Ramon Fernandez Astudillo, Unbabel and ''Instituto de Engenharia de Sistemas e Computadores''
+
!colspan=3|Monday, February 4, 2019
 
|-
 
|-
−
| Isabelle Augenstein, University of Copenhagen
+
| 13:30 || 14:30 || '''Session 1''' Introduction (Graham Neubig)<ul><li>Introduction to Neural Networks<li>What Neural Nets can Do To Help<li>Computation Graphs and Training<li>Tutorial Outline</ul>
 
|-
 
|-
−
| Johannes Bjerva, University of Copenhagen
+
| 14:30 || 14:50 || '''Coffee Break'''
 
|-
 
|-
−
| Murhaf Fares, University of Oslo
+
| 14:50 || 16:10 || '''Session 2''' Language Modeling and Neural Net Training (Chris Dyer)<ul><li>Feed-Forward Language Modeling<li>SGD and Friends<li>Generalization, Evaluation, Regularization</ul>
 
|-
 
|-
−
| Mareike Hartmann, University of Copenhagen
+
| 16:10 || 16:30 || '''Coffee Break'''
 
|-
 
|-
−
| Taraka Rama Kasicheyanula, University of Oslo
+
| 16:30 || 17:50 || '''Session 3''' Distributional Semantics and Word Vectors (Graham Neubig)<ul><li>Describing a Word by the Company that it Keeps<li>Counting and Predicting<li>Skip-Grams and CBOW<li>Evaluating and Visualizing Word Vectors<li>Advanced Methods for Word Vectors</ul>
 
|-
 
|-
−
| Andrei Kutuzov, University of Oslo
+
| 18:00 || 19:00 || '''Session 4''' NLPL Walk-Through (Project Team)
 
|-
 
|-
−
| Miryam de Lhoneux, Uppsala University
+
| 19:30 ||  || '''Dinner'''
 +
|}
 +
 
 +
 
 +
{| class="wikitable"
 
|-
 
|-
−
| Bjørn Lindi, Nordic e-Infrastructure Collaboration
+
!colspan=3|Tuesday, February 5, 2019
 
|-
 
|-
−
| Andre Martins, Unbabel and Instituto Superior Técnico
+
|colspan=3 | Breakfast is served from 07:30
 
|-
 
|-
−
| Farhad Nooralahzadeh, University of Oslo
+
| 08:30 || 10:00 || '''Session 5''' Computational Efficiency (Graham Neubig)<ul><li>Batching<li>CPU vs. GPU Issues</ul>
 
|-
 
|-
−
| Stephan Oepen, University of Oslo
+
|colspan=3| Lunch is served from 13:00-14:30
 
|-
 
|-
−
| Barbara Plank, IT University of Copenhagen
+
| 15:00 || 16:30 || '''Session 6''' RNNs (Chris Dyer)<ul><li>Unconditional LMs<li>Sentence Representation, Representation of Longer Sequences</ul>
 
|-
 
|-
−
| Yves Scherrer, University of Helsinki
+
| 16:30 || 16:50 || Coffee Break
 
|-
 
|-
−
| Natalie Schluter, IT University of Copenhagen
+
| 16:50 || 17:50 || '''Session 7''' Debugging (Graham Neubig)<ul><li>Identifying Problems<li>Debugging Training Time Problems<li>Debugging Test Time Problems</ul>
 
|-
 
|-
−
| Yan Shao, Uppsala University
+
| 17:50 || 19:20 || '''Session 8''' Sequence Prediction (Chris Dyer)<ul><li>Tagging with Bi-LSTMs<li>Conditional LMs and Applications</ul>
 +
 
 
|-
 
|-
−
| Aaron Smith, Uppsala University
+
| 19:30 ||  || '''Dinner'''
−
|-
+
|}
−
| Jörg Tiedemann, University of Helsinki
+
 
 +
{| class="wikitable"
 
|-
 
|-
−
| Aleksi Vesanto, University of Turku
+
!colspan=3|Wednesday, February 6, 2019
 
|-
 
|-
−
| Maria Barrett, University of Copenhagen
+
|colspan=3| Breakfast is served from 07:30
 
|-
 
|-
−
| Manex Zabaleta, University of Copenhagen
+
| 09:00 || 10:20 || '''Session 9''' Structured Prediction (Chris Dyer)<ul><li>RNNGs<li>Parsing or CRFs</ul>
 
|-
 
|-
−
| Senka Droba, University of Helsinki
+
| 10:20 || 10:40 || Coffee Break
 
|-
 
|-
−
| Mika Hämäläinen, University of Helsinki
+
| 10:40 || 12:00 || '''Session 10''' Current Research Questions (Chris Dyer)<ul><li>Transfer Learning<li>Structure Induction<li>Emergent Communication<li>Language Grounding</ul>
 
|-
 
|-
−
|Yova Kementchedjhieva, University of Copenhagen
+
| 12:30 || 13:30 || '''Lunch'''
 
|}
 
|}
 +
 +
slides: https://phontron.com/class/nn4nlp2019/schedule.html
 +
 +
= Software Environment =
 +
 +
Graham uses code from  https://github.com/neubig/nn4nlp-code .
 +
 +
There is an installation of DyNet (so far without GPU support) available on Abel:
 +
 +
<pre>
 +
module use -a /projects/nlpl/software/modulesfiles
 +
module load nlpl-dynet
 +
</pre>
 +
 +
= Subsized Participants =
 +
 +
In early December 2018, there has been great interest in participation
 +
already (and registration technically remains open), and we will regrettably
 +
have to decline numerous prospective participants.
 +
In a first round of allocations, we are currently confirming participation
 +
from members of the core consortium (including associate partners), who
 +
will have their cost of staying at the conference site covered directly
 +
by NeIC.
 +
Everyone else, please expect to hear more from us by mid-December.
 +
 +
# Mostafa Abdou (University of Copenhagen)
 +
# Rahul Aralikatte (University Copenhagen)
 +
# Victor Petrén Bach Hansen (University of Copenhagen)
 +
# Eivind Alexander Bergem (University of Oslo)
 +
# Leon Derczynski (Steering Group)
 +
# Chris Dyer (Lecturer)
 +
# Filip Ginter (Steering Group)
 +
# Suwisa Kaewphan (University of Turku)
 +
# Jenna Kanerva (University of Turku)
 +
# Sigrid Klerke (IT University Copenhagen)
 +
# Artur Kulmizev (Uppsala University)
 +
# Andrey Kutuzov (University of Oslo)
 +
# Sharid Loáiciga (Gothenburg University)
 +
# Akseli Leino (University of Turku)
 +
# Bjørn Lindi (Project Manager)
 +
# Miryam de Lhoneux (Uppsala University)
 +
# Graham Neubig (Lecturer)
 +
# Joakim Nivre (Steering Group)
 +
# Stephan Oepen (Steering Group)
 +
# Vinit Ravishankar (University of Oslo)
 +
# Yves Scherrer (University of Helsinki)
 +
# Umut Sulubacak (University of Helsinki)
 +
# Gongbo Tang (Uppsala University)
 +
# Andre Tättar (University of Tartu)
 +
# Jörg Tiedemann (Steering Group)
 +
# Raul Vazquez (University of Helsinki)
 +
 +
= Self-Funded Participants =
 +
 +
Our public call for participation was met with great interest by
 +
NLP researchers from all over Northern Europe (and beyond).
 +
A total of 54 registrations were received by the deadline, of which
 +
six have been declined for various reasons.
 +
Everyone else (who as of December 10 has yet to hear from us)
 +
will be invited to participate on a ''self-funding basis'',
 +
meaning that these participants will have to pay their own
 +
accomodation and meals while at Skeikampen (but are of course
 +
not expected to contribute to the cost of transfer between the
 +
Oslo airport and the hotel or otherwise pay a participation fee).
 +
Bjørn Lindi, the NLPL project manager, will email pre-registered
 +
participants by December 15, 2018, with more information on the
 +
cost for self-funded participation and the process for confirming
 +
your involvement.
 +
 +
# Chiara Argese (University of Tromsø)
 +
# Jeremy Barnes (University of Oslo)
 +
# Sidsel Boldsen (University of Copenhagen)
 +
# Amaru Cuba Gyllensten (KTH Stockholm)
 +
# Andrew Dyer (University of Uppsala)
 +
# Mark Fishel (University of Tartu)
 +
# Fredrik Jørgensen (University of Oslo)
 +
# Robin Kurtz (University of Linköping)
 +
# Jan Tore Lønning (University of Oslo)
 +
# Ailsa Meechan (University of Uppsala)
 +
# Farrokh Mehryary (University of Turku)
 +
# Luis Nieto Piña (Gothenburg University)
 +
# Asad Sayeed (Gothenburg University)
 +
# Felipe Soares (Barcelona Supercomputing Center)
 +
# Steinþór Steingrímsson (University of Reykjavik)
 +
# Trond Trosterud (University of Tromsø)
 +
# Erik Velldal (University of Oslo)
 +
# Antti Virtanen (University of Turku)
 +
# Lilja Øvrelid (University of Oslo)
 +
 +
[[File:skeikampen.2019.png|center]]

Latest revision as of 11:58, 4 December 2019


Neic.png



Background

A desirable side-effect of the NLPL cooperation is community formation, i.e. strengthening interaction and collaboration among Nordic research teams in NLP and advancing a shared level of knowledge and experience in using national e-Infrastructures for large-scale NLP research. Towards these goals, the project organizes an annual three-day winter school. For additional background, please see the archival pages from the 2018 NLPL Winter School.

For early 2019, NLPL will hold its winter school from Monday, February 4, to Wednesday, February 6, 2019, at a mountain-side hotel (with skiing opportunities) about two hours north of Oslo. The project will organize group bus transfer from and to the Oslo airport Gardermoen, leaving the airport at 9:30 on Monday morning and returning there around 17:30 on Wednesday afternoon.

The main instructors in 2019 will be Chris Dyer and Graham Neubig, core developers of the DyNet framework, for a combination of lectures and hands-on exercises in applications of ‘dynamic’ deep neural networks to NLP. The winter school programme will be complemented with a ‘walk-through’ of available software, data, and service resources from NLPL, as well as with a ‘research bazar’ involving all participants.

The winter school is subsidized by the project (with financial support from NeIC). There will be a limited number of fully funded ‘slots’ available to members of the core NLPL consortium, which each partner is free to allocate among their staff and student as they see fit. Additionally, the winter school will welcome partly self-funded participants from project partners and other Nordic NLP research centers, who will have to cover their own travel expenses to and from Oslo airport, as well as their cost of accommodation at the winter school hotel (the latter is expected to come to at most NOK 2,500). In total, we anticipate 30–35 participants in the 2019 Winter School.

Logistics

NLPL Winter School participants will meet at the meeting point in the Arrival hall at OSL. On this map of the OSL Arrival hall the meeting point is marked below the kiosks numbered 27 and 34: https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals-2018-3.pdf. Be at the meeting point at 09:00 (CET) 4. February, 2019. We will walk to the bus terminal together, leaving OSL shortly after 09:00. The bus will arrive at Skeikampen around 12:00 (CET). Lunch will be served upon our arrival, which currently is scheduled for 12:30(CET).

The Winter School will end with lunch on Wednesday 6. February, before we get on the bus to the Airport. The bus will leave Skeikampen at 14:00(CET). Expected arrival time at OSL Airport is 17:00(CET). When planning your flights, you should leave room for possible delays due to the season (slippery roads, heavy snow fall and so on).

Registration

We ask all interested parties (from NLPL project sites, associates, or other NLP research groups in Northern Europe) to register their interest in participation through the on-line registration form The deadline for requesting involvement was Friday, December 7, 2019; registration is now closed. Registration through this form does not automatically qualify for participation in the winter school, however.

Each NLPL partner site will have available three ‘fully subsidized’ (within Norway) slots, which will be confirmed by the local site manager: Anders Sørgaard at Copenhagen University, Leon Derczynski at IT University Copenhagen, Jörg Tiedemann at Helsinki University, Stephan Oepen at the University of Oslo, Filip Ginter at the University of Turku, and Joakim Nivre at Uppsala University).

In addition to these slots, five to ten additional participants will be welcome to participate partly at their own expense, i.e. covering the cost of accomodation at Skeikampen (and, of course, travel to and from Oslo airport) themselves. The NLPL outreach task force will review registrations in mid-December 2019 and then finally confirm participation by non–project members, based on a combination of the first come–first served principle and a goal of institutional and regional diversity.

Scientific Programme

Monday, February 4, 2019
13:30 14:30 Session 1 Introduction (Graham Neubig)
  • Introduction to Neural Networks
  • What Neural Nets can Do To Help
  • Computation Graphs and Training
  • Tutorial Outline
14:30 14:50 Coffee Break
14:50 16:10 Session 2 Language Modeling and Neural Net Training (Chris Dyer)
  • Feed-Forward Language Modeling
  • SGD and Friends
  • Generalization, Evaluation, Regularization
16:10 16:30 Coffee Break
16:30 17:50 Session 3 Distributional Semantics and Word Vectors (Graham Neubig)
  • Describing a Word by the Company that it Keeps
  • Counting and Predicting
  • Skip-Grams and CBOW
  • Evaluating and Visualizing Word Vectors
  • Advanced Methods for Word Vectors
18:00 19:00 Session 4 NLPL Walk-Through (Project Team)
19:30 Dinner


Tuesday, February 5, 2019
Breakfast is served from 07:30
08:30 10:00 Session 5 Computational Efficiency (Graham Neubig)
  • Batching
  • CPU vs. GPU Issues
Lunch is served from 13:00-14:30
15:00 16:30 Session 6 RNNs (Chris Dyer)
  • Unconditional LMs
  • Sentence Representation, Representation of Longer Sequences
16:30 16:50 Coffee Break
16:50 17:50 Session 7 Debugging (Graham Neubig)
  • Identifying Problems
  • Debugging Training Time Problems
  • Debugging Test Time Problems
17:50 19:20 Session 8 Sequence Prediction (Chris Dyer)
  • Tagging with Bi-LSTMs
  • Conditional LMs and Applications
19:30 Dinner
Wednesday, February 6, 2019
Breakfast is served from 07:30
09:00 10:20 Session 9 Structured Prediction (Chris Dyer)
  • RNNGs
  • Parsing or CRFs
10:20 10:40 Coffee Break
10:40 12:00 Session 10 Current Research Questions (Chris Dyer)
  • Transfer Learning
  • Structure Induction
  • Emergent Communication
  • Language Grounding
12:30 13:30 Lunch

slides: https://phontron.com/class/nn4nlp2019/schedule.html

Software Environment

Graham uses code from https://github.com/neubig/nn4nlp-code .

There is an installation of DyNet (so far without GPU support) available on Abel: 

module use -a /projects/nlpl/software/modulesfiles
module load nlpl-dynet

Subsized Participants

In early December 2018, there has been great interest in participation already (and registration technically remains open), and we will regrettably have to decline numerous prospective participants. In a first round of allocations, we are currently confirming participation from members of the core consortium (including associate partners), who will have their cost of staying at the conference site covered directly by NeIC. Everyone else, please expect to hear more from us by mid-December.

  1. Mostafa Abdou (University of Copenhagen)
  2. Rahul Aralikatte (University Copenhagen)
  3. Victor Petrén Bach Hansen (University of Copenhagen)
  4. Eivind Alexander Bergem (University of Oslo)
  5. Leon Derczynski (Steering Group)
  6. Chris Dyer (Lecturer)
  7. Filip Ginter (Steering Group)
  8. Suwisa Kaewphan (University of Turku)
  9. Jenna Kanerva (University of Turku)
  10. Sigrid Klerke (IT University Copenhagen)
  11. Artur Kulmizev (Uppsala University)
  12. Andrey Kutuzov (University of Oslo)
  13. Sharid Loáiciga (Gothenburg University)
  14. Akseli Leino (University of Turku)
  15. Bjørn Lindi (Project Manager)
  16. Miryam de Lhoneux (Uppsala University)
  17. Graham Neubig (Lecturer)
  18. Joakim Nivre (Steering Group)
  19. Stephan Oepen (Steering Group)
  20. Vinit Ravishankar (University of Oslo)
  21. Yves Scherrer (University of Helsinki)
  22. Umut Sulubacak (University of Helsinki)
  23. Gongbo Tang (Uppsala University)
  24. Andre Tättar (University of Tartu)
  25. Jörg Tiedemann (Steering Group)
  26. Raul Vazquez (University of Helsinki)

Self-Funded Participants

Our public call for participation was met with great interest by NLP researchers from all over Northern Europe (and beyond). A total of 54 registrations were received by the deadline, of which six have been declined for various reasons. Everyone else (who as of December 10 has yet to hear from us) will be invited to participate on a self-funding basis, meaning that these participants will have to pay their own accomodation and meals while at Skeikampen (but are of course not expected to contribute to the cost of transfer between the Oslo airport and the hotel or otherwise pay a participation fee). Bjørn Lindi, the NLPL project manager, will email pre-registered participants by December 15, 2018, with more information on the cost for self-funded participation and the process for confirming your involvement.

  1. Chiara Argese (University of Tromsø)
  2. Jeremy Barnes (University of Oslo)
  3. Sidsel Boldsen (University of Copenhagen)
  4. Amaru Cuba Gyllensten (KTH Stockholm)
  5. Andrew Dyer (University of Uppsala)
  6. Mark Fishel (University of Tartu)
  7. Fredrik Jørgensen (University of Oslo)
  8. Robin Kurtz (University of Linköping)
  9. Jan Tore Lønning (University of Oslo)
  10. Ailsa Meechan (University of Uppsala)
  11. Farrokh Mehryary (University of Turku)
  12. Luis Nieto Piña (Gothenburg University)
  13. Asad Sayeed (Gothenburg University)
  14. Felipe Soares (Barcelona Supercomputing Center)
  15. Steinþór Steingrímsson (University of Reykjavik)
  16. Trond Trosterud (University of Tromsø)
  17. Erik Velldal (University of Oslo)
  18. Antti Virtanen (University of Turku)
  19. Lilja Øvrelid (University of Oslo)
Skeikampen.2019.png
Retrieved from "http://wiki.nlpl.eu/index.php?title=Community/training/2019&oldid=806"