2025-02-03T11:12:00Z

Andreku: /* Schedule */

'''HPLT & NLPL 2025 Winter School on Pretraining Data Quality and Multilingual LLM Evaluation'''

[[File:HPLT and NLPL Winter School 2024.jpg|center|thumb|upright=2.0]]

= Background =

Since 2023, the NLPL network and Horizon Europe
project ''[https://hplt-project.org High-Performance Language Technologies]'' (HPLT)
have joined forces to organize the successful winter school series on Web-scale NLP.
The winter school seeks to stimulate ''community formation'',
i.e. strengthening interaction and collaboration among
European research teams in NLP and advancing a shared level of knowledge
and experience in using high-performance e-infrastructures for large-scale
NLP research.
This 2025 edition of the winter school puts special emphasis on
NLP researchers from countries who participate in the EuroHPC
[https://www.lumi-supercomputer.eu/lumi-consortium/ LUMI consortium].
For additional background, please see the archival pages from the
[https://wiki.nlpl.eu/index.php/Community/training/2018 2018],
[https://wiki.nlpl.eu/index.php/Community/training/2019 2019],
[https://wiki.nlpl.eu/index.php/Community/training/2020 2020],
[https://wiki.nlpl.eu/index.php/Community/training/2023 2023], and
[https://wiki.nlpl.eu/index.php/Community/training/2024 2024]
NLPL Winter Schools.

For early 2025, HPLT will hold its winter school from Monday, February 3, to
Wednesday, February 5, 2025, at a
[https://www.thonhotels.com/our-hotels/norway/skeikampen/ mountain-side hotel]
(with skiing and walking opportunities) about two hours north of Oslo.
The project will organize group bus transfer from and to the Oslo
airport ''Gardermoen'', leaving the airport at 9:45 on Monday morning
and returning there around 17:30 on Wednesday afternoon.

The winter school is subsidized by the HPLT project: there is no fee for
participants and no charge for the bus transfer to and from the
conference hotel.
All participants will have to cover their own travel and accommodation
at Skeikampen, however.
Two nights at the hotel, including all meals, will come to NOK 3855 (NOK 3455 per person in a shared double room),
to be paid to the hotel directly upon arrival.

= Programme =

The 2025 winter school will have a thematic focus on ''Pretraining Data Quality and Multilingual LLM Evaluation''.
The programme will be comprised of in-depth technical presentations (possibly including some
hands-on elements) by seasoned experts, with special emphasis on open science and European languages,
but also include critical reflections on current development trends in LLM-focussed NLP.
The programme will be complemented with a ‘walk-through’ of example experience
reports on the shared EuroHPC LUMI supercomputer.

Confirmed presenters and talks include:

* [https://sites.google.com/view/alexandra-birch Alexandra Birch], University of Edinburgh'''EuroLLM – A language model for Europe'''
* [https://laion.ai/team/ Jenia Jitsev] and [https://laion.ai/team/ Marianna Nezhurina], Jülich Supercomputing Centre / LAION'''Open Foundation Models: Scaling Laws and Generalization'''
* [https://huggingface.co/guipenedo Guilherme Penedo], Huggingface'''FineWeb2: Creating a Large Multilingual Dataset for LLM Pre-Training'''
* [https://scholar.google.com/citations?user=f5FSgPwAAAAJ&hl=en Gema Ramírez-Sánchez], Prompsit Language Engineering'''A look at Pre-Training Data through the Stats Glass'''
* [https://annargrs.github.io Anna Rogers], IT University of Copenhagen'''Large Language Models and Factuality'''
* [https://portizs.eu Pedro Ortiz Suarez] and [https://commoncrawl.org/team/sebastian-nagel-engineer Sebastian Nagel], Common Crawl'''Data Quality, Language Coverage and Ethical Considerations in Web Crawling'''
* [https://scholar.google.com.tr/citations?user=fvotcRIAAAAJ&hl=tr Ahmet Üstün], Cohere AI'''Recipe for multilingual post-training: How to collect high-quality data and use them?'''

= Schedule =
{| class="wikitable"
|-
!colspan=3|Monday, February 3, 2025
|-
| 13:00 || 14:00 || Lunch
|-
| 14:00 || 15:30 || '''Session 1''' Pedro Ortiz Suarez & Sebastian Nagel '''Data Quality, Language Coverage and Ethical Considerations in Web Crawling''' 
Common Crawl is a free, open repository of web crawl data that can be used by anyone, crawled since 2008. Throughout the years the foundation has focused on achieving a balance in a diversity and representative sample of web sites while operating an efficient and polite crawler. In recent years, with the advent of LLMs and multimodal models, the interest in obtaining large amounts of high quality data has skyrocketed, while also raising concerns about the ethical considerations of large scale data curation. After a quick introduction into the history of the Common Crawl Foundation, we present our recent efforts to respond to this new data requirements while also expanding the language and cultural coverage of our dataset, and addressing the practical and ethical questions that have arisen around web crawling in the era of LLMs.
|-
| 15:30 || 15:50 || Coffee Break
|-
| 16:00 || 17:30 || '''Session 2''' Anna Rogers '''LLMs and Factuality: facts from LLMs''' 
This lecture focuses on the workflows for using LLMs as information sources, the types of problems that may result from that, and the main current mitigation strategies (RAG and CoT). Finally, I will discuss the problem of detecting generated texts, and the impact of LLMs on the information ecosphere and content economy.
|-
| 17:30 || 17:50 || Coffee Break
|-
| 17:50 || 19:20 || '''Session 3''' Alexandra Birch '''EuroLLM – A language model for Europe'''
|-
| 19:30 || || Dinner
|}

{| class="wikitable"
|-
!colspan=3|Tuesday, February 4, 2025
|-
|colspan=3 | Breakfast is available from 07:30
|-
| 09:00 || 10:30 || '''Session 4''' Guilherme Penedo
|-
|colspan=3| Free time (Lunch is available between 13:00 and 14:30)
|-
| 15:30 || 17:00 || '''Session 5''' Gema Ramírez-Sánchez
|-
| 17:00 || 17:20 || Coffee Break
|-
| 17:20 || 19:20 || '''Session 6''' Jenia Jitsev & Marianna Nezhurina
|-
| 19:30 || || Dinner
|-
| 21:00 || || '''Evening Session: Findings from HPLT'''
|}

{| class="wikitable"
|-
!colspan=3|Wednesday, February 5, 2025
|-
|colspan=3| Breakfast is available from 07:30
|-
| 08:30 || 10:00 || '''Session 8''' Ahmet Üstün
|-
| 10:00 || 10:30 || Coffee Break
|-
| 10:30 || 12:00 || '''Session 9''' Anna Rogers
|-
| 12:30 || 13:30 || Lunch
|-
| 13:45 || 16:45 || Bus transfer to OSL Airport
|}

= Registration =

In total, this year we welcome 62 participants at the 2025 winter school.
The winter school is [https://nettskjema.no/a/381438 over-subscribed] and no longer accepting registrations.
We have processed requests for participation on a first-come, first-served basis, with an eye toward regional balance.
Interested parties who had submitted the registration form have been confirmed in three batches, on '''December 6''', on '''December 13''',
and on '''December 20''', which was also the closing date for winter school registration.

Once confirmed by the organizing team, participant names are published
on this page, and registration establishes a
''binding agreement'' with the hotel.
Therefore, a cancellation fee will be incurred (unless we can find someone else to ‘take over’ last-minute
spaces), and no-shows will be charged the full price for at least one night
by the hotel.

= Logistics =

With a few exceptions, winter school participants travel to and from the conference hotel
jointly on a chartered bus (the HPLT shuttle).
The bus will leave OSL airport no later than 9:45 CET on Monday, February 3.
Thus, please meet up by 9:30 and make your arrival known to your assigned
‘tour guide’ (who will introduce themselves to you by email beforehand).

The group will gather near the DNB currency exchange booth in the downstairs
arrivals area, just outside the international arrivals luggage claims and slightly
to the left as one exits the customs area:
the yellow dot numbered (18) on the
[https://avinor.no/globalassets/_oslo-lufthavn/ankomst-arrivals.pdf OSL arrivals map].
The group will then walk over to the bus terminal, to leave the airport not long after 9:40.
The drive to the Skeikampen conference hotel will take us about three hours, and the bus
will make one stop along the way to stretch our legs and fill up on coffee.

The winter school will end with lunch on Wednesday, February 5, before the group returns
to OSL airport on the HPLT shuttle.
The bus will leave Skeikampen at 14:00 CET, with an expected arrival time at OSL
around 17:00 to 17:30 CET. After stopping at the OSL airport, the bus will continue to central Oslo.

= Organization =

The 2025 Winter School is organized by a team of volunteers at the University
of Oslo, supported by a programme committee from the HPLT and NLPL network and beyond,
please see below.
For all inquiries regarding registration, the programme, logistics,
or such, please contact <code>hplt-training@ifi.uio.no</code>.

The programme committee is comprised of:

* Barry Haddow (University of Edinburgh, UK)
* Andrey Kutuzov (University of Oslo, Norway)
* Stephan Oepen (University of Oslo, Norway)
* Sampo Pyysalo (University of Turku, Finland)
* Jörg Tiedemann (University of Helsinki, Finland)

= Participants =

# Nikolay Arefev, University of Oslo (Norway)
# Maria Barrett, Silo AI (Finland)
# Toms Bergmanis, Tilde (Latvia)
# Alexandra Birch, University of Edinburgh (UK)
# Laurie Burchell, University of Edinburgh (UK)
# Lucas Charpentie, University of Oslo (Norway)
# Pinzhen (Patrick) Chen, University of Edinburgh (UK)
# Hannah Clausen, University of Oslo (Norway)
# Lucia Domenichelli, University of Pisa (Italy)
# Aleksei Dorkin, University of Tartu (Estonia)
# Kenneth Enevoldsen, Aarhus University (Denmark)
# Tita Enstad, National Library (Norway)
# Mariia Fedorova, University of Oslo (Norway)
# Yanzhu Guo, INRIA Paris (France)
# Arzu Burcu Güven, IT University of Copenhagen (Denmark)
# Barry Haddow, University of Edinburgh (UK)
# Jan Hajič, Charles University (Czech Republic)
# Jindřich Helcl, Charles University (Czech Republic)
# Bertram Højer, IT University Copenhagen (Denmark)
# Sekh Mainul Islam, University of Copenhagen (Denmark)
# Jenia Jitsev, Jülich Supercomputing Centre / LAION (Germany)
# Márton Kardos, Aarhus University (Denmark)
# Anastasiia Klimashevskaia, University of Bergen (Norway)
# Mateusz Klimaszewski, The University of Edinburgh (UK)
# Ville Komulainen, University of Turku (Finland)
# Markus Koskela, CSC – IT Center for Science (Finland)
# Martins Kronis, Tilde (Latvia)
# Vimal Kumar Kumar, University of Limerick (Ireland)
# Andrey Kutuzov, University of Oslo (Norway)
# Hengyu Luo, University of Helsinki (Finland)
# Farrokh Mehryary, University of Turku (Finland)
# Vladislav Mikhailov, University of Oslo (Norway)
# Andreas Motzfeldt, IT University of Copenhagen (Denmark)
# Zain Muhammad Mujahid, University of Copenhagen (Denmark)
# Sebastian Nagel, Common Crawl Foundation (Germany)
# Marianna Nezhurina, Jülich Supercomputing Centre / LAION (Germany)
# Stephan Oepen, University of Oslo (Norway)
# Guilherme Penedo, HuugingFace (France)
# Irina Proskurina, University of Lyon (France)
# Taido Purason, University of Tartu (Estonia)
# Marie Roald, National Library (Norway)
# Anna Rogers, IT University Copenhagen (Denmark)
# Ismaël Rousseau, Orange (France)
# David Samuel, University of Oslo (Norway)
# Gema Ramírez Sánchez, Prompsit Language Engineering (Spain)
# Marta Sartor, University of Pisa (Italy)
# Ipek Baris Schlicht, Universitat Politècnica de València (Spain)
# Étienne Simon, University of Oslo (Norway)
# Pavel Stepachev, The University of Edinburgh (UK)
# Pedro Ortiz Suarez, Common Crawl Foundation (France)
# Otto Tarkka, University of Turku (Finland)
# Kushal Tatariya, KU Leuven (Belgium)
# Jörg Tiedemann, University of Helsinki (Finland)
# Samia Touileb, University of Bergen (Norway)
# Elke Vandermeerschen, KU Leuven (Belgium)
# Raul Vazquez, University of Helsinki (Finland)
# Ramón Carreño Villar, University of Oslo (Norway)
# Fedor Vitiugin, Aalto University (Finland)
# Tea Vojtěchová, Charles University (Czech Republic)
# Artūrs Znotiņš, IMCS at University of Latvia (Latvia)
# Elaine Zosa, Silo AI (Finland)

Community/training

2025-02-03T10:55:51Z

Andreku: /* Schedule */

Community/training

2025-02-02T20:57:38Z

Andreku: /* Participants */

Community/training

2025-01-31T16:16:51Z

Andreku: /* Schedule */