Polish Large Language Model (PLLuM)

Innovative Polish large language model for the public and the private sector

PLLuM

Challenge

Challenge

 

The PLLuM (Polish Large Language Model) project aims to create a Polish large language model in line with the principles of responsible AI development. The model is fully open and free, and the licensing system will allow its implementation not only in public administration, but also in business. However, the project goes beyond the creation of the model. PLLuM will be enriched with various auxiliary processes, such as preference adaptation and output correction. The ecosystem created within the project for training and evaluating LLMs can serve as a foundation for building future language models. An additional outcome of the project will be a prototype of an intelligent assistant to support the Polish public administration.

Agnieszka Karlinska
Projekty_StyloMetrix_ikona
Project leader
Dr Agnieszka Karlińska

What have we done?

The PLLuM project is a unique collaboration between leading Polish scientific institutions, bringing together experts from different fields. It was carried out from January 22 to December 2024 by a consortium of six institutions: Wroclaw University of Technology (project leader), NASK – National Research Institute, the Information Processing Center – National Research Institute (OPI PIB), the Institute of Computer Science Foundations of the Polish Academy of Sciences, the University of Lodz and the Institute of Slavic Studies of the Polish Academy of Sciences – under the mandate of the Ministry of Digital Affairs.

 

The work included:

 

  1. Collecting diverse language data and creating high-quality corpora for model training and fine-tuning.
  2. Training a large language model for the Polish language.
  3. Fine-tuning the model using original instruction datasets.
  4. Aligning the model based on original preference datasets.
  5. Developing an output correction module to improve the quality of the model’s responses.
  6. Designing a virtual assistant to support public administration.

 

The model has been made available at the end of the project in December 2024. We describe the project’s results on its website: https://pllum.org.pl/blog/posts/zakonczenie-projektu-pllum.