Polish Large Language Model (PLLuM)

Challenge

The PLLuM (Polish Large Language Model) project aims to create a Polish large language model in line with the principles of responsible AI development. The model will be fully open and free, and the licensing system will allow its implementation not only in public administration, but also in business. However, the project goes beyond the creation of the model. PLLuM will be enriched with various auxiliary processes, such as preference adaptation and output correction. The ecosystem created within the project for training and evaluating LLMs can serve as a foundation for building future language models. An additional outcome of the project will be a prototype of an intelligent assistant to support the Polish public administration.

There are many ways to support our initiative. At this time, we especially encourage you to contact us about donating text data to train the model. Please fill out the contact form on the official PLLuM website.

The PLLuM project is a unique collaboration between leading Polish scientific institutions, bringing together experts from different fields. It will be carried out from January 22 to December 2024 by a consortium of six institutions: Wroclaw University of Technology (project leader), NASK – National Research Institute, the Information Processing Center – National Research Institute (OPI PIB), the Institute of Computer Science Foundations of the Polish Academy of Sciences, the University of Lodz and the Institute of Slavic Studies of the Polish Academy of Sciences – under the mandate of the Ministry of Digital Affairs.

The work includes:

1. Collecting diverse language data and creating high-quality corpora for model training and fine-tuning.
2. Training a large language model for the Polish language.
3. Fine-tuning the model using original instruction datasets.
4. Aligning the model based on original preference datasets.
5. Developing an output correction module to improve the quality of the model’s responses.
6. Designing a virtual assistant to support public administration.

The model will be made available at the end of the project in December 2024.