Data-Driven Digital Innovation Lab

Type of event: Hands-on workshops
Organisation: Faculty of Mathematics, Informatics and Natural Sciences
Funding period: 01.08.2024 to 30.09.2025
Short title: D3 Innovation Lab

Orientation of the D³ Innovation Lab

Language models in relation to generative AI are a central part of the digital landscape today and are used in many areas of natural language processing, such as text generation, revision, or translation. General-purpose models such as GPT (Generative Pre-trained Transformer) and BERT (Bidirectional Encoder Representations from Transformers) are particularly versatile and can be used for a variety of tasks without adjustments. However, they do not always perform best for specific use cases. By using customization methods such as fine-tuning, prompt engineering, or integrating new knowledge sources, these models can be tailored to specific requirements to increase their performance for certain use cases.

Such customized instances make it possible to use the versatility of general-purpose models even more effectively. By fine-tuning them to specific data or tasks, better results can be achieved, and the models can be adapted to particular needs. The objective of this project was to develop a guideline to help users select the language model that best suits their individual requirements and to customize it. This should help to maximize the potential of generative AI systems.

Review and results

The project yielded several key results that will have a lasting impact in terms of both content and methodology. It is particularly noteworthy that six workshops reached 105 students, who engaged intensively with the functioning, possibilities, and limitations of generative language models. The workshops not only imparted technical knowledge, but also promoted a reflective approach to tools supported by generative AI.

The students acquired numerous skills in the process: they learned to recognize differences, for example in performance, between general and fine-tuned language models and to critically evaluate their possible applications. Practical exercises introduced them to topics such as data preparation, model training, data-supported decision-making for language models, fine-tuning approaches, and evaluation of results. In addition, important interdisciplinary skills were also taught, including critical reflection on ethical issues (bias, fairness, data protection) and the ability to classify the potential and risks of AI-supported applications in various fields of application.

Another key result was the development and piloting of a guide for selecting and adapting language models. Although this guide has not yet been made publicly available due to necessary quality assurance measures, it has already been tested and iterated in workshops. Students provided valuable feedback on this, emphasizing that the guidelines offer orientation in an otherwise confusing model landscape and helped them make informed decisions about a) the choice of a base language model, b) the choice of whether fine-tuning is necessary, and c) the most suitable fine-tuning approach for an underlying use case.

Tips from lecturers for lecturers

New teaching formats were tested, in particular workshop-based approaches with a strong practical focus (hands-on learning). One practical lesson learned was how important it is to involve heterogeneous groups, from students with technical knowledge to those from other disciplines, through differentiated tasks, clear structuring, and accompanying reflection phases – ideally based on a common starting point. This helped to hone skills in moderating interdisciplinary discussions and dealing with different learning prerequisites. This led to particularly exciting discussions with and among the students, who contributed and exchanged their perspectives.

In addition, the project raised awareness of the importance of continuous feedback loops, even in ongoing workshop series: the direct integration of student feedback into the further development of the concept proved to be particularly valuable.