Track A: Large AI Models

Christophe Cerisara is a French researcher at CNRS (National Centre for Scientific Research), specialized in machine learning models for natural language processing (NLP). He has created and is leading the SYNALP research team composed of about 20 NLP researchers since 2012. He is also the leader of the AI-NLP axis of the LORIA laboratory since 2019, and he has been referent for the French National Plan in AI in 2020. He has supervised more than 12 Ph.D. thesis, and has lead several projects about AI and training Large Language Models in the past few years.

Introduction to Large Language Models”] The first part of this course will give the basic principles of the transformer architecture and how the decoder can be trained to build a Large Language Model (LLM), including a short overview of its scaling laws. The second part will present how to use such a trained LLM, either directly through zero-shot and in-context learning, or through fine-tuning to adapt the LLM to a given task, but with a focus on the direct usage of the LLM on either low-end and high-end hardware and without going into the details of parameter-efficient fine-tuning and other advanced adaptation strategies. The third part (30′) will consist of a practical session about how to implement this with the huggingface transformers library. The prerequisites for this course are a good knowledge of python and of fundamentals of machine learning; some experience with pytorch is useful.

Dr. Malte Ostendorff is a senior research engineer at Deutsche Telekom where he works on large language models (LLMs) and related topics. Previously, Malte was a senior researcher at the German Research Center for Artificial Intelligence (DFKI) and a Ph.D. student in the Scientific Information Analytics group at the University of Göttingen. Furthermore, Malte is a co-founder of Occiglot, a research collective for open-source language models for and by Europe, and a co-founder of Open Legal Data.

Large language models (LLMs) have emerged as a powerful technology underpinning state-of-the-art chatbots and various other natural language processing applications. Model sizes and computing resources that are used for building LLMs dominate the public discourse around these models, whereas one crucial aspect is often neglected – the LLM training data. LLMs are statistical models that learn from data and, therefore, the training data is crucial for LLMs and one of the main differentiators between different models. In this course, we will explore the datasets that were used by existing LLMs, automated tools for data curation and processing at the Web scale, and the most prominent sources where the data is coming from. We will discuss why commercial LLM providers are secretive about their data and what issues arise from training models on large-scale datasets. A basic understanding of what LLMs are and how they are trained is a prerequisite for this course.

xxxxxxxxxxxxxxxxxxxx

Dr. Jindong Gu is a senior research fellow at University of Oxford. He also partially works in Google DeepMind as a faculty researcher in Gemini Safety team. Prior to that, He received his Ph.D. Degree from University of Munich. His research focus is to build Responsible AI systems. Specifically, he is interested in the interpretability, robustness, privacy, and safety of visual perception, foundation models, robotic policy and planning, and their fusion towards general intelligent systems.

In recent years, generative AI (GenAI), like large language models and text-to-image models, has received significant attention across various domains. However, ensuring the responsible generation of content by these models is crucial for their real-world applicability. This raises an interesting question: What should responsible GenAI generate, and what should it not? This course will introduce the practical responsible requirements of both textual and visual generative models, outlining five key considerations: generating truthful content, avoiding toxic content, refusing harmful instruction, leaking no training data-related content, and ensuring generated content identifiable.

TBA

Alexandre is part of the founding research team at Kyutai, a leading non profit research lab in Paris. Before he was a research scientist for 3 years at Meta AI Research, leading in particular the development of the AudioCraft framework (EnCodec, AudioGen, MusicGen). Alexandre completed his PhD at Facebook AI Research and INRIA Paris, working in particular on music source separation (Demucs).

In this course, we will learn the theory around discrete audio modeling, covering the different components and techniques used (neural audio codec, multi stream transformers, etc), as well as the specificities of the audio domain. Then, we will apply theses techniques to fine tuning pre trained audio models to new datasets of audio. Attendees should have previous experience with Pytorch for the practical part of the class, along with a setup google account to use google Colab. Previous experience working with audio will help but is not required!