Deep Learning: Models, Training, and Adaptation

From linear models to modern neural networks

Author

Affiliation

Christoph Lippert

Hasso Plattner Institute, University of Potsdam

Deep learning connects a simple idea—learning from examples—to models that recognize images, understand language, and generate new content. This book develops that connection step by step. You will begin with linear regression, build and train neural networks, and explore the architectures behind modern vision, language, and generative models.

Throughout the book, we return to three questions: What can a model represent? How can we train it? How can we adapt it to a new task? Understanding these questions will help you make informed choices when a new architecture, dataset, or application comes along.

0.1 About this book

This is an executable textbook for a 15-week bachelor-level course. Mathematical explanations sit alongside runnable examples, visualizations, exercises, and experiments. You can read a derivation, inspect its implementation, and then change the assumptions to see what happens.

Examples from digital health connect the methods to concrete problems. These applications also raise questions that matter well beyond medicine: How should we split data? What happens when the data distribution changes? Which errors matter? And how far do our results generalize?

0.2 Contents at a glance

The book moves from foundations to specialized architectures, then brings them together through adaptation and comparison. Training and evaluation accompany the models throughout.

Part	Weeks	Main topics	Guiding question
I. Foundations of Learning	1–4	Linear regression; linear classification; optimization and regularization; multilayer perceptrons and backpropagation	How do we define, train, and evaluate a model?
II. Spatial, Generative, and Relational Models	5–8	CNNs and ResNets; transfer learning and U-Net; diffusion; sets, multiple-instance learning, and graph neural networks	How do we build models for images, collections, and relationships?
III. Sequence Models and Transformers	9–12	RNNs and GRUs; attention and Transformer blocks; GPT, BERT, and self-supervised learning; encoder–decoder models and fine-tuning	How do models learn from sequences and use context?
IV. Efficient Models and Synthesis	13–15	Vision Transformers and visual self-supervised learning; Mamba and distillation; architecture selection and reproducible comparison	How do we adapt and compare models for a task and a computing budget?

import matplotlib.pyplot as plt
from dlbook.figures import plot_curriculum_map

plot_curriculum_map()
plt.show()

Figure 1: A roadmap through the course: foundations support increasingly structured models and their comparison.

Start with Getting Started for the Python setup, then read Linear Regression and work through its lab. For a first reading, follow the course order: later chapters build on the notation, training methods, and experimental habits introduced earlier. When working on your project, return to the relevant chapters as a reference.

0.3 What you will learn to do

• Reason about models: connect mathematical expressions, computational operations, and architectural choices.
• Train models: choose an objective and optimizer, inspect learning curves, and investigate unstable or unsuccessful training.
• Evaluate results: establish baselines, separate training from validation and testing, and interpret performance in relation to the task.
• Adapt existing models: use pretrained representations and decide which parts to freeze, fine-tune, or replace.
• Design experiments: change one factor at a time, perform ablations, and distinguish evidence from speculation.
• Communicate findings: explain your choices, document reproducible experiments, and state the limits of your conclusions.

0.4 Before you begin

You do not need prior experience with deep learning or PyTorch. Familiarity with basic Python, vectors and matrices, derivatives, and elementary probability will help you get started. The chapters revisit mathematical ideas as they become relevant; use the accompanying examples to connect the notation to the code, and bring questions about unfamiliar steps to the sessions.

Keep track of tensor shapes, what each symbol represents, and which quantities are learned. These small habits make both derivations and implementations easier to understand.

0.5 How to study with this book

The course uses a flipped classroom format with two sessions each week. Reading and preparation introduce the ideas; the sessions give you time to apply them, ask questions, and work through difficult steps together.

1. Before the first session: read the assigned concepts, study the worked example, and attempt the readiness questions. Write down anything you cannot yet explain.
2. In the first session: work through derivations, discuss model choices, and apply the ideas to exercises. Short lectures clarify the points that need more attention.
3. Between sessions: complete the implementation checkpoint and check that you understand what the code computes.
4. In the second session: work through the lab, compare results, and investigate how changes affect the model.
5. After the second session: finish the exercises, run an ablation, and write a short explanation of what you learned.

When running an example, first predict what you expect to happen. Then change a parameter, an architectural choice, or a data assumption and compare the outcome with your prediction. A surprising result is an opportunity to inspect the model and revise your understanding.

If you use coding assistants, check their output against the mathematics and the experiment’s purpose. You should be able to explain the objective, data flow, and evaluation procedure in your own words.

0.6 Your course project

At the end of the semester, you will complete a coding project that brings together methods from the course. You will apply what you have learned to a concrete problem, evaluate your approach, and explain your results and their limitations. The chapters and labs provide examples and techniques you can return to when developing your solution. The project requirements and assessment criteria will be provided separately.

Begin with the foundations, and keep the three recurring questions in view: representation, training, and adaptation.