---
title: "Week 3: Feed-forward Networks and Backpropagation"
subtitle: "Composition, nonlinear representations, and computation graphs"
description: "Extend a linear classifier just enough to solve a problem with a nonlinear decision boundary."
jupyter: python3
---

::: {.callout-tip}
## Weekly workflow

| Phase | Student work |
|---|---|
| Before Session 1 | Trace one worked computation graph and review the chain rule bridge when needed. |
| Session 1 | Hidden representations, nonlinearities, computation graphs, chain rule, and manual backpropagation. |
| Between sessions | Implement one manual backward pass and compare it with automatic differentiation. |
| Session 2 | Paired MLP implementation, gradient checking, and debugging of broken derivative code. |
| After Session 2 | Complete the MLP and test one architectural change under a fixed training budget. |
:::

## Opening project

Extend a linear classifier just enough to solve a problem with a nonlinear decision boundary.

**Principal artifact:** A gradient-checked MLP and an activation or width ablation.

## Learning outcomes

- **W03-01:** Explain why stacked affine maps require nonlinear activations.
- **W03-02:** Trace forward and backward values through a small computation graph.
- **W03-03:** Implement and gradient-check a two-layer MLP.
- **W03-04:** Relate width, activation choice, and representation capacity.

## Prerequisites

Weeks 1–2: linear prediction, cross-entropy, gradients, and vectorized tensor operations.

## Design questions

1. What structure in the data or task should the model exploit?
2. Which computation is essential enough to implement from first principles?
3. Which baseline would reveal whether the added complexity is justified?
4. What controlled change could falsify the current design hypothesis?

## Conceptual model

::: {.callout-note}
## Authoring placeholder

Replace this callout with the final narrative, equations, worked example, and prediction questions. Keep the first executable example small enough to run in seconds.
:::

```{python}
#| label: fig-w03-concept
#| fig-cap: "A two-layer feed-forward network represented as a sequence of differentiable operations."
#| fig-alt: "Boxes for input, affine transformation, nonlinear activation, second affine transformation, and output connected by arrows."

import matplotlib.pyplot as plt
from dlbook.figures import plot_week_03

plot_week_03()
plt.show()
```

## From mechanism to implementation

The lab for this week follows a fixed progression:

1. reproduce the minimal mechanism;
2. test dimensions, invariances, masks, or gradients;
3. assemble the larger model;
4. state an experimental prediction;
5. change one factor under controlled conditions;
6. interpret the result rather than only reporting a score.

## D2L reuse and new material

Strong reuse candidate: MLPs, computational graphs, forward/backward propagation, initialization, and numerical stability. Add explicit gradient checking and failure diagnosis.

## Knowledge check

- Explain the module's central inductive bias or training principle in one paragraph.
- Identify one plausible implementation that would appear correct but violate the intended computation.
- State one metric that is necessary but insufficient for evaluating the weekly artifact.

## Summary

This page is deliberately a skeleton. Preserve its learning outcomes, runnable figure contract, and weekly workflow while replacing the authoring placeholders with reviewed content.