---
title: "Experimental Design and Reproducibility"
jupyter: python3
---

A comparison is interpretable only when the independent variable is isolated and the evaluation budget is explicit.

```{python}
#| label: fig-app-repeated-runs
#| fig-cap: "Repeated runs expose uncertainty hidden by a single result."
#| fig-alt: "Points from five simulated seeds for two methods with horizontal mean markers."

import matplotlib.pyplot as plt
import numpy as np

rng = np.random.default_rng(21)
a = rng.normal(0.78, 0.015, 5)
b = rng.normal(0.80, 0.025, 5)
plt.figure(figsize=(5.5, 3.4))
plt.scatter(np.zeros_like(a), a, label="A")
plt.scatter(np.ones_like(b), b, label="B")
plt.hlines(a.mean(), -0.15, 0.15)
plt.hlines(b.mean(), 0.85, 1.15)
plt.xticks([0, 1], ["method A", "method B"])
plt.ylabel("validation metric")
plt.tight_layout()
plt.show()
```