The Recipe
Scaling laws are the interview math of the training world.
Chinchilla plus the Epoch replication is the assigned pair because the replication is how you learn to read scaling-laws papers critically — it corrected the original's parametric fit and strengthened the ~20-tokens-per-parameter headline. Then Beyond Chinchilla-Optimal breaks the spell: once you serve a model at volume, compute-optimal is the wrong target, and training small models far past it (SmolLM3: 3B parameters, 11.2T tokens) is the industry default. Being able to argue both sides with arithmetic is precisely what xAI lists as a basic qualification.
The Smol Training Playbook is the modern synthesis — the SmolLM team's actual decisions with their reasoning — and the closest thing to shadowing a pretraining team. μP is deliberately framed as the optional deep end: know what hyperparameter transfer buys and who uses it, and know that the flagship open recipes mostly don't. The drill at the end is pure arithmetic, like the KV-cache math on the other path: 6ND, token budgets, epoch limits, precision effects.
Then the replication (arxiv.org/abs/2404.10102) — how the Approach-3 fit was corrected, and why ~20 tokens/param survived the audit.
Why deployable small models train 100×+ past compute-optimal — Llama 3 8B at ~1,875 tok/param, SmolLM3 at ~3,700.
The SmolLM team's decision log for building world-class small models: architecture, data mixture, schedule — with reasons attached.
Low-precision training as an effective-parameter discount — and the finding that overtrained models quantize worse afterward, which ties back to Precision Games.
Tune on a small proxy, transfer the LR to the big run. Know the idea and its successors (u-μP, CompleteP) — and that most flagship open recipes ship without it.
6ND arithmetic, Chinchilla vs overtraining, epochs on limited data, precision effects. 75% to pass.
drill+80 XPauto-verified