Teach It to Chat
Base model → assistant: SFT, LoRA, DPO on one consumer GPU.
Deliberately reachable without the pretraining capstone: post-training starts from open base models, and it's where the most jobs are. TRL v1.0 is the stack — its 2026 shape is itself a lesson (SFT/DPO/GRPO/Distillation stable, PPO demoted to experimental). “LoRA Without Regret” is the modern citation that changed default practice: adapters on all layers including MLPs at ~10× the full-FT learning rate match full fine-tuning for post-training workloads, which is what makes one consumer GPU a legitimate post-training rig.
Read LIMA critically — its thousand-example minimalism is true for style and format, not for capabilities (reasoning SFT uses six-figure trace counts). DPO survives as the preference stage in every serious open recipe (Tulu 3, OLMo 3) while RL owns capabilities — that division of labor is a favorite interview probe. The chat-template rigor from the serving path applies here in reverse: you're now the one whose template bugs everyone else inherits.
SmolLM-class base, a real instruct dataset, the chat template applied and hand-verified (render one example yourself and diff it). Compare before/after generations.
All layers including MLP, ~10× LR, modest rank — the Thinking Machines recipe. QLoRA (arxiv.org/abs/2305.14314) puts 7B-class fine-tuning inside 8 GB.
“Quality over quantity” — now understood as true for style/format, false for capabilities. Knowing the 2026 read on it beats knowing the abstract.
TRL DPOTrainer on your SFT checkpoint. Understand the implicit-reward derivation well enough to whiteboard it — it gets asked.
The harness trains its own base model, then hands it to you frozen with a budget of “language B” — 8 tokens the base has never seen. Return a rank-≤8 LoRA (the harness applies it to ITS base): language B must reach the calibrated band while language A regresses ≤0.1. The two lessons are rehearsal and learning-rate discipline — skip either and watch A collapse.
build+200 XPauto-verifiedTokenizer → pretrain → SFT → chat UI, one repo, small enough to finish. The integrative build: every stage you've now studied, touched in sequence.