InferQuest
← Quest map
Training Phase 3 · SFT & RL

The Reasoning Engine

RLVR on one GPU — the skill the biggest hiring category wants.

0/5 · 660 XP
You’re previewing the curriculum. Sign in (top right) to track progress, take knowledge checks, and unlock the verifiers.
Briefing

RL post-training is now the largest training-side hiring category; Anthropic alone lists around ten RL reqs. GRPO is why it's learnable on your hardware: no value network, no reward model for verifiable tasks. Sample groups, score against a checker, normalize within the group. Read the lineage in order — DeepSeekMath invents it, R1 proves it at scale, DAPO, Dr. GRPO, and GSPO fix its biases — then run it. Documented setups get reasoning RL going in 5 GB of VRAM, and a 0.5B model gains about 10 GSM8K points in an epoch.

The bridge task is where the two paths meet. Measure how much of your RL wall-clock is rollout generation: it dominates, which is why TRL runs vLLM inside the trainer and why RL-infra postings require inference skills. Your serving knowledge is a hiring edge here, not a detour. On-policy distillation closes the quest because it's the current cost/quality frontier for small models — the teacher grades student tokens by reverse KL, delivering RL-grade gains at a tenth of the compute.

Tasks
  • The algorithm, the scale proof (arxiv.org/abs/2501.12948), then the fixes: DAPO (2503.14476), Dr. GRPO's length-bias correction (2503.20783), GSPO for MoE stability (2507.18071).

    paper+100 XPresource ↗
  • TRL or Unsloth, ≤1.5B model, GSM8K-style verifiable tasks. Documented from 5 GB VRAM; log reward curves and watch for length hacking.

    build+180 XPresource ↗
  • ≥8–10 point pass@1 lift on a held-out problem split. Auto-verifier (secret 500-problem split, disjoint from public sets) is in calibration — self-check on your own holdout for now.

    build+200 XP
  • Instrument a GRPO run: what fraction is generation vs gradient steps? This number is why RL teams hire inference engineers — and why your serving-path skills compound here.

    bench+60 XP
  • Student samples, teacher grades per-token (reverse KL) — RL-grade gains at ~1/10 the compute. TRL's DistillationTrainer is stable; also read Apple's distillation scaling laws (2502.08606).

    build+120 XPresource ↗