InferQuest
← Quest map
Training Phase 3 · SFT & RL

The Reasoning Engine

RLVR on one GPU — the skill the biggest hiring category wants.

0/5 · 660 XP
You’re previewing the curriculum. Sign in (top right) to track progress, take knowledge checks, and unlock the verifiers.
Briefing

RL post-training is now the largest training-side hiring category (Anthropic alone lists ~10 RL reqs), and GRPO is why it's learnable on your hardware: no value network, no reward model for verifiable tasks — sample groups, score against a checker, normalize within the group. Read the lineage in order (DeepSeekMath invents it, R1 proves it at scale, DAPO/Dr. GRPO/GSPO fix its biases) and then run it: documented setups get reasoning RL from 5 GB of VRAM, and a 0.5B model gains ~10 GSM8K points in an epoch.

The bridge task is the punchline of the whole two-path platform: measure how much of your RL wall-clock is rollout generation — it dominates, which is why TRL runs vLLM inside the trainer and why RL-infra postings require inference skills. Your serving-path knowledge is a hiring edge here, not a detour. On-policy distillation closes the quest because it's the cost/quality frontier for small models: teacher grades student tokens by reverse KL, delivering RL-grade gains at a tenth of the compute.

Tasks
  • The algorithm, the scale proof (arxiv.org/abs/2501.12948), then the fixes: DAPO (2503.14476), Dr. GRPO's length-bias correction (2503.20783), GSPO for MoE stability (2507.18071).

    paper+100 XPresource ↗
  • TRL or Unsloth, ≤1.5B model, GSM8K-style verifiable tasks. Documented from 5 GB VRAM; log reward curves and watch for length hacking.

    build+180 XPresource ↗
  • ≥8–10 point pass@1 lift on a held-out problem split. Auto-verifier (secret 500-problem split, disjoint from public sets) is in calibration — self-check on your own holdout for now.

    build+200 XP
  • Instrument a GRPO run: what fraction is generation vs gradient steps? This number is why RL teams hire inference engineers — and why your serving-path skills compound here.

    bench+60 XP
  • Student samples, teacher grades per-token (reverse KL) — RL-grade gains at ~1/10 the compute. TRL's DistillationTrainer is stable; also read Apple's distillation scaling laws (2502.08606).

    build+120 XPresource ↗