Phase 2 · Engine Core

The KV Cache

The single most important idea in LLM serving.

0/3 · 410 XP

You’re previewing the curriculum. Sign in (top right) to track progress, take knowledge checks, and unlock the verifiers.

Tasks

Read kipply's “Transformer Inference Arithmetic” and do the math by hand
KV bytes/token, FLOPs, latency floors. Do it once by hand for a model you serve and you'll never be fooled by a benchmark again.
read+80 XPresource ↗
Pass the grader: KV-cached incremental decoding
Implement a cached decoder for the harness's reference GPT: prefill + O(1)-per-token decode. Graded on exact logit match AND ≥2× measured speedup over naive recompute.
build+250 XPauto-verified
Pass the KV-cache sizing drill
The interview staple: cache-per-token math, GQA arithmetic, quantized-KV capacity. 75% to pass.
drill+80 XPauto-verified