The Gauntlet

Mixed drill across everything: cache math, rooflines, spec-decode acceptance, parallelism, SLOs. 80% to pass — interview bar, not course bar.

Seven scenarios distilled from REAL reported prompts: Anthropic's single-GPU batcher and GPU-credit scheduler, Fireworks' multi-tenant SLAs, Baseten's no-thrash autoscaling, Together's 100-model fleet and shared KV cache, the 100K-RPS p99 design. 75% to pass.

The reported prompts, verbatim: 'Design an inference batching system — one GPU, up to 100 inputs per batch, users waiting' (Anthropic); 'serving path for a 70B under 200ms TTFT on H100' (Fireworks); 'serve 100+ open-source models on shared GPUs' and 'multi-tenant LoRA fine-tuning service' (Together); 'GPU autoscaling on queue depth without thrashing, handling cold starts' (Baseten); '100K QPS with strict p99' (NVIDIA). Whiteboard each in 35 minutes, alone, out loud.

Real loops are bimodal: NVIDIA still asks LeetCode (reports of 2 hards in 40min) while Baseten/Modal ask practical infra. Drill the infra-flavored classics (interval merging as 'GPU idle windows', DAG cycle detection as 'pod dependencies', LRU cache, rate limiter, beam search against unit tests), practice the rising buggy-file round (Mistral hands you 300 lines with a bug in attention masking/sampling/batching — 30 minutes), and work a question bank (StackScholar's LLM-inference set, github.com/llmgenai/LLMInterviewQuestions).

Fleet ops, verified endpoints, cost-per-token wins, merged PRs, published benchmarks, harness receipts. Cut everything else. Then rehearse the attribution drill: NVIDIA asks 'what percentage of the total improvement came from the specific thing YOU optimized?' — know the decomposition of every speedup you claim.