Phase 2 · Engine Core

Build Your Own Engine

The capstone: a real serving engine, probed live by InferQuest.

0/4 · 900 XP

You’re previewing the curriculum. Sign in (top right) to track progress, take knowledge checks, and unlock the verifiers.

Tasks

Read nano-vllm end to end
~1.2k lines: prefix caching, TP, CUDA graphs. The nanoGPT of inference engines — your blueprint.
read+80 XPresource ↗
Build a toy engine: continuous batching + paged-ish KV + streaming
Requests join/leave the batch at token boundaries; block-allocated KV; SSE streaming out with per-request cancellation and timeouts (the 'streaming token generator with cancellation' is a recurring Fireworks/Together coding exercise). Any small model.
build+300 XP
Add structured output (JSON mode) via token masking
Outlines-style FSM masking, or integrate xgrammar. Read the vLLM structured-decoding intro first.
build+120 XPresource ↗
VERIFIED CAPSTONE: your engine passes the OpenAI conformance probe
Expose your engine as an OpenAI-compatible API and point InferQuest's prober at it: /v1/models, chat completions, usage accounting, max_tokens cutoff, SSE streaming framing with [DONE], and error shapes — all live-tested.
build+400 XPauto-verified