The Serving Surface
Where production bugs actually live: templates, tools, adapters, modalities.
Template mismatch is a top source of silent quality regressions. Render a template by hand for one model and diff it against apply_chat_template.
How model-emitted markup becomes OpenAI tool_calls JSON — per-model parsers (hermes/llama/mistral), and where they break. Skim MCP as the emerging layer above.
Deploy one reasoning model on the fleet: separate reasoning_content, cap thinking budgets, and measure what long decodes do to your ITL and cost math.
S-LoRA/LoRAX-lineage batched adapters. Serve ≥2 adapters on one vLLM instance, hot-load a third, and check adapters appear in /v1/models.
Image inputs via content parts (encoder scheduling + image-token budgets), plus a TEI-style embedding server — a different serving profile: no KV cache, latency-critical.
API keys (and what stays unauthenticated: /health, /metrics), prefix-cache timing side channels + cache salting, and a guardrails/prompt-injection primer for the gateway layer.