Phase 5 · Compression

Compress & Prove It

Quantization without evals is vandalism.

0/5 · 740 XP

You’re previewing the curriculum. Sign in (top right) to track progress, take knowledge checks, and unlock the verifiers.

Tasks

Pass the grader: build a group-wise W4 quantizer from scratch
Scale/zero-point derivation, per-group granularity, dequantize — no quantization libraries. Graded on 4-bit validity, beating the per-tensor baseline ≥3× on outlier-heavy weights, and keeping the reference GPT's predictions intact end-to-end.
build+200 XPauto-verified
Run a per-layer quantization sensitivity analysis
MIT 6.5940-style: quantize one layer (or group) at a time, measure the damage, and let the scan choose where precision goes — before reaching for uniform recipes.
bench+90 XP
Quantize a real model with llm-compressor (FP8 + W4A16)
The production pipeline for vLLM. Produce both checkpoints from the same base model.
build+150 XPresource ↗
Run lm-eval-harness on base vs quantized, against a vLLM endpoint
Quality deltas on real tasks, plus throughput deltas from your benchmark harness. Red Hat's half-million-eval study is your methodology template.
bench+150 XPresource ↗
VERIFIED: publish your quantization study
Quality × speed × memory, real numbers, honest about regressions.
write+150 XPauto-verified