Under the Tensor

ezyang's post packs years of PyTorch core knowledge into one read. The idea to hold onto is strides: a tensor is a flat buffer plus indexing math, and view/permute/slice never copy anything. The same idea shows up later as paged-KV block tables, Triton pointer arithmetic, and coalesced-access analysis, which is why the strided-tensor build is here even though it feels like a detour.

py-spy is here because the third bottleneck, overhead, is usually Python. In a real engine the model runs on GPU while the scheduler, API server, and detokenizer are host code, and profiling is how you find out when they're the actual ceiling. Run it against something real you own, not a toy.