NVIDIA CUTLASS: High-Performance CUDA Templates for AI Linear Algebra

If you've trained a transformer in the last three years, your GPU spent most of its wall-clock time inside a matrix multiplication. The kernels doing that work were probably written by cuBLAS, generated by a compiler stack like Triton, or hand-assembled on top of NVIDIA's CUTLASS templates. CUTLASS is the one most people don't see directly, but it sits underneath a surprising amount of modern AI infrastructure — from FlashAttention to vLLM to several internal kernels inside PyTorch. Wh...