All pages on this site · 13 published · newest first
You now know what every line does. This chapter shows how the numbers in this guide were measured, how to read the judge's profile, and how to run real experiments on the judge's own H200 without…
gemvz_kernel (lines 275–472) is the workhorse of the submission. It runs 12 of the 30 scored shapes and produces most of our speedup. Structurally it is gemv_kernel from chapter 6 with three changes:
This chapter covers the last code region, lines 475–583. It holds two Triton GEMM kernels and the split-K "epilogue" they share. They run the router at m = 128 and 256 and the pooler at m = 128. The…
This chapter reads lines 41–244: the plain BF16 tensor-core GEMV, its launch helper, and its Python entry point. The kernel runs the router at m ≤ 32 and the unpooler slice at m ≤ 8. It is also the…
At m ≤ 16 the GEMV runs at nearly full memory bandwidth, so it can only go faster by reading fewer bytes. This chapter shows how each BF16 weight is stored in 12 bits with no loss of information. It…
Four lines of Python make the GEMV fast. _pack reorders each weight matrix once, in prepare() . After that, every lane can fetch its whole tensor-core operand with a single 16-byte load, and every…
This chapter reads every line of submission.py that is plain Python: the header, the configuration tables, how the CUDA code gets compiled, prepare() , and the dispatcher kernel_fn() . The CUDA…
submission.py is not one clever kernel. It is a dispatcher that sends each (layer, m) to whichever of four engines was fastest when we measured it. This chapter shows the whole map first, then…
This chapter explains the handful of hardware facts the kernels are built around. You don't need the whole CUDA programming guide: just how work is split into threads, how bytes reach those threads,…
Before reading any kernel code, you need three things: what is computed, how it is timed, and the fastest time physics allows. This chapter covers all three. It ends with a lab that recomputes our…
This guide walks through our Open Kernel Challenge submission for muse_bf16_proj one line at a time. It is written for someone who knows Python and roughly what a GPU kernel is. Every other idea is…
Short definitions of every term this guide uses, in alphabetical order, with a pointer to the chapter where it matters most.
Result: 1.33–1.34× vs SGLang (was 1.25×). Round 5 added a lossless 12-bit compressed-weight GEMV for the decode shapes. Scored run 1.331× (95% CI 1.311–1.351×, 3,628.9 µs/step, rank #167 , job…