Prior art, community tips and measured facts

Every public OKC solution I found, what to take from each, and the corrections the deepest data forced on this plan. Updated Nov 14 after a second research pass.

Public solutions, ranked by usefulness

RepoStanding (their report)Take from it
keell0renzdecode #38, bf16 #26, prefill #29, attention #35, MoE #58 (Oct 31). Best public on all five boards.Start here. Code for all five problems. 12-bit BF16 codec, 6-bit scales, stream-K wgmma MoE, per-M builds, PDL trigger placement. LESSONS.md has about 460 experiments, failures included.
alygg77overall #40; MoE 2.40, decode 2.21The best measurement corpus published: docs/H200_MEASURED_FACTS.md and docs/NEGATIVE_RESULTS.md (summarized below). KPERM3 scale-sharing re-layout.
vjeuxthe base most teams forkedNative CUDA, one file per problem. Origin of the lossless BF16 weight format. Its prefill and attention files were still near the top unchanged.
yesthatwilly#147 overall; prefill #50Closing-board leader numbers and rules. A 792-line claims ledger (landed / lead / shelf / refuted) per problem. Their verdict: the public keell0renz and alygg77/vjeux files are the fastest known, and own work added 0.1–2% per row.
georgxzhmid-boardPer-shape SGLang µs for every problem, the rule-change timeline, and the DeepGEMM path for 16K prefill experts.
Kushagrabainslamid-boardClear designs: cluster/DSMEM attention, PDL MoE, norm folding. Lessons on cold code, compiler-sunk loads and PDL __ldcg.
hariappiattention 2.657×, MoE 2.381×, bf16 1.584×A hybrid attention dispatcher (own kernel for full layers, keell0renz for sliding). Proof that mixing per-layer-type kernels pays.
trxphxle#191 overallRestoring SGLang's BF16 rounding points. Exact KV gather-read +0.63%. Pre-warming +0.16%.
lee101lower-midDiagnosed full verify as CTA-barrier bound. Ones-wgmma row sums. Heaviest-first MoE queue. Router [hi;lo] stacking.
ruwwww · tech-cosmos · shashank-100 · infraax · maximenewman · chenwainuovarioustech-cosmos: Gluon warp-specialized full-attention kernels (2.54×, rank 8 early on). infraax: an AI-agent campaign with a "predict, then kill" method. maximenewman: clear H100 per-shape tables. chenwainuo: serving-track requirements and patches.

Not public: ZhimaoL's sglang fork branches (decode #29), which were deleted Nov 1; their verify kernels survive inside keell0renz's files. Closing #1s (pjmullin3 attention 3.339×, gogowhy decode 2.577× at one point, dolanzhao MoE 2.758× on Oct 31, arashattar bf16, zzz0906 prefill) have published no code that I could find.

Serving-track prior art (added Nov 14)

SourceWhat it givesCaveat
Official rules summary (llms.txt)The serving contract: v0.5.23 base, lossless, score definition, loads, TTFT and memory gates, patch scope and the 400 KB cap, maintainer review, 1 scored run and 3 checks a day, the prize split.A summary. The full rules page and the serving API need a browser or the CLI.
chenwainuo/okcThe only public serving repo found. Active patches: small-batch fused finalization, and Hopper EP8 expert-tile caps (16 through 256 tokens, 32 through 512, stock selection above). Validation gates: baseline, profile, hypothesis, isolated change, paired measurement, re-profile, full 8-GPU check. Earlier routing candidates were parked for showing no useful gain.No serving results published. The speedup ranges in its rewrite plan are labelled priors.
SGLang low-latency mode (Oct 18)What the maintainers themselves ship for low concurrency: dependent kernels launched early with PDL, no separate read-back of the attention projection output, less routing work in MoE, draft sampling combined into one kernel per GPU. Per-user decode 12–38% faster at 1–8 requests.GB300 and a different model, delivered as an external plugin. A guide to what they consider reasonable, not a number to expect on H200.
LMSYS megakernel postAbout 1,450 launches per stock step, 12.9 → 5.9 ms, draft 1.16 → 0.44 ms, sync and fence costs.Batch 1 only, Blackwell only, 8,000 lines.
SGLang cookbookHopper specifics: Marlin W4A16 experts, static memory fraction 0.85, no MegaMoE or sequence-parallel prefill. DFlash2 with 8 draft tokens in the Blackwell recipes.No complete H200 launch line. Its high-throughput profile drops speculative decoding, which this track does not allow.
Quesma: reports from the top 50The closing spread (#1 / #10 / #50), how twelve top-50 teams worked, costs, and where profiling counters are available.Self-reported; kernel round only.

What the top 50 reported about process

Community discussion

Measured facts that matter most (alygg77, H200)

FactValueImplication
DRAM read ceiling4.60 TB/s on a 2 GiB stream, but size-dependent: 4.33 at 256 MiB, 3.63–3.81 at 66 MiB (NCU)Per-call streams of 50–600 MB can't reach 4.6. Use size-matched ceilings for floors.
Per-SM HBM pullbare TMA ring 140–207 GB/s; plain 16 B loads 111–125; consume-before-refill ring ~60–64Never let consumers gate refills.
Per-SM write to L2~40–50 GB/s (a 128 KB FP32 tile takes 2.6–4.1 µs)Why split-K partials lose at m256 and in big merges.
Integer ALU vs HGMMAALU alone 286 clk, MMA alone 256, together 489Dequant does not overlap wgmma on the same SM. Instruction count per weight is the MoE and codec lever.
Empty graph replay4.70–4.74 µs on their timerThe judge's ~4 µs subtraction leaves a bit of fixed cost in each shape.
Decode layer budget, d32s (154 µs)MoE 84 · dense GEMV 51.5 (floor 34.9) · glue 8.8 · attention 9.8MoE is over half the layer; dense has ~16 µs of headroom.
Decode layer budget, v32s (306 µs)MoE 188 · dense 83 (floor 72) · attention 14.8 · glue 18.4; idle between kernels <1.1 µsInter-kernel bubbles are already small on the verify path.
MoE loss budget (WGS build)ramp 3–6 · phase-A waves 3–6 · shadows 4–12 · end 7–11 µs. Stream phase 3.87–4.05 TB/s.Matches keell0renz: the losses are the edges, not the stream.
Phase-B unit vs end spreada unit is 18.5–19.4 µs; end spread is 2.7–8.2 µsDynamic claiming can't fix the tail. Make the last units smaller.
NVFP4 entropyE2M1 codes 4.000 bits; scales 3.71 bits (34 codes)Codes are incompressible; best lossless total saving is ~5%.
BF16 weight entropy10.46 bits; escapes 1.3% at 12 bitsEntropy coding below 12 bits: upper bound +2.4% even with free decode, realistically +0.5–1%.
Full verify consumer loop (v16f)exp 40.5% · QK + V transpose 17.5% · PV barrier 12.7% · data waits 12.5%. 1 CTA per SM is 16–18% slower than 2.exp offload is the lever for full verify. Keep 2 CTAs per SM and a 104-register budget.
Verify attention beside a GEMMv32s 14.9 µs alone vs 22–27 µs co-runningDon't overlap attention with weight streams.
Prefill first-turn attention12.3% of the first-turn load; a kernel with no exp still takes 87–91% of the timeFP8/exp tricks have a small ceiling for the first-turn load.
Prefill glue1,108 µs memory-bound glue (13.5% of load); GEMMs at 1.53 GHz, slowed 12–18% by other trafficFusing glue into GEMM epilogues beats hiding it under GEMMs, which lost.
prepare() time273–275 s typical, once 311.9 s (limit 300)Variant sweeps can time out. Budget compile time.
Judge noisethe same file spread 1.6–2.2% across runs; cross-cluster comparisons inflated a +2.4% gain to +4.19%; memory-layout pads shift v32 by 1.5–2.5 µsCompare only within one cluster, ≥2 runs per arm, and test ≥4 layout pads.

Corrections from the Nov 14 rules review

PageEarlier claimCorrection
overview, servingServing runs Nov 10–19; rules taken from public notesNov 12, 9 am PT to Nov 19, 11:59 pm PT. Since Nov 11: SGLang v0.5.23 base and lossless patches. 1 scored run and 3 checks a day.
overview, servingKernel work transfers almost one-to-one; every kernel can drop into SGLangWithdrawn. 400 KB of python/sglang source, +2 GB per GPU of build memory, nothing below stock precision, and a maintainer review that can disqualify load-specific code.
overviewPhases 0–6: 10–13 days of kernel-board work, then servingReversed. The boards are closed and unpaid. Serving comes first; the kernel pages are a parts catalogue.
servingDFlash tuning per concurrency, fused all-reduce and N-sharded experts ranked high or mediumDowngraded to check-first or out: fixed flags, load-level special-casing, patch size and review cost.
attention_decode, prefill_layer, bf16_projFP16 accumulation, emulated exp2, FP8-stored P, hi/lo BF16 routerKernel-track allowances only. Assume out in serving until RULES.md says otherwise.
bf16_proj, moe_w4a16Re-encode weights and keep per-shape buildsQualified. One layout per weight, converted in place. About 32 MB of duplicated weights per layer is all the memory cap allows.

Corrections this pass forced on the plan

PageEarlier claimCorrection
decode_layerRouter-first / pooler-concurrent reorder and o-proj shared-memory prefetch during attentionDowngraded. An early pooler/router stream at d32s measured 2.1–7.7 µs slower; L2 prefetch from sliding attention was 2.6–5.1 µs slower; 2 pre-wait weight stages are optimal and 4–6 slow attention by 1.7 µs. Only an in-launch variant is still untested.
moe_w4a16Dequant into wgmma A registers "fixes" instruction-bound large-M shapesQualified. Integer ALU and HGMMA don't overlap, so the win must come from fewer instructions per weight (KPERM: 20.12 → 18.44 per word; −12 instructions per k-block gave +5.8% on prefill experts), not from overlap.
bf16_projEntropy-coded ~11-bit exponents; stream-K for qkvDowngraded. ≤+2.4% upper bound (realistically +0.5–1%). qkv GEMV saturates at ~96 CTAs; stream-K's ceiling is 0.6 µs vs a 1.5–4 µs fix-up. 12-bit decode at m≥128 loses 2.4–12 µs.
prefill_layerFP8 attention as a top leverReduced to medium. Attention is only 12.3% of the first-turn load. Top levers become the expert kernel at T=753 (423 TFLOP/s, ~25% padding, instruction-bound) and fusing the 1.1 ms of glue.
attention_decodeEmulated exp2 only at v32 fullStrengthened. exp is 40.5% of the full-verify consumer loop at v16 too. Try it on v16f and v32f.

Couldn't read (GitHub rate limit): yesthatwilly's per-problem CLAIMS.md ledgers. These are the next sources to mine. Also unread: each repo's per-experiment files (Kushagrabainsla EXPERIMENTS.md, infraax HYPOTHESES.md, trxphxle full tables).