Attention mechanisms  /  02 — GLM's sparse attention
← Compare: Kimi Delta Attention
GLM-5 · Zhipu AI · MLA + DeepSeek Sparse Attention

Keep every token. Read 2,048 of them.

Kimi's answer to the quadratic wall is to compress the past into a fixed-size state. GLM's answer is the opposite: keep the full cache, exact and lossless, and put a cheap little model in front of it whose only job is deciding which few thousand tokens are worth attending to. Nothing is blurred. Things are skipped.

744Btotal params, ~40B active
MLAlatent KV compression
DSAindexer + top-k selector
1Mcontext, GLM-5.2
the premise

Attention is already sparse. It just doesn't know it yet.

Look at a trained model's attention weights over a long document and almost all of the mass sits on a handful of places: the first few tokens, the recent window, and a scattering of genuinely relevant positions far back. The other 99% of the softmax is computed, normalised, multiplied by values, and contributes nearly nothing.

So the sparse-attention bet is: predict where the mass will land, cheaply, and only compute there. Keep exact softmax over the survivors — no approximation of the attention operation itself, just a smaller argument list.

A · the triangle you don't compute cells computed
computed, negligible weightselected by the indexer

The same 56-token causal matrix, twice. Dense computes the whole lower triangle. Sparse computes eight cells per row — and the pattern it picks is not arbitrary: attention sinks at the start, a local window, and a couple of far-off anchors.

layer one of two

First, make the cache small.

Before sparsity, GLM inherits DeepSeek's Multi-head Latent Attention. Rather than caching a key and value per head, cache a single low-rank latent vector per token and reconstruct the heads on the fly:

multi-head latent attention
\[ c_t=W^{DKV}h_t\in\mathbb{R}^{d_c},\qquad k^{C}_t=W^{UK}c_t,\qquad v_t=W^{UV}c_t \]

Only ct is stored, plus one small decoupled key that carries the rotary position information — the up-projections fold into the query side at inference, so the heads are never materialised in cache. In DeepSeek-V3's configuration that is 576 numbers per token per layer instead of 32,768.

B · KV cache per token, per layerillustrative · DeepSeek-V3 shape, 128 heads × 128 dim

MLA doesn't reduce attention compute at all — the triangle is still there. It reduces what you have to hold in memory, which is what makes a million-token cache survivable in the first place.

layer two of two

Then, decide what to read.

DeepSeek Sparse Attention runs in two stages, and the split is the whole trick: something very cheap scores everything, then something expensive runs on almost nothing.

  1. A lightning indexer scores every past token. A handful of tiny heads, run in low precision, produce a relevance score for each candidate position. It is quadratic in sequence length like normal attention, but with a head dimension an order of magnitude smaller and no softmax, so the constant in front is tiny.
  2. A top-k selector keeps the winners. The k highest-scoring positions per query — 2,048 in DeepSeek's published configuration — become the entire context for the real attention layer. Everything else is skipped, not approximated.
stage 1 · the indexer score
\[ I_{t,s}=\sum_{j=1}^{H_I}w_{t,j}\cdot\mathrm{ReLU}\!\left(q^{I}_{t,j}\cdot k^{I}_{s}\right) \]
stage 2 · selection
\[ \mathcal{S}_t=\operatorname*{top\text{-}k}_{\,s\le t}\;I_{t,s},\qquad |\mathcal{S}_t|=k \]
and then ordinary attention, over a shorter list
\[ o_t=\sum_{s\in\mathcal{S}_t}\frac{\exp\!\left(q_t^{\top}k_s/\sqrt{d}\right)}{\sum_{r\in\mathcal{S}_t}\exp\!\left(q_t^{\top}k_r/\sqrt{d}\right)}\,v_s \]

Note what didn't change. The softmax is exact. The values are the real values. If the indexer picks the right 2,048 tokens, the output is nearly identical to full attention — and if it picks wrong, the model has simply not seen something, which is a different and more legible failure than a blurred memory.

the part that's easy to miss

The indexer has to be taught what full attention would have looked at. In DeepSeek's recipe it is trained to match the dense model's attention distribution before the model is fine-tuned to run sparse. The selector isn't a heuristic bolted on at inference — it is a learned component, and its quality is the ceiling on the whole scheme.

the signature

Watch one query pick its context.

Forty-four tokens of history, one query at the right. The indexer scores everything, the threshold drops to the k-th best, and only the survivors get a beam. Move k and watch the needle at position 9 fall out of the selection.

C · the selection strip k = 8 of 44 · attention work
indexer score (cheap)selected · exact softmaxskipped entirely

Position 9 is the needle — semantically relevant, far away, and invisible to any fixed sliding window. It survives only because the indexer scored it, which is exactly the case a local-window scheme gets wrong.

the arithmetic

Where the saving actually comes from.

attention cost, per token
\[ \underbrace{O(L\,d)}_{\text{dense}}\quad\longrightarrow\quad\underbrace{O(k\,d)}_{\text{main attention}}\;+\;\underbrace{O(L\,d_I)}_{\text{indexer},\;d_I\ll d} \]

The main attention term stops growing with context — past k tokens, adding history costs nothing. But the indexer term is still linear per token, which means quadratic over a sequence. It is smaller by the ratio of head dimensions, not by an order of complexity. At a million tokens the indexer is no longer a rounding error; it becomes the dominant term, which is precisely why the newer work is about making the indexer itself cheaper.

D · cost per generated tokenat L = 1M · cheaper than dense
densesparse totalindexer component

Log–log, arbitrary units, indexer head dimension taken as ~1.8% of the main head dimension. The flat cyan stretch is the win; the upward bend at the right is the indexer reasserting itself.

glm's own addition

IndexShare: stop recomputing the same decision.

Naively, every sparse layer builds its own index — its own scores, its own top-k, its own memory round-trips over the whole cache. GLM-5.2's reported change is to amortise that: compute the selection once and reuse it across several layers rather than rebuilding it at each one.

E · indexer work, per layer stackreported: ~50% fewer indexer computations · ~1.2× end-to-end

Violet blocks are indexer passes over the cache; cyan is the attention itself. Dashed blocks reuse the selection from above rather than recomputing it.

Zhipu has described this both as sharing across layers and as sharing across heads within a layer; the published GLM-5.2 material emphasises reuse of the indexer, and the reported result is roughly half the indexer computations removed for around a 1.2× end-to-end speedup with comparable long-context and reasoning scores. Alongside it sits multi-token prediction, which raises speculative-decoding acceptance and buys throughput on the generation side rather than the attention side.

the fork in the road

Two answers to one problem.

 GLM · sparseKimi · linear (KDA)
the past iskept in full, compressed by MLAfolded into a fixed-size matrix
memory / tokenstill grows — O(L)constant
attention computeO(k) + indexerO(1)
failure modeindexer misses a tokenstate blurs old detail together
recall of a hitexactapproximate
hard engineeringtop-k kernels, training the indexerchunkwise kernels for a diagonal-plus-low-rank recurrence

The GLM-5 report is worth reading on this directly: the team continual-trained a 9B baseline into several efficient-attention variants, including a linearised one, and published the comparison. Naively interleaved sliding-window attention collapsed on long-context benchmarks; searching for which layers keep full attention recovered most of the gap. They shipped sparse. MiniMax ran a similar bake-off, found hybrids looked fine on standard benchmarks but degraded on multi-hop reasoning at scale, and shipped full attention for M2 before moving to sparse for M3.

Kimi K3 is the case for the other branch, at a scale nobody had tried. Neither side has produced a result that settles it — which is why both stacks are still being built.

the other branch ← Kimi Delta Attention, explained A fixed-size memory, the delta rule as gradient descent, and eight forget rates instead of one.