arXiv:2411.09852  ·  Zeng, Liu, Hang, Liu et al.  ·  UIUC · Meta AI · UIC

InterFormer,
drawn out

A click-through-rate model built as two parallel arches that keep trading notes, instead of one pipeline that crushes your data early.

Task  predict P(click)
Inputs  static features + behavior history
Shipped  Meta Ads, 2024 pilots
Claim  +0.15% NE, +24% QPS vs prior SOTA
NON-SEQUENCE SEQUENCE 3 LAYERS OF MUTUAL EXCHANGE

01 — the job

Two kinds of evidence about one person

Click-through-rate prediction asks a narrow question: given this user and this ad, what's the probability of a click? The interesting part is that the evidence arrives in two incompatible shapes.

NON-SEQUENCE FEATURES static · unordered · a handful to several hundred user id age gender ad id ad category price device no ordering between them — the model has to learn which pairs and triples matter TELLS YOU: LONG-RUN, STABLE INTEREST BEHAVIOR SEQUENCES ordered · T = 50 to 1,000 events · several streams NOW time clicks, likes, conversions — noisy, bursty, most of it irrelevant to this particular ad TELLS YOU: WHAT THEY WANT RIGHT NOW
The heterogeneity problem. A set and a sequence want different machinery. Feature-interaction models (factorization machines, DCNv2, DHEN) are good at the left. Transformers and RNNs are good at the right. The whole design question is how to run both without one wrecking the other.

The paper's example is a nice one: your profile says you're into electronics in general, but the last twenty things you browsed were all phones. Neither signal is enough alone. The profile is stable but vague; the history is specific but noisy.

02 — the diagnosis

Two failures the authors name

InterFormer is basically a reaction to two habits in earlier sequential CTR models. Understanding them is most of understanding the architecture, because every piece exists to undo one of them.

Failure one: the information only flows one way

The standard pattern is to use the static features to steer sequence modeling — the target ad becomes a query, attention pulls out the relevant slice of history. Useful. But the reverse never happens: what the model learned from the sequence never gets folded back into the feature-interaction side. The static branch keeps computing crosses between age and category as though the last hour of browsing didn't exist.

TYPICAL SEQUENTIAL CTR MODEL static X features sequence S history QUERY the feature-interaction side never hears back. "insufficient inter-mode interaction" INTERFORMER static X features sequence S history CONTEXT BEHAVIOR both sides get revised, every layer. the paper calls this "interleaving"
Ablation backs this up. The authors ran the same model with the exchange turned off, one-way, and two-way. Performance climbs monotonically as more information crosses: no sequence arch < separate arches < one direction < both directions — up to 1.46% AUC between the ends.

Failure two: the sequence gets crushed before anything can use it

Because attending over hundreds of features and a thousand events is expensive, models usually compress the sequence early — sum it, pool it, run it through an MLP — and hand a single vector to the interaction module. Everything downstream sees that one vector. Whatever the pooling threw away is gone before the model had the context to know what mattered.

EARLY SUMMARIZATION T tokens SUM / POOL / MLP 1 vector everything after this point — every layer, the interaction module, the head — sees only this. DECISION MADE BEFORE THE MODEL KNEW ANYTHING SELECTIVE AGGREGATION T tokens still T SUMMARY ONLY 8 tokens the full sequence survives to the last layer. a small summary is branched off for exchange. COMPRESSION IS A SIDE CHANNEL, NOT THE MAIN LINE
This is the single most important structural idea in the paper. Summarization still happens — you can't cheaply cross-attend a thousand events against hundreds of features — but it happens in a separate branch whose output is consumed, not substituted. The trunk keeps its shape.

03 — getting to the starting line

Everything becomes a d-dimensional token

Before any of the clever machinery runs, both modes get flattened into the same currency: matrices of d-dimensional column vectors. This is what makes it possible to concatenate a sequence summary onto a feature matrix later without any special casing.

NON-SEQUENCE PATH m dense values n sparse IDs concat, then one linear map embedding table per feature X⁽¹⁾ ℝ ᵈ ˣ ⁽¹⁺ⁿ⁾ ONE COLUMN PER FEATURE · HEIGHT d SEQUENCE PATH clicks likes conversions k STREAMS, EACH EMBEDDED TO d MaskNet S ⊙ MLPmask(S) then MLPlce: kd → d DENOISE, THEN MERGE S⁽¹⁾ ℝ ᵈ ˣ ᵀ ONE COLUMN PER EVENT · HEIGHT d
MaskNet earns its place here. Real systems have several behavior streams from different surfaces and action types. MaskNet computes a mask from the sequence itself, multiplies it back in to suppress irrelevant events, then linearly combines the k streams down to one d-tall matrix — so downstream code only ever handles a single sequence.

04 — the block

Three arches, repeated L times

One InterFormer layer has three named parts. Two of them are the load-bearing columns, one is the span between them. They all run inside the same layer, and the layer stacks.

Interaction Arch

Feature crossing over the static side. Whatever backbone you like — dot product, DCNv2, DHEN. Takes the sequence summary as extra input columns.

out: behavior-aware X

Sequence Arch

Transformer-flavored sequence modeling. Multi-head attention with rotary position embeddings, preceded by a feed-forward whose weights come from the static side.

out: context-aware S

Cross Arch

The span. Compresses each side into a handful of tokens and hands them across. Kept separate precisely so the two columns never have to shrink.

out: X_sum, S_sum

Interaction Arch — feature crossing, told what just happened

The trick is almost anticlimactic: take the sequence summary and glue it on as extra columns of the feature matrix, then run your normal interaction module over the combined thing.

X⁽ˡ⁺¹⁾ = MLP⁽ˡ⁾( Interaction⁽ˡ⁾( [ X⁽ˡ⁾S_sum⁽ˡ⁾ ] ) )eq. 7

Because the sequence summary is now just more columns, the interaction module produces three families of crosses for free: static×static (the usual explicit interests), static×sequence (does this ad match what they're doing right now), and sequence×sequence (which of the summary tokens are actually active — low-scoring ones fade out). The trailing MLP restores the original column count so the layer can stack.

Backbone-agnostic by design. The experiments swap in dot product, DCNv2, and DHEN and the interleaving gains hold across all three. In the reported runs, DHEN is the one used.

Sequence Arch — attention, told who this person is

Two moves. First a Personalized FFN, and this is the part worth slowing down on, because it isn't the usual "concatenate the context and hope."

X_sum d × n_sum WHO THIS USER IS × W W_PFFN d × d A WEIGHT MATRIX, BUILT PER SAMPLE · S (d × T) still d × T Every event in the history is rotated by a transform built from this user. HENCE "PERSONALIZED" Algebraically this is a dot-product interaction between every sequence token and the static summary — but expressed as a linear layer, so it composes cleanly with the attention that follows. CONTRAST WITH A VANILLA TRANSFORMER FFN, WHOSE WEIGHTS ARE THE SAME FOR EVERY USER
PFFN in one line: PFFN(X_sum, S) = f(X_sum) · S, where f is an MLP producing a d×d matrix. The static context doesn't get appended to the sequence — it becomes the weights that process the sequence.

Then ordinary multi-head attention runs over the result, with rotary position embeddings so ordering survives. And at the very first layer only, the non-sequence summary is prepended to the sequence as CLS tokens — four of them, in the reported configuration. Those CLS slots start out as "who this user is," and attention lets them absorb the sequence around them. On the way out they're the natural place to read a context-aware sequence summary from.

S⁽ˡ⁺¹⁾ = MHA⁽ˡ⁾( PFFN( X_sum⁽ˡ⁾, S⁽ˡ⁾ ) )eq. 9

Cross Arch — the only place anything shrinks

Both columns keep their full width, which is exactly why they can't talk to each other directly: too noisy, too big. The Cross Arch is the narrow channel between them, and it's deliberately built as a separate structure so that compression is a read, never a write.

SUMMARIZING THE STATIC SIDE X⁽ˡ⁾ — n columns (hundreds, in production) LINEAR COMPRESSED EMB. mixes across columns: n → n_sum SELF-GATING σ( X ⊙ MLP(X) ) — a soft mask the input masks itself X_sum — n_sum ≪ n SUMMARIZING THE SEQUENCE SIDE — THREE READS, NOT ONE CLS ×4 HISTORY, OLDEST → NEWEST S⁽ˡ⁾ after attention 1 · CLS TOKENS 4 slots seeded with static context, then filled by attention FILTERED BY WHO THEY ARE 2 · PMA TOKENS 2 learned queries that pool the sequence on their own terms INSURANCE IF CLS GOES BAD 3 · RECENT TOKENS the last 2 events, passed through raw, no pooling at all NO POOLING, NO LOSS CONCATENATE, THEN SELF-GATE the same σ( · ⊙ MLP(·) ) mask, deciding which of the 8 summary tokens survive S_sum — 8 columns wide
Why three reads instead of one. The CLS tokens are only as good as the static context that seeded them, so PMA tokens with independently learned queries provide a fallback view, and the most recent events bypass pooling entirely. Ablations put PMA as the most valuable of the three — removing it costs up to 0.004 AUC, the largest single drop measured.

05 — one layer, step by step

Watch a single block execute

This is the paper's Algorithm 1, one line at a time. The order matters: summaries are computed first, from last layer's outputs, and then both arches consume them in parallel.

X⁽ˡ⁾ non-sequence d × n · full width, uncompressed S⁽ˡ⁾ sequence d × (4+T) · every event still here CROSS ARCH X_sum⁽ˡ⁾ LCE + gate S_sum⁽ˡ⁾ cls+pma+recent STATIC CONTEXT → ← BEHAVIOR SUMMARY INTERACTION ARCH concat [ X ‖ S_sum ] DHEN / DCNv2 / dot MLP back to n columns SEQUENCE ARCH PFFN: weights from X_sum multi-head attention rotary positions these two run at the same time — that is a systems decision, see §08 X⁽ˡ⁺¹⁾ d × n — same shape as it came in S⁽ˡ⁺¹⁾ d × (4+T) — same shape as it came in BOTH FEED STRAIGHT INTO LAYER l+1 after layer L: ŷ = MLP([X_sum ‖ S_sum])

Step 1 of 7

Use ← and → once the walkthrough has focus.

06 — the invariant

Follow the shapes and the design explains itself

Everything above can be checked against one table. Both trunks preserve their dimensions end to end; only the side channels are small. That property is what lets you stack layers at all, and it's the mechanical form of "no aggressive aggregation."

TensorShapeWhere it livesFate
X⁽ˡ⁾d × nInteraction Arch trunkpreserved through all L layers
S⁽ˡ⁾d × (4+T)Sequence Arch trunkpreserved through all L layers
X_sum⁽ˡ⁾d × n_sum, n_sum ≪ nCross Arch, recomputed each layerconsumed by PFFN
S_sum⁽ˡ⁾d × 8  (4 cls + 2 pma + 2 recent)Cross Arch, recomputed each layerconcatenated onto X
ŷscalarhead, after layer Lthe click probability
for contrast — a typical earlier model:
Sd × T  →  d × 1collapsed at layer 0the rest of the network never sees T again

The paper tests this directly: replace selective aggregation with average pooling, then an MLP, then multi-head attention, each feeding a DHEN interaction module. Performance improves as the compression gets gentler, and InterFormer's arrangement — where the trunk isn't compressed at all — comes out ahead of all three.

07 — depth

Each layer looks at a different time scale

Since the sequence keeps its full length, every layer's attention gets a fresh crack at it, informed by a static summary that has itself been revised. The published attention maps show the layers specializing.

SCHEMATIC RECREATION OF THE PATTERN DESCRIBED IN THE PAPER — NOT THE PUBLISHED FIGURE'S ACTUAL VALUES
Reading it. Layer 1 attends broadly and near-uniformly, which acts like wide pooling over long-run interest. Layer 2 fixates on particular columns — the most recent events, plus one specific mid-history item. Layer 3 forms tight blocks along the diagonal, pooling within small neighborhoods of adjacent events, which is short-term interest. The clustering is also doing noise reduction: a burst of related events gets averaged together, a stray click doesn't.

On the internal dataset this shows up as clean depth scaling. Going from one layer to two is the big jump — around 0.13% normalized entropy — with a third layer adding roughly 0.05% and a fourth about 0.04% on top.

08 — model–system co-design

The two arches have opposite bottlenecks, which turns out to be lucky

This is the part of the paper that's least about machine learning and most about why the architecture is shaped this way. Running the arches side by side isn't just a modeling choice — it's what makes the hardware behave.

Interaction Arch (DHEN)

Lots of parameters, comparatively little arithmetic. Under sharded data-parallel training, it spends its time waiting on network — gathering and scattering weights between GPUs.

communication-bound

Sequence Arch (Transformer)

Few parameters relative to the FLOPs it burns. It spends its time in the matrix units, with almost nothing to synchronize.

compute-bound

Run them one after the other and you pay for both serially. Run them concurrently — which the architecture permits, because neither arch consumes the other's output within a layer, only the previous layer's summaries — and the Interaction Arch's network stalls hide underneath the Sequence Arch's math.

SEQUENTIAL — ARCHES RUN ONE AFTER THE OTHER DHEN compute exposed communication sequence compute wall clock PARALLEL — COMMUNICATION HIDDEN UNDER COMPUTE DHEN compute communication sequence compute, overlapping saved +20% QPS FROM THIS ONE CHANGE
Plus kernel-level work. Separately, the team moved FLOPs out of small low-return modules into large high-return ones and fused small kernels together. Model FLOPs utilization went from 11% to 16% on the interaction modules and 38% to 45% on DHEN — about 19% better for the InterFormer layer overall. Overlap and kernel work together account for more than 30% higher training throughput.

09 — does it work

Numbers

CTR metrics move in small absolute increments and the field cares about the third decimal place, because at Meta's volume a hundredth of a percent is real money. Read these as basis points, not as percentages.

Public benchmarks — AUC, higher is better

ModelAmazonTaobaoAdsKuaiVideo
non-sequential
DCNv20.88070.64720.7426
DHEN0.87900.65090.7424
Wukong0.87650.64780.7423
sequential
DIN0.88480.65070.7437
DIEN0.88560.65190.7451
TransAct0.88510.64980.7448
InterFormer0.88650.65280.7453

Sequential methods beat non-sequential ones everywhere, which is the paper's first point: the two modes want different machinery. InterFormer then leads the sequential group, by up to 0.14% AUC and 0.9% grouped AUC over the strongest competitor.

Production — Meta Ads, 70 billion training samples

0.15%
NE gain over the internal SOTA model at comparable FLOPs
+24%
queries per second, serving
+30%
training throughput from the co-design work
0.6%
topline metric improvement, 2024 pilot launches

The scaling behavior is arguably the more interesting result. Adding two long sequences of 1,000 events to the existing six improved NE by 0.14%, and InterFormer's loss curve keeps descending with more training data where the internal cross-attention baseline flattens — a 0.06% NE gap by the end. There's also a pure efficiency lever: merging six sequences into one of length 600 bought 20% QPS and 17% MFU for a 0.02% NE cost.

10 — carry this away

The whole thing in five sentences

Static features and behavior history are kept in two separate trunks that never shrink. Once per layer, a dedicated Cross Arch compresses each trunk into a handful of tokens and hands them to the other side. The static summary becomes the weight matrix of the sequence model's feed-forward layer; the behavior summary becomes extra columns for the feature-interaction module. Stack that block three or four times, read a prediction off the final summaries, and you get better accuracy than one-way designs. Run the two trunks concurrently and you also get the throughput, because one is network-bound and the other is compute-bound.

Interaction Arch
Feature-crossing column. Pluggable backbone; DHEN in the reported runs.
Sequence Arch
PFFN then multi-head attention with rotary positions.
Cross Arch
The bridge. Where — and only where — anything is compressed.
PFFN
Feed-forward whose d×d weights are generated from the static summary.
LCE
Linear map across the feature axis, n → n_sum. Fewer columns, same height.
Self-gating
σ(X ⊙ MLP(X)). A soft mask a tensor computes on itself.
CLS tokens
4 slots prepended to the sequence at layer 1, seeded with static context.
PMA
Pooling by multi-head attention: learned queries summarize a set.
MaskNet
Denoises and merges the k raw behavior streams into one.
NE
Normalized entropy — log loss divided by the base-rate entropy. Lower is better.
MFU
Model FLOPs utilization — what fraction of peak the GPUs actually reach.
gAUC
Per-user AUC, weighted by click count. Rewards ranking within a user.