Practice and exams
Everything in the six primer pages, turned into questions. Nothing here needs a GPU: a pencil or a calculator is enough. Your progress is kept in this browser only.
Topic quizzes
Pick a page of the primer. Each answer gives its reason at once, and the order of the answers is shuffled every time. A question counts as done when your latest answer to it was right.
Hands-on labs
Each lab is one small investigation worked in steps, where later steps use your earlier answers. Type a number and press Enter. Stuck? "Show working" gives the full sum. Counts must be exact; other answers are accepted within about 1%.
Number drills
The same kinds of sum with fresh numbers every time, so they never run out. Only your first try at each question counts toward the streak.
Timed exams
An exam draws a fresh set of questions each time: about three quarters multiple choice and one quarter calculation. Nothing is marked until you submit or the clock runs out. The pass mark is 80%. The formula sheet stays open to you, as it would in an open-book exam.
Formula sheet
Every number and formula the questions rely on. Sizes use 1 MB = 1,000,000 bytes.
| Formula | In words |
|---|---|
| Size | bytes = rows × columns × bits ÷ 8 |
| Floor | shortest time = bytes ÷ road speed |
| Step | step time = 46 × sliding layer + 16 × full layer |
| Speed | tokens per second = tokens out per step ÷ step time |
| Amdahl | new time = (1 − p) + p ÷ s. Speedup = 1 ÷ new time. Ceiling = 1 ÷ (1 − p). |
| Arithmetic | operations = 2 × rows × columns × tokens. Time = operations ÷ calculator speed. |
| Geomean | multiply the n ratios, then take the n-th root |
| Rounds of tiles | whole rounds = tiles ÷ workers, rounded up. Even shares = tiles ÷ workers. |
| Naming things | bits needed for N values = the smallest b with 2 to the power b at least N |
| 4-bit weights | scale = largest weight in the block ÷ 6. Stored value = nearest allowed size × scale. |
| Users carried | (stock's step time − your fixed part) ÷ your per-user part. Ratio = users carried ÷ stock's users. |
| Fact | Value |
|---|---|
| Layers in one pass | 62: 46 sliding (last 2,048 tokens) and 16 full (the whole conversation) |
| Machine | 8 GPUs. Each has 132 workers (SMs), 141 GB of main memory, an L2 cache of about 50 MB and about 228 KB of shared memory per worker. A warp is 32 threads. |
| Road speed | about 4.3 MB per µs in practice, 4.8 rated |
| Calculators | tensor cores peak at about 989 million operations per µs |
| Token and experts | a token's state is 8,192 numbers. 256 experts per layer, 8 chosen per token, 32 held on each GPU. |
| Notes on earlier tokens | 256 bytes per token, per layer, per GPU |
| Draft and check | 8 tokens per user per step, so 1, 4, 16 and 32 users mean 8, 32, 128 and 256 tokens |
| Step time, one user | stock 16.6 ms, best kernels about 6.4 ms, floor about 3.9 ms. Stock at 16 users 35.2 ms, at 32 users 51.1 ms. |
| Tables in a layer | qkv_gate 2,304 × 8,192 (37.7 MB). o 8,192 × 1,024 (16.8 MB). pooler 4,096 × 8,192 (67.1 MB). router 256 × 8,192 at 32 bits (8.4 MB). All but the router are 16-bit. |
| Number formats | sign + exponent + mantissa bits: FP32 1+8+23, BF16 1+8+7, FP8 1+4+3, FP4 1+2+1. FP4 sizes: 0, 0.5, 1, 1.5, 2, 3, 4, 6. |
| Page 6's simple model | stock step = 15.5 ms + 1.11 ms per user, with 5 tokens out per step (an assumed figure) |
| Serving rules | loads of 1, 4, 16 and 32 users. First-token wait at most 2× stock. At most 2 GB more memory per GPU. Change under 400 KB. |