Module 12. Diffusion Language Models

Solution. AR emits one token per pass, so its ceiling is $1/0.004776119=209.375$ tok/s. A diffusion model reveals a 1,024-token canvas in $S$ full passes, giving average rate $1024/(S\cdot0.016549495)$. It is faster when

This gives $S<295.52$, so the largest integer is 295; at 296 calls it has already fallen below the AR boundary in this arithmetic model.

Solution. Eight billion dense weights occupy $8\text{B}\cdot2=16$ GB in bf16 and $8\text{B}\cdot0.5=4$ GB at ideal four-bit packing. At 3.35 TB/s, one weight read takes $16\,\mathrm{GB}/(3350\,\mathrm{GB/s})=0.004776\,\mathrm{s}=4.776$ ms or $4\,\mathrm{GB}/(3350\,\mathrm{GB/s})=0.001194\,\mathrm{s}=1.194$ ms. A 256-position pass has dense arithmetic floor

$\frac{2\cdot8\text{B}\cdot256}{990\text{ TFLOP/s}}=4.137\text{ ms}.$

The bf16 case is therefore memory-limited by the larger 4.776-ms floor, while ideal four-bit weights expose the 4.137-ms compute floor. These are lower bounds, not their sum under perfect overlap.

DiffusionGemma is instead a 25.2B-total/3.8B-active sparse MoE. Different canvas positions can select different experts, so one pass may read the union of many experts; tensor precisions are heterogeneous and runtime placement matters. Its H100 result uses FP8, while NVFP4 refers to Blackwell. Dense-8B arithmetic therefore cannot be relabeled as a DiffusionGemma prediction.

Solution. There is no defensible winner without a shared model-quality target, prompt/output distribution, batch size, precision, sampler, and end-to-end latency boundary. Gemini makes the boundary issue explicit: for $N$ output tokens, a scenario that applies its reported averages to that length gives approximately $0.84+N/1479$ seconds, so 1,479 tok/s is not the end-to-end rate of a short response. A matched comparison should report time to first token, complete-request latency, and steady-state throughput separately.

Solution. With rows and columns ordered P1 P2 P3 B1 B2 B3 B4, the allowed mask is

P1  1 0 0 | 0 0 0 0
P2  1 1 0 | 0 0 0 0
P3  1 1 1 | 0 0 0 0
    -------+--------
B1  1 1 1 | 1 1 1 1
B2  1 1 1 | 1 1 1 1
B3  1 1 1 | 1 1 1 1
B4  1 1 1 | 1 1 1 1

Prefix queries are causal and independent of the active block, so their K/V can be reused across denoising steps. Every active-block query reads the complete prefix and block; when masks change, those four states must be recomputed.

Demo for this project exercise: English project-demo notebook for Module 12, section “Demo 1”. It shows a compact reproducible example on small inputs, not a completed solution to the full-scale project.

Demo for this project exercise: English project-demo notebook for Module 12, section “Demo 2”. It shows a compact reproducible example on small inputs, not a completed solution to the full-scale project.