Module 12. Diffusion Language Models — quizzes and answer checking · Modern LLMs

Question 12.1.1

12.1. The sequential unit of work

What can reduce the sequential depth of diffusion generation?

One correct optionAA forward pass can update several positionsBEvery diffusion model has fewer parametersCAll tasks need only one forward pass

A) A forward pass can update several positions

B) Every diffusion model has fewer parameters

C) All tasks need only one forward pass

Answer and explanation

Correct answer: A. Several positions can progress together, but each denoising iteration still costs a model evaluation.

Question 12.5.1

12.5. The absorbing-mask forward process

With a linear masking schedule at t=0.25, how likely is a token to remain visible?

One correct optionA1.25B0.75C0.25

A) 1.25

B) 0.75

C) 0.25

Answer and explanation

Correct answer: B. The mask probability is t; survival has probability 1−t.

Question 12.5.2

12.5. The absorbing-mask forward process

Why is the forward-process mask called absorbing?

One correct optionAThe reverse model cannot reconstruct the tokenBThe entire prompt must be erasedCOnce masked, a position stays masked in the forward process

A) The reverse model cannot reconstruct the token

B) The entire prompt must be erased

C) Once masked, a position stays masked in the forward process

Answer and explanation

Correct answer: C. Absorption is a property of corruption. Reconstruction belongs to the reverse process.

Question 12.6.1

12.6. Worked example A: a mask schedule

Which mask counts describe four equal reveal steps over eight positions?

One correct optionA8, 6, 4, 2, 0B8, 4, 0, 0, 0C0, 2, 4, 6, 8

A) 8, 6, 4, 2, 0

B) 8, 4, 0, 0, 0

C) 0, 2, 4, 6, 8

Answer and explanation

Correct answer: A. Each step reveals two positions. The list includes the initial fully masked state.

Question 12.7.1

12.7. Training objective and likelihood

At t=0.5, four masked targets receive probabilities 0.9, 0.7, 0.6, and 0.8. What is the weighted sum loss?

One correct optionAExactly 3.0BApproximately 2.3920CApproximately 0.2990

A) Exactly 3.0

B) Approximately 2.3920

C) Approximately 0.2990

Answer and explanation

Correct answer: B. Add the four negative natural log probabilities, then multiply by 1/t=2.

Question 12.7.2

12.7. Training objective and likelihood

Keeping the time weight fixed, what happens if the four masked losses are averaged rather than summed?

One correct optionAIt stays unchangedBIt becomes four times largerCIt becomes one quarter as large

A) It stays unchanged

B) It becomes four times larger

C) It becomes one quarter as large

Answer and explanation

Correct answer: C. The reduction rule changes the numerical scale; it must be reported with the objective.

Question 12.7.3

12.7. Training objective and likelihood

Independent masking hides no tokens. What is the summed loss contribution?

One correct optionA0BNecessarily NaNCSilently force one token to be masked

A) 0

B) Necessarily NaN

C) Silently force one token to be masked

Answer and explanation

Correct answer: A. The empty sum is zero. Forcing a mask changes the training-input law.

Question 12.8.1

12.8. Any-order training and its limits

Does bidirectional masking guarantee that a model learns every reversed relation?

One correct optionANo; data coverage and entity representation still matterBYes, for every factCYes, provided the vocabulary is large enough

A) No; data coverage and entity representation still matter

B) Yes, for every fact

C) Yes, provided the vocabulary is large enough

Answer and explanation

Correct answer: A. More symmetric conditioning does not supply facts that the data never establish.

Question 12.9.1

12.9. Sampling and worked example B

What is the purpose of remasking a revealed position?

One correct optionAChange the number of model parametersBReconsider the token using the improved surrounding contextCGuarantee fewer model evaluations

A) Change the number of model parameters

B) Reconsider the token using the improved surrounding context

C) Guarantee fewer model evaluations

Answer and explanation

Correct answer: B. Remasking enables revision; the extra opportunity can also consume additional computation.

Question 12.9.2

12.9. Sampling and worked example B

A source emits only AA and BB equally. How much probability do two independent marginal draws assign to AB and BA together?

One correct optionA0B0.5C1

A) 0

B) 0.5

C) 1

Answer and explanation

Correct answer: B. Both marginals are (0.5,0.5); each invalid pair receives 0.25.

Question 12.9.3

12.9. Sampling and worked example B

Logits and greedy candidates are fixed; all positions will be revealed. What can changing the order affect?

One correct optionAIntermediate states, but not final candidatesBIt necessarily corrects the outputCIt changes trained weights

A) Intermediate states, but not final candidates

B) It necessarily corrects the output

C) It changes trained weights

Answer and explanation

Correct answer: A. Context can change the result only when predictions depend on the evolving state.

Question 12.10.1

12.10. A generic roofline calculation

A pass needs at least 4.776 ms of weight transfer and 16.549 ms of arithmetic. What is the ideal roofline time bound?

One correct optionA4.776 msBThe measured latency is exactly 21.325 msC16.549 ms

A) 4.776 ms

B) The measured latency is exactly 21.325 ms

C) 16.549 ms

Answer and explanation

Correct answer: C. The ideal overlapping model takes the larger bound. Real latency may be higher.

Question 12.11.1

12.11. Steps, batching, and weight precision

Why does a single-request diffusion speed advantage not establish a large-batch advantage?

One correct optionAAR can amortize weight reads across concurrent requests tooBBatching removes AR factorizationCA batch eliminates all KV traffic

A) AR can amortize weight reads across concurrent requests too

B) Batching removes AR factorization

C) A batch eliminates all KV traffic

Answer and explanation

Correct answer: A. Concurrency changes both weight amortization and per-request state traffic.

Question 12.12.1

12.12. Level-1 code

How should the teaching loss handle a target ID of −1?

One correct optionAReplace it with the most likely tokenBReject the invalid targetCUse the last logits column

A) Replace it with the most likely token

B) Reject the invalid target

C) Use the last logits column

Answer and explanation

Correct answer: B. NumPy permits negative indexing, but a vocabulary target must lie in the declared ID range.

Question 12.14.1

12.14. Block diffusion, length, and caching

Which state can be reused exactly in the block-diffusion setup described here?

One correct optionAAny state in the changing bidirectional canvasBAll logits, regardless of token updatesCThe completed prefix state, whose computation excludes the future block

A) Any state in the changing bidirectional canvas

B) All logits, regardless of token updates

C) The completed prefix state, whose computation excludes the future block

Answer and explanation

Correct answer: C. The current block changes. Exact prefix reuse depends on the conditioning and cache contract.

Question 12.14.2

12.14. Block diffusion, length, and caching

What happens to the progression order when the block size is one?

One correct optionAProgress becomes token-by-token, resembling AR orderBAll positions are revealed at onceCEvery training objective becomes identical

A) Progress becomes token-by-token, resembling AR order

B) All positions are revealed at once

C) Every training objective becomes identical

Answer and explanation

Correct answer: A. A shared unit of progress does not by itself equate every objective or sampler.

Question 12.14.3

12.14. Block diffusion, length, and caching

The last denoising call saw masks; final block tokens were chosen afterwards. How should the history cache be prepared?

One correct optionAKeep the old K/V unchangedBCopy logits into K/VCCompute states for the finalized clean block

A) Keep the old K/V unchanged

B) Copy logits into K/V

C) Compute states for the finalized clean block

Answer and explanation

Correct answer: C. The cache must represent the final values, not the earlier noisy input.

Question 12.15.1

12.15. Reading the DiffusionGemma passport

Why is the per-token active parameter count insufficient to predict a MoE canvas's weight traffic?

One correct optionAEvery token always activates every expertBDifferent positions may select different expert setsCMoE models have no weights

A) Every token always activates every expert

B) Different positions may select different expert sets

C) MoE models have no weights

Answer and explanation

Correct answer: B. Traffic depends on the union of selected experts as well as storage format and execution.

Question 12.17.1

12.17. Multimodal diffusion systems

What must be specified when a multimodal system is described as diffusion-based?

One correct optionAOnly the LLM layer countBOnly the text vocabulary sizeCWhich component performs diffusion, and over which representation

A) Only the LLM layer count

B) Only the text vocabulary size

C) Which component performs diffusion, and over which representation

Answer and explanation

Correct answer: C. Text-token reconstruction and image generation may use distinct diffusion processes.