Module 11. Speculative Decoding and Quantization — quizzes and answer checking · Modern LLMs

Question 11.4.1

11.4. The exact propose–verify–correct cycle

How is a proposed token accepted in exact speculative decoding?

One correct optionAWith probability min(1, p/q) for that tokenBWhenever its target probability exceeds a fixed 0.5 thresholdCWith the normalized product of p and q

A) With probability min(1, p/q) for that token

B) Whenever its target probability exceeds a fixed 0.5 threshold

C) With the normalized product of p and q

Answer and explanation

Correct answer: A. A proposal is always accepted when the target assigns at least as much probability; otherwise it is accepted with probability p/q. Residual correction completes the target distribution.

Question 11.4.2

11.4. The exact propose–verify–correct cycle

Temperature and top-p are enabled. Which probabilities belong in the exact rule?

One correct optionARaw softmax before processingBOnly the highest-probability tokenCActual normalized sampling laws after processing

A) Raw softmax before processing

B) Only the highest-probability token

C) Actual normalized sampling laws after processing

Answer and explanation

Correct answer: C. q is the proposal law and p the intended target law at the same history.

Question 11.5.1

11.5. Worked example A: seeing the balance component by component

What defines the correction distribution after the first rejection?

One correct optionAThe target distribution with the rejected token removedBThe normalized positive part of p − q over the vocabularyCThe draft distribution multiplied by p/q

A) The target distribution with the rejected token removed

B) The normalized positive part of p − q over the vocabulary

C) The draft distribution multiplied by p/q

Answer and explanation

Correct answer: B. Correction returns probability mass where the target exceeds the proposer, hence norm((p−q)+).

Question 11.6.1

11.6. Expected progress per cycle

Why does the constant-acceptance model give E = 1 + α + … + α^γ?

One correct optionABecause all γ proposals are accepted independentlyBBecause the bonus token appears only after full acceptanceCReaching position i requires all previous proposals to be accepted

A) Because all γ proposals are accepted independently

B) Because the bonus token appears only after full acceptance

C) Reaching position i requires all previous proposals to be accepted

Answer and explanation

Correct answer: C. Successive reach probabilities are 1, α, α², and so on under constant conditional acceptance. An arbitrary average of observed rates is insufficient.

Question 11.7.1

11.7. A cost model, an optimum, and an illustrative MTP point

In the scenario α = 0.8 and c = 0.1, which γ≤10 is best?

One correct optionAγ=6 with S≈2.47, and the optimum is broadBγ=10 because a longer proposal is always betterCγ=1 because every extra proposal always costs more than it returns

A) γ=6 with S≈2.47, and the optimum is broad

B) γ=10 because a longer proposal is always better

C) γ=1 because every extra proposal always costs more than it returns

Answer and explanation

Correct answer: A. γ=5 gives about 2.46 and γ=9 about 2.35. Proposal length matters, but acceptance and proposal cost often matter more than exact tuning.

Question 11.7.2

11.7. A cost model, an optimum, and an illustrative MTP point

What does the 1.79× result at γ=1, α=0.875, and c=0.05 mean?

One correct optionAA universally measured speedup for every MTP modelBThe output of the module’s scenario model for those parametersCThe exact ceiling of all multi-step MTP architectures

A) A universally measured speedup for every MTP model

B) The output of the module’s scenario model for those parameters

C) The exact ceiling of all multi-step MTP architectures

Answer and explanation

Correct answer: B. E=1.875 and S=1.875/1.05=1.78571. It calibrates a scenario rather than reproducing a particular external benchmark.

Question 11.7.3

11.7. A cost model, an optimum, and an illustrative MTP point

For γ=2, conditional acceptance rates are 0.9 and 0.5. What is expected output without EOS?

One correct optionA2.19B2.35C1.40

A) 2.19

B) 2.35

C) 1.40

Answer and explanation

Correct answer: B. E=1+0.9+0.9·0.5=2.35. Averaging rates loses the reach probabilities.

Question 11.8.1

11.8. A taxonomy of proposers

What distinguishes tree-based proposal from one linear draft continuation?

One correct optionAIt always preserves the target distribution without correctionBIt does not require target verificationCIt proposes multiple branches that can be verified together

A) It always preserves the target distribution without correction

B) It does not require target verification

C) It proposes multiple branches that can be verified together

Answer and explanation

Correct answer: C. A tree can raise the chance of a useful surviving path, while adding mask, selection, and verification complexity.

Question 11.9.1

11.9. Native multi-token prediction at inference time

What operational advantage does native MTP provide?

One correct optionAA compatible model needs no separate full draft checkpointBThe MTP distribution automatically matches the target distribution on every workloadCProposal requires no compute or extra memory

A) A compatible model needs no separate full draft checkpoint

B) The MTP distribution automatically matches the target distribution on every workload

C) Proposal requires no compute or extra memory

Answer and explanation

Correct answer: A. MTP reuses multi-token prediction modules. It need not serve a separate full draft model; some families package modules in a compatible auxiliary checkpoint.

Question 11.10.1

11.10. Verification as a short, wider decode operation

Why can verifying several proposals be more efficient than ordinary decode?

One correct optionAIt eliminates target-weight readsBOne pass processes several positions and raises arithmetic intensityCIt always uses the same kernel and has the same cost as the original prefill pass

A) It eliminates target-weight reads

B) One pass processes several positions and raises arithmetic intensity

C) It always uses the same kernel and has the same cost as the original prefill pass

Answer and explanation

Correct answer: B. Several positions amortize target-weight movement more effectively. KV traffic, batching, and implementation still determine the realized gain.

Question 11.11.1

11.11. What low-bit quantization actually changes

How does post-training quantization differ from quantization-aware training?

One correct optionAPTQ is for dense models and QAT only for MoEBPTQ always quantizes weights only, while QAT must quantize weights and activations togetherCPTQ transforms a trained checkpoint; QAT includes quantization effects during training

A) PTQ is for dense models and QAT only for MoE

B) PTQ always quantizes weights only, while QAT must quantize weights and activations together

C) PTQ transforms a trained checkpoint; QAT includes quantization effects during training

Answer and explanation

Correct answer: C. When quantization is introduced and which tensors are quantized are separate axes. PTQ is applied after training; QAT incorporates the effect during optimization.

Question 11.12.1

11.12. Microscaled FP4, W4A4, and the role of hardware

Why does the module count 4.25 bits for MXFP4 and 4.50 for NVFP4?

One correct optionAOne eight-bit local scale serves 32 versus 16 four-bit valuesBEvery NVFP4 value stores an extra half exponent bit inside its own four-bit codeCMXFP4 stores no scale metadata

A) One eight-bit local scale serves 32 versus 16 four-bit values

B) Every NVFP4 value stores an extra half exponent bit inside its own four-bit code

C) MXFP4 stores no scale metadata

Answer and explanation

Correct answer: A. (32·4+8)/32=4.25 and (16·4+8)/16=4.50. This counts local-scale metadata, not every byte in a real checkpoint.

Question 11.12.2

11.12. Microscaled FP4, W4A4, and the role of hardware

What does the notebook’s 3.32 MSE ratio establish?

One correct optionAEvery FP4 format is always 3.32 times more accurate than every INT4 formatBOn one outlier-containing teaching vector, the selected FP4 grid had lower MSECNVFP4 universally preserves downstream metrics better than INT4

A) Every FP4 format is always 3.32 times more accurate than every INT4 format

B) On one outlier-containing teaching vector, the selected FP4 grid had lower MSE

C) NVFP4 universally preserves downstream metrics better than INT4

Answer and explanation

Correct answer: B. It is a local numerical illustration of range allocation, not a general format or model benchmark.

Question 11.13.1

11.13. Code: make the probability contract explicit

How should the teaching grouped quantizer handle an all-zero block?

One correct optionADrop it and reconstruct its position from an indexBReuse the previous group’s scaleCKeep it zero with a safe nonzero service scale

A) Drop it and reconstruct its position from an index

B) Reuse the previous group’s scale

C) Keep it zero with a safe nonzero service scale

Answer and explanation

Correct answer: C. Dividing by a zero maximum would create NaNs. A service scale of one preserves zeros safely.

Question 11.13.2

11.13. Code: make the probability contract explicit

p and q nearly coincide, but their positive difference is nonzero. How should the residual be handled?

One correct optionANormalize the positive difference by its sumBDiscard any sufficiently small differenceCResample from p after rejection

A) Normalize the positive difference by its sum

B) Discard any sufficiently small difference

C) Resample from p after rejection

Answer and explanation

Correct answer: A. Small mass makes correction rare; it does not eliminate the correction distribution.

Question 11.15.1

11.15. Combining the levers without multiplying away the bottlenecks

Why are quantization and speculative-decoding multipliers not independent?

One correct optionAThe first optimization changes the bottleneck seen by the secondBQuantization always breaks exact acceptance and therefore requires approximate verificationCSpeculation requires bf16 weights

A) The first optimization changes the bottleneck seen by the second

B) Quantization always breaks exact acceptance and therefore requires approximate verification

C) Speculation requires bf16 weights

Answer and explanation

Correct answer: A. Reducing weight traffic can expose compute, KV, or scheduling limits. The combined configuration must be profiled again.

Question 11.15.2

11.15. Combining the levers without multiplying away the bottlenecks

Which distribution is preserved when the deployed target is quantized?

One correct optionAThe original bf16 checkpoint regardless of logit changesBThe distribution of the actually executed quantized targetCThe average of the quantized and original distributions

A) The original bf16 checkpoint regardless of logit changes

B) The distribution of the actually executed quantized target

C) The average of the quantized and original distributions

Answer and explanation

Correct answer: B. The acceptance rule reconstructs the p produced by the verifier. A quantized verifier therefore defines the preserved distribution.

Question 11.15.3

11.15. Combining the levers without multiplying away the bottlenecks

The target step becomes four times faster while draft time is unchanged. What happens to c?

One correct optionAIt falls by a factor of fourBIt rises by a factor of fourCIt stays constant

A) It falls by a factor of four

B) It rises by a factor of four

C) It stays constant

Answer and explanation

Correct answer: B. c is draft time divided by target-step time. Reducing the denominator increases the ratio.

Question 11.17.1

11.17. Reading a quant/spec run passport

What must a reproducible quant/spec passport contain?

One correct optionAOnly the format name and best advertised multiplierBOnly checkpoint size, GPU version, memory capacity, and average tokens per secondCFormat and scales, proposer, verification settings, workload, metrics, and full stack

A) Only the format name and best advertised multiplier

B) Only checkpoint size, GPU version, memory capacity, and average tokens per second

C) Format and scales, proposer, verification settings, workload, metrics, and full stack

Answer and explanation

Correct answer: C. Without configuration, workload, hardware, and measurements, a speedup cannot be reproduced or compared fairly.