Module 15. Reasoning Models and Test-Time Compute — quizzes and answer checking · Modern LLMs

Question 15.1.1

15.1. A second scaling axis

How does sequential depth differ from parallel breadth in test-time compute?

One correct optionADepth increases model size while breadth extends the context windowBDepth develops one trajectory; breadth creates several candidatesCDepth applies only to mathematics and breadth only to code

A) Depth increases model size while breadth extends the context window

B) Depth develops one trajectory; breadth creates several candidates

C) Depth applies only to mathematics and breadth only to code

Answer and explanation

Correct answer: B. Sequential compute continues one search path. Parallel compute widens the set of independent or partly independent solutions and therefore needs a selection rule.

Question 15.5.1

15.5. What DeepSeek-R1 teaches us about training reasoning

What did DeepSeek-R1-Zero primarily demonstrate?

One correct optionALong-form reasoning emerges only after SFT on human-written reasoning chainsBDistillation always produces a stronger model than reinforcement learningCRL on verifiable rewards can produce reasoning strategies without prior chain-format SFT

A) Long-form reasoning emerges only after SFT on human-written reasoning chains

B) Distillation always produces a stronger model than reinforcement learning

C) RL on verifiable rewards can produce reasoning strategies without prior chain-format SFT

Answer and explanation

Correct answer: C. R1-Zero isolated the RL mechanism. Longer paths and self-checking emerged without a cold SFT start, while readability and format stability remained weaker. The starting checkpoint was the pretrained DeepSeek-V3-Base.

Question 15.6.1

15.6. Parallel scaling, worked example A: voting can get worse first

For independent attempts with p = 0.4 and six equiprobable distractors, what is voting accuracy at N = 3 when a tie counts as failure?

One correct optionA0.352 — lower than one attemptB0.400 — exactly equal to one attemptC0.596 — higher than with five attempts

A) 0.352 — lower than one attempt

B) 0.400 — exactly equal to one attempt

C) 0.596 — higher than with five attempts

Answer and explanation

Correct answer: A. A top-mode tie occurs with probability 0.48. Tie handling is therefore part of the algorithm; random top-mode selection would raise success to 0.472.

Question 15.6.2

15.6. Parallel scaling, worked example A: voting can get worse first

What must a self-consistency protocol specify?

One correct optionAOnly the temperature and sampling settings used for generationBAnswer normalization and the tie-breaking ruleCOnly the order in which candidates run on one accelerator

A) Only the temperature and sampling settings used for generation

B) Answer normalization and the tie-breaking rule

C) Only the order in which candidates run on one accelerator

Answer and explanation

Correct answer: B. Textually different answers can be mathematically equivalent, and tie policy materially changes accuracy. Without both rules the result is not reproducible.

Question 15.7.1

15.7. Worked example B: coverage is not selection accuracy

What does 1 − (1 − p)^N represent?

One correct optionAThe probability of a correct result after majority votingBThe probability that the verifier scores all N candidates without errorCThe probability that at least one correct answer appears among N candidates

A) The probability of a correct result after majority voting

B) The probability that the verifier scores all N candidates without error

C) The probability that at least one correct answer appears among N candidates

Answer and explanation

Correct answer: C. For independent, identically distributed attempts this is oracle coverage, an upper bound on selection rather than a guarantee of identifying a correct candidate.

Question 15.7.2

15.7. Worked example B: coverage is not selection accuracy

Why can a weak verifier plateau or even degrade as the candidate count grows?

One correct optionAWith larger N, a persuasive wrong answer has more chances to receive a high verifier scoreBAs N grows, the probability of generating a correct candidate must decreaseCAfter N = 8, the verifier stops considering the earliest candidates

A) With larger N, a persuasive wrong answer has more chances to receive a high verifier score

B) As N grows, the probability of generating a correct candidate must decrease

C) After N = 8, the verifier stops considering the earliest candidates

Answer and explanation

Correct answer: A. Search optimizes the verifier signal. A wider pool expands both the space of correct solutions and the space of false positives or exploitable loopholes. Uniform accepted-candidate selection with constant TPR > FPR grows toward a limit; degradation needs another mechanism, such as extreme noisy-score selection.

Question 15.7.3

15.7. Worked example B: coverage is not selection accuracy

With p=0.4, TPR=0.9 and FPR=0.05, what is the large-N limit for uniform selection among accepted candidates?

One correct optionA12/13 ≈ 0.923B0.9, always equal to TPRC1, regardless of the verifier

A) 12/13 ≈ 0.923

B) 0.9, always equal to TPR

C) 1, regardless of the verifier

Answer and explanation

Correct answer: A. The limit is p·TPR / [p·TPR + (1−p)·FPR], correctness conditional on acceptance.

Question 15.8.1

15.8. Sequential scaling and budget forcing

What does budget forcing do in the s1 work?

One correct optionAIt updates model weights while processing the user requestBIt controls when generation ends and therefore the length of one trajectoryCIt launches several independent models and averages their answers

A) It updates model weights while processing the user request

B) It controls when generation ends and therefore the length of one trajectory

C) It launches several independent models and averages their answers

Answer and explanation

Correct answer: B. The procedure intervenes in decoding by appending Wait or forcing the final-answer transition. No additional weight update occurs at request time.

Question 15.9.1

15.9. Failure modes of longer reasoning

What does overthinking mean for reasoning models?

One correct optionAThe KV cache completely fills all available accelerator memoryBThe model always exhausts its token limit before it can produce an answerCContinued reasoning damages an answer that is already or potentially correct

A) The KV cache completely fills all available accelerator memory

B) The model always exhausts its token limit before it can produce an answer

C) Continued reasoning damages an answer that is already or potentially correct

Answer and explanation

Correct answer: C. Quality can be non-monotonic in trace length: the model may revise away from a correct result, repeat itself, or amplify an early error.

Question 15.10.1

15.10. Worked example C: an illustrative budget ladder

How should the module’s 1K, 4K, 16K, and 64K token ladder be interpreted?

One correct optionAAs a transparent cost scenario, not official tiers of a particular APIBAs an exact mapping to low, medium, high, and max across providersCAs a speed measurement of one published production system

A) As a transparent cost scenario, not official tiers of a particular API

B) As an exact mapping to low, medium, high, and max across providers

C) As a speed measurement of one published production system

Answer and explanation

Correct answer: A. The ladder fixes course assumptions: an 8.03B model at 209.4 tokens/s. Provider labels and budgets are not standardized.

Question 15.11.1

15.11. Worked example D: the KV cost of a thought

For 512 tokens, the teaching model gives a 0.0625 GiB KV footprint. What does (80 − 14.96) / 0.0625 ≈ 1040 represent?

One correct optionAThroughput of about 80 generated tokens per secondBA KV-only upper bound of about 1040 concurrent sequencesCA minimum context length of about 512 thousand tokens

A) Throughput of about 80 generated tokens per second

B) A KV-only upper bound of about 1040 concurrent sequences

C) A minimum context length of about 512 thousand tokens

Answer and explanation

Correct answer: B. About 14.96 GiB is first reserved for the bf16 weights of the 8.03B model; the remainder of an 80 GiB device is divided by one 512-token KV footprint. This is only an upper bound because workspaces, fragmentation, and scheduling reduce achievable concurrency. This scenario assumes a zero input prefix and no runtime reserve.

Question 15.11.2

15.11. Worked example D: the KV cost of a thought

KV uses 128 KiB per token. How much is needed for a 4096-token prompt and 4000-token continuation with full caching?

One correct optionA0.98828125 GiBB0.48828125 GiBC128 GiB

A) 0.98828125 GiB

B) 0.48828125 GiB

C) 128 GiB

Answer and explanation

Correct answer: A. 128·1024·(4096+4000) / 2³⁰ = 0.98828125. The second option counts only the continuation.

Question 15.12.1

15.12. Worked example E: why adaptive allocation matters

In the teaching mixture, 70% of requests receive a 500-token budget, 20% receive 4K, and 10% receive 32K. What is the average budget?

One correct optionA8.00K tokens per requestB32.00K tokens per requestC4.35K tokens per request

A) 8.00K tokens per request

B) 32.00K tokens per request

C) 4.35K tokens per request

Answer and explanation

Correct answer: C. The weighted mean is 0.70·500 + 0.20·4K + 0.10·32K = 4.35K tokens. Relative to a fixed 32K budget, this teaching scenario reduces the average generated-token budget by about 7.36×. Interpreting this as consumption rather than assigned caps additionally assumes full use of every budget.

Question 15.12.2

15.12. Worked example E: why adaptive allocation matters

Why is a fixed maximum budget not a universally safe choice?

One correct optionAWhen the extra budget is used, it costs time and memory and may harm answersBIt automatically disables verifiers, tools, and repeated samplingCIt inevitably lowers the probability that at least one correct candidate appears

A) When the extra budget is used, it costs time and memory and may harm answers

B) It automatically disables verifiers, tools, and repeated sampling

C) It inevitably lowers the probability that at least one correct candidate appears

Answer and explanation

Correct answer: A. Actual consumption matters, not just the cap. Continuing generation costs time and KV memory and may lead to overthinking. A model with a high cap can still stop early; compare quality with measured costs.

Question 15.12.3

15.12. Worked example E: why adaptive allocation matters

A simulation precomputes all sixteen candidates, then models stopping at the fourth. What actually decreased?

One correct optionACandidates inspected by the modeled policyBGenerator calls already executedCMemory of the existing array automatically

A) Candidates inspected by the modeled policy

B) Generator calls already executed

C) Memory of the existing array automatically

Answer and explanation

Correct answer: A. Avoiding generation requires stopping before the next call. The program-search lab implements that control flow.

Question 15.15.1

15.15. Verifier families

How does a process reward model differ from an outcome reward model?

One correct optionAIt scores intermediate steps rather than only the final outcomeBIt must use a formal proof checkerCIt is used only during training and never for selection

A) It scores intermediate steps rather than only the final outcome

B) It must use a formal proof checker

C) It is used only during training and never for selection

Answer and explanation

Correct answer: A. Step-level feedback can localize errors and prune paths. Human annotation can be expensive, but process labels can also be constructed automatically. Either source requires checks of label quality and transfer across tasks.

Question 15.16.1

15.16. Distilling reasoning traces

What does reasoning-trace distillation primarily transfer?

One correct optionAThe teacher’s hidden weights, internal states, and full computational capacityBObservable formats, operations, and procedures represented in training tracesCThe teacher API’s official budget controls and private parameters

A) The teacher’s hidden weights, internal states, and full computational capacity

B) Observable formats, operations, and procedures represented in training traces

C) The teacher API’s official budget controls and private parameters

Answer and explanation

Correct answer: B. Training on traces can teach new procedures and knowledge because student weights change. It does not guarantee the teacher’s full capability; generalization needs separate evaluation.

Question 15.17.1

15.17. Reading a model's reasoning passport

What should a reasoning-model passport distinguish?

One correct optionAAn average score on one public benchmark under one fixed configurationBThe maximum answer length, context size, and model nameCTrajectory depth, search breadth, visibility, billing, and tool-continuity protocol

A) An average score on one public benchmark under one fixed configuration

B) The maximum answer length, context size, and model name

C) Trajectory depth, search breadth, visibility, billing, and tool-continuity protocol

Answer and explanation

Correct answer: C. One effort label does not describe the compute contract. We need to know what scales, what the user sees and pays for, and how state survives across calls.

Question 15.14.1

15.14. Level 2: the module notebook

Why does score 1 establish correctness in the polynomial lab?

One correct optionAThree matching values identify a polynomial of degree at most twoBTwo tests suffice for arbitrary codeCThe generator saw hidden-test answers

A) Three matching values identify a polynomial of degree at most two

B) Two tests suffice for arbitrary code

C) The generator saw hidden-test answers

Answer and explanation

Correct answer: A. One value is fixed for the generator and two more are checked. The guarantee is specific to this grammar.