Module 13. Supervised Fine-Tuning and Data Work — quizzes and answer checking · Modern LLMs

Question 13.4.1

13.4. Chat templates are executable contracts

Why is a chat template part of the training contract rather than visual formatting?

One correct optionAIt decides which roles may be stored in the dataset at allBIt creates new vocabulary tokens during SFTCIt defines the token serialization used in both training and inference

A) It decides which roles may be stored in the dataset at all

B) It creates new vocabulary tokens during SFT

C) It defines the token serialization used in both training and inference

Answer and explanation

Correct answer: C. The template turns roles, message boundaries, tool calls, and turn endings into concrete tokens. A mismatch can degrade behavior without raising a runtime error.

Question 13.5.1

13.5. Label masking: a 56-token example

Under an assistant-only objective, what goes into labels for system and user tokens?

One correct optionA−100, so those positions contribute no direct loss termBThe padding token ID, so all rows have equal lengthCThe original token ID multiplied by a zero weight

A) −100, so those positions contribute no direct loss term

B) The padding token ID, so all rows have equal length

C) The original token ID multiplied by a zero weight

Answer and explanation

Correct answer: A. With ignore_index=-100, the tokens remain in the input context but are not direct cross-entropy targets. Full-sequence loss is possible in a different training recipe.

Question 13.5.2

13.5. Label masking: a 56-token example

A dialogue has 56 tokens and a 20-token assistant reply. What fraction is supervised if the reply terminator is also a target?

One correct optionA35.7%: only reply content is supervisedB37.5%: the terminator is added to the twenty reply tokensC71.4%: the reply and preceding user turn are supervised

A) 35.7%: only reply content is supervised

B) 37.5%: the terminator is added to the twenty reply tokens

C) 71.4%: the reply and preceding user turn are supervised

Answer and explanation

Correct answer: B. $(20+1)/56=37.5\%$ . The extra position is the assistant turn terminator in this teaching template.

Question 13.5.3

13.5. Label masking: a 56-token example

What is the main risk when the assistant turn terminator is excluded from the SFT targets?

One correct optionATraining loss must become infinite as soon as the first batch reaches the terminatorBEvery response will stop after its first tokenCSFT gives no direct signal for ending the turn at the demonstrated boundary

A) Training loss must become infinite as soon as the first batch reaches the terminator

B) Every response will stop after its first token

C) SFT gives no direct signal for ending the turn at the demonstrated boundary

Answer and explanation

Correct answer: C. The base model may still stop, but the demonstrations do not reinforce the required terminator. Inspect target tokens at boundaries rather than relying on aggregate loss alone.

Question 13.5.4

13.5. Label masking: a 56-token example

Which row predicts labels[t] in causal training?

One correct optionAlogits[t−1]Blogits[t+1]CAny row in the batch

A) logits[t−1]

B) logits[t+1]

C) Any row in the batch

Answer and explanation

Correct answer: A. The target is stored at its own position and predicted by the preceding position. Shift exactly once.

Question 13.6.1

13.6. Packing variable-length examples safely

What does the FFD result “24 examples → 5 containers” mean at a length of 2,048?

One correct optionAFixed-length rows fall, while utilization rises from 17.7% to 84.9%BTraining throughput is guaranteed to improve by exactly 4.8×CEvery container holds the same number of source examples

A) Fixed-length rows fall, while utilization rises from 17.7% to 84.9%

B) Training throughput is guaranteed to improve by exactly 4.8×

C) Every container holds the same number of source examples

Answer and explanation

Correct answer: A. The calculation measures fixed-length rows and removed padding. Runtime also depends on attention kernels, batch shape, device occupancy, and overhead.

Question 13.6.2

13.6. Packing variable-length examples safely

Why do packed examples need a block-diagonal attention mask?

One correct optionASo that positions of different examples get independent RoPE indicesBOtherwise examples in one bin see each other and train on foreign textCSo that FlashAttention can process the bin in a single kernel call

A) So that positions of different examples get independent RoPE indices

B) Otherwise examples in one bin see each other and train on foreign text

C) So that FlashAttention can process the bin in a single kernel call

Answer and explanation

Correct answer: B. Without isolation, a token in one dialogue can depend on unrelated text in the same container. EOS marks a boundary but does not by itself remove cross-example attention edges.

Question 13.6.3

13.6. Packing variable-length examples safely

How do you remove a target pair crossing from A into B?

One correct optionAIgnore the first label of each segmentBOnly reset position_idsCOnly insert EOS between examples

A) Ignore the first label of each segment

B) Only reset position_ids

C) Only insert EOS between examples

Answer and explanation

Correct answer: A. Attention masking cannot repair a cross-boundary logit–label pair created by shifting.

Question 13.7.1

13.7. Compute accounting and hyperparameters

What follows from comparing $3.47\cdot10^{19}$ FLOP for SFT with $7.23\cdot10^{23}$ FLOP for pretraining?

One correct optionAEvery SFT run is always exactly 20,833× cheaper than every pretraining runBThe two stages require equal compute at equal model sizeCThe scenarios differ by 20,833; 24.33 hours is a separate estimate

A) Every SFT run is always exactly 20,833× cheaper than every pretraining run

B) The two stages require equal compute at equal model size

C) The scenarios differ by 20,833; 24.33 hours is a separate estimate

Answer and explanation

Correct answer: C. The ratio belongs to the stated token and epoch counts. The time estimate additionally assumes 990 TFLOP/s and MFU 0.4; it is not a measured production run.

Question 13.7.2

13.7. Compute accounting and hyperparameters

Why is an excessive learning rate especially risky in a short SFT run?

One correct optionALarge updates can erase useful pretrained behavior within a few stepsBAdamW cannot finish allocating optimizer state within only a few training epochsCA large step automatically unmasks user tokens

A) Large updates can erase useful pretrained behavior within a few steps

B) AdamW cannot finish allocating optimizer state within only a few training epochs

C) A large step automatically unmasks user tokens

Answer and explanation

Correct answer: A. The dataset is comparatively small, so large updates can cause regressions or overfitting. Validate the learning-rate range on held-out capabilities and behavior.

Question 13.8.1

13.8. Organizing SFT data by capability

Why does row count alone not determine the quality of a public instruction dataset?

One correct optionALarge datasets are always entirely syntheticBRows may be duplicate, wrong, or unrelated to the target decisionsCSFT depends only on example count and not on token count

A) Large datasets are always entirely synthetic

B) Rows may be duplicate, wrong, or unrelated to the target decisions

C) SFT depends only on example count and not on token count

Answer and explanation

Correct answer: B. Correctness, diversity, target-skill coverage, licensing, and contamination matter. Scale does not compensate for systematically poor demonstrations.

Question 13.9.1

13.9. Reading LIMA correctly

Which careful conclusion is supported by LIMA?

One correct optionAModern models no longer need instruction tuning after pretrainingBExactly 1,000 examples suffice for every model and every domainCA strong base model can adapt from 1,000 carefully curated demonstrations

A) Modern models no longer need instruction tuning after pretraining

B) Exactly 1,000 examples suffice for every model and every domain

C) A strong base model can adapt from 1,000 carefully curated demonstrations

Answer and explanation

Correct answer: C. LIMA demonstrates the value of curation for an already capable 65B model. It does not establish a universal minimum dataset size.

Question 13.9.2

13.9. Reading LIMA correctly

When should source-task families be split?

One correct optionABefore generating paraphrases and synthetic answersBAfter shuffling every generated answerCOnly after training

A) Before generating paraphrases and synthetic answers

B) After shuffling every generated answer

C) Only after training

Answer and explanation

Correct answer: A. Derived examples from one source stay in one split to prevent content leakage.

Question 13.10.1

13.10. Synthetic data as a search-and-filter system

Why does a synthetic data pipeline need a verifier?

One correct optionAA generator model confidently produces both correct and wrong examplesBIt is needed to deduplicate the generated examples against each otherCIt brings the style of generated answers to one common template

A) A generator model confidently produces both correct and wrong examples

B) It is needed to deduplicate the generated examples against each other

C) It brings the style of generated answers to one common template

Answer and explanation

Correct answer: A. Incorrect demonstrations provide a learning signal in the wrong direction. A verifier applies explicit acceptance criteria, although incomplete tests and imperfect judges can still admit errors.

Question 13.11.1

13.11. Reasoning demonstrations and the s1 compute scenario

How does SFT on worked solutions differ from training on final answers alone?

One correct optionAIt always requires a separate reasoning role in the templateBIntermediate steps are targets, so the student imitates the solution pathCIt must run after preference optimization

A) It always requires a separate reasoning role in the template

B) Intermediate steps are targets, so the student imitates the solution path

C) It must run after preference optimization

Answer and explanation

Correct answer: B. When the worked solution is supervised, the model receives token-level signal over intermediate steps. Useful strategies and teacher artifacts can transfer together.

Question 13.12.1

13.12. Teaching a model to use tools

What must be represented in a complete call–observation–response demonstration?

One correct optionAThe tool must be reachable during training so the call can be verifiedBThe example must contain at least two different tools in sequenceCThe call itself, the tool response and the model's conclusion after it

A) The tool must be reachable during training so the call can be verified

B) The example must contain at least two different tools in sequence

C) The call itself, the tool response and the model's conclusion after it

Answer and explanation

Correct answer: C. The trajectory should cover the decision to call, the structured action, the environment observation, and the assistant's next step. The tool result is usually context rather than a model target.

Question 13.15.1

13.15. Distillation channels

How does distribution distillation differ from training on one teacher response?

One correct optionAThe student matches teacher probabilities, not only one sampled sequenceBTeacher and student must use the same architecture and parameter countCThe student must choose the same most likely token at every step

A) The student matches teacher probabilities, not only one sampled sequence

B) Teacher and student must use the same architecture and parameter count

C) The student must choose the same most likely token at every step

Answer and explanation

Correct answer: A. Soft targets expose probabilities over alternatives, either directly or through teacher logits. The token events must be aligned. Matching only the argmax does not match the distribution.

Question 13.16.1

13.16. LoRA and a concrete parameter count

Why does LoRA reduce optimizer-state memory?

One correct optionAAll optimizer states are automatically stored in four bitsBOptimizer state is needed only for trainable adapter parametersCMoments are recomputed from scratch at every step

A) All optimizer states are automatically stored in four bits

B) Optimizer state is needed only for trainable adapter parameters

C) Moments are recomputed from scratch at every step

Answer and explanation

Correct answer: B. The base weights are frozen, so gradients and optimizer state are maintained only for adapters. Total memory still includes base weights, activations, buffers, and overhead.

Question 13.16.2

13.16. LoRA and a concrete parameter count

If 208 MiB already includes adapter weights, what is added to the 4-bit base?

One correct optionA208 MiBB208 MiB plus another 26 MiB of weightsCOnly 26 MiB

A) 208 MiB

B) 208 MiB plus another 26 MiB of weights

C) Only 26 MiB

Answer and explanation

Correct answer: A. The chosen 16 bytes include the weight, gradient, master copy and moments, each once.