Modern LLMs1 / 20

Module 1. The Modern LLM Landscape

Choose your answers and select “Check”. Numbers accept a decimal point, comma, or simple fraction. Answers and drafts are saved in this browser and shared between RU/EN.

Checked: 0 / 19 · Correct: 0

Question 1.1.1

1.1. Start with a decision

What does an autoregressive language model compute at each step?

One correct optionAAll tokens of the completed answer simultaneouslyBIndependent external verification of every statementCScores for the next token given the available context

A) All tokens of the completed answer simultaneously

B) Independent external verification of every statement

C) Scores for the next token given the available context

Answer and explanation

Correct answer: C. The model scores the next token; the system selects it and repeats the step. Source retrieval and answer verification are separate system responsibilities.

Question 1.2.1

1.2. From a small calculation to an implementation you can read

What does the third form of work—reading an open implementation—mean in this course?

One correct optionADeveloping distributed training and specialized kernels for a production serviceBConnecting a studied mechanism to its configuration and selected source codeCCertifying operational readiness by running the program on a GPU

A) Developing distributed training and specialized kernels for a production service

B) Connecting a studied mechanism to its configuration and selected source code

C) Certifying operational readiness by running the program on a GPU

Answer and explanation

Correct answer: B. We inspect selected operations and relate them to formulas. Kernel development, distributed-training implementation, and production operation require separate study. The MiniLLM final project remains a trainable teaching model.

Question 1.5.1

1.5. Two passports: model and system

What most reliably answers the question "how is the computation graph built"?

One correct optionAA verified config.json together with the model's implementation codeBThe model's name and the size class claimed by the vendorCThe model's position on a public leaderboard at release time

A) A verified config.json together with the model's implementation code

B) The model's name and the size class claimed by the vendor

C) The model's position on a public leaderboard at release time

Answer and explanation

Correct answer: A. A verified configuration and implementation let you trace the computation graph. A brand name or leaderboard position does not determine KV-head geometry, MoE routing, or positional encoding.

Question 1.5.2

1.5. Two passports: model and system

Why does the model passport need a system passport beside it?

One correct optionATo collapse every model metric into one numberBTo conceal the exact weight revision in a published resultCTo record tools, permissions, isolation, memory, and evaluation

A) To collapse every model metric into one number

B) To conceal the exact weight revision in a published result

C) To record tools, permissions, isolation, memory, and evaluation

Answer and explanation

Correct answer: C. The same weights produce different capabilities and risks under different tools, network access, shared memory, and controls. Without a system passport, a result or incident cannot be reproduced or interpreted correctly.

Question 1.6.1

1.6. Axis I up close: dense and MoE

An announcement reads “1.6T total / 49B active.” What primarily determines weight storage?

One correct optionAThe 49B active: only experts selected for the current token need storageBThe 1.6T total: the full bank must remain available for executionCThe 1.6T-to-49B ratio: memory is set only by the sparsity factor

A) The 49B active: only experts selected for the current token need storage

B) The 1.6T total: the full bank must remain available for execution

C) The 1.6T-to-49B ratio: memory is set only by the sparsity factor

Answer and explanation

Correct answer: B. The router selects different experts for different tokens, so the full parameter bank must be available, locally or through distributed placement. Active parameters provide a first estimate of weight-side compute per token, not total weight storage.

Question 1.6.2

1.6. Axis I up close: dense and MoE

A card reports different active counts for prefill and decode. How should the passport record them?

One correct optionAUse the smaller count for every phaseBKeep both counts with their phase and counting conventionCAdd them and call the sum the stored weight count

A) Use the smaller count for every phase

B) Keep both counts with their phase and counting convention

C) Add them and call the sum the stored weight count

Answer and explanation

Correct answer: B. The active path can vary by phase. Neither count replaces total parameters or directly measures latency.

Question 1.7.1

1.7. Worked comparison: gpt-oss-120b and Llama 4 Scout

gpt-oss-120b has 117B parameters and ships with MXFP4 weights. How should the artifact size be read?

One correct optionA117B × 4.25/8 ≈ 57.89 GiB is the exact complete size of the official filesBMXFP4 affects speed only, so the artifact remains about 218 GiBCIdealized: 57.89 GiB; official original artefact: about 60.77 GiB

A) 117B × 4.25/8 ≈ 57.89 GiB is the exact complete size of the official files

B) MXFP4 affects speed only, so the artifact remains about 218 GiB

C) Idealized: 57.89 GiB; official original artefact: about 60.77 GiB

Answer and explanation

Correct answer: C. 57.89 GiB assumes every parameter costs the same 4.25 bits. The official index reports 65,248,815,744 bytes, about 60.77 GiB, because tensor formats, packing, and auxiliary data are heterogeneous.

Question 1.7.2

1.7. Worked comparison: gpt-oss-120b and Llama 4 Scout

What does the ratio 34/10.2 ≈ 3.33 mean in the Scout versus gpt-oss-120b teaching comparison?

One correct optionAA weight-matrix FLOP ratio; the latency ratio depends on the runtimeBA guaranteed latency ratio at every context length and on every deviceCOnly a checkpoint-size ratio; the two models perform identical compute

A) A weight-matrix FLOP ratio; the latency ratio depends on the runtime

B) A guaranteed latency ratio at every context length and on every device

C) Only a checkpoint-size ratio; the two models perform identical compute

Answer and explanation

Correct answer: A. The 2·N_active estimate gives about 34 versus 10.2 GFLOP per token for weight-side matrix operations. Actual latency also depends on attention, memory, routing, communication, batching, and kernels.

Question 1.8.1

1.8. Axis II: context means length, quality, and cost

What exactly does an advertised 1M-token context window mean?

One correct optionAThe KV cache for such a request is guaranteed to fit on one acceleratorBThe model accepts that length; per-position quality is measured separatelyCThe model makes equally good use of every one of the million positions

A) The KV cache for such a request is guaranteed to fit on one accelerator

B) The model accepts that length; per-position quality is measured separately

C) The model makes equally good use of every one of the million positions

Answer and explanation

Correct answer: B. The advertised window specifies supported length. It does not guarantee uniform quality across positions or tasks. Evaluate how well the system uses distant evidence on the intended workload, and account for KV memory separately from that quality measurement.

Question 1.9.1

1.9. Axis III: modalities are directional

Why is the label "multimodal" not enough to compare two models?

One correct optionAIt says nothing about input and output modalities or where fusion happensBMultimodal models all rely on one and the same vision encoderCModalities affect neither the token price nor the context budget

A) It says nothing about input and output modalities or where fusion happens

B) Multimodal models all rely on one and the same vision encoder

C) Modalities affect neither the token price nor the context budget

Answer and explanation

Correct answer: A. The same label covers both "understands an image" and "generates audio". Axis III of the passport requires input, output and the fusion method to be stated separately.

Question 1.10.1

1.10. Axis IV: who controls the reasoning budget?

How does configurable reasoning effort differ from adaptive thinking?

One correct optionAAdaptive mode switches reasoning off entirely in favour of answer speedBConfigurable mode exists only in multimodal model familiesCThe caller selects an effort level; the model may allocate computation adaptively

A) Adaptive mode switches reasoning off entirely in favour of answer speed

B) Configurable mode exists only in multimodal model families

C) The caller selects an effort level; the model may allocate computation adaptively

Answer and explanation

Correct answer: C. Configurable effort lets the caller select an effort level; adaptive thinking allocates computation through the model or controller. Both can coexist. Neither alone guarantees fixed cost, latency, or better quality.

Question 1.11.1

1.11. Axis V: access, licensing, and practical openness

Which statement most precisely describes the open-weights status?

One correct optionAThe model is cleared for any commercial use with no restrictions at allBWeights are published, but data and the training stack may stay closedCData, training recipe, checkpoints and training logs are all reproducible

A) The model is cleared for any commercial use with no restrictions at all

B) Weights are published, but data and the training stack may stay closed

C) Data, training recipe, checkpoints and training logs are all reproducible

Answer and explanation

Correct answer: B. Published weights make a specific artefact inspectable and runnable under its licence. Training data, code, and logs require separate checks: openness has several independent dimensions.

Question 1.12.1

1.12. Axis VI: from token prices to accepted answers

One instruction requires keeping experiment logs for 30 days; another requires deleting temporary exports after 7 days. The assistant answers “30 days” about a log but cites the export instruction. What is wrong?

One correct optionAThe source exists, but an export rule does not support the log-retention claimBNothing: a correct number is enough for acceptanceCThe number of days must also be wrong

A) The source exists, but an export rule does not support the log-retention claim

B) Nothing: a correct number is enough for acceptance

C) The number of days must also be wrong

Answer and explanation

Correct answer: A. The number is correct, but the cited passage concerns a different object. Answer correctness and source support need separate checks; an existing document is not enough.

Question 1.12.2

1.12. Axis VI: from token prices to accepted answers

What do you need to know to estimate the real cost of an API task?

One correct optionAApplicable rates, billed tokens, caching, tools, and all execution attemptsBThe input token price and the size of the system prompt in charactersCThe model's total parameter count and the accelerator class in use

A) Applicable rates, billed tokens, caching, tools, and all execution attempts

B) The input token price and the size of the system prompt in characters

C) The model's total parameter count and the accelerator class in use

Answer and explanation

Correct answer: A. Include every billed attempt and tool call under the applicable cache and pricing rules. To compare cost per accepted task, divide by the number of results that meet the same acceptance criterion.

Question 1.12.3

1.12. Axis VI: from token prices to accepted answers

A model has 550B total and 55B active parameters. What are its idealized four-bit weight payload and weight-matmul estimate per generated token?

One correct optionA275 GB and roughly 110 GFLOP/tokenB27.5 GB and roughly 55 GFLOP/tokenC1.10 TB and roughly 550 GFLOP/token

A) 275 GB and roughly 110 GFLOP/token

B) 27.5 GB and roughly 55 GFLOP/token

C) 1.10 TB and roughly 550 GFLOP/token

Answer and explanation

Correct answer: A. At four bits per parameter, 550B × 4/8 = 275 GB. The decode weight-matmul estimate is 2 × 55B = 110 GFLOP/token. Format metadata, attention, state, and system overhead are separate.

Question 1.12.4

1.12. Axis VI: from token prices to accepted answers

A system spent 2.80 USD on 100 tasks, including failures, and accepted 80 results. What is the cost per accepted task?

One correct optionA0.035 USDB0.028 USDC0.280 USD

A) 0.035 USD

B) 0.028 USD

C) 0.280 USD

Answer and explanation

Correct answer: A. Divide total spend by accepted tasks: 2.80 / 80 = 0.035 USD. Dividing by 100 would give cost per submitted task.

Question 1.13.1

1.13. Dated reference: APIs and product availability

A model is announced but accessible only through a restricted programme. What is its status for an ordinary API customer?

One correct optionAGenerally available: an announcement is sufficientBOpen weights: someone has accessCRestricted access; general API availability is unconfirmed

A) Generally available: an announcement is sufficient

B) Open weights: someone has access

C) Restricted access; general API availability is unconfirmed

Answer and explanation

Correct answer: C. Announcement, restricted access, general API availability, and weight release are separate events. Record only the status supported by evidence.

Question 1.15.1

1.15. From public signal to engineering evidence

Why can provider-reported scores from different models not simply be merged into one ranking?

One correct optionABenchmark scores are random and carry no useful signal about a modelBOnly models with open weights can be compared against one anotherCThe harnesses, budgets, dataset versions, and attempt counts may differ

A) Benchmark scores are random and carry no useful signal about a model

B) Only models with open weights can be compared against one another

C) The harnesses, budgets, dataset versions, and attempt counts may differ

Answer and explanation

Correct answer: C. A reported number may be valid for its setup yet incomparable with another harness, budget, or dataset version. One ranking requires one protocol—the subject of Module 19.

Question 1.17.1

1.17. Code, level 1: a passport from a validated config.json

Which way of fetching config.json makes the result reproducible?

One correct optionADownloading the main branch and recording the access date alongsideBResolving the branch to an exact commit SHA with schema and hash checksCCopying the config out of a model card or a survey blog post

A) Downloading the main branch and recording the access date alongside

B) Resolving the branch to an exact commit SHA with schema and hash checks

C) Copying the config out of a model card or a survey blog post

Answer and explanation

Correct answer: B. A branch moves: after its next update, the same URL may serve a different file and silently change the passport arithmetic. Reproducibility comes from an immutable revision recorded with the result.