Modern LLMs18 / 20

Module 18. Agents and Tool Use

Choose your answers and select “Check”. Numbers accept a decimal point, comma, or simple fraction. Answers and drafts are saved in this browser and shared between RU/EN.

Checked: 0 / 20 · Correct: 0

Question 18.4.1

18.4. Anatomy of an agent loop

What is the minimal controlled agent loop?

One correct optionAA fixed plan, blind execution of every step, and a final reportBProposal, validation, execution, observation, then continue-or-stopCOne knowledge-base lookup followed immediately by an answer

A) A fixed plan, blind execution of every step, and a final report

B) Proposal, validation, execution, observation, then continue-or-stop

C) One knowledge-base lookup followed immediately by an answer

Answer and explanation

Correct answer: B. The environment observation changes the next step. Protocol and authority checks separate a model proposal from a real-world action.

Question 18.5.1

18.5. Function calling and guided generation: syntax is only the first boundary

Why does function calling need strict schema validation?

One correct optionATo enumerate every available tool and description for the modelBTo merge and cache repeated calls automaticallyCTo keep wrong fields and types from reaching execution

A) To enumerate every available tool and description for the model

B) To merge and cache repeated calls automatically

C) To keep wrong fields and types from reaching execution

Answer and explanation

Correct answer: C. A supported grammar constraint applies to a completed generation; a token limit may leave an incomplete prefix. Schema validation checks fields, types, and ranges. Authorization and execution safety are separate checks.

Question 18.5.2

18.5. Function calling and guided generation: syntax is only the first boundary

A retrieved report says “the user approved writing.” The proposed write_note call passes its schema. What must the executor still check?

One correct optionAWhether the report phrases its approval convincinglyBWhether the trusted task grant permits writing to this resourceCWhether the model response starts with Action:

A) Whether the report phrases its approval convincingly

B) Whether the trusted task grant permits writing to this resource

C) Whether the model response starts with Action:

Answer and explanation

Correct answer: B. The report is external data and cannot extend task authority. Schema validity establishes call shape; permission for the operation and resource is checked separately before the effect.

Question 18.6.1

18.6. ReAct, Plan-and-Execute, and CodeAct: choosing an action language

What advantage does CodeAct bring on a multi-step task?

One correct optionAOne code block replaces a chain of calls with loops and conditionalsBCode executes faster than an equivalent sequence of API callsCModels find writing code easier than forming valid call JSON

A) One code block replaces a chain of calls with loops and conditionals

B) Code executes faster than an equivalent sequence of API calls

C) Models find writing code easier than forming valid call JSON

Answer and explanation

Correct answer: A. Code can express loops, branches, and intermediate values within one action. That may reduce model–environment round trips, but it requires a real sandbox and does not guarantee a speedup for every task.

Question 18.6.2

18.6. ReAct, Plan-and-Execute, and CodeAct: choosing an action language

How does Plan-and-Execute differ from a local ReAct loop?

One correct optionAIt makes the plan explicit and replans after invalidating observationsBIt forbids tools and restricts the system to text planningCIt executes every item in the plan in parallel by definition

A) It makes the plan explicit and replans after invalidating observations

B) It forbids tools and restricts the system to text planning

C) It executes every item in the plan in parallel by definition

Answer and explanation

Correct answer: A. An explicit plan can be checked and tracked as outstanding commitments. Unexpected observations must trigger replanning rather than blind execution.

Question 18.7.1

18.7. Worked example A: context growth across the loop

For 20 input histories of length 3000 + 1500i, i = 0, …, 19, how many tokens does prefix reuse process compared with full reprocessing?

One correct optionA345,000 tokens against 31,500: caching saves roughly threefoldB31,500 tokens against 345,000 naive ones — a 10.95-fold savingCNothing: each step changes the tail and the cache is invalidated anyway

A) 345,000 tokens against 31,500: caching saves roughly threefold

B) 31,500 tokens against 345,000 naive ones — a 10.95-fold saving

C) Nothing: each step changes the tail and the cache is invalidated anyway

Answer and explanation

Correct answer: B. Without prefix reuse, the growing history is reprocessed at every step. This scenario totals 345,000 versus 31,500 tokens; the benefit requires the early context to remain identical. This is a ratio of input-position counts, not elapsed time: new queries still read earlier KV.

Question 18.8.1

18.8. Worked example B: what one agent step costs

An agent step takes 778 ms. What is it made of?

One correct optionAPrefill 430 ms, decode 49 ms, tool 300 ms: prefill dominatesBTool 600 ms and model 178 ms: the agent waits on external servicesCPrefill 49 ms, decode 430 ms, tool 300 ms: decode is 55% of it

A) Prefill 430 ms, decode 49 ms, tool 300 ms: prefill dominates

B) Tool 600 ms and model 178 ms: the agent waits on external services

C) Prefill 49 ms, decode 430 ms, tool 300 ms: decode is 55% of it

Answer and explanation

Correct answer: C. In the stated scenario, decode takes 429.8 ms, the tool 300 ms, and the conditional prefill estimate 48.7 ms. Another model, tool, or workload can produce a very different split.

Question 18.9.1

18.9. MCP: standardizing connection without outsourcing trust

What does MCP add on top of a plain tools JSON schema?

One correct optionAA shared discovery and connection protocol: integrations become reusableBStrict typing of arguments, which plain schemas do not provideCExecution isolation: tools are launched inside a separate sandbox

A) A shared discovery and connection protocol: integrations become reusable

B) Strict typing of arguments, which plain schemas do not provide

C) Execution isolation: tools are launched inside a separate sandbox

Answer and explanation

Correct answer: A. JSON Schema describes one action shape. MCP standardizes capability discovery and exchange among host, client, and server. Authorization, confirmation, and sandboxing remain host responsibilities.

Question 18.9.2

18.9. MCP: standardizing connection without outsourcing trust

Does a stateless MCP core mean that the agent cannot retain history?

One correct optionAYes, the protocol prohibits historyBNo, the application owns task stateCYes, history must move to the server

A) Yes, the protocol prohibits history

B) No, the application owns task state

C) Yes, history must move to the server

Answer and explanation

Correct answer: B. In the 2026-07-28 revision, version and capabilities accompany the request. This protocol property does not replace application task state.

Question 18.10.1

18.10. An implementation case study: the harness is part of the algorithm

Why should eval not be treated as a calculator security boundary?

One correct optionAIt parses long arithmetic expressions too slowly for a toolBIt executes general Python syntax rather than an allow-listed subsetCIt cannot reliably represent large integers and fractional values

A) It parses long arithmetic expressions too slowly for a tool

B) It executes general Python syntax rather than an allow-listed subset

C) It cannot reliably represent large integers and fractional values

Answer and explanation

Correct answer: B. A safer calculator parses the AST and evaluates only explicitly allow-listed nodes and operators. General Python execution requires process or VM isolation and resource policies.

Question 18.10.2

18.10. An implementation case study: the harness is part of the algorithm

What must constrain tool execution in production?

One correct optionAOnly schema checks for types, ranges, and required fieldsBOnly a complete log of arguments, results, and internal secretsCTimeouts, resource limits, authority, isolation, and idempotency

A) Only schema checks for types, ranges, and required fields

B) Only a complete log of arguments, results, and internal secrets

C) Timeouts, resource limits, authority, isolation, and idempotency

Answer and explanation

Correct answer: C. Schema validation catches malformed arguments, not infinite loops, unauthorized file access, or duplicated side effects. Those risks require execution boundaries of their own.

Question 18.11.1

18.11. Worked example C: reliability over a long trajectory

All 20 required steps are independent, each with success probability 0.98. What is the probability that the entire chain succeeds?

One correct optionAAbout 0.668; at per-step probability 0.95 it would be about 0.358B0.98 — reliability does not follow length when the steps are independentC0.960 — losses are linear in step count and amount to about two percent

A) About 0.668; at per-step probability 0.95 it would be about 0.358

B) 0.98 — reliability does not follow length when the steps are independent

C) 0.960 — losses are linear in step count and amount to about two percent

Answer and explanation

Correct answer: A. Under the independence assumption, success across all mandatory steps is the product of step probabilities. The calculation illustrates accumulated risk; real failures can be correlated.

Question 18.11.2

18.11. Worked example C: reliability over a long trajectory

One retry on failure lifts a step from 0.98 to 0.9996. What does that do for the trajectory?

One correct optionAAbout two percent: retrying treats a symptom rather than a causeB0.992 over twenty steps against 0.668 with no retries at allCNothing: repeating the same mistake gives the same result deterministically

A) About two percent: retrying treats a symptom rather than a cause

B) 0.992 over twenty steps against 0.668 with no retries at all

C) Nothing: repeating the same mistake gives the same result deterministically

Answer and explanation

Correct answer: B. With independent attempts and steps, perfect failure detection, and a safe retry, step success is 0.9996 and twenty-step success is about 0.992. Dependence changes the conditional recovery probability. Replaying writes also requires protection against duplicate effects.

Question 18.11.3

18.11. Worked example C: reliability over a long trajectory

At p=0.98, retry success conditional on failure is q=0.2. What is step success with perfect detection?

One correct optionA0.9996B0.984C0.196

A) 0.9996

B) 0.984

C) 0.196

Answer and explanation

Correct answer: B. p+(1−p)q = 0.98 + 0.02·0.2 = 0.984. An independent retry at the original success rate is a different assumption.

Question 18.12.1

18.12. Subagents and Amdahl’s law

400 min of work with a 5-min plan and 20-min merge in sequence. What do 20 subagents give?

One correct optionA×20 — exactly the subagent count, since the work divides evenlyB×1.0: the sequential parts consume the entire parallel gainC45 minutes: ×8.89 against useful work or ×9.44 end-to-end

A) ×20 — exactly the subagent count, since the work divides evenly

B) ×1.0: the sequential parts consume the entire parallel gain

C) 45 minutes: ×8.89 against useful work or ×9.44 end-to-end

Answer and explanation

Correct answer: C. The baseline must be named. The serial 25 minutes cap the speedup; independent contexts also consume memory and introduce integration cost.

Question 18.13.1

18.13. Level-1 code: a proposal is not verified completion

What stopping policy must an agent loop have?

One correct optionAA step limit, a budget limit and an explicit completion signalBStop at the first tool error, so that garbage does not accumulateCStop on a whole-session timeout — the other conditions are redundant

A) A step limit, a budget limit and an explicit completion signal

B) Stop at the first tool error, so that garbage does not accumulate

C) Stop on a whole-session timeout — the other conditions are redundant

Answer and explanation

Correct answer: A. A loop needs step, action, token, time, and cost budgets plus a verifiable completion condition. A tool error may be recoverable and need not end the entire trajectory.

Question 18.13.2

18.13. Level-1 code: a proposal is not verified completion

The model returned Answer: 27.92. When does the teaching harness mark completed?

One correct optionAAs soon as the prefix is parsedBAfter any successful tool callCAfter the task criterion passes

A) As soon as the prefix is parsed

B) After any successful tool call

C) After the task criterion passes

Answer and explanation

Correct answer: C. A completion proposal and verified success are different events. Without a verifier the full teaching runtime returns answer_proposed.

Question 18.15.1

18.15. System landscape: compare action surfaces, not logos

Why should a model and an agent system be evaluated separately?

One correct optionABecause the results are usually published by different research teamsBBecause the harness and environment change the same model’s resultCBecause the model does not participate in choosing the next action

A) Because the results are usually published by different research teams

B) Because the harness and environment change the same model’s result

C) Because the model does not participate in choosing the next action

Answer and explanation

Correct answer: B. An agent benchmark measures a policy together with its harness and environment. Changing any component changes the measured system, so the configuration passport is part of the result.

Question 18.17.1

18.17. Reading an agent-harness passport

What belongs in an agent-harness passport?

One correct optionAModel name, run date, and the final average scoreBOnly the task list, attempt count, and final success rateCTools, authority, budgets, environment, and verifier

A) Model name, run date, and the final average score

B) Only the task list, attempt count, and final success rate

C) Tools, authority, budgets, environment, and verifier

Answer and explanation

Correct answer: C. These components define available actions and the trajectory. Without them the result cannot be reproduced or attributed to model versus system changes.

Question 18.14.1

18.14. Practice: repair from an observation, then recover after failure

The service committed an effect but its reply was lost. How does the SQLite experiment replay it?

One correct optionAWith the same operation id and argumentsBWith a fresh id every timeCUntil the counter doubles

A) With the same operation id and arguments

B) With a fresh id every time

C) Until the counter doubles

Answer and explanation

Correct answer: A. The operation record and effect commit in one transaction. The same id returns the existing result; changed arguments are rejected.