Module 18. Agents and Tool Use

Solution. Without reuse, step $i$ processes full history $3000+1500i$. The arithmetic progression gives $20\cdot3000+1500(0+\cdots+19)=345000$ tokens. Reuse processes initial prefix once and the 19 increments preceding calls once: $3000+19\cdot1500=31500$, a 10.9524 ratio of processed input positions, excluding the growing cost of reading old KV. Final KV is $33000\cdot128$ KiB = 4.0283 GiB.

The conditional $2PT$ prefill estimate is $2\cdot8.03\text{B}\cdot1500/(495\text{ TFLOP/s})=48.667$ ms. Decode is $90/209.4=429.799$ ms and tool latency 300 ms. Total is 778.466 ms, with decode at 55.21%. This is a conditional scenario without queueing or runtime overhead.

Solution. Without retry, every step must succeed: $0.98^{20}=0.66761$ and $0.95^{20}=0.35849$. With one independent retry, per-step success is $1-(1-p)^2$, or 0.9996 and 0.9975; raising these to the twentieth power gives 0.99203 and 0.95117. Requiring $p^{50}=0.9$ gives $p=0.9^{1/50}=0.997895$.

Perfect partitioning makes worker time $400/20=20$ minutes and critical path $5+20+20=45$ minutes. Speedups are $400/45=8.89\times$ against useful work and $(5+400+20)/45=9.44\times$ end-to-end. Even infinitely many workers leave 25 serial minutes, imposing 16× and 17× ceilings. KV is $20\cdot30000\cdot128$ KiB = 73.242 GiB. Retry independence and perfect partitioning are strong assumptions.

Solution. The JSON path uses $5\cdot90=450$ tokens and five model/tool round trips; CodeAct uses 160 tokens and one. The ratios are $450/160=\mathbf{2.8125\times}$ and $5/1=\mathbf{5\times}$. The value $150000/(2\cdot8.03\text{B})=9.33998\cdot10^{-6}$ compares 150K simple mask checks with rough $2P$ arithmetic for one model token; the work units differ, so the ratio is not a measured time fraction.

The reproducible trace first calls lookup(sensor_bach) and receives tool_error; it corrects the key to sensor_batch and obtains 13.96; then it calls the calculator with 13.96 * 2, obtains 27.92, and returns Answer: 27.92. Event kinds are assistant_action → tool_error → assistant_action → tool_result → assistant_action → tool_result → assistant_final → verification_pass, with three registry invocations, one error, and status completed. The invariant is that the failure becomes an observation rather than uncontrolled execution.

Demo for this project exercise: English project-demo notebook for Module 18, section “Demo 1”. It shows a compact reproducible example on small inputs, not a completed solution to the full-scale project.

Demo for this project exercise: English project-demo notebook for Module 18, section “Demo 2”. It shows a compact reproducible example on small inputs, not a completed solution to the full-scale project.