Module 17. Embeddings, Retrieval, and RAG — quizzes and answer checking · Modern LLMs

Question 17.4.1

17.4. BM25 and learned sparse retrieval: the branch that protects exact terms

Why does BM25 remain useful alongside dense retrieval?

One correct optionAIt preserves rare literal terms when the analyzer retains themBIt runs faster and is therefore needed as the first filtering stageCIt needs no training and transfers between languages unchanged

A) It preserves rare literal terms when the analyzer retains them

B) It runs faster and is therefore needed as the first filtering stage

C) It needs no training and transfers between languages unchanged

Answer and explanation

Correct answer: A. Rare strings may distinguish semantically similar documents. The analyzer must retain the required identifiers; neither branch is guaranteed to succeed on every query.

Question 17.5.1

17.5. Dense bi-encoders: learning a semantic space

How is the bi-encoder trained in the lecture’s contrastive example?

One correct optionABy recovering masked tokens inside query-document pairsBBy contrasting pairs: the relevant pair is encouraged to outscore in-batch negativesCBy regression onto relevance scores labelled by human assessors

A) By recovering masked tokens inside query-document pairs

B) By contrasting pairs: the relevant pair is encouraged to outscore in-batch negatives

C) By regression onto relevance scores labelled by human assessors

Answer and explanation

Correct answer: B. The positive pair should receive a higher relative score. Hard negatives teach fine distinctions, while false negatives supply an incorrect training signal.

Question 17.6.1

17.6. Matryoshka Representation Learning (MRL), quantization, and worked example A: the price of storing meaning

How does Matryoshka Representation Learning allow the embedding size to vary?

One correct optionAIt stores several versions of the vector and picks one per queryBIt trains a separate projection compressing the vector to a target sizeCLosses applied to several prefixes train useful nested representations

A) It stores several versions of the vector and picks one per query

B) It trains a separate projection compressing the vector to a target size

C) Losses applied to several prefixes train useful nested representations

Answer and explanation

Correct answer: C. At inference a slice v[:k] with renormalization suffices — no separate model is needed. Hence the working pattern: coarse search on the short representation, rescoring with the full vector.

Question 17.6.2

17.6. Matryoshka Representation Learning (MRL), quantization, and worked example A: the price of storing meaning

Ten million 1024-component vectors: what is the payload?

One correct optionA38.15 GiB in fp32 against 1.19 GiB in binary — a ×32 savingB38.15 GiB in fp32 against 9.54 GiB in binary — a ×4 savingC4.10 GiB in fp32: one byte for each component of the vector

A) 38.15 GiB in fp32 against 1.19 GiB in binary — a ×32 saving

B) 38.15 GiB in fp32 against 9.54 GiB in binary — a ×4 saving

C) 4.10 GiB in fp32: one byte for each component of the vector

Answer and explanation

Correct answer: A. MRL-256 plus int8 gives 2.38 GiB — a ×16 compromise. The table counts vectors only: index metadata, graph edges and replication are added separately on top.

Question 17.7.1

17.7. One vector, one pair, or a matrix: where query-document interaction happens

How does a cross-encoder differ from a bi-encoder?

One correct optionAIt trains on the same pairs but with a different loss functionBIt encodes the pair jointly; its final score cannot be computed before the queryCIt emits a fixed-length vector for every document in the corpus

A) It trains on the same pairs but with a different loss function

B) It encodes the pair jointly; its final score cannot be computed before the query

C) It emits a fixed-length vector for every document in the corpus

Answer and explanation

Correct answer: B. Joint attention operates within the pair and requires new computation for each candidate. This expands what the scorer can model; target-query evaluation determines quality. Pairs can be batched; scoring each one does not require a separate sequential call.

Question 17.7.2

17.7. One vector, one pair, or a matrix: where query-document interaction happens

What does late interaction do in ColBERT-style systems?

One correct optionAIt defers comparison until the full answer context has been assembledBIt computes similarity using only the last layers of both encodersCIt keeps a vector per token and matches them between query and document

A) It defers comparison until the full answer context has been assembled

B) It computes similarity using only the last layers of both encoders

C) It keeps a vector per token and matches them between query and document

Answer and explanation

Correct answer: C. Document token vectors are precomputed. The index is generally larger than a single-vector index; the ratio depends on dimensions, vector count, and compression.

Question 17.8.1

17.8. Worked example B: exact search and the arithmetic of ANN

At 10 GFLOP/s, what is the arithmetic-only estimate for 10M 1024-dimensional vectors?

One correct optionA2.048 s against 0.4096 ms for a 2,000-distance scenario — a factor of 5,000B20.48 ms: modern libraries fit comfortably inside that budgetC0.4096 ms — and that is the measured latency of an HNSW index

A) 2.048 s against 0.4096 ms for a 2,000-distance scenario — a factor of 5,000

B) 20.48 ms: modern libraries fit comfortably inside that budget

C) 0.4096 ms — and that is the measured latency of an HNSW index

Answer and explanation

Correct answer: A. 2Nd = 2·10⁷·1024 ≈ 2.05·10¹⁰ operations. This is a transparent model of operation count, not a benchmark: graph traversal, memory access and recall are outside it.

Question 17.9.1

17.9. ANN under the hood: three indexes for three resource constraints

What is HNSW's main trade-off?

One correct optionAHigh recall at the price of slow index rebuilds during insertsBFast search at the price of graph memory and an approximate answerCAn exact answer at the price of time growing linearly with corpus size

A) High recall at the price of slow index rebuilds during inserts

B) Fast search at the price of graph memory and an approximate answer

C) An exact answer at the price of time growing linearly with corpus size

Answer and explanation

Correct answer: B. The efSearch knob moves the recall-versus-latency trade-off, and it must be measured on your own index. Graph edges meanwhile occupy memory on top of the vectors themselves.

Question 17.10.1

17.10. Hybrid retrieval, reciprocal-rank fusion, and worked example F

How does RRF merge rankings without comparable raw scores?

One correct optionAIt brings scores to a common scale by normalizing per branch maximumBIt takes documents that landed in the top of both branches at onceCIt sums 1/(k + rank): the position matters, not the score value

A) It brings scores to a common scale by normalizing per branch maximum

B) It takes documents that landed in the top of both branches at once

C) It sums 1/(k + rank): the position matters, not the score value

Answer and explanation

Correct answer: C. RRF replaces original scores with rank contributions. Score calibration is another valid strategy, but RRF requires none and guarantees no quality improvement.

Question 17.10.2

17.10. Hybrid retrieval, reciprocal-rank fusion, and worked example F

Why is hybrid search not obliged to win on every metric at once?

One correct optionAAnother branch can promote irrelevant items above relevant onesBHybrid is tuned for recall and therefore always loses on precisionCHybrid metrics are incomparable with single-branch metrics by construction

A) Another branch can promote irrelevant items above relevant ones

B) Hybrid is tuned for recall and therefore always loses on precision

C) Hybrid metrics are incomparable with single-branch metrics by construction

Answer and explanation

Correct answer: A. Fusion changes candidate order. Average quality can improve or worsen; measure it on fixed queries and inspect distinct error classes.

Question 17.10.3

17.10. Hybrid retrieval, reciprocal-rank fusion, and worked example F

Two documents are relevant; one reaches the top five. What is Recall@5?

One correct optionA1 if it ranks firstB0.5C0.2

A) 1 if it ranks first

B) 0.5

C) 0.2

Answer and explanation

Correct answer: B. Recall divides retrieved relevant documents by all relevant documents, not by ranking length.

Question 17.11.1

17.11. Chunking: deciding what the index is allowed to retrieve

What semantic relationship can be lost when chunks are too short and have no overlap?

One correct optionAIndexing speed: short chunks simply produce many more of themBCoherence: a definition and its use land in two separate chunksCThe option to use a reranker, which needs chunks of equal length

A) Indexing speed: short chunks simply produce many more of them

B) Coherence: a definition and its use land in two separate chunks

C) The option to use a reranker, which needs chunks of equal length

Answer and explanation

Correct answer: B. A boundary may separate a definition from its use. Both pieces can remain retrievable while neither is sufficient alone. Overlap, section headings, or wider context are possible remedies to evaluate.

Question 17.12.1

17.12. The full cascade and worked example D: where request latency lives

What must a context assembler guarantee under a hard token budget?

One correct optionAThat every retrieved document enters the context at least in fragmentsBThat the document order matches the order of their relevance scoresCDeterminism and an explicit decision about what exactly was dropped

A) That every retrieved document enters the context at least in fragments

B) That the document order matches the order of their relevance scores

C) Determinism and an explicit decision about what exactly was dropped

Answer and explanation

Correct answer: C. Deterministic truncation is reproducible, but can remove necessary context. The teaching packer accepts whole chunks and records every skip; it counts the budget after citations are added.

Question 17.12.2

17.12. The full cascade and worked example D: where request latency lives

Why is the 1.658 s total not a rigorous lower bound?

One correct optionADecode and MFU are assumed; some costs are omittedBStage times can never be addedC2PT counts only KV

A) Decode and MFU are assumed; some costs are omitted

B) Stage times can never be added

C) 2PT counts only KV

Answer and explanation

Correct answer: A. It is a conditional serial budget; assumed rates are not hardware guarantees.

Question 17.15.1

17.15. The evolution of RAG: repair the measured failure, not the acronym list

How does agentic RAG differ from a retrieve-once pipeline?

One correct optionAThe model itself decides whether to search again and how to rephraseBIt uses several indexes instead of one shared vector storeCIt searches after generation in order to verify the produced answer

A) The model itself decides whether to search again and how to rephrase

B) It uses several indexes instead of one shared vector store

C) It searches after generation in order to verify the produced answer

Answer and explanation

Correct answer: A. Another search can use newly read evidence to refine the query. This may help with multi-hop tasks, but the benefit needs measurement. Call count, latency, and cost depend on the chosen trajectory.

Question 17.16.1

17.16. Worked example E: RAG or long context

A 200K-token document against a 3,712-token RAG context. What does the gap give?

One correct optionANothing in the limit: long context is processed in parallel anywayB53.88× in input length and the linear prefill estimate: 6.489 s versus 0.120 sC×53.88 on input, but prefill is the same — it does not follow length

A) Nothing in the limit: long context is processed in parallel anyway

B) 53.88× in input length and the linear prefill estimate: 6.489 s versus 0.120 s

C) ×53.88 on input, but prefill is the same — it does not follow length

Answer and explanation

Correct answer: B. The ratio 200000/3712 belongs to the assumed linear model. Quadratic attention, caching, and answer quality require separate treatment.

Question 17.17.1

17.17. Evaluation and the embedder passport

Which pair of metrics separates the retrieval and generation stages?

One correct optionAAnswer completeness and its factual correctness in a judge's opinionBRequest latency and the cost of processing it in tokensCRetrieval recall and the answer's groundedness in the retrieved context

A) Answer completeness and its factual correctness in a judge's opinion

B) Request latency and the cost of processing it in tokens

C) Retrieval recall and the answer's groundedness in the retrieved context

Answer and explanation

Correct answer: C. Low recall can come from chunking, representations, filters, or the index. At high recall, inspect context assembly and generation: fluent prose does not replace evidence.

Question 17.17.2

17.17. Evaluation and the embedder passport

Why reject a repeated ID in an nDCG ranking?

One correct optionAIts gain can be counted twiceBLogarithms are undefined for repeated stringsCMRR always becomes zero

A) Its gain can be counted twice

B) Logarithms are undefined for repeated strings

C) MRR always becomes zero

Answer and explanation

Correct answer: A. The ideal ranking contains each document once. Counting its gain repeatedly can inflate nDCG above one.

Question 17.14.1

17.14. Level 2: what the notebook makes reproducible

What does matching an excerpt to source offsets establish?

One correct optionAAnswer correctnessBLLM generation qualityCThe excerpt's provenance

A) Answer correctness

B) LLM generation quality

C) The excerpt's provenance

Answer and explanation

Correct answer: C. Exact text and a citation establish provenance; relevance and completeness need separate checks.