- Embedding: turns semantic meaning into a position in space — the more similar the meaning, the closer the positions. The indexing side and the query side must use the same model.
- Retrieval is a "wide-to-narrow funnel": vector search (by meaning) paired with keyword search (by literal text) running in parallel → filter by permissions and timeliness → precision reranking. Vectors excel at semantics but not at exact literal matches, so in most scenarios the two tracks complement each other and work best together.
- ANN (Approximate Nearest Neighbor): trades a small amount of accuracy for a massive speed gain. HNSW is the default choice (the graph must fit in memory); for very large datasets, use IVF or DiskANN.
- Inference has two phases: prefill reads the entire input in parallel (compute-bound); decode generates one token at a time (memory-bandwidth-bound). Total wait time is usually dominated by decode.
- KV cache: trades GPU memory for eliminated redundant computation, bringing quadratic computation down to near-linear; prompt caching further allows reuse of identical prefixes across requests.
- System communication: the backend / orchestration layer is the only party that actively initiates actions; the vector database, model services, reranking service, and other downstream components never communicate with each other directly — all coordination runs through the backend.
- Compliance: the single most sensitive moment is when confidential content is assembled into a prompt and sent to a cloud-hosted model; logs are the most easily overlooked plaintext retention point.
- Agents: Thought → Action → Observation in a continuous loop; the model is the brain, the harness is the body — the quality of the harness matters as much as the model.
- Generational distinction: when the loop rules are hard-coded, it's classic RAG; when "whether to search again, and how" is handed to the model to judge on the fly, that's agentic. What changes is not the components, but who's in command.
- The full landscape: RAG is just one of many LLM usage patterns. Missing knowledge → supply material (modest volume with long context, large volume with RAG); missing autonomy → agent; missing capability or behavior → fine-tuning. Which form to choose depends on what the task is missing.
You've reached the end. The whole guide is here, free, and will stay that way.
If you'd rather have it off the browser: 43 English pages / 30 Chinese pages, typeset as PDF and EPUB, 7 original diagrams, 11 footnotes to primary sources — four files in one download, $9.