After the AI customer support bot went live, the team received three distinct kinds of complaints.

The first kind: it can't answer. A customer asked what changed in the new release published last week, and the bot said it didn't know — the release notes hadn't been added to the knowledge base yet.

The second kind: the answer is correct, but unreadable. The content was accurate, but a wall of text with no bullet points, no source citations, and a tone that drifted unpredictably. The team wanted it to always lead with a conclusion, then list steps, and append document links at the end.

The third kind: fluent prose, completely beside the point. A semiconductor customer asked a question loaded with industry jargon, and the bot answered eloquently — about the entirely wrong thing. It fundamentally didn't understand what those terms mean in that industry.

All three sound like "the AI isn't working," but the root causes are entirely different: the first is a knowledge gap, the second is a behavior gap, the third is a comprehension gap. Any general-purpose model deployed for a specific use case will show cracks at these three seams; the work of closing those gaps and turning a general-purpose base model into a system that's genuinely usable for a given scenario is called model adaptation.

Only Two Intervention Points: the Input, or the Weights

Three kinds of gaps, three kinds of solutions. The simplest way to tell them apart is to ask: what does the solution actually change — does it leave the model untouched and only change what's placed in front of it each time, or does it go in and alter the model itself?

One approach intervenes on the input: the model's parameters stay completely unchanged; you just feed in the right material with each question. The RAG from Chapter One is this type.

The other approach intervenes on the weights: it directly modifies the model's internal parameters. Parameters (also called weights) are the enormous collection of numbers inside the model — often billions or hundreds of billions of them — in which all of the model's abilities are stored: which words it recognizes, how it habitually phrases things, how it decides what word to write next given the preceding text. "Training" is the process of iteratively adjusting these numbers. Once this kind of change is done, it's baked into the model, and every subsequent answer carries it. Fine-tuning and continued pretraining fall into this category.

Three Approaches, Each in Its Place

Note: what does "retrain" mean? It means running the training process again — preparing a fresh batch of example data, having the model learn on that data for another round, and producing a new version of the weights (or a new version of the adapter). It's not something you can do as casually as changing a line of configuration, but it's also not as out of reach as many people imagine.

When it comes to fine-tuning, the compute barrier fell long ago. Thanks to techniques like LoRA that train only a small fraction of the parameters, fine-tuning a 7–8B model in 2026 costs roughly a few dollars to low tens of dollars in GPU rental, taking two to four hours; using a managed service with pay-per-use pricing, a typical job runs to a few tens of dollars.¹²

The real effort is before and after training. The industry consensus is: GPUs are no longer the main expense — data preparation and evaluation are. Taking a fine-tuning project from start to finish (curating examples, cleaning and labeling data, iterating on experiments, building an evaluation set) typically lands in the range of a few thousand to tens of thousands of dollars, most of which goes to people and data, not compute. So the burden of "to change it you have to retrain" is mainly not about how expensive a single training run is, but that every change means redoing the entire preparation-and-validation cycle.

Continued pretraining is an entirely different order of magnitude: it consumes massive domain corpora and large-scale compute, and typically requires a dedicated team and infrastructure.

Using the wrong tool is a common trap, and the most typical version is: wanting the model to learn new facts, so you fine-tune it. Research has repeatedly shown that models struggle to learn new factual knowledge through fine-tuning — training examples that introduce new knowledge are learned noticeably more slowly than others; and by the time this new knowledge is finally absorbed, the model's tendency to fabricate content actually increases.³ The conclusion: factual knowledge is primarily acquired during pretraining; what fine-tuning teaches the model is to more effectively deploy the knowledge it already has. So when the model can't answer something (the first kind of complaint at the start of this chapter), the right fix is to retrieve the material and put it in front of the model, not to bake the material into the weights.

The reverse is equally true: RAG will not change the model's behavior, tone, or output format. If the model is verbose, RAG can't fix it; if the format is wrong, RAG can't fix it either. The second kind of complaint can only be addressed through fine-tuning.

In Practice, They're Usually Stacked

These three approaches aren't pick-one. A common combination in production systems is: use an adapter (a lightweight fine-tuning add-on module that doesn't modify the entire model but hangs a small block of new parameters alongside it — the representative technique is LoRA) to handle tone and citation formatting, while RAG manages knowledge — the adapter gets updated once a quarter, the knowledge base is updated continuously. The principle can be summed up in one sentence: language through training, facts through retrieval; anything that changes should not be baked into the weights.

The order of operations matters too. The lowest-cost approach is to adjust the prompt; next comes RAG; fine-tuning only comes after that. The prevailing industry advice is: before deciding to fine-tune, make sure you already have an evaluation set, and that the "prompt + RAG" approach genuinely failed to pass it — otherwise you'll likely spend the money on training without actually solving the real problem.

Regardless of which path you take, the endpoint is the same place: the model's weights (the base model, plus any stacked adapter) → the inference service (the process where the model actually "runs" and generates an answer) → the answer. The next chapter covers how this machinery actually works.