RAG vs. fine-tuning: which does your product actually need?

Two different bets

Retrieval and fine-tuning get bundled into “we’ll add AI” and they are not the same bet. Retrieval searches material you already have and answers from it. Fine-tuning changes how a model behaves, so a style, a format, or a judgement shows up even when the facts are new. Mixing them up is how a project spends its budget on the wrong problem.

The chalkboard we use is a three-row table. Fresh data. Cites its sources. Tone and format. The first two rows almost always belong to retrieval. The third is the only row where training starts to earn its place. If you cannot point at the row you are buying, you are not ready to choose.

If the answer has to cite a document you already maintain, start with retrieval. Do not train a model to memorise a wiki.

When retrieval is enough

Policies change. Prices change. Yesterday’s ticket is not today’s answer. A retrieved passage can be swapped the hour the source changes, and the reply can show where it came from. That is the product property clients actually want when they say “it should know our business.”

Retrieval also gives you a decline. No passage, no answer, hand it to a person. A tuned model that has “learned” the handbook will still speak when the handbook does not cover the question. For support, quotes, and anything with a compliance trail, that habit is the defect.

retrieve.ts

Copy

const hits = await index.query({
  vector: await embed(question),
  topK: 8,
  filter: { product: ticket.product },
});
const context = rerank(hits, question).slice(0, 3);

When training is the point

Train when the value is the behaviour, not the fact. A consistent structure for a clinical note. A tone a brand has already approved, applied to text the model has never seen. A classifier that must sort your own categories. Those are format problems. Stuffing last quarter’s PDF into the weights will not keep them fresh, and it will make them harder to audit.

Even then we keep the facts outside the model. Tune the shape of the answer. Retrieve the content. The two techniques are allowed to sit together. They are not allowed to be a substitute for each other.

Write the decision down

Before anyone opens a notebook, we write one paragraph: what must be fresh, what must be cited, and what must sound a certain way. The paragraph picks the approach. If a vendor demo cannot honour it, the demo is the wrong product, however fluent it is.

“Search what you already know. Train only the part that has to feel the same every time.”M Jhanzaib, CEO