What you'll learn
Quick Answer
Prompting changes the instructions you send with each request. RAG adds a retrieval step that pulls relevant documents into the prompt at query time. Fine-tuning retrains the model's weights on your examples to shape its behaviour and output format. Start with prompting, add RAG when the model lacks knowledge, and fine-tune when you need consistent style or structure rather than new facts.
What each one actually changes
The three techniques operate at different layers of the stack, and that difference is the whole basis for choosing between them.
Prompting touches nothing but the text you send. The system prompt, your instructions, and any worked examples all sit in the context window for that single call. No training runs, no extra infrastructure, and the model is byte-for-byte the same afterwards.
RAG, retrieval-augmented generation, adds a step before the model runs. A retriever searches an external index, pulls the passages most relevant to the question, and those passages are pasted into the prompt. OpenAI's accuracy guide describes it as retrieving external documents to augment the prompt before generation; it does not modify model weights.
Fine-tuning continues training the model on your own dataset of input and output pairs. Hugging Face describes it as identical to pretraining except that you start from existing weights rather than random ones. The weights change, and the result is a new model checkpoint you host or call separately.
So prompting and RAG shape one request at a time and leave the model alone. Fine-tuning produces a different model.
Prompting: start here
OpenAI's accuracy guide is blunt about the order: prompting is typically the best place to start, and often the only method needed for tasks such as summarisation, translation, and code generation.
The levers are clear instructions, a few worked examples (few-shot), an explicit output format, and splitting a hard task into smaller steps. A vague extract the key points becomes return exactly three bullet points, each under 15 words, quoting the source figure where one exists. Iteration is instant because there is nothing to rebuild between attempts.
The limits are real. Everything you add lives in the context window and is re-sent, and paid for, on every call, so a 2,000-token instruction block is a permanent tax on throughput and latency. Long instructions also compete with the user's actual input for the model's attention. And prompting cannot supply information the model was never trained on.
The practical rule from the same guide: squeeze as much accuracy out of basic methods as you can before reaching for anything more complex. Most teams stop here, and for well-defined tasks they are right to.
RAG: when the model lacks knowledge
Reach for RAG when the failure is missing context: the answer was not in the training data, the model's knowledge is stale, or the information is private to your organisation.
The mechanics are a pipeline. Split your documents into chunks, convert each chunk to a vector with an embedding model, and store the vectors in an index. At query time, embed the question, retrieve the closest chunks by similarity, and place them in the prompt. Hugging Face's RAG documentation frames this as pairing the model's parametric memory with an external non-parametric memory, and notes that you can update knowledge by changing the index instead of retraining.
The gotcha: retrieval quality is the ceiling on answer quality. If chunks are too coarse, or the embedding model is a poor fit for your jargon, the model receives irrelevant passages and answers from them confidently. RAG can even hurt when misapplied. OpenAI's Icelandic-language case study found that bolting RAG onto a model that had already been fine-tuned for the task dropped accuracy by four points. The lesson they draw is to match the tool to the actual failure.
Fine-tuning: behaviour, not facts
The most common mistake is fine-tuning to teach the model new information. OpenAI's guide states it plainly: fine-tuning does not add new knowledge, it teaches consistency and behaviour.
What it is genuinely good for: enforcing a rigid output structure, locking in a house tone of voice, lifting accuracy on one narrow task, and efficiency. Once the behaviour is in the weights, prompts get shorter because you stop re-explaining the rules on every call. A training example is a full input paired with the exact output you want back, and you typically need hundreds of them, cleaned and consistent.
Parameter-efficient methods make this cheaper. Hugging Face's PEFT library, and LoRA in particular, fine-tune only a small set of extra parameters while freezing the rest of the model, which sharply cuts compute and storage while staying close to full fine-tuning quality.
The costs that bite later: building and maintaining that dataset, the risk of the model getting worse at things outside your examples, and retraining every time the base model is upgraded. Any facts you baked in at training time still go stale on their own schedule.
A decision path
Take a support bot that answers from your product documentation.
Before: the team fine-tunes on a year of resolved support tickets. Answers sound on-brand but drift the moment the docs change, and every documentation update needs another training run to stay current. Wrong tool, because the problem was knowledge, not behaviour.
After: prompting plus RAG over the live documentation. Edit a page, and the next answer reflects it with no retraining. If the tone still reads too casual for your brand, then add a small fine-tune, scoped only to voice and formatting, on top of the RAG setup.
The order that holds up in practice: establish a baseline with prompting alone and measure it. If it fails because the model is missing context, add RAG. If it fails because the behaviour or format is inconsistent, fine-tune. Combine the two only once each has been pushed as far as it goes on its own. Reaching for fine-tuning first is how projects lose a month.