Short answer
Fine-tuning means taking a model that has already been pre-trained on a huge general dataset and training it a little more on a smaller, specific dataset so it gets better at a particular task, style or format. Modern methods such as LoRA and QLoRA train only a tiny set of extra weights, so a 7B or 8B model can be fine-tuned on a single 16 to 24 GB GPU. Fine-tuning is best for teaching behaviour and format; for adding up-to-date facts, RAG is usually the better tool.
How fine-tuning fits into model training
Most language models go through several stages:
1. Pre-training: the model learns language and general knowledge by predicting the next token over trillions of tokens. This is the expensive part (see AI training vs inference). 2. Supervised fine-tuning (SFT): the model is trained on examples of prompts and ideal responses, which turns a raw text predictor into a helpful assistant. 3. Preference tuning: methods such as RLHF (reinforcement learning from human feedback) or DPO (direct preference optimization) teach the model which of two answers people prefer.
When companies and developers talk about fine-tuning, they usually mean doing step 2, and sometimes step 3, again on their own data, starting from an existing open model or through a provider's fine-tuning API.
What fine-tuning is good for
- Consistent format: always returning valid JSON, a specific report layout or a fixed tone.
- Domain style and terminology: legal drafting, medical coding conventions, a brand voice.
- Narrow tasks: classification, extraction, routing or tagging, where a small fine-tuned model can match a much larger general model.
- Shorter prompts: behaviour you would otherwise describe in a long system prompt gets built into the model, saving tokens.
- Smaller, cheaper models: fine-tuning a small model on outputs from a large one (distillation) cuts serving costs.
What fine-tuning is not good for
- Adding fresh facts: knowledge learned by fine-tuning is hard to update and the model may still hallucinate details. Use RAG for documents, prices and anything that changes.
- Citing sources: a fine-tuned model cannot point to where a fact came from.
- Fixing a weak base model: fine-tuning sharpens existing abilities more than it creates new ones.
Fine-tuning vs RAG vs prompt engineering
| Prompt engineering | RAG | Fine-tuning | |
|---|---|---|---|
| What changes | The instructions | The information supplied | The model's weights |
| Best for | Quick behaviour changes | Up-to-date or private knowledge | Style, format, narrow tasks |
| Setup effort | Minutes | Days (index, embeddings, retrieval) | Days to weeks (data, training, evaluation) |
| Needs GPUs | No | For embeddings and serving only | Yes, for training |
| Updating | Edit the prompt | Re-index documents | Retrain |
| Cites sources | No | Yes | No |
A practical order is: try better prompts first, add RAG if the model lacks knowledge, and fine-tune when you need consistent behaviour that prompting cannot deliver. Many production systems combine RAG and fine-tuning.
Full fine-tuning vs parameter-efficient fine-tuning
Full fine-tuning
All of the model's weights are updated. This gives the most control but needs the same memory as training: about 16 bytes per parameter with mixed precision and the Adam optimizer, so a 7B model needs over 100 GB plus activations. That means several data center GPUs, and you must store a full new copy of the model for every fine-tune.
LoRA (Low-Rank Adaptation)
LoRA freezes the original weights and adds small trainable matrices to selected layers. Instead of updating a large weight matrix directly, it learns two thin matrices whose product represents the change. The LoRA paper reported cutting the number of trainable parameters for GPT-3 by 10,000 times and GPU memory needs by about 3 times, with quality on par with full fine-tuning. The resulting "adapter" is often only tens or hundreds of megabytes and can be swapped on and off the base model.
QLoRA
QLoRA goes further by loading the frozen base model in 4-bit precision and training LoRA adapters on top. Its authors fine-tuned a 65-billion-parameter model on a single 48 GB GPU. QLoRA is the most common way hobbyists and small teams fine-tune models today. For more on 4-bit formats, see LLM quantization.
How much GPU memory do you need?
Rough guide for fine-tuning language models (exact needs depend on sequence length, batch size and settings):
| Model size | QLoRA | LoRA (16-bit base) | Full fine-tuning |
|---|---|---|---|
| 1B to 3B | 6 to 8 GB | 12 to 16 GB | 24 to 48 GB |
| 7B to 8B | 12 to 16 GB | 24 GB+ | 100 GB+ (multi-GPU) |
| 13B to 14B | 16 to 24 GB | 40 GB+ | 200 GB+ |
| 70B | 48 GB+ | 160 GB+ | Over 1 TB (cluster) |
A 24 GB card such as the RTX 4090 or RTX 5090 covers most hobby fine-tuning; see the best GPU for local LLMs. For bigger jobs, renting an H100 or A100 by the hour is usually cheaper than buying; check today's rates in our GPU cloud price tracker.
The fine-tuning process, step by step
1. Define the task and success metric. What exactly should improve, and how will you measure it? 2. Collect examples. Hundreds to a few thousand high-quality prompt-response pairs often beat a larger, messy dataset. 3. Format the data in the chat template your base model expects. 4. Choose a base model that already does reasonably well on the task. 5. Train with a library such as Hugging Face TRL and PEFT, Unsloth or Axolotl, or with a provider's fine-tuning API. 6. Evaluate on held-out examples the model never saw, and compare against the base model with a good prompt. 7. Deploy by merging the adapter or loading it at inference time, and keep monitoring.
Risks to watch
- Overfitting: the model memorises training examples and generalises poorly.
- Catastrophic forgetting: it gets worse at things it used to do well.
- Safety regression: research has shown fine-tuning can weaken a model's safety training, even with harmless data, so re-test safety behaviour.
- Licences: check the base model's licence allows your use.
Frequently asked questions
How many examples do I need to fine-tune?
For format and style, a few hundred good examples can be enough. Complex tasks may need thousands. Quality and consistency matter more than volume.
Can I fine-tune ChatGPT or Claude?
OpenAI offers fine-tuning for several of its models through its API, and some providers offer fine-tuning on their cloud platforms. Open-weight models such as Llama, Mistral, Qwen and Gemma can be fine-tuned on your own hardware.
Is fine-tuning the same as training from scratch?
No. Training from scratch builds all of a model's knowledge from random weights and costs millions of dollars for large models. Fine-tuning starts from a capable model and adjusts it, usually in hours.
Does fine-tuning make a model smarter?
It makes a model better at the specific task you trained it on. It rarely improves general reasoning and can sometimes reduce performance on unrelated tasks.
Sources and further reading
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (2021)
- Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (2023)
- Ouyang et al., Training language models to follow instructions with human feedback (InstructGPT, 2022)
- Rafailov et al., Direct Preference Optimization (2023)
- Qi et al., Fine-tuning Aligned Language Models Compromises Safety (2023)
- Hugging Face, PEFT documentation




Leave a Reply