A base model has broad skills but not your behavior. Fine-tuning updates the model on your task data so it adopts a style, format, or capability by default. Modern parameter-efficient fine-tuning (LoRA, QLoRA, adapters) does this by training a tiny set of added weights instead of the whole network, so it is cheap, fast, and swappable.
Fine-tuning continues the training of a pretrained model on a smaller, task-specific dataset so its weights shift toward the behavior you want. Where prompting steers a frozen model at inference time, fine-tuning bakes a pattern into the weights themselves, so the model produces the right tone, format, or decision without you re-explaining it in every prompt.
Every weight is updated. Most powerful and most expensive - you need the full model in memory, big GPUs, and a fresh copy of all parameters per task.
The base weights are frozen; you train a small set of added parameters. Same effect for most tasks at a fraction of the compute and storage.
Best for teaching behavior: consistent style, strict output formats, tool-use conventions, or a narrow classification skill - not fresh facts.
| Piece | What it does | Common choices |
|---|---|---|
| Base model | The pretrained network you adapt | Llama, Mistral, Qwen, small SLMs |
| Dataset (SFT) | Instruction / response pairs teaching the target behavior | JSONL, chat-format, preference pairs |
| PEFT method | How you inject trainable parameters | LoRA, QLoRA, adapters, prefix tuning |
| Trainer | Runs the optimization loop over the data | trl SFTTrainer, Axolotl, Unsloth |
| Adapter artifact | The small trained weights you ship and serve | LoRA weights (tens of MB), registry |
Full fine-tuning of a large model means updating billions of parameters, storing a full copy per task, and burning serious GPU hours. PEFT keeps the base frozen and trains a tiny fraction of new weights, which changes the economics entirely.
LoRA trains well under 1% of the parameters, so it fits on a single GPU. QLoRA quantizes the frozen base to 4-bit, shrinking memory further so large models fine-tune on modest hardware.
An adapter is tens of megabytes, not tens of gigabytes. You keep one base model and hot-swap many adapters, one per customer, tone, or task.
Because the original weights are frozen, the model keeps its general abilities and you sidestep the worst of catastrophic forgetting compared to updating everything.
Once a format or style lives in the weights, you stop paying for long few-shot prompts on every call - lower latency and token cost at inference.
Same idea, four ways to picture it, so it clicks whoever you are.
Prompting is giving worked examples on the exam sheet every single time. Fine-tuning is practicing hundreds of problems beforehand so the method becomes second nature and you no longer need the examples in front of you.
LoRA is like a patch or plugin layered on a shared library. You do not fork and recompile the whole dependency; you ship a small diff that overrides behavior, and you can load a different patch per deployment.
It is an experienced hire taking a short specialization course. They already know the field; a focused course teaches your house style and process, so they stop needing a manual re-read for every task.
A skilled musician rehearsing one setlist until it is muscle memory. The general talent was already there; the practice locks in the specific songs so the performance is automatic.
Fine-tuning has two halves: an offline preparation and training pipeline that produces an adapter, and an inference path where that adapter runs alongside the frozen base to change its behavior.
Curate clean instruction / response pairs, split train and eval, then run supervised fine-tuning. LoRA freezes the base and learns two small low-rank matrices per target layer.
At serving time the LoRA weights are added into the frozen base activations. The math is W + (B·A)·scale, so the tiny learned matrices nudge the original model.
flowchart LR
subgraph Train["📚 Offline training"]
D["📄 SFT dataset"] --> PP["🧹 Clean + format"]
PP --> BASE["🧊 Frozen base"]
BASE --> LR["🪶 LoRA matrices A,B"]
LR --> AD["📦 Adapter artifact"]
end
subgraph Serve["🔌 Online serving"]
REQ["❓ Prompt"] --> RT["🧠 Base + adapter"]
AD --> RT
RT --> OUT["✅ Tuned response"]
end
from peft import LoraConfig
from trl import SFTTrainer, SFTConfig
lora = LoraConfig(
r=16, # rank of the low-rank update
lora_alpha=32, # scaling factor for the adapter
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM",
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)
args = SFTConfig(
output_dir="out/brand-tone-lora",
num_train_epochs=3,
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
learning_rate=2e-4, # higher LR is fine for LoRA
lr_scheduler_type="cosine",
warmup_ratio=0.03,
bf16=True,
eval_strategy="steps",
eval_steps=50,
)
trainer = SFTTrainer(
model="mistralai/Mistral-7B-v0.3",
args=args,
peft_config=lora, # freezes base, trains only LoRA
train_dataset=train_ds, # chat-format instruction pairs
eval_dataset=eval_ds,
)
trainer.train()
trainer.save_model() # saves ~tens of MB of adapter weightsNotice only the LoRA matrices carry gradients - the 7B base is frozen. That is why the saved artifact is tiny and a single GPU is enough, while the eval split guards against overfitting during the run.
Three views: the offline training pipeline, LoRA adapters at inference, and how you choose between prompting, RAG, and fine-tuning.
sequenceDiagram
autonumber
participant Eng as 👩🔧 ML engineer
participant Data as 🧹 Data pipeline
participant Tr as 🏋️ Trainer
participant Ev as 🧪 Eval harness
participant Reg as 📦 Model registry
Eng->>Data: Collect instruction and response pairs
Data->>Data: Clean, dedupe, split train and eval
Data->>Tr: Formatted SFT dataset
loop each epoch
Tr->>Tr: Update LoRA matrices, base frozen
end
Tr->>Ev: Candidate adapter
Ev-->>Tr: Scores on held out eval
alt quality above bar
Tr->>Reg: Register versioned adapter
Reg-->>Eng: Adapter ready to serve
else regressed or overfit
Ev-->>Eng: Reject, adjust data and hyperparams
end
sequenceDiagram
autonumber
participant U as 👤 User
participant Srv as 🖥️ Serving runtime
participant Base as 🧊 Frozen base weights
participant Lora as 🪶 LoRA adapter
participant Dec as 🔤 Decoder
U->>Srv: Prompt
Srv->>Base: Forward pass through layers
Base->>Lora: Layer activations
Lora-->>Base: Add low rank update B times A times scale
Note over Base,Lora: Base unchanged, adapter nudges output
Base->>Dec: Adjusted logits
Dec-->>Srv: Tokens in the tuned style
Srv-->>U: Response with learned format
sequenceDiagram
autonumber
participant Team as 🧭 Team
participant P as ✍️ Prompt engineering
participant R as 📚 RAG
participant F as 🪶 Fine-tuning
Team->>P: Try a better prompt and few shot examples
alt prompt alone is good enough
P-->>Team: Ship it, cheapest option
else needs fresh or private facts
Team->>R: Add retrieval over knowledge base
R-->>Team: Grounded, always current answers
else needs consistent behavior, format or style
Team->>F: Fine-tune with LoRA on task data
F-->>Team: Behavior baked into the weights
Note over Team,F: Often combine RAG for facts plus a tuned adapter for style
end
From a base model and a goal to a served, monitored adapter - the whole journey in order.
Write down exactly what should change: a tone, a strict JSON format, a classification, a tool-use convention. If it is really missing knowledge, stop and use RAG instead.
Try system prompts and few-shot examples. Fine-tune only when prompting cannot get consistent enough results or the prompt is too long and costly.
Assemble representative input and ideal-output pairs from logs, human writers, or a stronger model. Cover the edge cases you care about.
Deduplicate, fix bad labels, and convert to chat or instruction format. A few hundred clean examples beat thousands of noisy ones.
Reserve a representative slice the model never trains on, so you can measure real generalization and catch overfitting honestly.
Choose a base model sized to your latency and quality needs, then set LoRA rank, alpha, target modules, and learning rate. Use QLoRA if memory is tight.
Run a few epochs while watching train and eval loss. The base stays frozen; only the adapter learns. Stop before eval loss turns back up.
Compare the adapter to the base model on task metrics and on general capability, so you confirm a gain without regressing everything else.
Run a broad capability suite to detect catastrophic forgetting. If general skills dropped, lower the learning rate or mix in general data.
Store the adapter with its data snapshot, config, and eval report in a registry so the result is reproducible and auditable.
Load the adapter onto the shared base, optionally serving many adapters at once, and watch production metrics to trigger the next data and retrain cycle.
Most fine-tuning failures come from the data and the goal, not the optimizer. These are the usual culprits.
Trying to teach fresh facts by fine-tuning is slow, expensive, and goes stale. Facts belong in retrieval; fine-tune for behavior, format, and style instead.
Noisy labels, duplicates, and inconsistent formatting teach the wrong pattern. A small, clean, consistent dataset beats a large messy one every time.
Too many epochs or too high a learning rate memorizes the training set. Eval loss rising while train loss falls is the classic warning sign.
Aggressive updates on a narrow task erode general ability. Keep learning rates modest, prefer PEFT, and mix in some general examples.
Rank too low underfits the behavior; targeting the wrong modules or a huge alpha destabilizes training. Tune rank, alpha, and target modules deliberately.
Judging by a few cherry-picked prompts hides regressions. Without a held-out set and a capability suite you cannot tell improvement from luck.
Track training health and downstream quality separately, so you know whether to fix the run or the data.
| Metric | What it tells you | Good sign |
|---|---|---|
| Eval loss / perplexity | How well the adapter fits held-out data | Falls then flattens, no upturn |
| Task success rate | Does output meet the target format or label? | High: the behavior actually learned |
| Win rate vs base | Preference of tuned vs base outputs (human or judge) | Clearly above 50 percent |
| Capability retention | General benchmark scores after tuning | Unchanged: no catastrophic forgetting |
| Train vs eval gap | Distance between training and eval loss | Small: not overfitting |
| Inference cost / latency | Serving overhead of the adapter per request | Within SLA, prompts got shorter |
Three representative patterns showing fine-tuning in production-style use.
A marketing team wants every generated snippet to match a precise brand voice and layout without a giant style prompt on each call.
A team runs a large model to label support tickets, but the per-call cost and latency are too high at scale.
An agent must emit strict JSON tool calls, but the base model occasionally drifts, breaking the downstream parser.
Adaptation is moving from a heavyweight training project toward fast, composable, preference-aware tuning.
Runtimes that host many LoRA adapters on one base and route each request to the right one, making per-customer or per-task models economical at scale.
Lightweight alignment methods like DPO and its successors tune models toward preferred behavior directly from comparison data, without a separate reward model.
Advances beyond QLoRA keep shrinking the memory and time needed, pushing fine-tuning of large models onto commodity hardware.
The winning pattern pairs retrieval for facts with a tuned adapter for behavior, rather than treating them as either-or choices.
Research into merging and stacking adapters points to mixing learned skills at serve time, closer to modular, composable capabilities.
Synthetic data generation and automatic filtering make the dataset - the real bottleneck - faster to build and easier to keep clean.