Full fine-tune vs PEFT
PEFT is the default; full fine-tuning is for big shifts with data and hardware. The memory math explains why — and why QLoRA exists. Plus merging and multi-adapter serving.
Learning objectives
- Compare full fine-tuning and PEFT on memory, cost, and when each is right.
- Do the memory math for full fine-tuning (why it's so heavy).
- Explain adapter merging and multi-adapter serving.
- Choose an approach for a given constraint.
code/ft4-full-vs-peft/ in the course, with a README. Run the scripts or copy the configs directly.When you'd go full intermediate
PEFT (LoRA/QLoRA) is the default — cheaper, faster, swappable. Full fine-tuning updates every weight and is worth it only when you need a large behavior shift, have lots of quality data, and have the hardware. For most applied work, PEFT is enough.
| Full fine-tune | PEFT (LoRA/QLoRA) | |
|---|---|---|
| Params trained | all (billions) | <1% |
| Memory | very high (weights+grads+optimizer) | low (QLoRA: one GPU) |
| Artifacts | a whole new model | a small adapter |
| Swappable | no | yes — many adapters, one base |
| Use when | big shift + data + hardware | almost always |
The memory math intermediate
Full fine-tuning holds, per parameter: the weight, its gradient, and optimizer state (Adam keeps two moments). That's roughly weights × (2 + optimizer) — often >16 bytes/param in mixed precision. A 7B model can need 100+ GB; that's why full tuning means multi-GPU, and why QLoRA (which trains <1% of params on a 4-bit base) fits on one card.
Merging and serving adapters advanced
A trained LoRA adapter can be merged into the base weights to produce a standalone model (simplest to serve), or kept separate and loaded at runtime — which lets one base serve many task adapters, hot-swapped per request.
merge.pyfrom peft import AutoPeftModelForCausalLM
# Load base + adapter, merge into one set of weights, save a standalone model.
model = AutoPeftModelForCausalLM.from_pretrained("out/adapter", device_map="auto")
merged = model.merge_and_unload()
merged.save_pretrained("out/merged-model") # now serve it like any HF model (IC6/LM3)
After training you have a base model plus a separate adapter. This tiny script bakes the adapter into the base to produce one standalone model that's simplest to serve.
AutoPeftModelForCausalLM.from_pretrained("out/adapter", device_map="auto")loads the base model and your trained adapter together, in one step.device_map="auto"places it on whatever hardware you have.model.merge_and_unload()is the key line: it merges the adapter's changes into the base weights and unloads the separate adapter, leaving a single ordinary model.merged.save_pretrained("out/merged-model")writes that combined model to disk — from here it behaves like any normal Hugging Face model and can be served directly (FT6, IC6/LM3).
What the output means: A new folder out/merged-model appears containing a full standalone model — no adapter needed at serving time. It's bigger than the adapter alone but the simplest thing to deploy.
Try this: Merge when you serve one tuned behavior. If you have several adapters (support tone, code style, …), skip merging and keep them separate so one base model can hot-swap between them per request.
Exercise FT4.1 — Decide and defend
Context: The PEFT-vs-full decision is one you should be able to defend with numbers, and merging your adapter is the step that turns a tune into something servable in FT6.
Your task: For your task, decide PEFT vs full and justify it with the memory math and your data volume, then merge your FT3 adapter into a standalone model and confirm it loads and generates.
Requirements:
- State the PEFT-vs-full decision for your task
- Justify it with the per-param memory math and your data volume
- Merge the FT3 adapter into a standalone model
- Confirm the merged model loads and generates
- Note it's now ready to serve in FT6
💡 Hint: Reuse the merge step from the Professional rung; the defence is just the memory numbers you computed earlier applied to your own case.
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: PEFT beats full fine-tuning on nearly every practical axis — params trained, memory, artifact size, and swappability — so full FT is the justified exception, not the default.
Your task: Encode the lesson's full-vs-PEFT comparison table and print the row you'd reach for by default.
Requirements:
- Rows compare params trained (all vs <1%), memory, artifact, and swappability
- Print the table in aligned form
- Declare PEFT (LoRA/QLoRA) as the default
- Make the case that PEFT wins on every practical axis
💡 Hint: Every column favours PEFT except raw capacity — that lopsidedness is the whole point of the row you pick.
Show solution
The comparison table as data, with the default called out:
rows = {
"params trained": ("all (billions)", "<1%"),
"memory": ("very high", "low (QLoRA: one GPU)"),
"artifact": ("whole new model","small adapter"),
"swappable": ("no", "yes — many adapters, one base"),
}
print(f"{'dimension':16}{'full':18}PEFT")
for k,(full,peft) in rows.items():
print(f"{k:16}{full:18}{peft}")
print("\nDEFAULT: PEFT (LoRA/QLoRA) — cheaper, faster, swappable")PEFT wins on every practical axis for applied work; full fine-tuning is the exception you justify, not the default you assume.
Context: Full fine-tuning holds four things per parameter — the weight, its gradient, and Adam's two moments — and it's the last three, not the weight, that force multi-GPU.
Your task: Compute full-FT memory for a 7B model in mixed precision and show why it needs more than one card.
Requirements:
- Account weight 2 + grad 2 + Adam 4 + 4 ≈ 12 bytes/param
- Print totals for 1B, 7B, and 13B models
- Show 7B (≈78 GB) exceeds a single card
- Note weights are only ~1/6 of the cost; grads + optimizer dominate
- Frame this as the motivation for PEFT
💡 Hint: The weight itself is the cheap part — it's the gradient plus Adam's m and v that multiply the footprint by six.
Show solution
Sum the per-parameter memory the way the lesson describes it:
def full_ft_gb(params_b, w=2, grad=2, adam_m=4, adam_v=4):
bytes_per = w + grad + adam_m + adam_v # ~12 bytes/param, mixed precision
return params_b * bytes_per / 1024**3
for p in (1e9, 7e9, 13e9):
print(f"{p/1e9:.0f}B params -> {full_ft_gb(p):.0f} GB")
# 1B -> 11 GB 7B -> 78 GB 13B -> 145 GBWeights are only a sixth of the cost — gradients and Adam's two moments dominate. A 7B model needs ~78 GB, well past one card, which is the entire motivation for PEFT.
Context: QLoRA fits one GPU because freezing + quantizing removes the weight, gradient, and optimizer cost for 99.6% of the model.
Your task: Model full FT (all params at 12 bytes) against QLoRA (4-bit frozen base + <1% trained at full optimizer cost) for a 7B model and print the ratio.
Requirements:
- Full =
params × 12 bytes - QLoRA = frozen base at 0.5 bytes/param + adapter at 12 bytes/param on ~0.4% of params
- Print both totals and the ratio (~20× lighter)
- Attribute the win to freezing + quantizing the base
💡 Hint: Only the tiny trained fraction still pays the full 12-byte optimizer tax; the frozen 99.6% drops to half a byte each.
Show solution
Put both footprints in one function:
def compare(params_b, trained_frac=0.004):
full = params_b * 12 / 1024**3
base_4bit = params_b * 0.5 / 1024**3 # frozen, quantized
adapter = params_b * trained_frac * 12 / 1024**3 # only the adapter pays optimizer
qlora = base_4bit + adapter
return round(full,1), round(qlora,1), round(full/qlora,1)
full, qlora, ratio = compare(7e9)
print(f"full FT : {full} GB") # 78.2
print(f"QLoRA : {qlora} GB") # ~3.5
print(f"QLoRA is ~{ratio}x lighter") # ~22xFreezing + quantizing the base removes the weight/grad/optimizer cost for 99.6% of the model. QLoRA is roughly 20x lighter, which is the difference between a datacenter and a single consumer GPU.
Context: A trained adapter can be merged into the base (one standalone model, simplest and fastest to serve) or kept separate and loaded at runtime (one base serves many adapters).
Your task: Write a chooser that picks merged vs hot-swapped serving from the serving need.
Requirements:
- Choose HOT-SWAP for many adapters when latency isn't critical (one base, load per request)
- Choose MERGE for ops simplicity or latency-critical paths (one standalone model)
- Default a single-task deployment to MERGE
- Explain the tradeoff: one-model-simplicity vs many-adapters-on-one-base
💡 Hint: Count the adapters and check the latency bar: many + relaxed latency leans hot-swap, one-or-latency-critical leans merge.
Show solution
Decide the serving mode from how many tasks share the base:
def serve_mode(num_task_adapters, latency_critical, ops_simplicity):
if num_task_adapters >= 2 and not latency_critical:
return "HOT-SWAP adapters — one base serves many tasks, load per request"
if ops_simplicity or latency_critical:
return "MERGE — bake adapter into base -> one standalone model, simplest & fastest"
return "MERGE — default for a single-task deployment"
print(serve_mode(1, True, True)) # MERGE
print(serve_mode(5, False, False)) # HOT-SWAPMerge when you serve one task and want the simplest, fastest artifact; keep adapters separate when one base must serve many tasks and you can pay a small per-request load cost to hot-swap them.
Context: Merging folds an adapter into the base weights, producing a single standalone model with no runtime adapter load — the simplest path onto vLLM/TGI.
Your task: Show the lesson's real merge step so a teammate can produce a standalone model, using only documented PEFT APIs and labelling the required libraries.
Requirements:
- Load base + adapter together with
AutoPeftModelForCausalLM.from_pretrained - Fold with
merge_and_unload() - Write one standalone model with
save_pretrained - Result has no runtime adapter load
- Note the libs:
peft transformers
💡 Hint: merge_and_unload() is the whole step — it bakes the adapter into the weights so what you save is just a normal model.
Show solution
The real merge — needs pip install peft transformers:
from peft import AutoPeftModelForCausalLM
# Load base + trained adapter together, fold the adapter into the weights, save.
model = AutoPeftModelForCausalLM.from_pretrained("out/adapter", device_map="auto")
merged = model.merge_and_unload() # adapter baked into base weights
merged.save_pretrained("out/merged-model") # now serve like any HF model (IC6)merge_and_unload() collapses the adapter into the base so there is a single set of weights to serve — no runtime adapter loading, simplest path onto vLLM/TGI.
Context: PEFT is the default, but occasionally a project genuinely needs a full fine-tune; before spending the hardware, several conditions must all hold.
Your task: Write the checklist gate that must all be true before a full fine-tune is approved and print GO/NO-GO with the missing conditions named.
Requirements:
- Require a large behaviour shift, ample clean data, AND multi-GPU hardware
- Collect any unmet conditions into a list
- Return GO only when all are present
- Return NO-GO with the specific missing items otherwise
- Fall back to PEFT when the gate fails
💡 Hint: Treat it as an AND of three conditions — any single missing one flips the verdict to NO-GO and names itself.
Show solution
Gate full FT on all three conditions the lesson requires:
def full_ft_gate(big_behavior_shift, lots_of_quality_data, have_multigpu):
need = {"large behavior shift": big_behavior_shift,
"lots of quality data": lots_of_quality_data,
"multi-GPU hardware": have_multigpu}
missing = [k for k,v in need.items() if not v]
return ("GO — full FT justified" if not missing
else "NO-GO — PEFT instead; missing: " + ", ".join(missing))
print(full_ft_gate(True, True, True)) # GO
print(full_ft_gate(True, False, True)) # NO-GO ... missing: lots of quality dataFull fine-tuning is worth its cost only when a large behavior shift, ample clean data, and the hardware are all present. Miss any one and PEFT is the right call — the gate makes that explicit before the spend.
✓ Checkpoint — you can move on when you can…
- Compare full and PEFT on memory, cost, and fit.
- Do the full-fine-tune memory math.
- Merge an adapter and explain multi-adapter serving.
- Choose an approach for a stated constraint.
Knowledge check check yourself
Do the memory math: why does full fine-tuning of a 7B model need 100+ GB while QLoRA fits on one GPU?
Show answer
When should you merge a LoRA adapter into the base weights versus keep it separate, and why?