Fine-tuning · Chapter FT1

When fine-tuning wins

xt4 answered should you? (usually no). This track is the hands-on how, for when the answer is yes: style, strict format, latency, and narrow tasks.

⏱️ ~1 hour🧪 1 lab🎯 Beginner→Expert

🌱 Start here — from zero Fine-tuning, from scratch — before any training code, understand what fine-tuning is — and why the answer is usually "don't".

A base model knows a lot in general. Fine-tuning means continuing to train it a little on your examples so it picks up a specific style, format, or narrow skill. It's powerful but often overkill — prompting and RAG (giving the model context) solve most problems more cheaply. This section teaches you to tell when tuning is worth it, and then how to do it efficiently on modest hardware.

The words you'll hear (in plain terms):

Term	What it actually means
fine-tuning	further-training a model on your examples to change how it responds.
base vs instruct model	base = raw next-word predictor; instruct = already tuned to follow instructions. You tune from one of these.
LoRA / QLoRA	cheap fine-tuning that trains tiny add-on pieces instead of the whole model — fits on one GPU.
dataset	your training examples, usually pairs of input → desired output.
DPO / RLHF	ways to tune a model toward preferred answers, not just imitate examples.

What you need before starting:

Read xt4 (fine-tune vs RAG vs prompt) first — it frames the decision.
Python basics and comfort installing packages.
A GPU for the training labs (QLoRA runs on a single consumer card); concepts read without one.

New to the topic? Read this box, then take the chapters in order — each section is tagged essential → expert so you always know the depth you're at.

Learning objectives

Recall the fine-tune vs prompt vs RAG decision and where tuning genuinely wins.
Name the four cases tuning is for: style, format, latency, and narrow tasks.
Explain what tuning can and cannot change (behavior, not new facts).
Set up the training stack you'll use across the track.

▶ Runnable companionThe code in this lesson is also saved under code/ft1-when-why/ in the course, with a README. Run the scripts or copy the configs directly.

The decision, revisited essential

xt4 made the case that the answer to "should we fine-tune?" is usually no, not yet — prompting and RAG solve most problems cheaper and faster. This track is for the cases where the answer is yes, and it's hands-on: you'll actually tune a model.

🗺️ How to read this diagram

This flow shows where fine-tuning sits in your toolbox: it is the last thing you reach for, not the first. Read the four boxes left to right — each arrow means "if this didn't solve it, move on".

Need better output (left) is the starting point — you have a task and the model's answers aren't good enough yet.
Try prompt + RAG comes next because it is cheap first: rewording the prompt or feeding the model relevant documents (RAG) fixes most problems with no training at all.
Still failing on style/format/latency? is the trigger box — you only continue if prompting and RAG genuinely fell short on one of those specific issues.
Fine-tune (right) is the final step, reached only after the cheaper options failed. The sub-labels ("cheap first", "the trigger", "hands-on here") tell you the role each stage plays.

In short: Fine-tuning is the bottom of the ladder, not the top. Climb the cheap rungs (prompt, then RAG) first, and only tune when a real style/format/latency problem remains.

Tuning changes behavior, not knowledgeFine-tuning teaches a model how to respond — tone, format, a narrow skill. It is a poor way to add facts (that's RAG's job) and it can't make a small model as smart as a big one. Match the tool to the problem.

The four cases tuning wins essential

Case	Example	Why tuning beats prompting
Style/voice	always answer in your brand voice	consistent without a huge prompt
Strict format	always emit this exact JSON/DSL	higher reliability than instructions
Latency/cost	a small tuned model matches a big prompted one	cheaper per call at scale
Narrow task	classify your domain's tickets	beats general model on the niche

Set up the stack intermediate

The track uses the Hugging Face ecosystem: transformers, peft (LoRA), trl (SFT/DPO trainers), datasets, and bitsandbytes (4-bit, from IC2). One install covers the whole track.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.

Lab FT1.1 · Environment

setup.shpip install "transformers>=4.44" peft trl datasets accelerate bitsandbytes

# Sanity check the GPU stack:
python -c "import torch; print('cuda:', torch.cuda.is_available())"
# QLoRA (FT3) needs a GPU; small runs fit on a single consumer card.

▶ How this works

Before you can train anything, you install the toolbox and confirm your machine can actually use its GPU (the graphics chip that makes training fast). This is a shell script — the lines starting with # are just comments explaining what's happening.

pip install "transformers>=4.44" peft trl datasets accelerate bitsandbytes downloads the whole fine-tuning toolkit in one go: transformers (the models), peft (LoRA), trl (the trainers), datasets (loading data), and bitsandbytes (4-bit math for QLoRA).
The python -c "..." line runs a one-line Python program that imports torch and prints whether a GPU is available. -c means "run this string as code" without making a file.
torch.cuda.is_available() returns True if PyTorch can see a CUDA-capable NVIDIA GPU, and False otherwise.

What the output means: You'll see a line like cuda: True. True means the GPU is ready and the training labs will run; False means no usable GPU — you can still read the lessons, but the training steps won't run.

Try this: Run just the python -c line on your machine. If it prints cuda: False on a laptop with no NVIDIA card, that's expected — the QLoRA lab (FT3) is what needs the GPU.

You can follow along cheaplyFull fine-tuning needs serious hardware, but QLoRA (FT3) tunes a 7B model on a single consumer GPU — and the concepts transfer up. This track is designed to be runnable, not just readable.

Exercise FT1.1 — Justify a tuning decision

Context: The most valuable fine-tuning skill is knowing when not to fine-tune; the four-case test is how you check yourself against your own temptation.

Your task: Take a real task you're tempted to fine-tune and write down which of the four cases it hits (if any) and what you'd try with prompting or RAG first.

Requirements:

Name the concrete task honestly
Classify it against the four winning cases
State the prompting/RAG approach you'd try before tuning
If none of the four apply, conclude explicitly: don't tune it

💡 Hint: Being unable to place your task in any of the four cases is a result, not a failure — it means a cheaper lever is the right call.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · The four cases tuning winsBeginner

Context: Fine-tuning changes behaviour, not knowledge, and it genuinely wins in only four cases: style/voice, strict format, latency/cost, and one narrow task.

Your task: Write a function that takes a goal and reports whether fine-tuning is even a candidate and which of the four winning cases it hits.

Requirements:

Map goal phrasings to the four cases (style, format, latency, narrow task)
Match case-insensitively on substrings of the goal
Return the matched case, or a “probably not — try prompt + RAG first” verdict
Reinforce that new facts belong in RAG, not a tune

💡 Hint: If a goal is really about knowing a fact rather than behaving a certain way, no case should match — that's the intended answer.

Show solution

Encode the lesson's four cases as a lookup:

CASES = {
    "consistent brand voice":     "style",
    "always emit strict JSON":    "format",
    "smaller/faster cheap model": "latency",
    "one narrow classification":  "narrow task",
}
def tuning_case(goal):
    for k, case in CASES.items():
        if k in goal.lower():
            return f"candidate — case: {case}"
    return "probably NOT — try prompt + RAG first"

print(tuning_case("we need a consistent brand voice"))   # style
print(tuning_case("answer questions about our new docs")) # NOT (that's RAG)

Tuning changes behavior (style/format/latency/narrow skill), not knowledge. New facts belong in RAG, not a fine-tune.

Exercise 2 · The decision gate: prompt -> RAG -> tuneIntermediate

Context: The right order is prompt, then RAG, then tune. Fine-tuning should only be recommended once the cheaper levers were actually tried and the need is genuinely behavioural.

Your task: Write the decision gate that walks prompt → RAG → tune in order and only lands on fine-tuning when the earlier options were exhausted.

Requirements:

Take flags for what was tried and whether the need is behavioural
Recommend prompting first if it was never tried
Recommend RAG when the need isn't behavioural, or before tuning if untried
Block tuning without clean data + a GPU
Only then recommend FINE-TUNE
Use ordered short-circuit guards so cheaper levers must be ruled out first

💡 Hint: Guard clauses in priority order read like the funnel itself — each return is a cheaper option you must eliminate before the next.

Show solution

The decision flow from the lesson, as short-circuit logic:

def decide(tried_prompting, tried_rag, need_is_behavioral, have_data_and_gpu):
    if not tried_prompting:
        return "PROMPT first — cheapest, no training"
    if not need_is_behavioral:
        return "RAG — the need is knowledge, not behavior"
    if not tried_rag:
        return "Try RAG before tuning — often enough"
    if not have_data_and_gpu:
        return "Tuning blocked — need clean data + a GPU"
    return "FINE-TUNE — cheaper options exhausted, need is behavioral"

print(decide(True, True, True, True))    # FINE-TUNE
print(decide(True, False, False, True))  # RAG

"Should we fine-tune?" is usually "no, not yet." The gate makes you prove the cheaper levers failed before spending training effort.

Exercise 3 · Base vs instruct starting pointAdvanced

Context: You tune from either a base model (a raw next-word predictor) or an instruct model (already follows instructions); most applied tunes start from instruct to keep instruction-following for free.

Your task: Write a selector that picks base vs instruct as the starting point from the scale of the behaviour change and the amount of data.

Requirements:

Pick BASE only for a full behaviour overhaul with a big dataset (must reteach instruction-following)
Otherwise pick INSTRUCT to inherit chat ability
Treat instruct as the safe default for applied tuning
Explain that starting from instruct teaches only the delta

💡 Hint: Base is the exception, not the rule — reserve it for when you have both a large dataset and a reason to relearn instruction-following from scratch.

Show solution

Pick the starting checkpoint from the goal:

def pick_base(want_full_behavior_control, big_dataset, keep_general_chat):
    if want_full_behavior_control and big_dataset:
        return "BASE — most control, but you must reteach instruction-following"
    if keep_general_chat:
        return "INSTRUCT — inherit chat ability, add your skill on top"
    return "INSTRUCT — the safe default for applied tuning"

print(pick_base(True, True, False))   # BASE
print(pick_base(False, False, True))  # INSTRUCT

Most applied fine-tunes start from an instruct model so you keep general instruction-following for free and only teach the delta. Base models are for large overhauls where you have the data to reteach everything.

Exercise 4 · What tuning can and cannot changeExpert

Context: A common failure is expecting a tune to inject facts. Tuning reshapes response style; knowledge and freshness are a retrieval problem.

Your task: Classify a set of goals into “tuning can do this” (behaviour) vs “use RAG/tools instead” (knowledge/freshness), and print the verdict for each.

Requirements:

Label each goal behaviour vs knowledge/fresh (e.g. terse tone = behaviour, today's price = knowledge)
Print TUNE for behaviour goals and RAG/tools for knowledge goals
Cover a strict-format goal (behaviour) and a cite-internal-docs goal (knowledge)
Reinforce that tuning reshapes style, not fact storage

💡 Hint: Ask whether the goal would still be right next month — if the answer moves with the world, it's a retrieval problem, not a tune.

Show solution

Sort goals along the behavior-vs-knowledge line:

goals = [
    ("adopt a terse legal tone",            "behavior"),
    ("know today's stock price",            "knowledge/fresh"),
    ("always answer in this JSON schema",   "behavior"),
    ("cite our internal wiki accurately",   "knowledge"),
    ("classify tickets into 5 buckets",     "behavior"),
]
for goal, kind in goals:
    verdict = "TUNE" if kind == "behavior" else "RAG/tools — NOT tuning"
    print(f"{goal:38} -> {verdict}")

Tuning reshapes how the model responds; it does not reliably store new or changing facts. Anything about freshness or specific documents is a retrieval problem.

Exercise 5 · Set up the training stack (real, needs GPU/libs)Professional

Context: A reproducible training environment is what lets a teammate rerun your fine-tune; the track uses a specific, real Hugging Face stack.

Your task: Show the exact environment the track uses so a teammate can reproduce it, using only the real libraries the lesson names and labelling it as needing a GPU.

Requirements:

Install transformers peft trl datasets accelerate bitsandbytes
Include a tiny probe that prints CUDA availability and the device name
Note each library's role (transformers=models, peft=LoRA, trl=trainers, bitsandbytes=4-bit)
State that a single consumer GPU suffices for QLoRA on a 7B
Note that no CUDA means data-prep prototyping only, not training

💡 Hint: The probe just needs to answer “can this box actually train?” — a one-line torch.cuda.is_available() check is enough.

Show solution

The reproducible stack — needs a GPU + these installs:

# A single consumer GPU is enough for QLoRA on a 7B model.
pip install transformers peft trl datasets accelerate bitsandbytes

# Sanity check the GPU is visible before you burn hours on data prep:
python - <<'PY'
import torch
print("CUDA:", torch.cuda.is_available())
print("device:", torch.cuda.get_device_name(0) if torch.cuda.is_available() else "CPU only")
PY

transformers (models), peft (LoRA), trl (SFT/DPO trainers), datasets (data), bitsandbytes (4-bit) are the whole stack. If CUDA is False you can still read and prototype data prep, just not train.

Exercise 6 · Write the fine-tune business caseIndustry scenario

Context: When a PM asks “why not just fine-tune?”, the answer is a scorecard, not an opinion — you lead with the cheapest lever that clears the bar.

Your task: Build a scorecard that compares prompt, RAG, and tuning across cost-to-build, cost-to-run, time-to-ship, and fit for a “consistent JSON output” need, then print the recommendation.

Requirements:

One row per option (prompt / RAG / tune) across the four dimensions
Print it as an aligned table
Recommend trying a strict-output PROMPT first (same-day, near-free)
Reserve tuning for when the prompt drifts under load
Frame tuning's win as run-cost + reliability, not as the first move

💡 Hint: The recommendation falls out of the table: whichever option ships today for almost nothing goes first, and tuning is the fallback if it drifts.

Show solution

Turn the decision into a scorecard the PM can read:

rows = [
    # option,  build,  run,     ship,     fits 'strict JSON'?
    ("prompt", "hours","cheap", "same day","maybe — can drift"),
    ("RAG",    "days", "medium","week",    "no — not a knowledge gap"),
    ("tune",   "days", "cheapest","1-2 wk","YES — bakes in the format"),
]
print(f"{'option':7}{'build':7}{'run':9}{'ship':10}fit")
for r in rows:
    print(f"{r[0]:7}{r[1]:7}{r[2]:9}{r[3]:10}{r[4]}")
print("\nRECOMMEND: try a strict-output PROMPT first (same day, near-free);")
print("if it drifts under load, TUNE to bake the format in and cut run cost.")

The honest recommendation leads with the cheapest lever (a prompt) and only escalates to tuning if it proves insufficient — tuning wins here on run-cost and reliability, not as a first move.

✓ Checkpoint — you can move on when you can…

Recall the fine-tune vs prompt vs RAG decision.
Name the four cases fine-tuning is for.
Explain why tuning changes behavior but not knowledge.
Have the training stack installed and GPU verified.

Knowledge check check yourself

✓ Knowledge check

Fine-tuning changes a model's behavior but not its knowledge. Given that, why is RAG (not fine-tuning) the right tool when you need the model to answer from new facts?

Show answer

Fine-tuning teaches how a model responds — tone, format, a narrow skill — it does not reliably inject new facts. RAG supplies the facts as retrieved context at inference time, which is the correct way to give the model current or proprietary knowledge.

✓ Knowledge check

Name the four cases where fine-tuning genuinely beats prompting, and say why tuning wins for one of them.

Show answer

Style/voice, strict format, latency/cost, and narrow task. For latency/cost, a small tuned model can match a big prompted one at lower cost per call at scale; for strict format, tuning yields higher reliability of exact output than instructions in a prompt.

Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.

Learning objectives

The decision, revisited essential

The four cases tuning wins essential

Set up the stack intermediate

Exercise FT1.1 — Justify a tuning decision

🪜 Practice ladder beginner → industry

✓ Checkpoint — you can move on when you can…

Knowledge check check yourself

More in Fine-tuning