Defend a tradeoff

1 · The decision framework essential

Every one of the forks below is worked with the same six-step frame. Interviewers and design reviews are really testing whether you have a frame — the specific answer matters less than whether the reasoning is legible. Learn the loop once; apply it to anything.

The criteria are almost always the same short list — quality, cost, latency, maintenance/ops, risk, time-to-ship — and the whole skill is in the weighting. The right answer for a nightly batch job is the wrong answer for a chat UI because the weights differ, not because the options changed. Make the weights explicit and the decision defends itself.

2 · Fork — RAG vs fine-tune essential

Decision: default to RAG. For a docs Q&A bot that must answer from a corpus that changes weekly, RAG puts the knowledge in context at query time; fine-tuning bakes it into weights you'd have to re-train every time the docs change. Fine-tuning changes behavior (style, format, a narrow skill) — it's a poor way to add knowledge. This is the XT4 decision, defended out loud.

The defense: weight freshness and time-to-ship high (the docs change weekly, we need it this sprint) and RAG dominates — it wins the two heaviest criteria and ties on quality. The scorer below makes that ranking concrete rather than asserted.

decide.pydef score_options(options, weights):
    """options: {name: {criterion: 0..5}}  (higher = better on that criterion).
    weights:   {criterion: weight}.  Returns options ranked best-first."""
    ranked = []
    for name, scores in options.items():
        total = sum(scores[c] * w for c, w in weights.items())
        ranked.append((name, round(total, 1)))
    ranked.sort(key=lambda x: x[1], reverse=True)
    return ranked

# Fork: RAG vs fine-tune vs long-context prompt for a docs Q&A bot.
weights = {"quality": 3, "cost": 2, "latency": 1, "maintenance": 2, "time_to_ship": 3}
options = {
    "RAG":        {"quality": 4, "cost": 4, "latency": 3, "maintenance": 4, "time_to_ship": 5},
    "Fine-tune":  {"quality": 4, "cost": 2, "latency": 5, "maintenance": 1, "time_to_ship": 1},
    "Big prompt": {"quality": 3, "cost": 1, "latency": 2, "maintenance": 5, "time_to_ship": 5},
}
for name, total in score_options(options, weights):
    print(f"{name:12} {total}")
print("decision:", score_options(options, weights)[0][0])

RAG          46
Big prompt   38
Fine-tune    26
decision: RAG

3 · Fork — build vs buy a component intermediate

Decision: buy the undifferentiated component; build only your moat. For a vector database, an eval harness, or an LLM gateway, the honest question is: is this the thing customers pay us for? Almost never. A managed vector DB, a hosted eval tool, or an off-the-shelf gateway is battle-tested, maintained by someone else, and lets your small team spend its scarce build-hours on the product no one else can ship.

The defense: engineering time is the scarcest resource on a small team. Every week spent building a vector DB is a week not spent on the differentiated product. Undifferentiated heavy lifting is precisely what you pay a vendor for — you're buying focus, not just software. This is the same capability-vs-control weighing as PE4's API-vs-self-host call, one layer up the stack.

4 · Fork — single agent vs pipeline of steps advanced

Decision: pipeline by default; reach for an agent only where the path is genuinely open-ended. A fixed pipeline (classify → retrieve → answer → validate) is deterministic, cheap, testable step-by-step, and easy to debug — you know exactly which stage failed. An agentic loop that plans its own tool calls buys flexibility for tasks whose shape you can't predict, at the cost of determinism, token spend, and debuggability.

The defense: most production tasks have a knowable shape. Weight determinism, debuggability, and eval surface high — as any team running this in production must — and the pipeline wins decisively. "Agent" is not a maturity level; it's a tool for open-ended tasks, and you pay for its flexibility in every dimension a pipeline is strong. When you do need an agent, contain it: bounded steps, a tool allowlist, and a validation stage on the way out — a pipeline wrapped around the loop.

5 · Fork — sync/realtime vs batch professional

Decision: route by whether a human is blocked on this exact response. If someone is waiting (chat, an API endpoint) it must be synchronous — no discount is worth a user staring at a spinner. If nothing is blocked (overnight classification, backfills, bulk summarization) batch it and take the ~50% token discount. This is the AP1 decision, defended with the gate question.

The defense: the same model, same prompt, two delivery modes — the decision is entirely about which criterion you weight. A chat turn weights latency critical, so the 50% saving is irrelevant. A 50,000-row nightly job weights throughput-per-dollar critical and latency at zero, so batching is close to free money. Naming the weights is the defense; the flip is built into the question "is anyone blocked?"

6 · Fork — small model + scaffolding vs big model tech-lead

Decision: push the cost/quality frontier — a small model with good scaffolding beats a big model raw, until it doesn't. A cheap, fast model wrapped in retrieval, tight prompts, validation, and a retry/repair loop often matches a frontier model on the common cases at a fraction of the cost and latency. The frontier model still wins on the hardest reasoning — so the lead's move is usually both: small model for the easy majority, escalate the hard tail to the big one (the hybrid from PE4).

The build-vs-buy-style flip here is a pure break-even: a big model has near-zero fixed cost but a high per-call price; a small tuned model (or a self-hosted one) carries an upfront build and cheaper calls. Below some volume the big model is simply cheaper; above it, the small one wins. Don't argue it — compute it.

flip.pydef when_finetune_flips(rag_infra_monthly, ft_fixed_cost, ft_infra_monthly,
                        rag_cost_per_call, ft_cost_per_call, months=12):
    """Big-model/RAG path: low fixed cost, higher per-call. Small-tuned path: big
    upfront build, cheaper per-call. Return the monthly volume where the cheap
    per-call path wins over the horizon."""
    per_call_gap = rag_cost_per_call - ft_cost_per_call          # expensive path pays more/call
    infra_gap = (rag_infra_monthly - ft_infra_monthly) * months
    if per_call_gap <= 0:
        return "cheaper-per-call path never wins at these rates"
    calls_total = (ft_fixed_cost - infra_gap) / (per_call_gap * months)
    if calls_total <= 0:
        return "cheaper-per-call path already wins from call 1"
    return f"small/tuned model wins above ~{round(calls_total):,} calls/month"

print(when_finetune_flips(
    rag_infra_monthly=300, ft_fixed_cost=40_000, ft_infra_monthly=200,
    rag_cost_per_call=0.004, ft_cost_per_call=0.0008))

small/tuned model wins above ~1,010,417 calls/month

The defense: at low volume the frontier model is both cheaper (no build cost) and higher quality — start there, ship this week. The break-even (~1M calls/month here) is where the small-model investment pays for itself; below it, building your own is premature optimization. A lead states the number, not a preference, and revisits it as volume grows — the same discipline as PE4's API-vs-self-host break-even.

7 · Tech-lead — defending it in the room tech-lead

A lead's job in a design review is not to be right — it's to make the decision legible and falsifiable. Walk the frame out loud: "here are the two or three options I weighed; here are the criteria and how I weighted them for this context; the evidence points to X; we're giving up Y; and the signal that would flip me is Z." That last clause is what separates a defense from a rationalization — a decision you'd never reverse under any evidence was never reasoned, it was chosen.