API vs self-host economics

The honest comparison advanced

Picture the meeting where this goes wrong. Someone pulls up the API invoice — a big, scary number that grows every month — and next to it drops the price of a GPU you could rent for a fraction of that. The room nods: obviously we should host it ourselves. Six months later you're paying for the GPU and the engineer who babysits it and the on-call pager at 3am, the open model you switched to is a notch worse than the frontier one you left, and the savings quietly evaporated. That meeting happens constantly, and it's almost always decided by the wrong comparison. This section is about making the right one.

Here's the core intuition, and it's worth holding onto before any math: a hosted API and a self-hosted GPU have completely different cost shapes. The API is a taxi — you pay per ride, nothing when you're idle, and the meter never stops when you're busy. Self-hosting is buying a car — a big fixed cost whether it sits in the driveway or runs all day, plus insurance, fuel, and your time keeping it road-legal. Nobody sensibly asks "is a taxi cheaper than a car?" in the abstract; the honest answer is "it depends how much you drive." The build-vs-buy decision is exactly that question, and the only way to answer it is to work out how much you drive and where the two bills cross.

The mistake people make — and the reason this earns an "honest" in its title — is that the self-host side of the ledger is easy to under-count. The GPU rental price is a big, visible number, so that's what gets typed into the spreadsheet. But a running model is not a bill, it's a system: it needs someone to patch it, someone to scale it when traffic spikes, someone to wake up when it falls over, plus electricity, networking, and the quiet fact that an open-weight model is usually a step behind the best frontier model you'd get from a hosted API. Leave any of those out and self-hosting looks like a slam dunk on paper — right up until reality sends the rest of the invoice. So the discipline here is to write down every cost on both sides, then let the numbers, not the vibe in the room, make the call.

Self-hosting looks cheaper (no per-token fee) but carries real costs: GPU rental/purchase, ops, and the quality gap vs frontier models. The right answer is a calculation, not a preference. This is the AWS cost-model lesson generalized.

Break-even math advanced

The word "break-even" sounds like finance jargon, but it's the most useful single number you can compute here — and it turns an argument into arithmetic. Break-even is simply the monthly usage at which the two bills are equal. Below it, the taxi (hosted API) is cheaper. Above it, the car (self-host) can pay for itself. Once you have that one number, every "should we host it?" debate collapses into a single check: is our real volume above or below the line?

Think about why a crossover has to exist. The hosted bill starts at zero and climbs in a straight line — a little usage, a little cost; ten times the usage, ten times the cost. The self-host bill starts high (you've paid for the GPU and a slice of an engineer before you've served a single token) but then stays almost flat as usage grows, because a GPU you already own doesn't charge you more to run it harder. One line starts low and rises; the other starts high and stays level. Two lines like that must cross exactly once, and that crossing point is your break-even. Everything left of it favors renting; everything right of it favors owning. If you remember nothing else, remember the shape: rising line vs flat line, they cross once.

The single most common way this math lies to you is by leaving a cost off the flat line. The classic error is to set the self-host cost equal to just the GPU price — that drops the flat line artificially low, which drags the crossover point far to the left and makes self-hosting look good at volumes where it absolutely isn't. The honest flat line is GPU + electricity + a real fraction of an engineer's salary, because patching, scaling, and 3am incidents are not free and do not manage themselves. Get that number right and the break-even you compute is a fact you can defend to finance; get it wrong and you've built a spreadsheet that tells you what you already wanted to hear. The steps below spell out exactly what goes into each side.

Compute your break-even

1. Hosted cost/month = tokens/month × $/token (from the provider's pricing).
2. Self-host cost/month = GPU (rental or amortized) + electricity + a fraction of an engineer.
3. The engineer time is the cost people forget — patching, scaling, incidents don't self-manage.
4. Break-even = the monthly volume where the two lines cross. Below it, hosted wins; above, self-host can.

The non-cost factors expert

Here's the trap that catches people who just learned the break-even math: they compute the crossover, see their volume is above it, and confidently recommend self-hosting — as if the spreadsheet were the whole decision. It isn't. Cost is one input, and often not the deciding one. Plenty of teams sit well past break-even and still use a hosted API, and plenty sit far below it and self-host anyway. If that sounds contradictory, it's because money is only one of the columns, and this section is about the other columns that quietly overrule it.

The useful analogy is buying a house versus renting. The rent-vs-buy calculator gives you a number, and it matters — but almost nobody chooses purely on that number. You factor in whether you'll be transferred next year (flexibility), whether the neighborhood has the school you need (quality of fit), whether you want to be the one fixing the roof (who owns the problems). A build-vs-buy decision for a model is the same: the break-even is real, but it lives alongside factors that can dominate it entirely. There are two directions this cuts, and it's worth naming both honestly.

Reasons to stay on a hosted API even past break-even. The first is quality: the best hosted models — frontier models like Claude — are typically a genuine step ahead of the open weights you'd self-host, and if a few points of accuracy change whether your product works, that gap can be worth far more than the token savings. The second is zero ops: the provider owns uptime, scaling, patching, and the pager, which means your engineers ship features instead of babysitting GPUs — a real cost you already learned to count on the self-host side.

Reasons self-hosting wins even below break-even. The decisive one is data residency: if your data legally or contractually cannot leave your own infrastructure — regulated industries, sensitive customer data, sovereignty rules — then no per-token price makes shipping it to a third party acceptable, and the cost comparison never even happens. The other is offline or air-gapped operation: if the model has to run somewhere with no reliable internet, or on a network deliberately cut off from the outside world, a hosted API simply isn't an option. In both cases cost isn't the tiebreaker — it's irrelevant, because only one side of the ledger is even allowed. The honest takeaway: run the break-even, then ask whether a non-cost factor overrides it before you write down the answer.

Even past break-even, hosted can still win on quality (frontier models) and zero ops. Self-host wins decisively when data cannot leave your infra or you need offline operation — sometimes cost isn't even the deciding factor.

def hosted_monthly(tokens_per_month, usd_per_1k_tokens):
    return tokens_per_month / 1000 * usd_per_1k_tokens

for toks in [5_000_000, 200_000_000]:
    cost = hosted_monthly(toks, usd_per_1k_tokens=0.90)  # blended in+out price
    print(f"{toks:>12,} tok/mo -> ${cost:,.2f}/mo")

def selfhost_monthly(gpu_hourly, num_gpus, kwh_price, gpu_watts,
                     engineer_salary, engineer_fraction):
    gpu = gpu_hourly * num_gpus * 24 * 30
    power = (gpu_watts/1000) * num_gpus * 24 * 30 * kwh_price
    people = engineer_salary/12 * engineer_fraction    # the forgotten cost
    return gpu + power + people, {"gpu": gpu, "power": round(power), "people": round(people)}

total, parts = selfhost_monthly(
    gpu_hourly=1.20, num_gpus=2, kwh_price=0.12, gpu_watts=400,
    engineer_salary=180_000, engineer_fraction=0.25)
print(f"${total:,.0f}/mo", parts)

def breakeven_tokens(usd_per_1k, selfhost_fixed_monthly):
    # tokens/1000 * price = fixed  ->  tokens = fixed/price * 1000
    return selfhost_fixed_monthly / usd_per_1k * 1000

be = breakeven_tokens(usd_per_1k=0.90, selfhost_fixed_monthly=8_000)
print(f"break-even ~ {be:,.0f} tokens/month")
for v in [5_000_000, 20_000_000]:
    verdict = "self-host can win" if v > be else "hosted wins"
    print(f"{v:>12,} tok/mo -> {verdict}")

def effective_breakeven(usd_per_1k, fixed_monthly, utilization):
    # you pay for the whole GPU but only 'utilization' of it does work
    effective_fixed = fixed_monthly / max(utilization, 0.01)
    return effective_fixed / usd_per_1k * 1000

for u in [1.0, 0.5, 0.2]:
    be = effective_breakeven(0.90, 8_000, u)
    print(f"utilization {int(u*100):>3}% -> break-even ~ {be:,.0f} tok/mo")
# lower utilization pushes break-even much higher -> hosted stays cheaper longer

def score(option, weights):
    return sum(option[k] * weights[k] for k in weights)

WEIGHTS = {"cost": 0.4, "quality": 0.25, "ops": 0.15, "residency": 0.2}
# scores 0-1, higher = better for that option
hosted   = {"cost": 0.5, "quality": 1.0, "ops": 1.0, "residency": 0.2}
selfhost = {"cost": 0.9, "quality": 0.7, "ops": 0.3, "residency": 1.0}

sh, ss = score(hosted, WEIGHTS), score(selfhost, WEIGHTS)
print(f"hosted={sh:.2f}  self-host={ss:.2f}")
print("recommend:", "self-host" if ss > sh else "hosted")

def recommend(tokens_per_month, usd_per_1k, selfhost_fixed, utilization,
              residency_required):
    hosted = tokens_per_month/1000 * usd_per_1k
    eff_fixed = selfhost_fixed / max(utilization, 0.01)
    be = eff_fixed / usd_per_1k * 1000
    decision = "SELF-HOST" if (tokens_per_month > be or residency_required) else "HOSTED"
    reasons = [
        f"hosted ~${hosted:,.0f}/mo at this volume",
        f"self-host effective fixed ~${eff_fixed:,.0f}/mo (util {int(utilization*100)}%)",
        f"break-even ~{be:,.0f} tok/mo",
    ]
    if residency_required:
        reasons.append("data residency forces self-host regardless of cost")
    return decision, reasons

dec, why = recommend(tokens_per_month=50_000_000, usd_per_1k=0.90,
                     selfhost_fixed=8_000, utilization=0.6,
                     residency_required=False)
print(dec)
for r in why: print(" -", r)

Knowledge check check yourself