Buy vs build & model selection

1 · The four options, honestly framed

Here's the meeting that goes wrong in a thousand companies. Someone demos a slick AI feature to leadership; everyone's excited; and then a senior engineer says the words that derail the next six months: "We should just build our own." Nobody in the room agrees on what "build" even means — one person pictures pre-training a model, another pictures fine-tuning, a third is thinking about self-hosting an open-weight model, and the exec is quietly wondering why they can't just call an API. They argue for weeks because they never named the actual choices. This section exists to make that never happen to you.

Think of it like housing, because you've already made this decision in your own life. You can rent an apartment (move in tomorrow, someone else fixes the plumbing, you pay monthly forever and can't knock down a wall), lease-and-renovate (customize the space but you don't own the building), or buy land and build a house from foundation up (total control, and a multi-year project with a construction crew you now employ). Nobody sane builds a house because they want a place to sleep next week — they rent. And nobody rents forever if they're running a hotel at massive scale. The right answer is entirely about your constraints, not about which option sounds most impressive in a meeting.

The AI version has the same shape, and the key word is ownership: the more of the stack you own, the more control and (eventually) the more cost-leverage you get — but the more you also own the burden of building, running, and maintaining it. The most expensive mistake in this whole topic is treating it as a binary — "buy" (call someone's API) versus "build" (do everything ourselves) — when in reality there's a continuous spectrum with at least five distinct stops, each winning under different conditions. The table below lays them out from least ownership to most. As you read it, resist the pull toward the right-hand columns: more ownership feels like maturity and independence, but for the overwhelming majority of teams the correct answer sits at the top of the table, and the whole rest of this chapter is machinery for proving that to yourself and to a skeptical board.

There is a spectrum, not a binary. From least to most ownership:

2 · A decision flow that ends the debate

You've seen a debate like this eat a whole quarter. Two smart people argue "API vs self-host" for weeks, each marshaling cost spreadsheets and quality anecdotes, and the argument never converges — because they're weighing everything at once, trading a cost point against a quality point against a latency point, with no agreed order. The fix is almost embarrassingly simple: stop optimizing and start eliminating. Most of these decisions aren't decided by the close call at the end; they're decided by a single hard constraint near the beginning that quietly removes most of the options before you ever compare prices.

The mental model is airport security versus a talent competition. A weighted scorecard (the next section) is a talent competition — everyone performs, you tally points, best average wins. But a decision flow is security screening: there are a few things that get you rejected at the door no matter how brilliant you are elsewhere. If your data legally cannot leave your network, it does not matter that the hosted API is cheaper, faster, and higher quality on every axis — it's disqualified at the first gate. A constraint like that isn't a factor to be weighed; it's a filter that decides the outcome by itself. Confusing the two — treating a hard legal or residency constraint as just another column with a weight — is exactly how teams talk themselves into an option they'll have to rip out during an audit.

So the discipline is: run your constraints in priority order, hardest and least-negotiable first, and let the first one that bites make the decision. This mirrors how the clinical case study resolved — the PHI (protected health information) constraint settled the deployment target before anyone got to argue about model quality. Only after the hard constraints have pruned the tree do you switch modes and optimize the survivors on the soft, tradeable things: cost, latency, quality. The flow below is that pruning made visible; read it left to right as a sequence of gates, not a menu.

Run the constraints in priority order — the first hard constraint that bites usually decides it, the same way the PHI constraint decided deployment in the clinical case study. Read the flow as: eliminate on hard constraints first, then optimize on cost and quality.

Worked reading of the flow: (1) If data legally cannot leave your boundary, you are pushed toward self-hosting regardless of cost. (2) If you need your own knowledge in answers, that's retrieval on top of an API — not fine-tuning; fine-tuning teaches behavior (format, tone, a narrow task), not facts. (3) Only steady, high volume amortizes the fixed cost of a serving team. (4) If your quality bar needs the current frontier, that usually means a hosted or managed-cloud frontier model, since your self-hosted open weights will trail it.

3 · The vendor / model scorecard

Every engineer has watched a decision get made by a demo. A vendor shows up, runs three cherry-picked prompts that dazzle the room, drops a benchmark chart where their model tops a public leaderboard, and suddenly the "decision" is made in the hallway afterward on vibes. Six months later you discover the model is mediocre on your actual workload, the pricing balloons at your real volume, and the impressive demo used a prompt nobody could reproduce. The demo was theater — and theater is designed to move you emotionally, not to inform a decision you'll have to defend to a board.

The antidote is the same tool a good hiring panel uses to avoid hiring the most charming candidate instead of the most capable one: a rubric you commit to before you meet anyone. A weighted scorecard forces two acts of honesty. First, it makes you write down what you actually value and how much — quality, cost, latency, security, portability — as explicit percentage weights that sum to 100. Second, it makes you score every candidate on the same axes, so a vendor who's brilliant at the one thing they demoed can't win on that alone. The number that falls out isn't magic, but it is legible: anyone can see why the winner won.

Here is the discipline that makes it work, and the one people skip: assign the weights first, before you've seen a single score. The reason is a well-documented human failing — motivated reasoning. If you set weights after you've fallen for a vendor, you'll unconsciously tune them to make your favorite win ("well, latency is actually the most important thing, and look, they're fastest"). Locking the weights in advance turns the scorecard from a rationalization engine into an honest one. The dimensions and starting weights below are a sensible default for a typical enterprise workload; treat them as a template, adjust the weights to your reality (a regulated industry pushes data-posture weight up; a latency-critical product pushes latency up), but adjust them before the demos start, not after.

Demos are theater; a weighted scorecard forces you to state what you actually value and score every candidate on the same axes. Assign weights first (before you see scores) so you can't rationalize your favorite afterward.

4 · Total cost of ownership — the costs teams forget

Picture the slide that wins the room and loses the year. An engineer puts up two numbers: the API costs $0.004 per request, and a GPU we could rent costs $3,000 a month. At our volume the API bill is $X, and — look — self-hosting is cheaper. Heads nod, the plan is approved, and eighteen months later the "cheaper" option is costing three times what the API ever did. Nothing on that slide was a lie. It was just radically incomplete — and incomplete cost models are the single most common way smart teams talk themselves into the wrong build decision.

The trap has a name from any car buyer: the difference between the sticker price and the total cost of ownership. The sticker on the windshield is the purchase price; TCO is what the car actually costs you over the years you own it — insurance, fuel, maintenance, the mechanic on speed-dial, depreciation, and the Saturday afternoons you spend dealing with it. A cheap car with expensive parts and terrible reliability can easily cost more to own than a pricier one that just works. Nobody who has owned a car twice confuses the two numbers again. Yet in AI infrastructure, the sticker price — per-token spend or GPU-hour — is exactly the number people quote, and the ownership costs are exactly the ones they forget.

So what's hiding below the sticker? For a self-host path it's the whole iceberg under the GPU line: the loaded salary of the MLOps engineers who build and babysit the serving stack (loaded means salary plus benefits, taxes, equipment — often 1.3–1.4× base), the on-call rotation that keeps a critical service alive at 3 a.m., the upgrade and re-tuning treadmill as base models improve and your adapters go stale, and the opportunity cost of months spent on infrastructure before the feature delivers a dollar of value. The API path has forgotten costs too — integration effort, eval-set upkeep, re-testing every time the provider ships a new model — but they're a fraction of the self-host burden. The table below sorts the buckets across both paths. Read it as a checklist: the honest cost of an option is the sum of every row, not the one row that fit on the exciting slide, and §4's warning box shows why this pushes the self-host break-even point much later than engineers hope.

The sticker price (per-token or GPU-hour) is the cost people quote. TCO is the cost they actually pay. A defensible model includes:

5 · Lock-in and the exit strategy

The worst version of this story starts with good news. Your AI feature is a hit. Then, at renewal, your provider quietly raises prices 40% — or deprecates the exact model your prompts were tuned to with 90 days' notice — or a new compliance rule says that model's region won't do. You go to switch, and discover you can't: your prompts are hand-fitted to one model's quirks, your tool-calling uses a vendor-specific format, your fine-tuned adapter only runs on their stack, and your embeddings live in a store you can't easily export. You're not a customer anymore; you're a hostage. That's vendor lock-in, and the time to deal with it is the day you sign — never the day you want to leave.

The useful way to think about lock-in is that it's coupling, the same disease software engineers already know to fear. When two parts of a system know too much about each other's internals, you can't change one without breaking the other; when your application knows too much about one provider's specific API, you can't change providers without a rewrite. And just as in code, the cure is not to avoid depending on anything — you can't, you have to ship — but to route the dependency through a seam you control. Lock-in comes in four flavors worth naming so you can spot each: prompt/behavior coupling (prompts fitted to one model's personality), proprietary features (a vendor-only tool, format, or endpoint with no standard equivalent), fine-tune artifacts (an adapter married to one serving stack), and data gravity (embeddings and indexes so large or so awkward to export that moving them is its own project).

Here's the mindset shift that separates teams who stay free from teams who get trapped: you design the exit before you need it, and you rehearse it. An escape route you've drawn on a whiteboard but never actually driven is a hope, not a plan — the parking-brake analogy is a fire drill you never run. Concretely that means an abstraction layer so swapping providers is a config change, a provider-agnostic eval set so you can re-qualify a challenger in a day, avoiding one-way proprietary formats where an open standard exists, and keeping the data you'd need to re-tune elsewhere. And here's the part that surprises people: the point of all this usually isn't to actually leave. It's that a credible, tested ability to leave is your single strongest piece of negotiating leverage on price and terms at every renewal. A vendor who knows you can walk away in a week treats you very differently from one who knows you're stuck.

Every choice creates dependency; the question is whether you can leave when you need to. Sources of lock-in: prompt/behavior coupling (prompts tuned to one model's quirks), proprietary features (a vendor-specific tool or format), fine-tune artifacts (an adapter you can't move), and data gravity (your embeddings/indexes in one store). You reduce lock-in the same way you reduce coupling in code: an abstraction seam and a tested escape route.