AI governance & compliance

1 · What a governance program actually is

Start with a Monday-morning story you can feel. A regulator's letter lands in your CEO's inbox: "Please describe the AI system that declined Ms. Okafor's loan, the data it was trained on, and who authorized its use." Your team scrambles. The model was shipped eighteen months ago by an engineer who has since left. No one can find the training set. There is no record of who approved it. The model still runs, quietly deciding people's finances, and the company cannot answer a single question about it. Nothing crashed, no alert fired — and yet this is the most expensive failure on the page, because it is a failure your organization cannot even see until someone outside asks.

A governance program is the machinery that makes that letter answerable on the first try. Think of it like the flight-safety system around an aircraft: the plane (your model) can be brilliantly engineered, but what lets an airline fly passengers is the maintenance logs, the pre-flight checklists, the named captain, and the black box — the standing apparatus that lets someone prove the plane was safe to fly and reconstruct what happened if it wasn't. Governance is that apparatus for AI. It doesn't write your model; it makes your model defensible.

Precisely, governance is the standing process that answers three questions before and after any AI reaches production: may this system exist (policy — the rules for what we will and won't build), what did it pass through to get here (process — the review gates), and can we prove it (evidence — the artifacts the process leaves behind). Notice the order matters: policy without process is a wish; process without evidence is theater. Only all three together turn "we think this is fine" into "here is the file that shows why this is fine."

The reason this belongs to engineers, not only lawyers, is that every one of those three parts becomes a concrete artifact you build: a policy is a document you version in git, a process is a CI gate that blocks a deploy, and evidence is a model card and an audit log your pipeline emits. The single most common — and most dangerous — mistake is treating governance as a compliance department's paperwork bolted on at the end. Bolted-on governance is always out of date and always bypassed. Governance that lives in the pipeline, generated by the same automation that ships the model, is the only kind that survives contact with a shipping team. Keep that framing as you read: each framework below is just a structured checklist for producing evidence you'd be glad to hand a regulator.

A governance program is the standing machinery that decides which AI systems may exist, under what controls, and who is accountable when one misbehaves. For an engineer it is not paperwork for its own sake — it is the difference between "we think this is fine" and "we can show a regulator exactly why this is fine." The three moving parts are policy (what we will and won't build), process (the gates a system passes before and after launch), and evidence (the artifacts that prove the process ran).

The failure mode governance prevents is the orphaned model: a system in production that no one owns, whose training data no one can describe, and whose behavior no one is monitoring. Every framework below is, at heart, a structured way to make orphaned models impossible.

2 · EU AI Act — risk tiers as a design input

Here is the trap that catches good engineers. Two teams build the same week. Team A ships a frontier LLM that writes dazzling marketing copy — huge model, clever prompts, months of work. Team B ships a fifty-line logistic regression that decides who gets a loan. In the hallway, everyone assumes Team A's system is the "serious" one that needs oversight and Team B's is trivial "just some maths." Under the EU AI Act, that instinct is exactly inverted: Team B is likely operating a high-risk system carrying heavy legal duties, while Team A's is probably minimal-risk. The lesson lands hard the first time you meet it — risk follows the stakes of the use, not the sophistication of the model.

The way to internalize the Act is to stop thinking of it as a legal document and start treating it the way you already treat a latency budget or a memory limit: a non-negotiable design input that decides what you must build before you write code. When a product manager says "this feature has a 200 ms budget," you don't argue — you architect around it. The risk tier works the same way. Learn it at intake, and it tells you whether you owe the system a documented risk assessment, human oversight, and immutable logging, or almost nothing. Discover it at launch, and you're re-architecting under a deadline with legal breathing down your neck.

The Act sorts systems into four tiers by ascending obligation. The mental model is a simple ladder: as you climb from minimal to prohibited, the amount of evidence and control the law demands rises sharply.

• Prohibited — a hard "no." Some uses (certain social-scoring and biometric practices are the commonly-cited examples) simply may not be placed on the market. This isn't a control you satisfy; it's a door that's locked.
• High-risk — allowed, but only with the full apparatus: documented risk management, data governance, logging, human oversight, technical documentation, and a conformity assessment. This is where credit, specified employment, and critical-infrastructure uses commonly land.
• Limited-risk — allowed with a transparency duty: tell people they're talking to an AI or that content is AI-generated. A chatbot on a public site typically lives here.
• Minimal-risk — everything else (spam filters, most game AI). No AI-Act-specific obligations beyond the law you already follow.

The single mistake to guard against is memorizing a tier boundary from any page — including this one — as if it were settled fact. The categories, the exact list of high-risk uses, and the phased application dates are legal definitions that evolve; classify against the current published Act and its annexes with your legal team. What is durable, and worth carrying forever, is the shape: higher stakes buy you more documentation, more human-in-the-loop, and more logging — and that shape is a design input you can act on the moment a feature is proposed.

The EU AI Act is public, adopted law that classifies AI systems by risk tier and attaches escalating obligations. As an engineer, treat the tier as a design input the way you treat a latency budget: it changes what you must build. The four commonly-described tiers:

Note the asymmetry engineers miss: an EU-facing product may be high-risk because of where it is used, not because of how clever the model is. A boring logistic-regression that decides loan eligibility can carry heavier obligations than a frontier LLM writing marketing copy.

3 · ISO/IEC 42001 & NIST AI RMF — the control catalogs

You've probably lived the version of this that goes wrong. The EU AI Act told you loan-scoring is high-risk and owes "risk management, data governance, human oversight." Great — but the Act is a law, not a manual. It tells you the outcome the state expects; it does not hand you a checklist of what to actually do on Tuesday. So a team reads "human oversight required," nods, and ships — and six months later an auditor asks "show me your documented oversight procedure and evidence it ran," and there's nothing but a good intention. The gap between "the law says we must be responsible" and "here is the operating manual that makes us responsible" is exactly the gap these two frameworks fill.

The clean way to hold the three together: the EU AI Act tells you what is illegal; ISO/IEC 42001 and the NIST AI RMF tell you how to run a shop that stays legal. If the Act is the building code, these are the contractor's playbook — the recurring practices that make sure every house you build passes inspection without heroics.

ISO/IEC 42001 is an international standard for an AI management system (AIMS). Don't let the acronym intimidate you — it's the same well-worn management-system pattern as ISO 27001 for security, just pointed at AI: define a policy, assign named roles, run a repeating plan-do-check-act loop, and keep records that prove you did. Its two practical superpowers are that it's about the system that produces models rather than any one model, so it doesn't rot every time you retrain, and that it's certifiable — a third party can audit you and issue a certificate, which is often what unblocks an enterprise sales deal.

The NIST AI RMF is a voluntary US framework, and its gift to a busy engineer is that it collapses to four verbs you can use as a literal checklist for any system: Govern (who's accountable, what's our risk appetite?), Map (what is this thing, its context, its risks?), Measure (how do we quantify performance, bias, robustness?), and Manage (how do we act on risk across the lifecycle?). Each verb points straight at an artifact you can build — a RACI, a model card, an eval suite, a monitoring dashboard.

The mistake here is thinking you must choose one framework, or that satisfying them means duplicating work N times. You don't pick — you map. Because the frameworks deliberately overlap, one well-built control usually satisfies several at once, which is the single most valuable move in this whole lesson: write your control set once and map it to every framework you're measured against, so you audit once and report many times. A team that treats ISO, NIST, and the EU Act as three separate compliance projects will do the same work three times and still have gaps; a team that maps them does it once.

Two public frameworks tell you how to run governance rather than what is illegal. ISO/IEC 42001 is an international standard for an AI management system (AIMS) — the same management-system pattern as ISO 27001 for security, applied to AI: define policy, assign roles, run a plan-do-check-act cycle, and keep records. It is certifiable, which matters for enterprise sales. The NIST AI RMF is a voluntary US framework organized around four functions you can use as a checklist:

4 · The artifacts auditors ask for: model cards & audit trails

Picture the audit room. A calm, unhurried external auditor sits across from you with a laptop and exactly two questions. First: "What is this system?" Second: "Can you prove it behaved the way you just told me it does?" Everything you've read so far — the tiers, the frameworks, the policy — evaporates in that room unless it has condensed into something you can slide across the table. Frameworks are abstract; auditors are concrete. The whole apparatus of governance becomes real, or becomes vapor, at the moment those two questions get asked.

Those two questions map onto exactly two artifacts, and it's worth seeing them as a passport and a flight recorder. A model card is the passport: a short, standardized identity document for the model — its intended use, where its training data came from, how it scored on evaluations, its known limitations, and the name of the human who owns it. It answers "what is this?" the way a passport answers "who is this traveler?" An audit trail is the flight recorder: the immutable, append-only log of what actually happened in production — which model version served which request, what it was asked, what it returned (recorded in a privacy-preserving way), and who signed off on its deployment. It answers "can you prove it?" the way a black box lets investigators reconstruct a flight after the fact.

Why two and not one? Because they prove different things and neither substitutes for the other. A model card is a claim — "this system is a support-ticket classifier that routes to six queues and was tested to 0.88 F1." An audit trail is the proof that the claim held in the wild — "and here are the eleven thousand tickets it actually routed last month, at this version, under this approval." A model card without an audit trail is an unverifiable promise; an audit trail without a model card is a pile of logs no one can interpret. Together they turn a governance narrative into something falsifiable, which is precisely what an auditor is there to test.

The mistake that quietly poisons the whole exercise is letting the model card drift out of date. An engineer writes a beautiful card at launch, the model gets retrained three times, and the card still describes v1 while v4 is in production. That is worse than having no card at all — because now the card is evidence that your process lies, and an auditor who catches one stale field will reasonably distrust every other artifact you present. The durable discipline, which the tip below expands on, is to make the machine-checkable fields (version, eval scores, data hashes) auto-generated by CI so they cannot drift, and to hand-write only the human-judgment fields (intended use, limitations) that genuinely require a person. A card that regenerates itself on every retrain can't lie; a card maintained by memory always eventually will.

Frameworks become real as two artifacts. A model card is a short, standardized document describing a model's intended use, training-data provenance, evaluation results, known limitations, and owner. An audit trail is the immutable record of what happened: which model version served which request, what it was asked, what it returned (privacy-preserving), and who approved its deployment. Together they answer the auditor's two questions — "what is this system?" and "can you prove it behaved as claimed?"

5 · Who owns the model? — the RACI

Recall the orphaned model from the very first section — the loan-decliner nobody could account for. Now watch how it's actually born, because it's never a villain's decision; it's a diffusion of responsibility that felt reasonable at every step. The data scientist assumed the ML engineer owned production behavior. The ML engineer assumed the product owner owned the risk decision. The product owner assumed "governance" — a committee that met twice — had it covered. Everyone acted in good faith, and precisely because everyone was a little bit responsible, no one was accountable. The orphaned model is not a technical failure. It is an ownership vacuum.

The fix is almost embarrassingly simple, which is why it's so often skipped: for every important decision, name exactly one person who is accountable. The tool that enforces this is a RACI matrix, and the four letters are worth pulling apart because people blur them. Responsible is who does the work (writes the classifier, runs the eval). Accountable is the single throat to choke — the one who answers for the outcome and whose signature means "ship it." Consulted are the experts you must ask before deciding (legal, security). Informed are the people you tell after the fact (execs, adjacent teams). Think of it like surgery: many people are Responsible for a good outcome — nurses, anesthesiologist, techs — but there is exactly one surgeon whose name is on the chart and who answers if it goes wrong.

The iron rule, the one thing to carry out of this section, is exactly one "A" per row. Zero Accountable owners means the decision is an orphan. But the subtler, more common failure is two Accountable owners — which is functionally the same as zero, because each quietly assumes the other has it, and the gap between them is exactly where the orphaned model slips through. If you cannot look at a decision and name a single accountable role without hedging, you have found a governance hole before it found you. The pipeline diagram below traces the gates a system passes through; read the RACI table under it as the answer to "and whose name is on each gate?"

The single most common governance gap is diffuse ownership. A RACI matrix (Responsible, Accountable, Consulted, Informed) fixes this by naming, for each decision, exactly one Accountable role. The pipeline below shows the flow a system passes through and who owns each gate.