Project · Governance & rollout plan
The capstone ties EL1–EL6 into one deliverable: stand up AI governance and a rollout for a representative mid-size company. You'll produce a governance charter, a risk register, a rollout plan, and an eval/monitoring gate — the exact artifact set a new head of AI is asked for in their first 90 days. Templates are provided; the scenario is a representative composite, and every regulatory specific must be verified against the current text with legal. This is engineering-leadership practice, not legal advice.
Learning objectives
- Author a governance charter: scope, risk tiers, review lanes, mandatory artifacts, and framework alignment.
- Build a risk register with likelihood, impact, owner, and mitigation per risk.
- Design a pilot → scale rollout plan with graduation gates, a RACI, and adoption metrics.
- Define the eval + monitoring gate that governs what may deploy and stay deployed.
- Self-assess the whole deliverable against a rubric at meets-bar and above-bar levels.
1 · The scenario
Representative scenario. "Meridian" is a ~600-person B2B software company. Teams have started shipping AI features ad hoc: a support-drafting agent, a doc-search assistant, and a prototype that drafts customer-facing content. There is no inventory, no review process, and no one accountable. A large EU customer just asked, in a security questionnaire, "describe your AI governance." You've been hired as the first head of AI engineering. Deliver the foundation in 90 days.
2 · Deliverable A — the governance charter
The charter is the constitution: short, durable, and the thing everything else references. It reuses EL1's policy shape. Minimum sections:
| Section | Content | Source lesson |
|---|---|---|
| Scope | What counts as an 'AI system' under governance | EL1 §1 |
| Risk tiering | EU-AI-Act-style tiers + blast-radius check | EL1 §2 |
| Review lanes | Fast path (low-risk) vs full path (high-risk) | EL1 §5 / expert rung |
| Mandatory artifacts | Owner + model card + audit trail, as deploy blockers | EL1 §4 |
| Framework alignment | ISO 42001 / NIST AI RMF mapping; verify-the-text stance | EL1 §3 |
| Roles / RACI | One accountable owner per decision | EL1 §5 |
3 · Deliverable B — the risk register
Inventory each in-flight AI system, then log its risks in the EL6 format. A starter register for Meridian's three systems:
| Risk | System | Likelihood | Impact | Owner | Mitigation |
|---|---|---|---|---|---|
| Hallucinated support answer | Support agent | Medium | Medium | Eng Lead | Grounding + citations + HITL send |
| Cross-tenant leak in doc search | Doc search | Low | Critical | Eng Lead | Tenant-filtered retrieval + canary test |
| Customer-facing content is wrong/off-brand | Content drafter | High | High | Product | Human approval before publish; decision-support only |
| PII in prompts/logs | All | Medium | High | Security | DLP boundary; PII-free logging |
| Vendor deprecation / lock-in | All | Medium | Medium | Platform | Provider abstraction + fallback |
| Untracked cost growth | All | Medium | Medium | FinOps | Tagging + budgets + anomaly alerts |
4 · Deliverable C — the rollout plan
For the system worth scaling first (the internal, grounded support agent), lay out the pilot → scale path from EL4 with graduation gates, a rollout RACI, adoption metrics, and the HITL policy. The flow below is the whole governed lifecycle, from a system entering governance to being monitored in production.
5 · Deliverable D — the eval + monitoring gate
Governance is only real if something can fail. Define the gate that governs deployment and continued operation, drawing eval discipline from the safety/case-study tracks and cost/adoption from EL3/EL4:
| Gate | Checks | Action on fail |
|---|---|---|
| Deploy gate (CI) | Quality evals pass; faithfulness/bias for high-risk; isolation canary; cost per success within budget | Block deploy |
| Runtime monitoring | Drift, error/refusal rate, cost anomalies, adoption trend | Alert owner; roll back on severe regression |
| Periodic re-review | Risk still tiered correctly? Regulation changed? Value realized? | Re-classify; update charter/register |
✓ Checkpoint — you can move on when you can…
- Produce a governance charter with all six sections and a stable charter-vs-procedures split.
- Build a risk register with likelihood, impact, owner, and mitigation, and identify the highest-risk system.
- Lay out a pilot→scale rollout with graduation gates, a RACI, and value (not vanity) adoption metrics.
- Define a deploy gate, runtime monitoring, and a re-review cadence.
- Self-assess the full deliverable against the rubric and know what 'above bar' looks like.
Meridian's CEO wants all three AI systems governed identically "to be fair to the teams." Why is uniform governance the wrong call, and what do you propose instead?
Show answer
The EU customer's questionnaire asks Meridian to "describe your AI governance." You've been in role three weeks. What's the honest, credible answer?
Show answer
6 · Self-assessment rubric
Grade your full deliverable. This is the whole track, integrated — score each dimension honestly at meets-bar or above-bar.
| Dimension | Meets bar | Above bar |
|---|---|---|
| Governance charter | A charter exists with scope and some rules. | Six sections; risk-tiered review lanes; artifacts are deploy blockers; frameworks mapped; stable charter-vs-procedures split; verify-the-text stance stated. |
| Risk register | A list of risks exists. | Every system inventoried; each risk has likelihood × impact, a single owner, and a concrete mitigation; the highest-risk system is identified and governed hardest. |
| Rollout plan | A plan to launch the agent. | Explicit pilot→scale graduation gates; rollout RACI incl. sponsor; value (not vanity) adoption metrics; tiered HITL policy matched to risk. |
| Eval + monitoring gate | Some testing before launch. | CI deploy gate (quality + high-risk evals + isolation canary + cost/success); runtime monitoring with rollback; periodic re-review; incidents become permanent gate checks. |
| Communication | Artifacts are technically sound. | Board-ready framing: BLUF ROI case, three-sentence risk framing, now/next/later roadmap; can say no to a bad idea with evidence and an alternative. |
| Honesty / compliance posture | Mentions regulations. | Public frameworks cited as fact and attributed; company specifics labelled representative; regulatory drift hedged with verify-against-current-text; framed as engineering leadership, not legal advice. |
Score each dimension Meets or Above. All six at least Meets = a governance foundation a new head of AI could actually present in their first 90 days and defend to that EU customer. If Risk register or Eval + monitoring gate is below Meets, systems can ship or drift with no owner and no way to fail — fix those before anything else. Above bar on Communication and Honesty is what separates a compliance binder no one reads from governance that leadership trusts and teams actually follow.
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: You can't govern what you haven't listed; the first 90-day move is an inventory.
Your task: Produce the AI-system inventory for Meridian and assign each a risk tier with justification.
Requirements:
- List the three systems with a one-line description each
- Assign a risk tier and blast-radius note to each
- Flag which needs the fast path vs full path
- Note the verify-against-current-text caveat
💡 Hint: Tier by stakes and blast radius, not by how impressive the model is.
Show solution
Inventory + tiering:
| System | Description | Tier / blast radius | Path |
|---|---|---|---|
| Support agent | Drafts support replies; human sends | Limited; internal, reversible | Fast |
| Doc search | Answers over internal + customer docs | Limited-to-high; cross-tenant risk | Full (isolation) |
| Content drafter | Drafts customer-facing content | High; customer-facing, unproven | Full |
The content drafter and doc search take the full review path (customer-facing / cross-tenant risk); the internal, grounded support agent takes the fast path. Tier boundaries verified against the current EU AI Act text with legal.
Context: The charter is the durable constitution everything else references.
Your task: Author Meridian's one-page governance charter with all six sections.
Requirements:
- Scope, risk tiering, review lanes, mandatory artifacts, framework alignment, roles
- Make artifacts deploy blockers
- State the verify-the-text stance and that it's not legal advice
- Keep procedures out of the charter (link them)
💡 Hint: Stable charter; changeable procedures live in linked runbooks.
Show solution
Meridian AI Governance Charter (v1):
1. Scope: any system using ML/LLMs to inform a decision, generate content, or act for Meridian — built or bought.
2. Risk tiering: each system classified minimal/limited/high using current EU AI Act tiers + blast-radius, recorded in the inventory.
3. Review lanes: low-risk → fast path (Eng Lead + model card); high-risk/customer-facing → full path (Governance board + full evidence).
4. Mandatory artifacts (deploy blockers): named owner, current model card, audit trail.
5. Framework alignment: controls mapped to ISO/IEC 42001 and NIST AI RMF; EU-facing systems classified under the EU AI Act; specifics verified against the current text with legal. Not legal advice.
6. Roles: one accountable owner per decision (RACI in linked procedure). Charter reviewed twice a year; templates and gates live in linked runbooks.
Context: The register is both an engineering artifact and the source of board risk framing.
Your task: Build Meridian's risk register across the three systems and identify the highest-risk one.
Requirements:
- Each risk: likelihood, impact, owner, mitigation
- Cover model, security, privacy, cost, and vendor risks
- Identify the highest-risk system and justify governing it hardest
- Note how each entry maps to a board three-sentence framing
💡 Hint: Likelihood × impact ranks the register; the top entry sets your hardest controls.
Show solution
Register (starter — see §3 table for the full set). Highest-risk row:
Content drafter — customer-facing content wrong/off-brand. Likelihood High (unproven, generative, public), Impact High (brand + possible compliance). Owner: Product. Mitigation: full review path, human approval before publish, faithfulness + brand-safety eval gate, decision-support framing.
Why govern it hardest: it's the only system that reaches customers unmediated and it's unproven — the product of likelihood and impact tops the register, so it earns the strictest controls while the internal support agent moves faster.
Board framing (from EL6): "Our AI content tool could publish an inaccurate or off-brand message to customers (high impact); today that's likely without controls, so nothing it produces publishes without human approval and an automated accuracy check, and Product owns it." Each register row collapses to that three-sentence shape.
Context: A launch plan without graduation gates is enthusiasm, not governance.
Your task: Design the rollout for the support agent from pilot to default tool with explicit gates, a RACI, adoption metrics, and the HITL policy.
Requirements:
- Define the four stages and the graduation criterion for each
- Give the rollout RACI incl. a sponsor
- Pick value (not vanity) adoption metrics
- State the tiered HITL policy for the agent's actions
💡 Hint: Each stage transition is a decision gated on evidence, not a calendar date.
Show solution
Stages + gates: Pilot (one squad hits target draft-acceptance and wants to keep it) → Limited (3–4 teams, support load manageable, no trust-destroying incident) → Broad (training + champions in place, adoption climbing, HITL holding) → Default (majority of eligible drafting flows through it, value realized).
RACI: Pilot — R: PM, A: Rollout lead. Training — R: Enablement, A: Rollout lead. HITL sign-off — A: Risk owner. Metrics — A: PM. Sponsor: a VP who visibly uses it and provides air cover.
Adoption metrics (value, not vanity): % of eligible drafts done via agent; draft-acceptance rate; time saved per ticket; weekly active on real tickets; realized value as the headline.
HITL policy: draft internal reply → autonomous (sample); post internal note → notify; send to customer → human approves (the agent never sends unattended); any account change → two-key. Uncertainty escalates one tier.
Context: Governance is only real if something can fail; the gate is where it can.
Your task: Specify Meridian's deploy gate, runtime monitoring, and re-review cadence across the three systems.
Requirements:
- CI deploy gate checks (incl. high-risk-only checks)
- Runtime monitoring signals and rollback triggers
- Re-review cadence and what forces an off-cycle review
- How an incident becomes a permanent gate check
💡 Hint: Match extra checks (faithfulness, isolation canary) to the high-risk systems only.
Show solution
Deploy gate (CI): all systems — quality evals pass, cost-per-success within budget, model card current. High-risk only — faithfulness + brand-safety evals (content drafter), cross-tenant isolation canary (doc search). Fail → block deploy.
Runtime monitoring: error/refusal rate, drift, cost anomalies, adoption trend, and (content drafter) sampled faithfulness. Rollback triggers: a quality regression past threshold, a cost anomaly, or any confirmed cross-tenant signal → auto-alert owner, roll back on severe.
Re-review: scheduled semi-annually; off-cycle forced by a regulatory change, a material incident, or a use-case change (e.g. someone wires the support agent to send unattended → immediate re-tier).
Incident → gate: every production incident produces a new check — a rail, an eval case, or a monitor — added to the deploy gate so that failure class can never silently recur. The gate strengthens monotonically.
Context: Representative scenario: 90 days in, you present the governance + rollout foundation to Meridian's board, which includes the skeptical CFO and the EU customer's concern on the agenda.
Your task: Deliver the board presentation that lands the whole deliverable — integrating every EL lesson.
Requirements:
- Open with BLUF: what you built and why it matters commercially
- Frame the top risks in three sentences each (what / likelihood×impact / owner+mitigation)
- Present the ROI of the support-agent rollout with a value range
- Give a now/next/later roadmap
- Address the EU customer's governance question truthfully
- Handle a board member pushing to 'go faster on the customer-facing tool'
💡 Hint: Board altitude throughout: money, risk, time, ownership — mechanism only if asked.
Show solution
BLUF. "In 90 days we've gone from ungoverned AI to a governance program that lets us ship faster and answer enterprise buyers: every AI system is inventoried, tiered, owned, and gated, and our first agent rollout is delivering measured value."
Top risks (three sentences each). Content drafter: could publish something wrong to customers (high impact); likely without controls, so nothing publishes without human approval + an accuracy check; Product owns it. Doc search: cross-tenant leak (critical impact); low likelihood because retrieval is tenant-filtered and a canary test blocks regressions; Eng owns it.
ROI. Support agent: conservative/expected/optimistic net value ~$180k/$330k/$540k a year at ~70% adoption; funded as a gated pilot before broader spend. Numbers verified against current pricing.
Roadmap. Now: support agent to broad rollout, governance live. Next: extend drafting, cost routing. Later (bet): autonomous back-office if reliability clears the bar.
EU customer. "We govern AI under a charter aligned to ISO/IEC 42001 and NIST AI RMF, with owners, model cards, audit trails, and human oversight on high-risk systems; EU AI Act obligations are verified with legal." Truthful, framework-based, contract-ready.
'Go faster on the customer-facing tool.' Affirm the goal, show the register entry (high × high), and offer the path: human-in-the-loop now, autonomy earned by evals — visibly ahead without betting the brand. The board owns the call; the risk is documented either way.