Regulated-industry build brief
The capstone of the enterprise case-study track. You are handed one mandate: produce the complete build brief for a regulated-industry AI system — the document a staff engineer and a compliance program sign off before production code is written. You tie together the regulated-industry discipline (CS7), the legacy retrofit (CS8), and the shared-platform posture (CS9) into one deliverable, with ready-to-use templates: a design-doc outline, a threat model, an eval plan, and a go/no-go gate. Engineering-leadership framing; not legal or financial advice.
Learning objectives
- Turn a regulated-industry mandate into a structured build brief with constraints, a quality bar, and a compliance surface.
- Compose CS7 (regulated discipline), CS8 (legacy retrofit), and CS9 (shared platform) into one coherent design.
- Produce four reusable artifacts: a design-doc outline, a threat model, an eval plan, and a go/no-go gate.
- Make human oversight, auditability, and explainability structural — not documentation after the fact.
- Self-assess the whole design at a staff level against an explicit rubric, and know which gap ships next.
- State honestly what you'd verify with legal/compliance and against current regulation before building.
A build brief is a sequence of decisions and the artifacts that make them reviewable, not prose. Read the arc left to right — it is the shape of this page and of the brief you assemble at the end. Every box is a section you fill in, and the four templates in this capstone are the reusable skeletons for the ones that matter most in a regulated setting.
1 · The mandate & the compliance surface
Start by pinning the mandate into constraints that will veto architectures, exactly as CS7 did — but now also name the compliance surface: which public frameworks plausibly apply and what each demands in general terms. This is the section you write with compliance, not for them.
| Element | Helios statement | Design consequence |
|---|---|---|
| Purpose | Assist loan officers deciding consumer credit; AI recommends, human decides. | Recommendation-only output; approval gate mandatory (CS7). |
| Constraints | Explainable, auditable, human-overseen; no rewrite of core banking; bounded spend. | Verified rationale + audit trail + strangler retrofit + gateway budgets. |
| Compliance surface | EU AI Act-style high-risk; GDPR/DPDP; PCI-DSS; SOC 2. | Record-keeping, minimization, card-data scoping, change control. |
| Quality bar | Citation validity 100%; consistency + cohort screening pass; refer-recall held. | Fairness-aware eval gate in CI (CS7). |
| Platform posture | Build on the internal AI gateway, not a bespoke stack. | Inherit guardrails, routing, cost governance, observability (CS9). |
2 · Template — the design-doc outline
The first deliverable is a design doc a reviewer can sign. Below is a reusable outline; fill each section for your system. It deliberately front-loads the constraints and the human-oversight model, because in a regulated build those decide everything downstream.
Design-doc outline (fill each section)
- 1. Problem & mandate — what the system does, who it's for, and the one-line non-goal ("the AI never decides").
- 2. Constraints & compliance surface — explainability, auditability, oversight, data-protection; the frameworks in play (verify with legal).
- 3. Architecture — the request path; where the approval gate and audit trail sit; how it grafts onto legacy (ACL/strangler, CS8) and rides the platform (CS9).
- 4. Output contract — recommendation-only fields, verified rationale, citation checks, prohibited-factor screen (CS7).
- 5. Data handling — PII minimization before egress, read model / CDC, advisory writes, retention.
- 6. Human oversight — the approval gate, who is accountable, how their action is logged.
- 7. Evals & monitoring — the eval gate (§4) and the production signals (override rate, cohort drift).
- 8. Rollout & rollback — the staged plan and the one-config kill switch.
- 9. Open questions for legal/compliance — everything you must not decide alone.
3 · Template — the threat model
The second deliverable is a threat model. For each asset and threat, name the control and where it lives in the architecture. In a regulated system the threats are as much compliance threats (unexplainable decision, unauditable trail, disparate treatment) as they are security threats (injection, data exfiltration). Cover both.
| Threat | Impact | Control (and where it lives) |
|---|---|---|
| Prompt injection | Model coerced to leak data or misbehave. | Gateway guardrail layer, on by default (CS9); output screen (CS7) |
| PII exfiltration | Personal/card data leaks into logs or prompts. | Minimize before egress; PCI scope-out; reference-only audit (CS7,CS8) |
| Unexplainable decision | Declined applicant gets no valid reason. | Verified structured rationale contract (CS7 §3) |
| Unauditable decision | Can't reconstruct a past decision. | Immutable, versioned, sealed audit trail (CS7 §4) |
| Disparate treatment | System treats cohorts inconsistently. | Consistency + cohort screening gate in CI (CS7 §5) |
| System-of-record corruption | AI writes bad data into core banking. | Advisory-write queue; legacy authoritative (CS8 §4) |
| Runaway spend | One feature blows the AI budget. | Per-team enforceable budgets + quotas at the gateway (CS9 §4) |
| No rollback | A regression can't be reversed. | Rehearsed kill switch → legacy/off in one config (CS8 §5) |
4 · Template — the eval plan & the quality bar
The third deliverable is the eval plan — the regression test that lets you change a prompt, model, or policy without praying, and the gate that blocks a bad change in CI. In a regulated build it measures more than accuracy: citation validity, consistency, cohort screening, and refer-recall, with the fairness definition owned by compliance.
| Eval dimension | What it measures | Gate |
|---|---|---|
| Recommendation quality | Accuracy vs a labeled golden set of representative applications. | Must hold vs prod baseline. |
| Citation validity | Every cited policy clause exists in the retrieved set. | Hard gate: must be 100%. |
| Refer / abstain recall | Cases that should punt to a human are correctly referred. | Must not fall vs baseline. |
| Consistency | Near-identical applications get near-identical recs (perturbation). | Perturbations must not flip the rec. |
| Cohort screening | Outcome-rate parity across synthetic cohorts. | Gap within the compliance-defined tolerance. |
| Prohibited-factor screen | No rationale references a disallowed attribute. | Hard gate: zero hits. |
5 · Template — the go/no-go gate
The fourth deliverable is the go/no-go gate: an explicit checklist that must be all-green before each rollout stage advances. It converts "it feels ready" into a decision a room can make together, with compliance in the room. Every item is a yes/no with evidence attached.
Go/no-go checklist (all must be YES, with evidence)
- Approval gate is un-bypassable and the human's accept/override is logged. (CS7)
- Audit trail reconstructs a decision from the record alone, and is tamper-evident. (CS7)
- Eval gate is green in CI: citation validity 100%, consistency, cohort screening within tolerance. (CS7)
- Kill switch was rehearsed in a drill: one config reverts to legacy/off in seconds. (CS8)
- System of record is never written directly by the AI; writes are advisory + validated. (CS8)
- Guardrails + budgets are inherited from the gateway and verified active for this feature. (CS9)
- Monitoring for override rate, refer rate, cohort drift, and citation validity is live. (CS7,CS9)
- Legal/compliance sign-off recorded, with the open-questions list resolved for this stage. (all)
6 · Rollout, oversight & monitoring
Sequence the rollout with the human firmly in the loop, borrowing CS7's staged model and CS8's retrofit safety. The AI earns scope one stage at a time, each gated by §5, and the human owns every decision throughout. Monitor the compliance signals, not just latency.
| Stage | AI scope | Go/no-go focus |
|---|---|---|
| 1 · Shadow | Recommends on one low-risk slice; logged, never shown. | Audit trail + eval gate green; agrees with baseline. |
| 2 · Assist | Shows rec + rationale to the officer for that slice. | Approval gate un-bypassable; guardrails active. |
| 3 · Fast-path | Preps low-risk, high-confidence recs for quick human sign-off. | Kill switch rehearsed; sampled audit of the fast-path. |
| 4 · Steady state | Enabled slices, human on every decision, full monitoring. | Cohort drift + override-rate monitoring; compliance review. |
7 · Assemble the build brief & self-assess
Assemble the four templates plus the rollout plan into one build brief — the deliverable a staff engineer and compliance sign before production code. Then grade it against the rubric at a staff level. The rubric is the point of the capstone: it tells you, honestly, which gap ships next.
| Dimension | Meets bar | Above bar (staff) |
|---|---|---|
| Mandate & compliance surface | Names constraints and the frameworks in play. | Maps each framework to a control and marks obligations "confirm with legal; verify against current regulation." |
| Design doc | Covers architecture and data handling. | Front-loads oversight; has an explicit open-questions-for-legal section; grafts onto legacy + platform coherently. |
| Threat model | Lists security threats + controls. | Treats compliance failures (unexplainable/unauditable/disparate) as first-class threats, each with a named control. |
| Eval plan | Has a golden set and an accuracy bar. | Hard gates on citation validity + prohibited factors; consistency + compliance-defined cohort screening in CI. |
| Go/no-go gate | Has a launch checklist. | Every item is yes/no with evidence; a single NO stops the stage; compliance sign-off is an item. |
| Rollout & oversight | Ships in stages. | Human owns every decision; kill switch rehearsed; monitors override/refer/cohort drift, not just latency. |
Score each dimension Meets or Above at a staff level. All six at least Meets = a build brief you could put in front of a staff engineer and a compliance program. Any dimension below Meets is your next task — and in a regulated build, the gap you skip is the finding an auditor writes up later.
Your build brief is complete and the rubric is all "Meets", but the go/no-go for stage 1 has one red item: the kill switch has never been drilled. Leadership wants to start shadow mode anyway "since it's only logging." What do you do?
Show answer
Why does this capstone build the regulated system on the shared platform (CS9) and as a retrofit onto legacy (CS8), instead of a clean-room bespoke build?
Show answer
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: Every regulated build starts with a mandate pinned into constraints and an honest list of the frameworks that plausibly apply. Get this wrong and the whole brief drifts.
Your task: Write the mandate block for Helios: purpose, the one-line non-goal, four constraints, and the plausible compliance surface — marking what legal must confirm.
Requirements:
- Purpose plus the non-goal ("the AI recommends; a human decides")
- Four constraints: explainability, auditability, human oversight, data protection
- The plausible frameworks (EU AI Act, GDPR/DPDP, PCI-DSS, SOC 2), each in correct general terms
- Mark the classification + exact obligations as "confirm with legal; verify against current regulation"
- Do not assert which laws apply — identify, don't rule
💡 Hint: Separate what you design (controls) from what legal decides (obligations).
Show solution
- Purpose: assist loan officers deciding consumer credit. Non-goal: the AI never issues a decision — it recommends; a named human decides and is accountable.
- Constraints: (1) explainable — verified structured rationale; (2) auditable — reconstructable, sealed trail; (3) human oversight — un-bypassable approval gate; (4) data protection — PII minimized, referenced not copied.
- Compliance surface: EU AI Act-style high-risk duties (record-keeping, oversight, transparency); GDPR/DPDP-style minimization + retention; PCI-DSS card-data scoping; SOC 2-style change control.
- To confirm with legal: whether this exact use is classified high-risk, retention periods, the fairness definition — verify against current regulation.
You identify the surface and design controls; legal rules on obligations. That split is what keeps the brief honest.
Context: A design doc a reviewer can sign is the core deliverable. Its shape encodes what a regulated build cares about most.
Your task: Produce the design-doc outline for Helios, front-loading constraints and human oversight, and including the open-questions-for-legal section.
Requirements:
- Sections in order: problem/mandate, constraints/compliance, architecture, output contract, data handling, human oversight, evals/monitoring, rollout/rollback, open questions
- Architecture section must show where the approval gate and audit trail sit
- Show how it grafts onto legacy (CS8) and rides the platform (CS9)
- Include an explicit open-questions-for-legal/compliance section
- Explain why that section makes the doc more credible, not less
💡 Hint: The ordering is the message: constraints and oversight before features.
Show solution
- Problem & mandate — assist credit decisions; the AI never decides.
- Constraints & compliance surface — the four constraints + frameworks (verify with legal).
- Architecture — minimize → RAG over versioned policy → recommendation → approval gate → decision → immutable audit; grafted via an anti-corruption layer/strangler onto core banking (CS8); built on the AI gateway (CS9).
- Output contract — recommendation-only, verified rationale, citation checks, prohibited-factor screen.
- Data handling — PII minimized before egress; CDC → read model; advisory writes; retention limits.
- Human oversight — the un-bypassable gate; who is accountable; how their action is logged.
- Evals & monitoring — the eval gate + production signals.
- Rollout & rollback — staged plan + one-config kill switch.
- Open questions for legal/compliance — classification, retention, fairness definition.
Section 9 makes the doc more credible: it shows you know your own edges and aren't quietly practicing law.
Context: A regulated threat model covers compliance threats as well as security threats — and maps each to a control already in the design.
Your task: Produce the threat model for Helios: at least six threats spanning security and compliance, each with its impact and the named control that handles it.
Requirements:
- Security threats: prompt injection, PII/card exfiltration
- Compliance threats: unexplainable decision, unauditable decision, disparate treatment
- Operational threats: system-of-record corruption, runaway spend, no rollback
- Each row names the control AND where it lives (CS7/CS8/CS9 §)
- Explain why compliance failures are first-class threats here
💡 Hint: If a threat's control isn't already in the architecture, that's a design gap, not a doc gap.
Show solution
| Threat | Impact | Control (where) |
|---|---|---|
| Prompt injection | Data leak / misbehavior. | Gateway guardrails + output screen (CS9,CS7). |
| PII/card exfiltration | Regulatory + trust harm. | Minimize before egress; PCI scope-out; reference-only audit (CS7,CS8). |
| Unexplainable decision | No valid reason for a decline. | Verified rationale contract (CS7 §3). |
| Unauditable decision | Can't reconstruct a past decision. | Immutable, sealed audit trail (CS7 §4). |
| Disparate treatment | Inconsistent cohort outcomes. | Consistency + cohort screening gate (CS7 §5). |
| SoR corruption | Bad data in core banking. | Advisory-write queue; legacy authoritative (CS8 §4). |
Compliance failures are first-class threats: "we can't explain this" and "we can't prove a human owned it" have severe impact and each map to a control already in the design. A control that isn't there yet is a design gap to close before ship.
Context: In a regulated build the eval plan measures more than accuracy, and the fairness definition belongs to compliance. This is the regression test that lets you change anything safely.
Your task: Produce the eval plan for Helios: the dimensions, the hard gates, and the fairness screen — stating clearly what compliance owns.
Requirements:
- Dimensions: recommendation quality, citation validity, refer/abstain recall, consistency, cohort screening, prohibited-factor screen
- Hard gates: citation validity = 100%, prohibited-factor hits = 0
- Consistency via perturbation; cohort screening within a compliance-defined tolerance
- State that cohorts + tolerance + fairness definition are compliance decisions
- Run in CI on every model/prompt/policy change; a red result blocks and names the case
- Hedge: verify the fairness definition against current regulation
💡 Hint: Map each dimension to a way the system harms someone, and treat citation validity + fairness as hard gates.
Show solution
| Dimension | Gate |
|---|---|
| Recommendation quality | Hold vs prod baseline on the labeled golden set. |
| Citation validity | Hard gate: 100% — every cited clause exists in the retrieved set. |
| Refer / abstain recall | Must not fall vs baseline (don't start guessing). |
| Consistency | Perturbations must not flip the recommendation. |
| Cohort screening | Outcome-rate gap within the compliance-defined tolerance. |
| Prohibited-factor screen | Hard gate: 0 rationales referencing a disallowed attribute. |
Run in CI on every model/prompt/policy change; a red result blocks and names the failing case. Which cohorts, what tolerance, and the definition of "fair" are compliance decisions you make gate-able — verify against current regulation.
Context: The go/no-go gate turns "it feels ready" into a decision a room makes together, with compliance present. It's only worth having if it holds under pressure.
Your task: Produce the go/no-go checklist for Helios: every item a yes/no with evidence, spanning CS7-CS9, with compliance sign-off as an item — and state the stop rule.
Requirements:
- Items: un-bypassable approval gate; reconstructable + sealed audit trail; green eval gate; rehearsed kill switch; advisory-only SoR writes; gateway guardrails + budgets active; monitoring live; legal/compliance sign-off
- Each item is yes/no with evidence attached (a drill log, a CI run, a signed record)
- State the stop rule: a single NO stops the stage
- Explain why overriding the gate destroys its value
- Tie each item back to CS7/CS8/CS9
💡 Hint: Make compliance sign-off a checklist item, not an afterthought — and make one NO a hard stop.
Show solution
- Approval gate un-bypassable; human accept/override logged (CS7). — evidence: code + audit sample
- Audit trail reconstructs a decision and is tamper-evident (CS7). — evidence: reconstruction drill
- Eval gate green in CI: citation 100%, consistency, cohort within tolerance (CS7). — evidence: CI run
- Kill switch rehearsed → legacy/off in seconds (CS8). — evidence: drill log
- System of record written only via advisory + validation (CS8). — evidence: architecture review
- Gateway guardrails + budgets active for this feature (CS9). — evidence: config check
- Monitoring for override/refer/cohort drift + citation validity live (CS7,CS9). — evidence: dashboard
- Legal/compliance sign-off recorded; open questions resolved for this stage (all). — evidence: signed record
Stop rule: all items must be YES with evidence; a single NO stops the stage, no overrides by schedule pressure. The first time you override the gate it stops being a gate — fix the system or fix the gate, never the discipline.
Context: This is the capstone deliverable: one build brief tying CS7-CS9 together, ready to put in front of a staff engineer and a compliance program, with a staged rollout and an honest self-assessment.
Your task: Assemble the complete build brief for Helios — design doc, threat model, eval plan, go/no-go gate, rollout — grade it at a staff level against the rubric, and name the one gap you'd close next.
Requirements:
- Design doc (§2), threat model (§3), eval plan (§4), go/no-go gate (§5), rollout (§6) assembled coherently
- Composes CS7 (regulated discipline), CS8 (retrofit), CS9 (platform) — each visibly present
- Human oversight, auditability, and explainability are structural, not documented after the fact
- Self-assess against the rubric; identify the lowest-scoring dimension as the next task
- State plainly what you'd verify with legal/compliance and against current regulation
- Defend one hard decision (e.g. recommendation-only, or platform-over-bespoke) in a short paragraph
💡 Hint: Lead with the constraints and the human, compose the three cases, and be honest about the gap that ships next.
Show solution
- Design doc — mandate, constraints/compliance surface, architecture (approval gate + audit trail, grafted via ACL/strangler onto core banking, built on the AI gateway), output contract, data handling, oversight, evals/monitoring, rollout/rollback, open-questions-for-legal.
- Threat model — security + compliance + operational threats, each mapped to a control already in the design (CS7/CS8/CS9).
- Eval plan — quality, citation validity (100%), refer-recall, consistency, cohort screening (compliance-defined), prohibited-factor screen; in CI, red blocks.
- Go/no-go gate — every item yes/no with evidence; a single NO stops the stage; compliance sign-off is an item.
- Rollout — shadow → assist → fast-path (human still signs) → steady state; human owns every decision; monitor override/refer/cohort drift; rehearsed kill switch.
- Self-assessment — grade against the rubric at a staff level; the lowest dimension is the next task (in a regulated build, the gap you skip is a future audit finding).
- Defend a decision. e.g. recommendation-only: it makes human oversight structural — the system is incapable of deciding, so oversight can't be skipped under load — which is exactly what an EU AI Act-style regime expects (verify against current regulation).
The through-line: lead with the constraints and the accountable human, compose the regulated discipline (CS7) onto a safe retrofit (CS8) atop a governed platform (CS9), and stay honest about what legal must confirm and which gap you close next. That brief is the deliverable a staff engineer and a compliance program can both sign.
✓ Checkpoint — you can move on when you can…
- Turn a regulated mandate into constraints and an honest, framework-attributed compliance surface.
- Produce a design-doc outline that front-loads oversight and includes an open-questions-for-legal section.
- Build a threat model treating compliance failures as first-class threats, each mapped to a named control.
- Write an eval plan with hard gates on citation validity + prohibited factors and a compliance-owned cohort screen.
- Write a go/no-go gate where a single NO stops the stage and compliance sign-off is an item.
- Assemble the full build brief composing CS7-CS9, self-assess at a staff level, and name the gap that ships next.