Every model you ship is a liability as much as an asset - it can be biased, drift, leak data, or break the law. AI governance is the operating system that keeps models accountable: an inventory of every use case, a risk tier for each, controls matched to that tier, and independent sign-off before anything reaches production. Model risk management (MRM) is the discipline underneath it - validation, documentation, and continuous monitoring.
AI governance is the set of policies, roles, and processes that decide which AI you build, how it gets approved, and who is accountable when it goes wrong. Model risk management is the technical core - the practice, formalized by the Fed's SR 11-7 guidance, of identifying, measuring, and controlling the risk that a model produces wrong or misused outputs. Together they turn "we built a model" into "we can defend this model to a regulator, an auditor, and the person it affects."
A single source of truth listing every AI use case - owner, purpose, data, and status. You cannot govern what you cannot see, so the registry comes first.
Each use is classified by impact and likelihood into tiers - minimal, limited, high - and controls are matched to the tier. Heavy scrutiny goes where the harm is.
Validation, documentation, and independent review gate deployment. A risk committee signs off, and monitoring keeps the model honest afterward.
| Piece | What it does | Typical artifacts / owners |
|---|---|---|
| Use-case registry | Catalogs every model and AI use in one inventory | Intake form, registry ID, business owner |
| Risk tiering | Scores impact × likelihood into a control tier | Risk assessment, tier label (minimal→high) |
| Validation | Independently tests soundness, bias, and fitness | Validation report, challenge tests |
| Documentation | Records design, data, limits, and decisions | Model card, datasheet, risk register |
| Monitoring & audit | Watches drift, bias, and decay in production | Dashboards, re-validation, audit trail |
A model that is accurate in the lab can still be discriminatory, opaque, or illegal in production. Governance exists because the cost of an ungoverned model - a wrongful loan denial, a regulatory fine, a reputational fire - dwarfs the cost of reviewing it first.
The EU AI Act, SR 11-7, and sector rules impose hard obligations on high-risk AI. Governance is how you meet them provably instead of hoping no one asks.
Bias, drift, data leakage, and unsafe outputs are managed deliberately - measured, mitigated, and monitored - rather than discovered by the people they hurt.
Clear owners, documented decisions, and an audit trail mean that when a model fails you know who signed off, on what evidence, and how to fix it.
A tiered process lets low-risk use cases move fast while concentrating scrutiny on the few that matter - governance enables adoption, it doesn't just block it.
Same idea, four ways to picture it, so it clicks whoever you are.
Governance is the proctored exam with a grading rubric. You cannot just hand in an answer; there are rules for what counts, an invigilator who checks you followed them, and a paper trail of your marks.
Think of it as code review plus CI gates for models. Nothing merges to production until an independent reviewer signs off, the checks pass, and the change is logged with an owner.
It is the risk and compliance function around a model: an inventory of what exists, a rating for how dangerous each item is, controls sized to that rating, and an auditor who verifies it all happened.
Like building inspections and permits. You cannot occupy a new building until an independent inspector certifies it meets code - and the higher the risk, the more inspections it takes.
Governance has two motions: an onboarding path that takes a new use case from intake through tiering, validation, and sign-off, and an ongoing path that monitors, re-validates, and audits deployed models over their whole lifecycle.
Register → tier by risk → develop with documentation → independent validation → risk-committee sign-off → deploy. The heavier the tier, the deeper each gate.
Monitor drift, bias, and performance → periodic re-validation → internal audit → refresh the model card and risk register, or trigger rollback and retirement.
flowchart LR
subgraph L1["🏗️ 1st line - Business & dev"]
UC["💡 New AI use case"] --> REG["🗂️ Register in inventory"]
REG --> TIER["🎚️ Risk tiering"]
TIER --> DEV["🔧 Build + document"]
end
subgraph L2["🔍 2nd line - Independent risk"]
VAL["🧪 Validation"]
RC["✍️ Risk committee"]
end
subgraph L3["📋 3rd line - Internal audit"]
AUD["🔎 Audit the process"]
end
DEV --> VAL
VAL --> RC
RC -->|approved| PROD["🚀 Production"]
RC -->|rejected| DEV
PROD --> MON["📈 Monitoring"]
MON -->|drift or breach| VAL
AUD -.-> L1
AUD -.-> L2
model_card:
id: MDL-2041
name: Retail credit scoring v3
owner: credit-risk@bank.example
purpose: Approve/decline consumer loan applications
risk_tier: high # minimal | limited | high
intended_use: Automated decisioning with human review on declines
data:
training: 2018-2024 booked loans (de-identified)
protected_attributes_excluded: [race, gender, age]
performance:
auc: 0.87
fairness: {metric: demographic_parity_diff, value: 0.03, threshold: 0.05}
limitations: Degrades for thin-file applicants; not for SME lending
risk_register:
- risk: Disparate impact on protected groups
likelihood: medium
impact: high
control: Quarterly bias test + adverse-action reason codes
- risk: Population drift post-deployment
likelihood: high
impact: medium
control: PSI monitoring, re-validation if PSI > 0.2
approval:
validated_by: model-validation@bank.example # independent 2nd line
approved_by: model-risk-committee
next_revalidation: 2027-01-31Notice the artifact is machine-readable and auditable: tier, owner, fairness thresholds, named controls, and the next re-validation date all live in one record. This is what an auditor or EU AI Act conformity assessment asks to see.
Three views: the approval workflow before deployment, risk-tiering that routes a use case to light or heavy controls, and the ongoing monitoring-and-audit loop.
sequenceDiagram
autonumber
participant Dev as 🔧 Developer
participant Val as 🧪 Validation
participant Rev as 🔍 Independent reviewer
participant RC as ✍️ Risk committee
participant Prod as 🚀 Production
Dev->>Val: Submit model + documentation
Val->>Val: Test soundness bias performance
Val->>Rev: Effective challenge of assumptions
Rev-->>Val: Findings and required changes
alt validation passed
Val->>RC: Validation report + risk rating
RC->>RC: Weigh residual risk vs controls
RC-->>Dev: Sign-off with conditions
Dev->>Prod: Deploy under monitoring
else material findings
Val-->>Dev: Remediate and resubmit
end
sequenceDiagram
autonumber
participant Own as 💡 Use-case owner
participant Reg as 🗂️ Registry
participant Risk as 🎚️ Risk assessor
participant Gov as 🏛️ Governance office
Own->>Reg: Submit intake form
Reg->>Risk: Score impact and likelihood
Note over Risk: Impact x likelihood grid
alt minimal or limited risk
Risk-->>Own: Light controls self-attest
Own->>Gov: Register and proceed
else high risk
Risk->>Gov: Route to full MRM
Gov-->>Own: Require validation and sign-off
end
sequenceDiagram
autonumber
participant Prod as 🚀 Production model
participant Mon as 📈 Monitoring
participant Val as 🧪 Validation
participant Aud as 📋 Internal audit
loop every scoring window
Prod->>Mon: Emit metrics and predictions
Mon->>Mon: Check drift bias decay vs thresholds
end
alt breach detected
Mon->>Val: Trigger re-validation
alt cannot remediate
Val->>Prod: Roll back or retire
else remediated
Val-->>Prod: Approve continued use
end
else within thresholds
Mon->>Aud: Periodic report
Aud-->>Prod: Confirm controls operating
end
From an idea for a model to a monitored, audited production system - the whole governance journey in order.
Every proposed AI use enters the registry with an owner, purpose, and the data it will touch. Nothing gets built off the books.
Score impact and likelihood to assign a tier - minimal, limited, or high - which sets how much validation, documentation, and oversight the use requires.
High-risk uses get full MRM; low-risk uses get lightweight self-attestation. Controls are proportionate to the harm, not one-size-fits-all.
The team builds the model while capturing the model card, datasheet for the dataset, and a risk register of known failure modes and their mitigations.
Evaluate accuracy, robustness, and fairness across subgroups against pre-set thresholds. Document limitations and out-of-scope uses explicitly.
A separate 2nd-line function reproduces results and applies effective challenge - SR 11-7's core idea - testing assumptions the builders may have missed.
Governance weighs residual risk against controls and either approves with conditions, requests changes, or rejects. The decision and evidence are recorded.
The model ships with human-in-the-loop where required, adverse-action explanations, and kill-switch or rollback plans wired in from day one.
Production metrics stream to dashboards. Data and concept drift, fairness metrics, and performance are tracked against thresholds continuously.
On a schedule or on a breach, the model is re-validated and internal audit checks the process ran as designed. Findings feed remediation.
Models that drift beyond repair are rolled back or retired; the registry, model card, and risk register are updated so the record stays true.
Most governance programs fail on follow-through, not on intent. These are the usual culprits.
Treating sign-off as a gate you pass once and forget. Risk is a lifecycle property; a model approved last year can be non-compliant today. Governance is ongoing or it is theater.
Without a single registry of every model and AI use, shadow AI proliferates. You cannot control, tier, or defend what you cannot even see. The inventory comes first.
Applying the same heavy process to a spellchecker and a credit model wastes effort and stalls adoption; applying the same light touch to both leaves real harm ungoverned. Match controls to tier.
A model card written at launch and never touched again slowly becomes fiction. Data shifts, thresholds change, owners leave. Stale docs fail the first audit that reads them.
Validating once and walking away misses the slow decay that follows. Drift, concept shift, and emerging bias only show up in production, so oversight has to continue there.
When "everyone" owns a model, no one does. Without a named accountable owner and independent challenge, decisions have no author and failures have no one to fix them.
Track coverage and follow-through, so you know whether the program actually runs or just exists on paper.
| Metric | What it tells you | Good sign |
|---|---|---|
| % of AI use cases inventoried | How much of your real AI footprint is on the books | Near 100%: little shadow AI |
| % with a risk tier assigned | Whether inventoried uses are actually classified | High: controls can be matched to risk |
| % with sign-off before deploy | Whether the approval gate is really enforced | High: few models slip past review |
| Audit findings closure rate | Whether identified issues get remediated, not just logged | High and timely: the loop closes |
| Drift / bias alerts resolved | How well post-deploy monitoring is acted on | High: signals lead to fixes, not backlog |
| Time-to-approval | How long governance takes per use case | Low for low-risk: the process enables, not blocks |
Three representative patterns showing governance and MRM in production-style use.
A retail bank deploys an ML model that scores consumer loan applications - a high-stakes, regulated, customer-facing decision.
A large enterprise has dozens of teams spinning up AI, from chatbots to forecasting, with no shared visibility or standards.
A company deploys an AI system used in hiring - explicitly classified as high-risk under the EU AI Act - into the EU market.
Governance is shifting from periodic paperwork toward automated, continuous, and standardized assurance embedded in the ML platform itself.
Policy checks, tiering, and model-card generation wired into CI/CD, so a model cannot deploy until its controls and documentation pass automatically.
New tiers and controls for LLMs and autonomous agents - hallucination, prompt injection, and tool-use risk that classic MRM never contemplated.
Real-time monitoring replaces annual review - drift, bias, and incident signals feed dashboards regulators and auditors can inspect on demand.
The EU AI Act, NIST AI RMF, and ISO/IEC 42001 are aligning into a common control vocabulary, letting one program satisfy many regimes.
Content credentials, watermarking, and transparency notices become baseline obligations so people know when they are interacting with AI.
Governance extends to foundation models and vendors - due-diligence, model-card demands, and contractual controls over AI you did not build.