{ "name": "AI governance operating model — intake, accountability, responsible AI", "description": "A complete operating model for governing AI: one intake front door with published triage rules, named accountability, nine control themes mapped to NIST AI RMF, ISO/IEC 42001, the OECD principles and the EU AI Act, an obligation map, and generative and agentic overlays.", "author": { "name": "Shailesh Kumar", "url": "https://vantessence.com/" }, "url": "https://vantessence.com/ai-governance/", "methodology": "https://vantessence.com/ai-governance/framework/", "license": "Free to adopt, adapt and cite with attribution to Shailesh Kumar (vantessence.com).", "deterministic": true, "usesLanguageModel": false, "sourceNote": "Anchored to public frameworks: the EU AI Act, NIST AI RMF 1.0 and the Generative AI Profile (NIST AI 600-1), ISO/IEC 42001:2023 and the OECD AI Principles. Obligations are described by category rather than by date, because the Act’s phase-in schedule has been amended — verify current effective dates against the Official Journal before relying on them.", "disclaimer": "An operating model, not legal advice. Obligations are described by category rather than by date because the EU AI Act phase-in schedule has been amended; verify current effective dates against the Official Journal.", "pillars": [ { "id": "intake", "name": "AI Intake", "question": "How does an AI idea enter the organisation safely and consistently?", "outcome": "Every proposed or discovered use case is captured, classified and routed, with traceability from idea to decision.", "failureMode": "Without it you have a backlog nobody owns, and a shadow estate you cannot measure." }, { "id": "accountability", "name": "AI Accountability", "question": "Who is answerable when an AI system behaves well — or badly?", "outcome": "Named individuals own each system across its life, with evidence, approvals and an escalation path.", "failureMode": "Without it you have owners who do not know what they own, and an hour lost to arguing who may press pause." }, { "id": "responsible", "name": "Responsible AI", "question": "Are these systems lawful, trustworthy and aligned with what we claim to value?", "outcome": "Risk is mapped and mitigated across validity, safety, fairness, transparency, privacy, oversight and the generative and agentic surfaces.", "failureMode": "Without it you get speed and no trust, which is the most expensive combination available." } ], "designPrinciples": [ { "name": "One front door", "detail": "Every AI idea reaches one intake. Nothing bypasses governance, including things bought rather than built." }, { "name": "Risk-proportionate review", "detail": "Low-risk uses pass in days. High-risk uses get depth. Reviewing everything equally means reviewing everything badly." }, { "name": "Named accountability", "detail": "Business, technical, model and data owners recorded at all times. Risk accepted by a committee is risk accepted by nobody." }, { "name": "Evidence as a first-class object", "detail": "Decisions are records, not email threads. If it is not in the register, it did not happen." }, { "name": "Lifecycle, not one-shot", "detail": "Material change reopens assessment. Monitoring loops back into governance rather than into a dashboard nobody reads." }, { "name": "Policy as configuration", "detail": "Controls, questions and thresholds are versioned and changeable without a release. Regulation moves faster than software." }, { "name": "Human oversight by design", "detail": "Oversight is designed at the start and evidenced in operation, not asserted in a policy document." } ], "scope": { "heading": "Does this reach you at all?", "points": [ "The EU AI Act applies extraterritorially. A provider or deployer established outside the EU is in scope where the system’s output is used in the EU — which catches a great many organisations that assume they are out of scope.", "The provider and deployer split is the first question, not a detail. Badging someone else’s system as your own can make you the provider.", "Free and open-source models carry partial exemptions, which do not extend to systems placed on the market as high-risk, nor to prohibited practices.", "Sectoral regimes — model risk management in financial services, medical device regulation, employment law — sit on top and frequently bite sooner." ] }, "lifecycle": [ { "stage": "Ideation", "scope": "Before anything formal exists", "output": "Optional idea capture, so demand is visible before it becomes commitment" }, { "stage": "Intake", "scope": "An idea becomes a submitted use case", "output": "Intake record, unique identifier, named sponsor and owners, initial classification" }, { "stage": "Triage", "scope": "Lightweight filtering and routing", "output": "Tier, lane, applicable modules and overlays, service level started" }, { "stage": "Assessment", "scope": "Depth proportionate to tier", "output": "Risk assessment, mitigation plan, reviewer sign-offs" }, { "stage": "Approval", "scope": "A formal decision", "output": "Decision record with rationale, conditions and an expiry date" }, { "stage": "Build or procure", "scope": "Delivery or acquisition", "output": "Technical documentation, model and system cards, data lineage, test records" }, { "stage": "Pre-release validation", "scope": "Evaluation, red teaming, bias and security testing", "output": "Validation evidence and sign-offs proportionate to tier" }, { "stage": "Conformity", "scope": "Where the system is high-risk and you are the provider", "output": "Conformity assessment, technical documentation, declaration of conformity, registration" }, { "stage": "Go-live", "scope": "Production enablement", "output": "Owner acceptance, inventory activation, monitoring plan live" }, { "stage": "Operation and monitoring", "scope": "Continuous", "output": "Telemetry, alerts, post-market monitoring, periodic attestations" }, { "stage": "Change", "scope": "Material change, including silent vendor model updates", "output": "Change records and re-approval" }, { "stage": "Incident", "scope": "Unexpected or harmful behaviour", "output": "Incident record, containment, root cause, remediation, regulator notification where required" }, { "stage": "Reassessment", "scope": "Scheduled or event-driven", "output": "Reassessment record and renewed attestations" }, { "stage": "Retirement", "scope": "Withdrawal from service", "output": "Decommissioning, data handling, documentation retention" }, { "stage": "Archival", "scope": "Retention after retirement", "output": "Immutable archive of the full record for the required period" } ], "accountability": { "roles": [ { "role": "Business Owner", "owns": "Business value, intended use, acceptable-use boundaries, and the continue, change or retire decision.", "line": "first" }, { "role": "Technical Owner", "owns": "Engineering integrity in operation, incidents, change control, technical evidence. Holds authority to pause without convening anyone.", "line": "first" }, { "role": "Model or Agent Owner", "owns": "Model selection, evaluation, fine-tuning, prompts, tool permissions, memory scope and content filters.", "line": "first" }, { "role": "Data Owner", "owns": "Dataset quality, provenance, licensing, consent, retention and right to use.", "line": "first" }, { "role": "Risk and Compliance", "owns": "Ongoing conformance to policy and obligation, and the triggers that reopen assessment.", "line": "second or third" }, { "role": "Privacy Officer / DPO", "owns": "Lawful basis, impact assessments, data-subject rights, cross-border transfer.", "line": "second or third" }, { "role": "Security Officer", "owns": "Threat model, vulnerability management, access control, non-human identity, logging.", "line": "second or third" }, { "role": "Legal Counsel", "owns": "Contractual exposure, intellectual property, liability, regulatory interpretation.", "line": "second or third" }, { "role": "Independent Review", "owns": "Second-line challenge and assurance. Not the same people who built it.", "line": "second or third" }, { "role": "Executive Sponsor", "owns": "Ultimate accountability for high-risk systems and for the portfolio as a whole.", "line": "second or third" } ], "threeLines": [ { "line": "First line", "who": "Business and technology delivery — the four owner roles", "does": "Owns the system, the risk and the day-to-day decisions." }, { "line": "Second line", "who": "Risk, compliance, privacy, security, responsible AI", "does": "Sets policy, challenges the first line, reports independently." }, { "line": "Third line", "who": "Internal audit", "does": "Provides independent assurance that the first two lines work as described." } ], "momentsRequiringAName": [ "At intake, when owners are first named", "At approval, when conditions are accepted", "At go-live, when the owner formally accepts the system", "At material change, including a vendor changing the model underneath you", "At incident, when someone must have standing authority to stop it", "At periodic review, when owners re-attest", "At retirement, when someone confirms the data is handled correctly" ], "principle": "Risk accepted by a committee is risk accepted by nobody. Owners are individuals." }, "controlFramework": { "description": "Nine themes, each mapped to the frameworks it satisfies. The mapping absorbs regulatory change: when an obligation moves, re-point a mapping rather than rewrite a programme.", "themes": [ { "id": "validity", "name": "Validity, reliability and robustness", "intent": "The system does what it claims, at the quality it claims, under conditions it will actually meet.", "controls": [ "Pre-release validation against a defined acceptance threshold", "Benchmark and task-level evaluation with a versioned golden dataset", "Regression suite covering prompts, retrieval configuration and model version", "Drift detection on inputs and outputs with a defined response action", "Safe failure behaviour — degrade or refuse rather than guess", "Adversarial and stress testing proportionate to consequence" ], "frameworkAnchors": [ "NIST AI RMF — Measure 2", "ISO/IEC 42001 — A.6 AI system life cycle", "ISO/IEC 23894 — AI risk management", "ISO/IEC 5338 — AI lifecycle processes", "EU AI Act — accuracy and robustness obligations for high-risk systems", "OECD — Robustness, security and safety" ] }, { "id": "safety", "name": "Safety and security", "intent": "The system cannot be made to cause harm, and cannot become a route into everything behind it.", "controls": [ "Threat model covering model theft, inversion, membership inference, data poisoning, prompt injection, supply chain and API abuse", "Prompt-injection defence: input segregation, least-privilege tool access from the generative layer, output filtering, detection telemetry", "Content filters and refusal behaviour tested, including jailbreak resistance", "Kill-switch with named operators, tested on a schedule rather than assumed", "Red teaming where the consequence of failure justifies it", "Supply-chain assurance for models, weights, datasets and components" ], "frameworkAnchors": [ "NIST AI 600-1 — Information Security", "OWASP Top 10 for LLM Applications", "MITRE ATLAS", "EU AI Act — cybersecurity obligations for high-risk systems", "ISO/IEC 27001", "OECD — Robustness, security and safety" ] }, { "id": "fairness", "name": "Fairness and non-discrimination", "intent": "The system does not distribute harm unevenly across the people it affects.", "controls": [ "Bias assessment before release, with the protected characteristics identified in advance", "Sub-group performance measurement, not just aggregate accuracy", "Mitigation applied and its effect measured rather than assumed", "Residual bias documented and explicitly accepted by a named owner", "Monitoring for divergence between sub-groups after release" ], "frameworkAnchors": [ "NIST AI RMF — Measure 2.11", "EU AI Act — data and data governance obligations", "ISO/IEC 42001 — A.5 Assessing impacts", "OECD — Human-centred values and fairness" ] }, { "id": "transparency", "name": "Transparency and disclosure", "intent": "People know when they are interacting with AI, and what it can and cannot do.", "controls": [ "User-facing disclosure where a person interacts with the system", "Marking of synthetic content and provenance signals where feasible", "Model and system cards covering intended use, limitations and out-of-scope use", "Published statement of what the system does not do" ], "frameworkAnchors": [ "EU AI Act — transparency obligations for certain AI systems", "ISO/IEC 42001 — A.8 Information for interested parties", "NIST AI RMF — Govern 4", "OECD — Transparency and explainability" ] }, { "id": "explainability", "name": "Explainability and contestability", "intent": "A decision that affects someone can be explained to them, and challenged.", "controls": [ "Decision-level explanation proportionate to consequence", "Model logic and feature documentation retained", "A route for an affected person to contest an outcome where law or decency requires it", "Explanations tested on the people who will actually receive them" ], "frameworkAnchors": [ "NIST AI RMF — Measure 2.9", "EU AI Act — right to explanation of individual decision-making", "OECD — Transparency and explainability" ] }, { "id": "privacy", "name": "Privacy and data protection", "intent": "Personal data is used lawfully, minimally, and does not leak back out of the model.", "controls": [ "Lawful basis recorded before processing begins", "Data protection impact assessment where required, and a fundamental rights impact assessment where the deployer is in scope", "Minimisation at collection and at prompt construction", "Memorisation and extraction testing for models trained or fine-tuned on personal data", "Data-subject rights handling that works when the data is inside a model, not only inside a database", "Cross-border transfer assessed, including where inference happens" ], "frameworkAnchors": [ "GDPR / UK GDPR", "EU AI Act — data and data governance; fundamental rights impact assessment", "NIST AI 600-1 — Data Privacy", "ISO/IEC 42001 — A.7 Data for AI systems" ] }, { "id": "oversight", "name": "Human oversight", "intent": "A human can understand, intervene in and stop the system — in practice, not on paper.", "controls": [ "Oversight mode chosen by rule rather than preference (see the oversight decision rule)", "Named roles able to override, with overrides logged and reviewed", "Automation-bias prevention: the interface must make disagreement easy", "Oversight evidence captured as a record, so the control can be audited", "Appeal path where the law requires one" ], "frameworkAnchors": [ "EU AI Act — human oversight obligations for high-risk systems", "NIST AI RMF — Govern 3.2", "ISO/IEC 42001 — A.9 Use of AI systems" ] }, { "id": "accountability", "name": "Accountability and audit trail", "intent": "Every consequential decision has a name against it and survives the people who made it.", "controls": [ "Named owners maintained and re-attested on a schedule", "Decisions, approvals, overrides and exceptions held as immutable records", "Retention aligned to the longest applicable obligation", "A point-in-time audit pack producible for any system on demand", "Exceptions time-boxed, owned and reviewed — never open-ended" ], "frameworkAnchors": [ "ISO/IEC 42001 — Clause 5 Leadership; A.3 Internal organization", "EU AI Act — record-keeping and documentation retention", "NIST AI RMF — Govern 1" ] }, { "id": "genai", "name": "Generative and agentic surfaces", "intent": "The risks that only appear once a system can generate, plan, remember and act.", "controls": [ "System prompts and templates versioned, reviewed and change-controlled", "Retrieval sources explicitly scoped; grounding verified as part of evaluation", "Confabulation measured against a threshold, with high-stakes output verified before use", "Vendor model updates treated as material change and revalidated", "Agent autonomy classified, tool permissions whitelisted, memory scope bounded", "Blast-radius containment on consequential actions" ], "frameworkAnchors": [ "NIST AI 600-1 — all twelve risk categories", "EU AI Act — general-purpose AI obligations", "ISO/IEC 42001 — A.10 Third-party relationships" ] } ] }, "oversightRule": { "intent": "Most frameworks say oversight should be \"proportionate to risk\" and stop there, which leaves the hardest question unanswered. This is a rule rather than a principle.", "modes": [ { "mode": "In the loop", "meaning": "A human approves each output before it has effect.", "when": "The output is binding on a person, materially affects their rights, access, pricing or employment, or is effectively irreversible." }, { "mode": "On the loop", "meaning": "The system acts; a human monitors and can intervene.", "when": "The output influences a consequential decision but a person still makes it, or actions are reversible within a useful window." }, { "mode": "Over the loop", "meaning": "The system acts; humans review samples and aggregate behaviour.", "when": "Output is internal, low-consequence and reversible, and volume makes per-item review meaningless." }, { "mode": "Out of the loop", "meaning": "No routine human involvement.", "when": "Only where the worst realistic outcome is trivial and recoverable. If you cannot describe that outcome, you are not in this category." } ], "note": "Oversight that exists only in a policy document is not a control. If there is no evidence a human ever disagreed with the system, assume the oversight is decorative." }, "triage": { "description": "Ten dimensions, evaluated against ordered rules. The first rule that matches decides the route. A prohibited-practice screen runs first and cannot be overridden by anything downstream.", "dimensions": [ { "id": "prohibited", "name": "Prohibited-practice screen", "question": "Does the system do any of the following?", "rationale": "These are not risk-rated. They are a hard stop, and they are screened first so that nothing downstream can override them.", "options": [ { "value": "none", "label": "None of these apply.", "flags": [] }, { "value": "unsure", "label": "Possibly, or I am not certain.", "flags": [ "prohibited-review" ] }, { "value": "yes", "label": "Yes, one or more of these describes the system.", "flags": [ "prohibited" ] } ] }, { "id": "providerRole", "name": "Your role", "question": "What is your organisation’s role in relation to this system?", "rationale": "Provider and deployer obligations differ substantially, and accountability must terminate inside your organisation either way.", "options": [ { "value": "deployer-approved", "label": "Using an approved tool already governed elsewhere in the organisation.", "flags": [] }, { "value": "deployer", "label": "Deploying a third-party system under our own control.", "flags": [ "deployer" ] }, { "value": "provider-internal", "label": "Building it ourselves for internal use.", "flags": [ "provider" ] }, { "value": "provider-market", "label": "Building it and placing it on the market, or putting our name on someone else’s.", "flags": [ "provider", "placing-on-market" ] } ] }, { "id": "aiType", "name": "Type of system", "question": "What kind of AI is this?", "rationale": "Generative and agentic systems carry whole categories of risk that classical machine learning does not.", "options": [ { "value": "rules", "label": "Rule-based automation with no learned model.", "flags": [] }, { "value": "classical", "label": "Classical machine learning — classification, regression, forecasting.", "flags": [] }, { "value": "genai", "label": "Generative AI, including anything built on a foundation model.", "flags": [ "genai" ] }, { "value": "agentic", "label": "Agentic — it plans, calls tools, holds memory, or acts without per-step approval.", "flags": [ "genai", "agentic" ] } ] }, { "id": "dataSensitivity", "name": "Data sensitivity", "question": "What is the most sensitive data the system touches?", "rationale": "Special-category data changes both the lawful basis and the assessment depth required.", "options": [ { "value": "public", "label": "Public or non-sensitive internal data only.", "flags": [] }, { "value": "confidential", "label": "Commercially confidential data, no personal data.", "flags": [] }, { "value": "personal", "label": "Personal data.", "flags": [ "personal" ] }, { "value": "special", "label": "Special-category personal data, or sector-regulated records.", "flags": [ "personal", "special" ] } ] }, { "id": "affectedPopulation", "name": "Who is affected", "question": "Who is on the receiving end of this system’s output?", "rationale": "Vulnerability and scale determine whether an error is an inconvenience or a harm.", "options": [ { "value": "none", "label": "Nobody directly — internal analysis or drafting only.", "flags": [] }, { "value": "staff", "label": "Our own staff.", "flags": [] }, { "value": "customers", "label": "Customers or members of the public.", "flags": [ "public-affected" ] }, { "value": "vulnerable", "label": "Groups in a position of dependency or vulnerability.", "flags": [ "public-affected", "vulnerable" ] } ] }, { "id": "materialConsequences", "name": "Material consequences", "question": "What does the output actually do to a person?", "rationale": "This is the dimension that most often moves a system into the high-risk category, and the one teams most often understate.", "options": [ { "value": "none", "label": "Informational only. A person makes every decision with other evidence.", "flags": [] }, { "value": "influences", "label": "It materially influences a decision a human then makes.", "flags": [ "influences" ] }, { "value": "determines", "label": "It effectively determines an outcome affecting a person’s rights, access, livelihood or safety.", "flags": [ "determines", "material-effect" ] } ] }, { "id": "autonomy", "name": "Human oversight", "question": "How much human involvement is there in practice?", "rationale": "Oversight claimed in a design document and oversight evidenced in operation are different things.", "options": [ { "value": "in", "label": "In the loop — a human approves each output before it has effect.", "flags": [] }, { "value": "on", "label": "On the loop — it acts, a human monitors and can intervene.", "flags": [ "semi-autonomous" ] }, { "value": "over", "label": "Over the loop — humans review samples and aggregate behaviour only.", "flags": [ "semi-autonomous" ] }, { "value": "out", "label": "Out of the loop — no routine human involvement.", "flags": [ "autonomous" ] } ] }, { "id": "impactOfError", "name": "Impact of error", "question": "If the system is confidently wrong, what is the worst realistic outcome?", "rationale": "Consequence of failure, not probability of failure, sets the depth of assessment.", "options": [ { "value": "low", "label": "Minor inefficiency or rework.", "flags": [] }, { "value": "moderate", "label": "Financial loss, or a poor experience for an individual.", "flags": [] }, { "value": "high", "label": "Significant financial, legal or reputational damage, or unfair treatment of a person.", "flags": [ "high-impact" ] }, { "value": "critical", "label": "Harm to health, safety, fundamental rights, or critical operations.", "flags": [ "high-impact", "critical" ] } ] }, { "id": "reversibility", "name": "Reversibility", "question": "If it acts wrongly, how easily can the effect be undone?", "rationale": "Reversible decisions deserve a fast yes. Irreversible ones deserve a slow one.", "options": [ { "value": "reversible", "label": "Trivially reversible before anything depends on it.", "flags": [] }, { "value": "recoverable", "label": "Recoverable with effort inside a useful window.", "flags": [] }, { "value": "costly", "label": "Costly and disruptive to unwind.", "flags": [ "hard-to-reverse" ] }, { "value": "irreversible", "label": "Effectively irreversible once it has acted.", "flags": [ "hard-to-reverse", "irreversible" ] } ] }, { "id": "deploymentContext", "name": "Deployment context", "question": "Where does the system sit?", "rationale": "External exposure brings disclosure obligations and removes your ability to quietly correct mistakes.", "options": [ { "value": "internal", "label": "Internal only.", "flags": [] }, { "value": "customer", "label": "Customer-facing.", "flags": [ "external" ] }, { "value": "public", "label": "Publicly accessible.", "flags": [ "external", "public" ] } ] } ], "rules": [ { "order": 1, "name": "Prohibited pre-screen", "condition": "any prohibited-practice flag", "result": "Stop — legal review. Nothing downstream can override this." }, { "order": 2, "name": "High-risk short-circuit", "condition": "the output determines an outcome affecting a person, OR special-category data is involved with real-world effect, OR impact of error is critical, OR the system is autonomous and hard to reverse", "result": "High-risk lane with independent review" }, { "order": 3, "name": "Generative overlay", "condition": "the system is generative, or built on a third-party foundation model", "result": "Adds the generative overlay — grounding, confabulation, prompt injection, provenance, IP, vendor model change" }, { "order": 4, "name": "Agentic overlay", "condition": "the system plans, calls tools, holds memory or acts without per-step approval", "result": "Adds the agentic overlay — autonomy classification, agent identity, tool permissions, memory, blast radius, kill-switch" }, { "order": 5, "name": "Fast lane", "condition": "no personal data AND internal only AND nobody externally affected AND not generative or agentic AND reversible AND low impact of error", "result": "Fast lane — attestation only" }, { "order": 6, "name": "Default", "condition": "anything else", "result": "Standard lane" } ], "lanes": [ { "id": "prohibited", "label": "Stop — prohibited-practice review", "meaning": "The screen flagged a practice that may be prohibited outright. This is not a risk rating to be mitigated; it goes to legal review before any further work, and may not proceed at all.", "serviceLevel": "Immediate — legal review before any build, procurement or pilot" }, { "id": "high", "label": "High-risk lane", "meaning": "Full assessment across every applicable module, plus independent second-line review. If you are the provider, a conformity route applies before this may be placed on the market.", "serviceLevel": "Scheduled assessment — expect weeks, and plan for it" }, { "id": "standard", "label": "Standard lane", "meaning": "Full assessment at proportionate depth, with the modules and overlays listed below.", "serviceLevel": "Typically 10–20 working days" }, { "id": "fast", "label": "Fast lane", "meaning": "Attestation-only. Register it, name the owners, confirm the boundaries, proceed. The point of a fast lane is that most AI is genuinely low-risk and should not queue behind the things that are not.", "serviceLevel": "Under 10 working days" } ], "assessmentModules": [ { "id": "data", "name": "Data governance", "scope": "Quality, provenance, consent, licensing, retention, cross-border transfer.", "reviewer": "Data governance", "attachesOnFlags": [ "personal", "special", "genai" ] }, { "id": "privacy", "name": "Privacy", "scope": "Lawful basis, impact assessment, data-subject rights, memorisation risk.", "reviewer": "Privacy officer / DPO", "attachesOnFlags": [ "personal", "special" ] }, { "id": "fria", "name": "Fundamental rights impact assessment", "scope": "Effects on rights of affected people, mitigations, and the oversight arrangements that support them.", "reviewer": "Privacy, legal and responsible AI jointly", "attachesOnFlags": [ "determines", "vulnerable" ] }, { "id": "security", "name": "Security", "scope": "Threat model, secure configuration, authentication, secrets, logging, supply chain.", "reviewer": "Security", "attachesOnFlags": [ "genai", "agentic", "external", "personal" ] }, { "id": "legal", "name": "Legal and regulatory", "scope": "Applicable law, sectoral obligations, intellectual property, contractual exposure.", "reviewer": "Legal counsel", "attachesOnFlags": [ "determines", "external", "placing-on-market", "genai" ] }, { "id": "rai", "name": "Responsible AI", "scope": "Fairness, transparency, explainability, oversight design, affected parties.", "reviewer": "Responsible AI function", "attachesOnFlags": [ "influences", "determines", "public-affected", "vulnerable" ] }, { "id": "validation", "name": "Model risk and validation", "scope": "Development, testing, evaluation design, conservatism, monitoring plan.", "reviewer": "Model risk / validation", "attachesOnFlags": [ "influences", "determines", "high-impact", "genai" ] }, { "id": "architecture", "name": "Architecture", "scope": "Fit with reference architecture, reuse, integration, feasibility.", "reviewer": "Enterprise architecture", "attachesOnFlags": [ "provider", "agentic" ] }, { "id": "tprm", "name": "Third-party risk", "scope": "Vendor viability, governance maturity, contract terms, concentration, model change notification.", "reviewer": "Procurement / third-party risk", "attachesOnFlags": [ "deployer" ] }, { "id": "oversight", "name": "Human oversight design", "scope": "Oversight mode, override authority, automation-bias prevention, appeal route.", "reviewer": "Responsible AI with the business owner", "attachesOnFlags": [ "influences", "determines", "semi-autonomous", "autonomous" ] }, { "id": "ops", "name": "Operational readiness", "scope": "Operating model, support, monitoring, cost, capacity.", "reviewer": "Operations", "attachesOnFlags": [ "provider", "external" ] } ] }, "obligationMap": { "description": "A risk tier tells you how much review to do. It does not tell you what you must produce. This maps answers to deliverables.", "obligations": [ { "id": "literacy", "name": "AI literacy", "applies": "Everyone. Providers and deployers alike, at every risk level, including fast-lane systems.", "produce": "Evidence that the people operating and overseeing the system have sufficient understanding to do so — role-appropriate training records, not a completion certificate.", "frameworkAnchor": "EU AI Act — AI literacy duty on providers and deployers", "attachesOnFlags": [ "always" ] }, { "id": "transparency", "name": "Transparency to people", "applies": "Where a person interacts with the system, or where it generates or manipulates content they may take to be genuine.", "produce": "User-facing disclosure, and marking of synthetic content where feasible.", "frameworkAnchor": "EU AI Act — transparency obligations for certain AI systems", "attachesOnFlags": [ "external", "genai", "public-affected" ] }, { "id": "fria", "name": "Fundamental rights impact assessment", "applies": "Deployers of certain high-risk systems, notably public bodies and those providing essential services.", "produce": "A documented assessment of the effect on affected people’s rights, the mitigations, the oversight arrangements, and who is accountable.", "frameworkAnchor": "EU AI Act — fundamental rights impact assessment for deployers", "attachesOnFlags": [ "determines" ] }, { "id": "conformity", "name": "Conformity assessment and declaration", "applies": "Providers of high-risk systems, before the system may be placed on the market or put into service.", "produce": "Technical documentation, the conformity assessment route, an EU declaration of conformity, and CE marking where applicable.", "frameworkAnchor": "EU AI Act — conformity assessment for high-risk systems", "attachesOnFlags": [ "provider+high" ] }, { "id": "registration", "name": "Registration", "applies": "High-risk systems, registered in the EU database before being placed on the market or put into service.", "produce": "A registration entry, kept current as the system changes.", "frameworkAnchor": "EU AI Act — registration of high-risk systems", "attachesOnFlags": [ "provider+high" ] }, { "id": "qms", "name": "Quality management system", "applies": "Providers of high-risk systems.", "produce": "A documented quality management system covering design control, data management, testing, post-market monitoring and incident handling. ISO/IEC 42001 is the natural certifiable vehicle.", "frameworkAnchor": "EU AI Act — quality management system; ISO/IEC 42001", "attachesOnFlags": [ "provider+high" ] }, { "id": "postmarket", "name": "Post-market monitoring", "applies": "High-risk systems, for the whole time they remain in service.", "produce": "A monitoring plan and the evidence it ran — metrics, thresholds, actions taken, and feedback into reassessment.", "frameworkAnchor": "EU AI Act — post-market monitoring", "attachesOnFlags": [ "high" ] }, { "id": "incident", "name": "Serious incident reporting", "applies": "Providers of high-risk systems, on becoming aware of a serious incident or malfunction.", "produce": "A reporting route to the relevant authority with a defined internal clock, plus the record of what was reported and when. Build the route before you need it.", "frameworkAnchor": "EU AI Act — reporting of serious incidents", "attachesOnFlags": [ "provider+high" ] }, { "id": "records", "name": "Record-keeping and logging", "applies": "High-risk systems, with automatic logging over the system’s lifetime.", "produce": "Automatically generated logs retained for the period required, and documentation retained after the system is withdrawn.", "frameworkAnchor": "EU AI Act — record-keeping and documentation retention", "attachesOnFlags": [ "high" ] }, { "id": "oversight", "name": "Human oversight", "applies": "High-risk systems, designed so oversight is effective rather than nominal.", "produce": "Oversight design, named roles able to override, and evidence the override has been exercised.", "frameworkAnchor": "EU AI Act — human oversight for high-risk systems", "attachesOnFlags": [ "high", "autonomous" ] }, { "id": "gpai", "name": "General-purpose model obligations, inherited", "applies": "Anyone building on a third-party foundation model. The provider’s obligations do not become yours, but their documentation is how you discharge your own.", "produce": "The model provider’s technical documentation, acceptable-use terms, training-data summary where published, and a contractual route to notification of model change.", "frameworkAnchor": "EU AI Act — general-purpose AI model obligations", "attachesOnFlags": [ "genai", "deployer" ] }, { "id": "retention", "name": "Retention versus minimisation", "applies": "Any high-risk system processing personal data.", "produce": "A documented position reconciling long record-retention obligations with data-protection minimisation — typically by separating the decision record from the personal data that informed it.", "frameworkAnchor": "EU AI Act record-keeping, read with GDPR minimisation", "attachesOnFlags": [ "high+personal" ] } ] }, "generativeRisks": { "source": "NIST Generative AI Profile — twelve risk categories", "categories": [ { "number": 1, "name": "CBRN information or capabilities", "concern": "Lowered barriers to chemical, biological, radiological or nuclear weapons information." }, { "number": 2, "name": "Confabulation", "concern": "Confidently stated false content, delivered in the same tone as correct content." }, { "number": 3, "name": "Dangerous, violent or hateful content", "concern": "Easier production of radicalising, violent or threatening material." }, { "number": 4, "name": "Data privacy", "concern": "Leakage of personal data memorised in training; inference of sensitive attributes." }, { "number": 5, "name": "Environmental impacts", "concern": "Energy and resource intensity of training and inference." }, { "number": 6, "name": "Harmful bias and homogenisation", "concern": "Amplification of bias and convergence of outputs toward a narrow range." }, { "number": 7, "name": "Human-AI configuration", "concern": "Over-reliance, misplaced trust and poorly designed oversight." }, { "number": 8, "name": "Information integrity", "concern": "Misleading content and synthetic media at scale." }, { "number": 9, "name": "Information security", "concern": "Prompt injection, data poisoning, model theft, AI-assisted attacks." }, { "number": 10, "name": "Intellectual property", "concern": "Use or reproduction of protected material without permission." }, { "number": 11, "name": "Obscene, degrading or abusive content", "concern": "Including non-consensual intimate imagery." }, { "number": 12, "name": "Value chain and component integration", "concern": "Risk inherited from third-party models, data and components you cannot see into." } ] }, "agentic": { "thesis": "Agentic governance assumes agents will occasionally be creative in unproductive ways. Controls therefore constrain blast radius, not just intent — because intent is the thing you cannot inspect.", "autonomyLevels": [ { "level": "Assistive", "meaning": "Proposes; a human does everything.", "governance": "Treat as a generative system. Oversight in the loop by construction." }, { "level": "Supervised", "meaning": "Acts within a narrow scope, each action confirmed.", "governance": "Tool permissions whitelisted. Confirmation cannot be bulk-approved." }, { "level": "Delegated", "meaning": "Completes multi-step tasks; a human reviews outcomes.", "governance": "Blast-radius limits, transaction caps, full action logging, tested kill-switch." }, { "level": "Autonomous", "meaning": "Plans and acts continuously without routine human involvement.", "governance": "Independent review before release. Sandbox by default. Named operator on call." } ], "controls": [ { "name": "Autonomy classification", "detail": "The single most important governance input for an agent. Classify before anything else, because it determines every other control." }, { "name": "Agent identity", "detail": "A dedicated non-human identity with scoped credentials, rotation and revocation. An agent sharing a human’s credentials is unauditable." }, { "name": "Tool permissions", "detail": "Explicitly whitelisted and least-privilege. A new tool is a change requiring approval, not a configuration tweak." }, { "name": "Memory governance", "detail": "Ephemeral, session and long-term memory categorised, with policy bounding what may be retained and for how long." }, { "name": "Transaction limits", "detail": "High-impact actions bounded by amount, count or category, and reducible immediately on anomaly." }, { "name": "Blast-radius containment", "detail": "Data-modifying actions reversible by default. Anything irreversible requires explicit approval at design time, not at runtime." }, { "name": "Multi-agent orchestration", "detail": "Which agent may call which, the message policy between them, how trust propagates, and how a compromised sub-agent is contained." }, { "name": "Human checkpoints", "detail": "Mandatory confirmation for consequential actions, placed where a human can still realistically intervene." }, { "name": "Behaviour monitoring", "detail": "Agent behaviour as a time series. Anomalies investigated rather than averaged away." }, { "name": "Kill-switch", "detail": "Disables cleanly without corrupting state. Tested periodically. Operators named in advance." }, { "name": "Sandboxing by default", "detail": "New agents run contained. Promotion to production scope requires sign-off." } ], "note": "Autonomy is classified first because it determines every other control." }, "shadowAi": { "problem": "Shadow AI (unsanctioned use) and ambient AI (quietly embedded by vendors into tools you already bought) are the largest real exposure in most organisations, and neither arrives through the front door. A governance programme that only governs what is submitted to it is measuring its own inbox.", "methods": [ { "method": "Procurement and expense analysis", "finds": "Paid AI tools bought on cards or through departmental budgets, below approval thresholds." }, { "method": "SaaS and network telemetry", "finds": "Traffic to model APIs and AI services from managed devices, including from applications you did not know called them." }, { "method": "Identity and OAuth grant review", "finds": "AI tools granted access to corporate data through single sign-on or third-party app consent." }, { "method": "Vendor disclosure requests", "finds": "Ambient AI already shipped into existing platforms — often the largest category and the least visible." }, { "method": "Code and repository scanning", "finds": "Model SDKs, API keys and inference calls embedded in internal applications." }, { "method": "Periodic amnesty", "finds": "Everything the above misses. Run without blame, or it returns nothing." } ], "note": "Discovery is a capability with an owner and a cadence, not a one-off audit. The number that matters is the ratio of discovered to submitted — if it is not falling, the front door is not working." }, "standardsLandscape": { "note": "Described by what each instrument is and what kind of thing it is — binding law, treaty, technical standard, voluntary framework, regulator guidance or industry resource. Deliberately without dates, version numbers or status claims: legislative status, scope and phase-in change, and are the first thing a careful reader will check against the primary source.", "insights": [ { "heading": "There is no single global regime, and there will not be one", "body": "The EU regulates horizontally by statute. The UK applies principles through existing sector regulators. The US has no comprehensive federal law and obligations arrive state by state and regulator by regulator. Asia-Pacific ranges from prescriptive filing regimes to voluntary guidance. Designing for one of these and assuming the rest will follow is the most common strategic error." }, { "heading": "Map controls to frameworks, not frameworks to programmes", "body": "Running a NIST programme, then an ISO programme, then an AI Act programme produces three registers and one exhausted team. Maintain one control set and map each control to every framework it satisfies. When an obligation moves, you re-point a mapping instead of launching a workstream." }, { "heading": "Binding obligations usually arrive through old law first", "body": "Data protection, consumer protection, employment law, product safety and model risk management already apply to AI and are already enforced. Organisations waiting for AI-specific legislation routinely discover they were non-compliant under law that predates the technology by decades." }, { "heading": "Certification is becoming the shorthand", "body": "ISO/IEC 42001 is the only AI management system you can currently certify against, which is why it is becoming the thing procurement functions ask for. Expect it to be requested in tenders before it is required by law." } ], "regions": [ "Global", "European Union", "United Kingdom", "United States", "Asia-Pacific", "Canada" ], "natures": [ "Intergovernmental instrument", "International treaty", "Voluntary framework", "Technical standard", "Binding law", "Regulator guidance", "Industry resource" ], "instruments": [ { "id": "oecd", "name": "AI Principles", "body": "OECD", "region": "Global", "nature": "Intergovernmental instrument", "covers": "Five values-based principles and five policy recommendations, endorsed by a large group of jurisdictions and revised to address safety, information integrity and intellectual property.", "soWhat": "The common vocabulary underneath most national AI policy. If your principles do not map to these, you will struggle to explain yourself internationally.", "supportsControlThemes": [ "fairness", "transparency", "accountability", "validity" ] }, { "id": "unesco", "name": "Recommendation on the Ethics of Artificial Intelligence", "body": "UNESCO", "region": "Global", "nature": "Intergovernmental instrument", "covers": "The broadest multilateral ethics instrument on AI, adopted by member states, covering human rights, environment, gender equality and readiness assessment.", "soWhat": "The reference point when operating in jurisdictions with no AI law of their own, and the one that takes environmental and societal impact most seriously.", "supportsControlThemes": [ "fairness", "transparency", "accountability" ] }, { "id": "coe", "name": "Framework Convention on AI and Human Rights, Democracy and the Rule of Law", "body": "Council of Europe", "region": "Global", "nature": "International treaty", "covers": "The first international treaty on AI, open to signature beyond Europe, obliging parties to adopt measures protecting human rights, democracy and the rule of law in the AI lifecycle.", "soWhat": "Treaty-level rather than guidance, so it shapes national law in signatory states over time rather than applying to you directly today.", "supportsControlThemes": [ "fairness", "accountability", "oversight" ] }, { "id": "g7", "name": "Hiroshima Process International Code of Conduct for Advanced AI", "body": "G7", "region": "Global", "nature": "Voluntary framework", "covers": "Voluntary commitments for organisations developing advanced AI systems: risk identification, red teaming, incident reporting, transparency reporting and content authentication.", "soWhat": "A useful external benchmark for frontier and foundation-model work, and the clearest statement of what \"responsible developer\" is taken to mean at state level.", "supportsControlThemes": [ "safety", "transparency", "genai" ] }, { "id": "iso42001", "name": "ISO/IEC 42001 — AI management system", "body": "ISO/IEC", "region": "Global", "nature": "Technical standard", "covers": "A certifiable management system standard for AI, with management-system clauses and an Annex A control set organised under control objectives spanning policy, organisation, resources, impact assessment, lifecycle, data, information for interested parties, use, and third-party relationships.", "soWhat": "The natural vehicle for a quality management system obligation, and the only AI standard you can currently certify against.", "supportsControlThemes": [ "accountability", "validity", "privacy", "transparency" ] }, { "id": "iso23894", "name": "ISO/IEC 23894 — AI risk management", "body": "ISO/IEC", "region": "Global", "nature": "Technical standard", "covers": "Guidance on managing risk specific to AI, aligned to the ISO 31000 risk management structure.", "soWhat": "The companion to 42001. Where 42001 tells you to manage risk, this tells you how — and it is the bridge to enterprise risk management your risk function already uses.", "supportsControlThemes": [ "validity", "safety", "accountability" ] }, { "id": "iso22989", "name": "ISO/IEC 22989 — AI concepts and terminology", "body": "ISO/IEC", "region": "Global", "nature": "Technical standard", "covers": "Standardised vocabulary and concepts for AI systems.", "soWhat": "Unglamorous and genuinely useful. Most governance arguments are definitional, and a shared vocabulary ends a surprising number of them.", "supportsControlThemes": [ "accountability" ] }, { "id": "iso5338", "name": "ISO/IEC 5338 — AI system lifecycle processes", "body": "ISO/IEC", "region": "Global", "nature": "Technical standard", "covers": "Lifecycle processes for AI systems, extending established software and systems lifecycle standards to AI-specific stages.", "soWhat": "Maps AI governance onto the engineering lifecycle your delivery teams already follow, rather than running alongside it.", "supportsControlThemes": [ "validity", "accountability" ] }, { "id": "iso27001", "name": "ISO/IEC 27001 — information security management", "body": "ISO/IEC", "region": "Global", "nature": "Technical standard", "covers": "Management system for information security, with an annexed control set.", "soWhat": "AI security obligations do not replace information security ones. Most organisations should extend an existing 27001 scope rather than build a parallel regime.", "supportsControlThemes": [ "safety", "privacy" ] }, { "id": "eu-ai-act", "name": "EU AI Act", "body": "European Union", "region": "European Union", "nature": "Binding law", "covers": "Risk-tiered regulation of AI systems — prohibited practices, high-risk obligations including conformity assessment, registration, quality management, logging, human oversight and post-market monitoring, transparency duties, general-purpose model obligations and an AI literacy duty.", "soWhat": "The only comprehensive binding AI law of its kind, and it applies extraterritorially where output is used in the Union. Phase-in has been amended; verify current dates against the Official Journal.", "supportsControlThemes": [ "accountability", "oversight", "transparency", "validity", "privacy", "genai" ] }, { "id": "gdpr", "name": "GDPR", "body": "European Union", "region": "European Union", "nature": "Binding law", "covers": "Lawful basis, data minimisation, purpose limitation, impact assessment, data-subject rights, automated decision-making safeguards and cross-border transfer.", "soWhat": "Predates AI and governs most of it anyway. The automated decision-making provisions and the right to explanation bite long before the AI Act does.", "supportsControlThemes": [ "privacy", "explainability", "oversight" ] }, { "id": "eu-sector", "name": "Sectoral product and safety regimes", "body": "European Union", "region": "European Union", "nature": "Binding law", "covers": "Medical devices, machinery, vehicles, financial services and similar regimes, into which AI obligations are embedded rather than standing alone.", "soWhat": "Where AI is a component of a regulated product, the product regime usually governs first and the AI obligations ride on top of its conformity route.", "supportsControlThemes": [ "validity", "safety" ] }, { "id": "uk-principles", "name": "Pro-innovation cross-sectoral AI principles", "body": "UK Government", "region": "United Kingdom", "nature": "Regulator guidance", "covers": "Five principles applied through existing regulators rather than a single statute: safety, security and robustness; appropriate transparency and explainability; fairness; accountability and governance; and contestability and redress.", "soWhat": "The UK deliberately chose regulator-led application over a horizontal law. That means your obligations come from your sector regulator, not from one AI act — which is easy to miss if you are reading EU material.", "supportsControlThemes": [ "safety", "transparency", "fairness", "accountability", "explainability" ] }, { "id": "ico", "name": "Guidance on AI and data protection", "body": "Information Commissioner’s Office", "region": "United Kingdom", "nature": "Regulator guidance", "covers": "How UK data protection law applies to AI, including lawful basis, fairness in AI, explaining decisions, and an AI and data protection risk toolkit.", "soWhat": "The most practically useful UK material, and the one a UK regulator will actually measure you against.", "supportsControlThemes": [ "privacy", "fairness", "explainability" ] }, { "id": "uk-finance", "name": "Model risk management expectations", "body": "Prudential Regulation Authority / Financial Conduct Authority", "region": "United Kingdom", "nature": "Regulator guidance", "covers": "Model identification, model risk governance, development and validation standards, independent review and ongoing monitoring for regulated firms.", "soWhat": "If you are a regulated firm, your AI governance must reconcile with model risk management rather than sit beside it. Two registers is a finding.", "supportsControlThemes": [ "validity", "accountability", "oversight" ] }, { "id": "uk-aisi", "name": "AI Safety Institute evaluations", "body": "UK AI Safety Institute", "region": "United Kingdom", "nature": "Industry resource", "covers": "Public evaluation research and open-source evaluation tooling for advanced models.", "soWhat": "A credible external reference for what model evaluation looks like when done seriously, and useful input to your own pre-release evaluation design.", "supportsControlThemes": [ "validity", "safety", "genai" ] }, { "id": "nist-rmf", "name": "AI Risk Management Framework", "body": "NIST", "region": "United States", "nature": "Voluntary framework", "covers": "A voluntary framework structured as Govern, Map, Measure and Manage, with categories and subcategories and an accompanying playbook.", "soWhat": "Voluntary, but it has become the de facto structure for AI risk programmes worldwide, including in organisations with no US presence.", "supportsControlThemes": [ "validity", "safety", "fairness", "accountability", "oversight" ] }, { "id": "nist-600-1", "name": "Generative AI Profile (AI 600-1)", "body": "NIST", "region": "United States", "nature": "Voluntary framework", "covers": "Twelve generative-AI-specific risk categories with suggested actions mapped to the AI RMF core functions.", "soWhat": "The best available completeness check for generative risk. If your assessment template cannot answer against all twelve, it has a gap.", "supportsControlThemes": [ "genai", "safety", "privacy", "transparency" ] }, { "id": "nist-800-53", "name": "SP 800-53 security and privacy controls", "body": "NIST", "region": "United States", "nature": "Technical standard", "covers": "A comprehensive catalogue of security and privacy controls for information systems.", "soWhat": "Where AI security controls need to land inside an existing control catalogue rather than a new one, this is usually the catalogue.", "supportsControlThemes": [ "safety", "privacy", "accountability" ] }, { "id": "us-state", "name": "State-level AI and automated decision laws", "body": "US states and cities", "region": "United States", "nature": "Binding law", "covers": "Obligations attaching to consequential automated decisions — notably in employment screening, insurance, lending and biometric processing — including bias auditing and candidate notification duties in some jurisdictions.", "soWhat": "The most commonly missed US exposure. There is no single federal AI law, so the binding obligations arrive state by state and sector by sector.", "supportsControlThemes": [ "fairness", "transparency", "explainability" ] }, { "id": "us-sector", "name": "Sectoral supervision and enforcement", "body": "Federal regulators", "region": "United States", "nature": "Regulator guidance", "covers": "Model risk management expectations in banking, medical device pathways for AI-enabled software, and consumer protection enforcement against unfair or deceptive AI claims.", "soWhat": "Existing regulators have been explicit that current law already applies to AI. Waiting for an AI-specific statute is not a strategy.", "supportsControlThemes": [ "validity", "fairness", "transparency" ] }, { "id": "sg-framework", "name": "Model AI Governance Framework, and the generative AI edition", "body": "Singapore — IMDA / PDPC", "region": "Asia-Pacific", "nature": "Voluntary framework", "covers": "Practical governance guidance covering internal structures, risk-based decision-making, operations management and stakeholder communication, extended for generative AI.", "soWhat": "The most implementable of the national frameworks, and the one most often borrowed by organisations that find NIST too abstract.", "supportsControlThemes": [ "accountability", "oversight", "transparency", "genai" ] }, { "id": "sg-verify", "name": "AI Verify", "body": "Singapore — IMDA", "region": "Asia-Pacific", "nature": "Industry resource", "covers": "An open-source testing framework and toolkit for validating AI system behaviour against governance principles.", "soWhat": "A rare example of governance expressed as runnable tests rather than a questionnaire. Worth studying even if you never adopt it.", "supportsControlThemes": [ "validity", "fairness", "transparency" ] }, { "id": "jp-guidelines", "name": "AI Guidelines for Business", "body": "Japan", "region": "Asia-Pacific", "nature": "Voluntary framework", "covers": "Consolidated guidance for developers, providers and users of AI, oriented to voluntary adoption and agile governance.", "soWhat": "Represents the light-touch, multi-stakeholder end of the spectrum, and distinguishes obligations by role in a way many frameworks do not.", "supportsControlThemes": [ "accountability", "transparency" ] }, { "id": "cn-measures", "name": "Generative AI measures and algorithm filing", "body": "China", "region": "Asia-Pacific", "nature": "Binding law", "covers": "Registration and filing requirements for algorithms and generative services, content labelling, security assessment and training-data requirements.", "soWhat": "Among the most prescriptive regimes in force. If you serve users there, filing and labelling obligations attach regardless of where you are based.", "supportsControlThemes": [ "transparency", "genai", "accountability" ] }, { "id": "ca-dadm", "name": "Directive on Automated Decision-Making", "body": "Government of Canada", "region": "Canada", "nature": "Regulator guidance", "covers": "An algorithmic impact assessment requirement with tiered obligations for federal automated decision systems, covering notice, explanation, human intervention and recourse.", "soWhat": "One of the earliest working examples of tiered impact assessment, and still one of the clearest models for how to scale obligations to consequence.", "supportsControlThemes": [ "explainability", "oversight", "fairness" ] }, { "id": "owasp-llm", "name": "Top 10 for Large Language Model Applications", "body": "OWASP", "region": "Global", "nature": "Industry resource", "covers": "The most common security weaknesses in LLM applications, including prompt injection, insecure output handling, training-data poisoning, model denial of service, supply-chain vulnerability and excessive agency.", "soWhat": "The single most practical security reference for generative systems, and \"excessive agency\" is the closest the security community has come to naming the agentic problem.", "supportsControlThemes": [ "safety", "genai" ] }, { "id": "mitre-atlas", "name": "ATLAS adversarial threat landscape", "body": "MITRE", "region": "Global", "nature": "Industry resource", "covers": "A knowledge base of adversary tactics and techniques against AI systems, structured like ATT&CK.", "soWhat": "Turns \"we did a threat model\" into something testable, using a vocabulary your security team already works in.", "supportsControlThemes": [ "safety" ] } ], "faq": [ { "question": "Which AI governance standards and regulations should we follow?", "answer": "Start with the binding ones in the jurisdictions where your output is used, then adopt a framework to structure the work and a standard to certify against. In practice that usually means the EU AI Act and data protection law where you have European exposure, your sector regulator’s expectations in the UK, state-level and sectoral obligations in the United States, the NIST AI Risk Management Framework as the structure, and ISO/IEC 42001 as the certifiable management system. Map controls to all of them at once rather than running separate programmes." }, { "question": "What is the difference between the EU, UK and US approaches to AI regulation?", "answer": "The EU regulates horizontally through a single risk-tiered statute with conformity assessment, registration and post-market obligations. The UK has deliberately not legislated horizontally, instead applying five cross-sectoral principles through existing regulators, so obligations arrive from your sector regulator and from data protection law. The United States has no comprehensive federal AI statute; binding obligations come from state and city laws on consequential automated decisions, from sectoral supervision, and from existing consumer protection enforcement, with NIST providing a voluntary framework that has become the de facto structure." }, { "question": "Can you be certified against an AI standard?", "answer": "ISO/IEC 42001 is the AI management system standard that organisations can be certified against. Other instruments in this space are frameworks, guidance or risk-management standards rather than certifiable management systems — ISO/IEC 23894 gives risk management guidance, ISO/IEC 22989 gives vocabulary, and the NIST framework is explicitly voluntary and not a certification scheme." }, { "question": "Do AI rules apply to organisations outside the jurisdiction that wrote them?", "answer": "Frequently, yes. The EU AI Act reaches providers and deployers established outside the Union where the system’s output is used within it. Data protection law has long had extraterritorial reach. Prescriptive regimes elsewhere attach to services offered to local users regardless of where the provider sits. Assuming you are out of scope because of where you are headquartered is the wrong test." } ] }, "maturityModel": [ { "level": 1, "name": "Ad hoc", "looks": "AI decisions are made case by case by whoever is in the room. No inventory. No consistent intake.", "risk": "You cannot answer \"how many AI systems do we run\" or \"who owns this one\"." }, { "level": 2, "name": "Documented", "looks": "A policy exists and an intake form exists. Routing is manual and inconsistent. Inventory is a spreadsheet.", "risk": "Governance depends on individuals remembering. It degrades the moment they are busy." }, { "level": 3, "name": "Operating", "looks": "Risk-proportionate triage with published rules. Named owners. Decisions held as records. Periodic review by tier.", "risk": "Still reactive to regulatory change; still weak on discovery of what never came through intake." }, { "level": 4, "name": "Measured", "looks": "Controls mapped to frameworks. Monitoring feeds back into governance. Metrics with targets. Discovery running continuously.", "risk": "The remaining gap is usually agentic systems and third-party model change." }, { "level": 5, "name": "Adaptive", "looks": "Policy-as-configuration absorbs regulatory change. Evaluation is automated and versioned. Agent behaviour monitored as a time series. Assurance is independent.", "risk": "The failure mode becomes complacency — treating the framework as finished." } ], "metrics": [ { "name": "Intake cycle time", "measure": "Submission to decision, by tier", "target": "Fast lane under 10 working days", "signal": "The best predictor of whether teams use the front door or route around it" }, { "name": "Discovered-to-submitted ratio", "measure": "Systems found by discovery versus systems submitted", "target": "Falling quarter on quarter", "signal": "The honest measure of whether intake has authority" }, { "name": "Inventory completeness", "measure": "Systems with all four owner roles named and current", "target": "Above 95%", "signal": "If you cannot name owners, nothing downstream is reliable" }, { "name": "Attestation currency", "measure": "Systems whose periodic review is in date", "target": "Above 90%", "signal": "Stale attestations mean the register describes a past organisation" }, { "name": "Exception ageing", "measure": "Open exceptions past their expiry", "target": "Near zero", "signal": "A growing exception register means the policy is wrong, not that teams are" }, { "name": "Evaluation coverage", "measure": "Production systems with a current pre-release evaluation and regression suite", "target": "Above 90% for medium and high tiers", "signal": "Distinguishes governance that tests from governance that asks" }, { "name": "Incident detection time", "measure": "Behaviour change to detection", "target": "Hours, not weeks", "signal": "Especially for silent vendor model updates" }, { "name": "Oversight evidence rate", "measure": "Share of in-the-loop systems with recorded human disagreement", "target": "Non-zero", "signal": "If humans never disagree with the model, the oversight is decorative" } ], "antiPatterns": [ { "name": "The policy without a register", "looksLike": "A well-written AI policy, no inventory of AI systems.", "costs": "Nothing is enforceable, because nothing is enumerated." }, { "name": "Governance by committee calendar", "looksLike": "Every use case waits for a monthly forum regardless of risk.", "costs": "Teams route around it, and you lose sight of the riskiest work first." }, { "name": "The ethics board with no teeth", "looksLike": "A senior panel that advises but cannot stop anything.", "costs": "Deliberation without consequence, which trains everyone to treat it as theatre." }, { "name": "Model-centric thinking", "looksLike": "Governing the model and ignoring the prompt, retrieval sources, tools and memory around it.", "costs": "The model is rarely what fails. The scaffolding is." }, { "name": "One-shot assessment", "looksLike": "Approved at go-live, never revisited. Vendor changes the model silently.", "costs": "Your approved system is not the system now running." }, { "name": "Shadow-blind governance", "looksLike": "Governing only what is submitted.", "costs": "You are measuring your own inbox while the exposure sits outside it." }, { "name": "Compliance as the ceiling", "looksLike": "Controls stop exactly where the regulation stops.", "costs": "Regulation is a floor written for the last generation of systems. It will not cover your agents." } ], "faq": [ { "question": "What is an AI intake process?", "answer": "An AI intake process is the single structured front door through which every proposed or discovered AI use case enters formal governance. It captures the context needed to understand the proposal, classifies its risk, routes it to the right depth of review, and ends in a documented decision. It handles both submitted intake, where a person proposes something, and discovered intake, where scanning, procurement data or vendor disclosure surfaces AI that never came through the door." }, { "question": "How do I know if my AI system is high-risk?", "answer": "Work through the classification dimensions in order. The decisive questions are whether the output determines an outcome affecting a person’s rights, access, livelihood or safety; whether special-category personal data is involved with real-world effect; whether the worst realistic error causes harm to health, safety or fundamental rights; and whether the system acts autonomously in ways that are hard to reverse. Any one of those routes the system to the high-risk lane. A prohibited-practice screen runs first and overrides everything." }, { "question": "What is the difference between an AI provider and an AI deployer?", "answer": "A provider develops an AI system, or has one developed, and places it on the market or puts it into service under its own name. A deployer uses an AI system under its own authority. The obligations differ substantially — conformity assessment, registration and serious-incident reporting largely fall on providers, while fundamental rights impact assessment and operational oversight fall on deployers. Putting your own name on a third party’s system can make you the provider." }, { "question": "Who should be accountable for an AI system?", "answer": "Four named individuals, not teams: a business owner accountable for value and intended use, a technical owner accountable for operational integrity and holding standing authority to pause, a model or agent owner accountable for model selection, prompts, tool permissions and memory, and a data owner accountable for provenance, licensing and right to use. Second-line functions challenge and report; internal audit assures. Risk accepted by a committee is risk accepted by nobody." }, { "question": "What is shadow AI and how do you find it?", "answer": "Shadow AI is unsanctioned use; ambient AI is capability quietly embedded by vendors into tools you already bought. Neither arrives through the front door, and together they are the largest real exposure in most organisations. Discovery combines procurement and expense analysis, network and SaaS telemetry, identity and OAuth grant review, vendor disclosure requests, repository scanning, and periodic blameless amnesties. The number that matters is the ratio of discovered to submitted — if it is not falling, the front door is not working." }, { "question": "How should agentic AI be governed differently?", "answer": "Classify autonomy first, because it determines every other control. Then give the agent its own non-human identity with scoped credentials, whitelist its tools under least privilege, bound what it may remember, cap consequential transactions, and contain blast radius so data-modifying actions are reversible by default. Agentic governance assumes agents will occasionally be creative in unproductive ways, so controls constrain blast radius rather than relying on intent — because intent is the thing you cannot inspect." }, { "question": "When should a human be in the loop rather than on the loop?", "answer": "In the loop, meaning a human approves each output before it has effect, where the output is binding on a person, materially affects rights, access, pricing or employment, or is effectively irreversible. On the loop, where the output influences a decision a person still makes, or where actions are reversible within a useful window. Over the loop, with sample and aggregate review, where output is internal, low-consequence and high-volume. Oversight that leaves no evidence of a human ever disagreeing with the system is decorative." }, { "question": "Does the EU AI Act apply to organisations outside the EU?", "answer": "Yes, in defined circumstances. The Act reaches providers and deployers established outside the EU where the output produced by the system is used within the EU. Organisations frequently assume they are out of scope on the basis of where they are headquartered, which is the wrong test." } ], "privacy": "The classifier runs entirely in the visitor’s browser. Nothing is transmitted to a server." }