AI Governance for Regulated Industries: A Practical Guide for 2027

AI governance in a regulated industry is the set of rules, controls and records that make every AI-assisted action accountable to a named person and provable to a regulator. It is not a policy document on its own, and it is not the safety settings inside a model. It is the system that decides what the AI may do, stops it when it should not, and leaves evidence that an auditor can follow.

This guide is for compliance, risk and engineering leads at banks, NBFCs, insurers, hospitals, universities and other institutions that answer to a supervisor. It covers the rules that apply in India and the EU, the building blocks of a working governance set-up, how to tell real governance from the appearance of it, and a 90-day plan to get started. It is not legal advice; check how each instrument applies to you with counsel.

Why is AI governance different in a regulated industry?

In most businesses, an AI mistake is a product problem. In a regulated institution it is also a compliance event, because the decision touches a customer’s money, health, education record or personal data, and someone will later be asked to justify it.

That changes the question. A product team asks “does the AI work?” A regulator asks “who was in charge when it acted, under what rule, and can you prove it?” We made that argument in Your AI Copilot Can’t Save You From an Audit. Governance is the answer to the second question, built before you need it.

Adoption is still early in many regulated sectors, which is the right time to get this right. When the Reserve Bank of India surveyed regulated entities and fintechs for its FREE-AI committee, 20.8% were deploying AI systems, mainly in customer support, sales, credit underwriting and cybersecurity, while 67% said they were interested in exploring AI use cases (Khaitan & Co, 2025). The gap between those two numbers is where most governance decisions will be made over the next two years.

Which rules apply to AI in regulated industries?

There is no single AI law in India. Instead, a regulated institution sits under several overlapping instruments, and EU rules reach Indian organisations that serve EU markets. The summaries below are deliberately general.

India: Digital Personal Data Protection Act, 2023 and DPDP Rules, 2025

The DPDP Act governs how personal data is processed, whether by people or by AI. MeitY notified the DPDP Rules in November 2025 with a staggered start: some provisions immediately, Consent Manager registration after 12 months, and the main operational duties, including notice and consent, security safeguards, breach reporting and data principal rights, from 14 May 2027 (AMS Shardul, 2025).

Two points matter most for AI. First, the Rules require data fiduciaries to retain personal data, associated traffic data and processing logs for at least one year from the date of processing, for specified purposes (AMS Shardul, 2025). AI processing logs are processing logs. Second, the penalties are significant: the Act’s Schedule sets a ceiling of up to ₹250 crore for failing to take reasonable security safeguards, up to ₹200 crore for failing to notify a breach, and up to ₹200 crore for breaching duties on children’s data (King Stubb & Kasiva). These are maximums set by the Data Protection Board case by case, not automatic fines.

India: CERT-In cyber security directions, 2022

CERT-In’s April 2022 directions require covered entities to report specified cyber incidents within six hours of noticing them, keep ICT system logs securely in India for a rolling 180 days, and synchronise clocks to National Informatics Centre or National Physical Laboratory time servers. They took effect on 27 June 2022 (Linklaters, 2022). An AI system that acts on your infrastructure produces exactly the kind of logs these directions cover, and an AI-related incident is still an incident.

India: RBI’s FREE-AI framework for the financial sector

In August 2025 the RBI published the report of its committee on a Framework for Responsible and Ethical Enablement of Artificial Intelligence (FREE-AI) in the financial sector, setting out 7 Sutras as foundational principles and 26 recommendations under six pillars (RBI press release, 2025). The pillars are Infrastructure, Policy, Capacity, Governance, Protection and Assurance (KPMG, 2025).

The recommendations most relevant to governance teams include board-approved AI policies covering governance, lifecycle management, risk controls and third-party vendor liabilities; AI inventories covering models, use cases, dependencies and risks; AI incident reporting and audit processes; and AI-related disclosures in annual reports (Khaitan & Co, 2025). A committee report is not a direction, but regulated entities would be unwise to treat it as optional reading.

EU: the AI Act, as amended by the Digital Omnibus

The EU AI Act applies to providers and deployers whose AI systems are placed on the EU market or whose outputs are used in the EU, which can include Indian institutions and their vendors. In July 2026 the Digital Omnibus on AI, Regulation (EU) 2026/1744, entered into force and moved the high-risk obligations: to 2 December 2027 for stand-alone high-risk systems listed in Annex III (which include areas such as employment and education) and to 2 August 2028 for AI embedded in regulated products (Hunton Andrews Kurth, 2026). The delay changes timing, not substance.

For high-risk systems, the Act requires automatic logging over the system’s lifetime (Article 12), log retention of at least six months by both providers and deployers (Article 19; Article 26), and human oversight assigned to people “who have the necessary competence, training and authority” (Article 26). Fines reach up to €35 million or 7% of worldwide annual turnover for prohibited practices, and up to €15 million or 3% for breaches of most other obligations, whichever is higher, with lower caps for SMEs (Article 99).

ISO/IEC 42001:2023, the AI management system standard

ISO/IEC 42001, published on 18 December 2023, “specifies the requirements and provides guidance for establishing, implementing, maintaining and continually improving” an AI management system, and is intended for any organisation that provides or uses AI-based products or services (Standards Norway listing). It is voluntary, but it gives you a recognised structure for policy, roles, risk assessment and continual improvement, and it is a useful common language with auditors and enterprise customers.

What does a working AI governance set-up include?

Whatever mix of rules applies to you, the same building blocks keep coming up. We think of them as eight controls. Each one should produce evidence, not just intent.

1. An AI inventory

You cannot govern what you have not listed. Record every AI system in use, including vendor tools and copilots: what it does, which data it touches, who owns it, which model and version it uses, and what it is allowed to change. The FREE-AI recommendations on AI inventories cover models, use cases, dependencies and risks (Khaitan & Co, 2025), which is a good minimum.

2. Risk classification

Not every use needs the same controls. A model that drafts internal meeting notes is not a model that recommends a credit limit. Classify each inventory entry by the consequence of a wrong output for a customer, patient or student, and by whether it can act on its own. Higher classes get stricter controls below.

3. A board-approved AI policy

The policy states what AI may be used for, who decides, what is prohibited, and how exceptions are handled. In financial services the FREE-AI report expects it to be board-approved and to cover governance, lifecycle management, risk controls and vendor liabilities. Elsewhere, board approval is still the simplest way to show that accountability sits at the top.

4. Enforcement outside the model

This is where many set-ups fall short. Rules written in a prompt, or switched on in a vendor’s settings page, can be bypassed by a bug, a clever input or a rushed administrator. Real governance evaluates the rule in a separate layer before the AI is allowed to act, and stops the action if the check fails or errors. We set out what that requires, and six questions to test any vendor, in Non-Bypassable AI Governance: What It Means and How to Test It.

5. Human oversight with named approvers

For consequential actions, a named person with the competence and authority to say no should approve before the action runs. “The system approved it” is not an answer an auditor will accept. A timeout should count as a rejection. The approver should see the AI’s proposal, the evidence behind it and the policy result, and their decision should be recorded.

6. An audit trail that can be replayed

Every AI-assisted action should leave a record that lets an outsider reconstruct it: who asked, what the AI saw and proposed, which rule applied, who approved, what changed. The record should be tamper-evident and kept for the longest retention period that applies. Our AI audit trail checklist lists the fields stage by stage and maps them to the EU AI Act, DPDP and CERT-In.

7. Data residency and vendor control

Know where every piece of data the AI touches is stored and processed, which model providers see it, and under what contract. For institutions holding student records, patient files or core banking data, that often rules out sending data to general-purpose public AI services. We have written about why in The 2026 AI Data Crisis: Why Public Clouds Are Failing Regulated Institutions and, from a national perspective, in Governance Is the New Product Category. Here Is Why India Cannot Borrow It.

8. Incident response and testing

Decide in advance what counts as an AI incident (a wrong decision at scale, a data exposure, a model acting outside its policy), who is told, and how fast. Align it with existing duties such as CERT-In’s six-hour window and DPDP breach reporting. Then test: run a tabletop exercise, try to bypass your own controls, and reconstruct a real action from the audit trail.

How should governance handle AI agents?

Agents raise the stakes because they do not just suggest; they act. An agent that can read a ticket, query a customer record, draft a reply and send it has crossed from advice into execution, and every step it takes is a potential compliance event.

The building blocks still apply, with three additions:

• Scoped, revocable authority. Give each agent the narrowest permissions that let it do its job, for a limited time, and make them easy to withdraw. Log any attempt to go beyond them.
• Declared actions. List the actions an agent may take, and the states a record may move between, in advance. Anything not on the list is refused rather than attempted.
• Approval at the point of consequence. An agent can gather information and prepare a proposal on its own. The step that changes a customer’s position, sends money or shares data should wait for a named person.

The test is the same as for any other AI system: can you reconstruct what the agent did, step by step, and show who authorised the steps that mattered?

What should you ask AI vendors?

Most regulated institutions will buy at least some of their AI. Contracts and due diligence questionnaires should cover governance, not just security. At a minimum, ask each vendor:

1. Which model providers see our data, where is it processed, and is it used for training?
2. Where do the rules that constrain the AI live, and can your staff or ours switch them off?
3. What happens when a policy check fails or times out?
4. Can we export a complete, tamper-evident record of every AI action on our data, and in what format?
5. How quickly will you tell us about an AI incident, and does that fit our CERT-In and DPDP timelines?
6. Who at your company can access our environment, and is their access logged in the same record?

Write the answers into the AI inventory entry for that system, and revisit them when the vendor changes models or providers.

How can you tell real governance from governance theatre?

Many AI products now describe themselves as governed. The test is whether each control produces evidence and whether it can be switched off.

Signs of real governance:

• The rules live outside the model and are versioned.
• A failed or errored check stops the action.
• Consequential actions have a named approver, visible in the record.
• You can replay what happened on a given day from the audit trail alone.
• Administrators and vendor staff pass through the same gate as everyone else, and their changes are logged.

Signs of theatre:

• Governance is a PDF, a training deck or a vendor’s “responsible AI” page.
• Controls are toggles an administrator can turn off without a trace.
• Logs exist but cannot be joined across the model, the approval and the system that acted.
• Nobody outside the vendor can say where the data went.

If you are assessing a vendor, ask for a live demonstration of a denied action, a timed-out approval and a replayed audit record. A vendor with a real design can show all three in one meeting.

How does this look in different sectors?

The building blocks are the same, but the emphasis shifts.

Banks, NBFCs and insurers. Expect supervisors to look for the FREE-AI themes: board-approved policy, inventory, incident reporting and audit. Credit, fraud and claims decisions are the obvious high-consequence uses. Third-party AI vendors need the same scrutiny as any outsourced critical service.

Hospitals and health providers. Patient data is personal data under the DPDP Act, and an AI suggestion that reaches a clinical decision needs a clinician as the named approver. Data residency questions are usually decisive.

Universities and schools. Admissions, assessment and student support involve personal data, often including minors, which carries the higher DPDP penalty band for children’s data. Where EU students or programmes are involved, education is one of the areas the EU AI Act treats as high-risk.

Public sector and critical infrastructure. CERT-In duties, data localisation expectations and public accountability usually push towards self-hosted or sovereign deployments with strict logging.

In every sector the core question is the same: for each AI action that matters, can you show who was in charge?

What does a 90-day starting plan look like?

You do not need a finished programme to make real progress. This sequence works for most mid-sized institutions.

Days 1 to 30: see what you have

• Build the AI inventory, including shadow use of public tools.
• Classify each entry by consequence and autonomy.
• Map which rules apply to each: DPDP, CERT-In, sector regulator, EU AI Act.
• Name an accountable owner for each high-consequence use.
• Draft the AI policy for board review.

Days 31 to 60: put controls on the highest-risk uses

• For the top three high-consequence uses, move rule checks outside the model and make them fail closed.
• Introduce named human approval for consequential actions.
• Start writing a joined-up audit trail with a shared action ID, using the audit trail checklist.
• Confirm where each system’s data is stored and processed, and close any gaps in vendor contracts.

Days 61 to 90: prove it works

• Have someone outside the team reconstruct three real AI actions from the audit trail.
• Run a tabletop AI incident exercise against your CERT-In and DPDP timelines.
• Try to bypass your own controls: as an administrator, as a vendor, with a malformed request.
• Take the policy and results to the board, with a plan for the remaining inventory.

What are the most common mistakes?

• Starting with a policy and stopping there. A policy without enforcement and evidence will not satisfy an auditor.
• Trusting model-level safety settings as controls. They help, but they sit inside the system they are meant to constrain.
• Logging everything except the approval. The approval is the part the regulator cares about most.
• Forgetting vendor and administrator paths. Integrators and privileged users should face the same gate as everyone else.
• Treating the EU delay as a reason to wait. The obligations did not change, and India’s DPDP operational duties start in May 2027.
• Over-collecting in logs. Prompts and outputs can contain personal data; the audit trail must be governed too.

Frequently asked questions

Is there an AI-specific law in India?

Not a single one. AI in a regulated Indian institution is governed through existing instruments, chiefly the DPDP Act and Rules, CERT-In’s directions and sector regulators’ guidance such as the RBI’s FREE-AI report. Check with counsel for your sector.

Does the EU AI Act apply to an Indian company?

It can, if your AI systems are placed on the EU market or their outputs are used in the EU. Get advice on your specific position, especially for education, employment and financial uses.

Do we need ISO/IEC 42001 certification?

It is voluntary. Many organisations use it as a framework without certifying; others certify because enterprise customers or partners ask for it. Either way, its structure maps well onto the eight controls above.

Can we use public AI tools at all?

Often yes, for low-consequence tasks that do not involve regulated data. For anything touching customer, patient or student records, check data residency, contracts and logging before use, and record the decision in your inventory.

Who should own AI governance?

Accountability should sit with a named senior owner and the board, with compliance, risk and engineering sharing the work. The worst outcome is governance owned by nobody because it sits between teams.

How QUAICU approaches it

Every QUAICU delivery runs on one enforcement model: policy checked before the AI acts, a named engineer approving every change, fail-closed behaviour, and an append-only, hash-chained ledger of every action. The full model is on our trust page, and our page for regulated teams explains how we work with institutions that answer to a regulator. If you want to test your current set-up against this guide, talk to us.

Sources

• Khaitan & Co, ERGO on the RBI FREE-AI framework (Aug 2025)
• RBI, press release on the FREE-AI committee report (13 Aug 2025)
• KPMG India, RBI FREE-AI committee report
• AMS Shardul, Enforcement of the DPDP Act and notification of the DPDP Rules
• King Stubb & Kasiva, Penalties and adjudication under India’s DPDP Act 2023
• Linklaters, India’s CERT-In cyber security directions (2022)
• Hunton Andrews Kurth, EU Digital Omnibus on AI enters into force (2026)
• EU AI Act, Article 12: Record-keeping
• EU AI Act, Article 19: Automatically generated logs
• EU AI Act, Article 26: Obligations of deployers
• EU AI Act, Article 99: Penalties
• Standards Norway, ISO/IEC 42001:2023 listing