Global regulators shift AI oversight into high gear, targeting ‘shadow AI’ and agentic risks in finance

Global financial regulators are converging on core principles for AI governance, yet their operational approaches vary widely, ranging from prescriptive mandates to principles-based and innovation-friendly frameworks. This divergence reflects the challenge firms face in maintaining consistent AI governance across jurisdictions while adapting to local regulatory nuances, underscoring the critical need for enhanced global coordination and adaptable compliance strategies. As highlighted in IRSG’s analysis, this spectrum of regulatory styles—from the UK’s FCA’s principles-led supervision to Singapore’s MAS SAFR framework—illustrates a dynamic landscape where innovation and risk management must be balanced.

Regulators are shifting their supervisory focus from static, point-in-time validations to continuous lifecycle oversight of AI systems, emphasizing ongoing testing, monitoring, and assurance to manage the evolving risks of autonomous and agentic AI. The Financial Stability Board’s non-prescriptive framework, endorsed by the FCA’s Brandon Bhatti as a milestone that avoids stifling innovation, exemplifies this trend by prioritizing quality assurance and operational resilience. Similarly, IOSCO’s new supervisory toolkit mandates continuous governance and third-party risk management, reflecting a coordinated global effort to embed systemic oversight into AI deployment across capital markets.

The emergence of autonomous AI agents in finance is exposing significant gaps in existing regulatory frameworks, prompting authorities like the Bank of England and MAS to pioneer sophisticated governance models that include real-time operational checkpoints, correlated-failure scenario planning, and cross-firm recovery strategies. Deputy Governor Sarah Breeden’s acknowledgment that current rules were not designed for AI systems making independent decisions has catalyzed explorations into market-wide 'kill switches' and enhanced recovery mechanisms to mitigate systemic risks such as market herding and concentration on a few AI vendors. These initiatives, including collaborative simulations under Project Logos, signal a proactive, globally coordinated response to the unique challenges posed by agentic AI.

Operational resilience and vendor risk management have ascended as central regulatory priorities amid the scaling adoption of AI in financial services, driven by concerns over dependency on a limited number of third-party providers and heightened cyber vulnerabilities. Regulators across the UK, US, and EU are intensifying scrutiny on firms’ due diligence processes and contingency planning, as reflected in updated U.S. model risk management guidance and the EU’s DORA regulation. This focus complements broader supervisory expectations for embedding governance controls within the first line of defense and redefining the second line’s role to emphasize guardrails and risk indicators, ensuring AI systems remain robust and accountable throughout their lifecycle.

Regulators worldwide, including the EU, FSB, IOSCO, and the Bank of England, are converging on a transformative approach to AI governance that mandates continuous validation, monitoring, and lifecycle oversight rather than traditional one-time compliance checks. The EU AI Act and IOSCO’s supervisory toolkit emphasize that AI systems, especially high-risk and agentic AI like large language models, must be treated as core operational infrastructure requiring ongoing quality assurance, transparency in risk identification, and demonstrable human oversight. Brandon Bhatti from the FCA highlights that testing and assurance are no longer optional technical disciplines but foundational pillars of operational resilience and risk management in financial services.

The evolving nature of AI systems, which can drift and generate unpredictable outputs post-deployment, challenges conventional validation paradigms and necessitates robust lifecycle governance frameworks. The Bank of England’s Deputy Governor Sarah Breeden underscores that frameworks designed for traditional model risk are insufficient for autonomous agents, advocating for sophisticated governance including observability, deterministic fallbacks, and rapid incident response. Similarly, MAS’s SAFR framework operationalizes these principles through real-time validation, policy-bound execution, and auditability, supported by collaborative industry pilots to refine continuous oversight mechanisms amid growing AI autonomy.

Operational resilience in AI governance is increasingly linked to proactive cybersecurity and third-party risk management, with regulators urging financial institutions to embed mature controls akin to those for production code changes. The Financial Supervisory Service’s intensified inspections and exemptions for minor disruptions during AI security tests reflect a pragmatic shift toward enabling rapid recovery and consumer protection. Incidents like the Woori Bank data leak, caused by external developer negligence, spotlight the critical need for continuous oversight of vendor risks and internal accountability, especially as larger banks enter comprehensive governance frameworks clarifying executive responsibilities.

Global regulatory bodies are not only tightening supervisory scrutiny but also providing practical, non-prescriptive toolkits to balance innovation with risk mitigation. The FSB’s consultation, described by Bhatti as a milestone that avoids constraining innovation, offers 12 sound practices focused on lifecycle oversight and governance controls. IOSCO’s toolkit similarly spans governance, third-party dependencies, disclosure, and recordkeeping, reflecting a coordinated effort to strengthen operational resilience while supporting the rapid scaling of AI deployment in capital markets.

The deployment of agentic AI in financial institutions demands a robust governance architecture that integrates deterministic control layers, auditable decision logs, and domain-specific knowledge models to ensure accountability throughout complex, multi-step workflows. Backbase’s acquisition of Kasisto exemplifies this approach by embedding banking-grade agentic AI with built-in regulatory controls into its AI-native Banking OS, fostering a 'United Frontline' model that unites shared context and governed authority across functions. This model underscores the necessity of cross-functional collaboration to maintain a unified source of customer truth and ensure compliant, intelligent execution of customer intents across channels.

Governance challenges in scaling agentic AI extend beyond technology capabilities to encompass continuous oversight, clear performance metrics, and realistic timelines that align with legacy benchmarks to secure leadership buy-in and manage expectations. As experts emphasize, tight governance processes with weekly tracking of error rates and incremental improvements are essential to pause and recalibrate deployments when necessary, ensuring that productivity gains materialize only after successful implementation and sustained governance.

Shadow AI—untracked and unauthorized AI tool usage—poses a significant governance risk within financial firms, exacerbated by a governance gap at the senior leadership level where nearly 70% of C-suite executives prioritize speed over security, often bypassing their own AI policies. This disconnect necessitates active monitoring, risk-based vendor oversight, and comprehensive training to enforce AI policies consistently across all organizational layers, as highlighted by Teramind’s report and CIPD guidance, to mitigate data security risks and costly breaches linked to unsanctioned AI use.

Effective accountability frameworks for agentic AI hinge on a two-loop governance model that enables real-time interaction between AI agents and subject matter experts to continuously refine decision logic, ensuring compliance and self-correction within regulated financial workflows. This approach requires clear ownership of AI tool accountability prior to build-versus-buy decisions, simultaneous coexistence of compliance, legacy infrastructure, and human oversight, and cross-functional collaboration among IT, legal, marketing, and business units to maintain transparency, control, and risk management. As Yoav Naveh of Reindeer AI notes, governance—not capability—is the primary bottleneck in scaling agentic AI, demanding practical management of decision logic and robust identity, permission, and escalation controls to reduce liability and build trust.