The rapid integration of artificial intelligence (AI) into the core operations of financial institutions has outpaced the development of robust governance frameworks, leaving many banks vulnerable to significant risks when AI systems falter. A critical oversight, highlighted by Elaine F. Duffus and Aoife May of Wolters Kluwer Financial & Corporate Compliance, is the lack of clearly designated individuals with the authority to stop a malfunctioning AI. This deficiency extends beyond technical solutions like a "kill switch," as regulators are increasingly emphasizing the need for named accountability for AI-driven errors, a requirement that many institutions are currently unprepared to meet.
The Governance Gap: Where AI Outpaces Oversight
Financial institutions have been proactive in deploying AI across various critical functions, including lending, fraud detection, and customer collections. However, this adoption has often preceded the establishment of comprehensive governance structures necessary to oversee these powerful technologies. The core issue, according to compliance experts, is that effective control over AI’s impact on customers relies not just on technological safeguards but on a defined human element: an individual with explicit authority to intervene. This authority must be clearly documented, supported by vendor contracts that grant such powers, and seamlessly integrated into the bank’s response mechanisms for regulatory inquiries.
A significant shortfall identified by industry observers is the inability of many banks to produce a current, risk-tiered inventory detailing where AI is currently deployed across their operations. This lack of comprehensive visibility is particularly concerning for an industry actively seeking to expand its AI adoption. A fundamental tenet of regulatory compliance is understanding the legal landscape governing business operations; consequently, institutions should possess an equally clear understanding of where automated decisions are being made. This deficiency becomes starkly apparent when an AI system malfunctions. In such scenarios, a significant number of banks struggle to identify who is empowered to halt the faulty system or, crucially, who is responsible for communicating with and responding to regulatory bodies. While industry attention has often focused on more complex AI failures, such as model drift or the emergence of autonomous agents, the more fundamental problem of individual accountability for AI malfunctions in high-risk functions remains significantly under-scrutinized.
The AI Kill Switch: A Mechanism Needing a Named Operator
The concept of an "AI kill switch" – the ultimate recourse for halting a technology that has deviated from its intended function – is only effective if accompanied by clear operational protocols. For a kill switch to be a viable safety mechanism, three critical elements must be in place: a designated individual with the authority to take a production model offline; clearly defined conditions that trigger this decision; and a responsible party tasked with explaining the failure to regulators. Industry analysis reveals that for their highest-risk AI systems, few banks have adequately established all three of these prerequisites.
Beyond the immediate act of deactivation, the efficacy of a kill switch is intrinsically linked to the existence of a robust business continuity plan. If institutional management hesitates to disable a failing AI system due to the potential disruption of critical business processes, then the kill switch remains merely a theoretical safeguard, existing only on paper. Financial institutions must proactively identify how affected activities will be sustained in the event a model is taken offline. This could involve implementing manual processing, deploying fallback rules, operating in a reduced-functionality mode, or transitioning to alternative systems. The crucial question is not simply who possesses the authority to stop the AI model but whether the institution can continue to serve its customers, fulfill its regulatory obligations, and effectively manage risks once that decision has been made. A kill switch without a thoroughly tested fallback plan represents an operational vulnerability rather than a genuine safety control.
Regulators have been unequivocal in their stance: when an AI capability goes awry, there must be a named individual who bears accountability. A diffusion of responsibility across committees or processes, where no single person has the explicit power to act, inevitably leads to delays when swift action is most critical. Unlike previous technological challenges where developing problems could be identified and corrected before they permeated entire workflows, AI’s nature allows a model to potentially replicate the same fault across thousands of decisions before a pattern is recognized. This necessitates that the authority to intervene must be firmly established before an incident occurs, rather than being hastily assembled in the midst of a crisis. A recent Wolters Kluwer survey of 230 US banking professionals underscored this preparedness gap. Approximately 72% of respondents identified either regulatory reporting for AI failures or model kill switch protocols as areas where they felt least prepared, highlighting a significant industry-wide concern.
Regulatory Landscape: A Shifting Ground Without a Firm Rulebook
The guidance that financial institutions typically rely upon to navigate complex technological challenges often leaves critical AI applications unaddressed. In April 2024, regulatory bodies such as the Federal Reserve, the Office of the Comptroller of the Currency (OCC), and the Federal Deposit Insurance Corporation (FDIC) issued revised model risk management guidance. While this update superseded a 2011 standard, it notably excluded generative and agentic AI from its purview. The stated rationale for this exclusion was the nascent and rapidly evolving nature of these technologies. The guidance itself is principles-based and non-binding, with a promise of a future request for information on banks’ AI usage. A binding regulatory framework specifically for AI in banking is not anticipated in the immediate future.
However, the exclusion of these advanced AI systems from formal guidance does not imply a lack of oversight. Regulators expect each bank to leverage its existing risk management and governance practices for any areas not explicitly covered by the new guidance. This places the onus squarely back on the institutions themselves, aligning with a supervisory approach that is increasingly risk-based and tailored, mirroring trends observed in European regulatory frameworks. In this environment, banks are encouraged to consult existing, applicable resources. The National Institute of Standards and Technology (NIST) AI Risk Management Framework offers a structured approach to managing AI risks, and the Treasury Department, in collaboration with the Cyber Risk Institute, has developed a financial services-specific AI framework. Despite its sector-specific relevance and free availability, this latter framework appears to be underutilized.
Vendor Dependencies: The Erosion of Internal Control
A significant complicating factor in the accountability chain is the prevalent reliance on third-party vendors for AI solutions. Banks often procure their AI capabilities from external providers, meaning that the power to detect a problem, deactivate a system, and report an incident may reside with the vendor, not the bank. This creates a critical disconnect, as it is the bank that ultimately remains answerable to regulators. Furthermore, a designated internal AI owner possesses no genuine authority if the contract with the third-party vendor does not explicitly grant it. Traditional software agreements typically lack such provisions and require thorough review.
Key contractual terms to scrutinize include stipulations for prompt notification of any AI-related incident, transparency into the model’s inner workings and underlying data, an obligation for the vendor to flag customer complaints indicative of AI issues, and the right for the bank to be informed of any model modifications made by the vendor. A common deficiency arises when banks’ oversight processes verify a vendor’s outputs but fail to examine the vendor’s methodology for achieving those outputs. This internal process, or lack thereof, is precisely where AI systems are most susceptible to failure.
The implications of this missing authority are starkly illustrated in the context of collection activities. A recent Wolters Kluwer survey identified collections and recovery as the functional area where AI poses the greatest risk of customer harm, surpassing credit and underwriting by approximately 10 percentage points. This heightened risk stems from the fact that customers in these situations often have limited disclosure and protection, and are frequently experiencing distress when interacting with AI-driven systems. Even a minor deviation in an automated collection system can lead to disparate treatment of similar customers long before the issue is detected. When a third party engages in AI-driven customer contact and lacks a contractual duty to report escalating complaints, and if the bank has neither internal oversight of the system nor the ability to halt the harm, the burden falls disproportionately on the most vulnerable customers.
Navigating the Path Forward: Establishing Clear Accountability
The path to effective AI governance within financial institutions begins with concrete, actionable steps. As a foundational element of their governance processes, every bank must clearly identify:
- The Designated Authority: Who possesses the explicit authority to halt a malfunctioning AI model?
- Triggering Events: What specific conditions or indicators will necessitate the activation of this authority?
- Reporting Protocols: Who is responsible for promptly reporting the incident and its resolution to regulatory bodies?
These initial steps are not theoretical but are practical and within the immediate reach of most institutions.
The process of establishing effective AI governance must commence with a clear and comprehensive understanding of AI’s footprint within the organization. This involves identifying precisely where AI systems are operating, the specific business functions they influence, and the individuals or teams accountable for their performance and oversight. Banks that proactively address these challenges can shape their AI governance on their own terms, implementing the necessary controls, oversight mechanisms, and resilience measures required to safeguard customers, maintain public trust, and fully realize the transformative benefits that AI can offer. Ultimately, the institutions that will achieve sustainable success in the AI era will not be those that deploy these technologies at the fastest pace, but rather those that can demonstrably maintain firm control over their AI systems when it matters most. This involves a commitment to ongoing evaluation, adaptation, and a clear understanding that human oversight and accountability remain paramount, even in an increasingly automated world.
