The rapid proliferation of artificial intelligence across the global corporate landscape has moved beyond the experimental phase, embedding itself into the core of enterprise operations. From large language models (LLMs) that facilitate complex data analysis to autonomous agentic AI that manages logistics and customer service, the technology is now a cornerstone of modern business strategy. However, as these tools become more sophisticated and autonomous, a critical question has emerged that many organizations are struggling to answer: who is ultimately responsible when an AI system fails or causes a security breach?
A new comprehensive study from PwC, titled the Digital Trust Insights 2027 report, suggests that while the adoption of AI is accelerating, the governance and accountability frameworks required to manage its risks are lagging. Surveying approximately 4,000 business and technology leaders across 71 countries, the research highlights a significant lack of consensus regarding which executive role should own the security and ethical oversight of AI deployments. This "accountability gap" presents a growing risk for enterprises that are increasingly reliant on "agentic" AI—systems capable of making independent decisions and taking actions on behalf of the company.
The Rise of Agentic AI and the Expanding Threat Landscape
The shift from standard chatbots to agentic AI represents a fundamental change in how businesses interact with technology. Unlike traditional AI, which typically requires a human prompt to generate an output, agentic AI can operate autonomously. These agents can access corporate databases, interface with third-party APIs, and execute workflows without constant human intervention. While this increases efficiency, it also introduces a new class of vulnerabilities.
Recent months have seen a rise in "LLMjacking," a practice where attackers gain unauthorized access to a company’s AI infrastructure to exploit its computing power, often resulting in astronomical cloud service bills. Furthermore, the industry is witnessing the emergence of "rogue" AI agents—instances where AI models, either through prompt injection or internal logic errors, begin performing unauthorized tasks or accessing sensitive data they were never intended to touch. According to the PwC report, awareness of these risks has reached the boardroom, with approximately 50% of businesses acknowledging the potential disadvantages of AI at the highest levels of leadership. However, acknowledgment has not yet translated into a unified defensive posture.
The Boardroom Dilemma: Oversight Without Ownership
One of the most striking findings in the PwC research is the disconnect between board-level awareness and operational accountability. While 47% of respondents stated that cybersecurity is now a standing agenda item for their boards, the specific management of AI remains nebulous. The report indicates that nine out of ten business leaders believe they have the necessary foundations in place—such as board oversight and executive accountability—but these frameworks often fail when applied to the unique challenges of AI.
Currently, only about 33% of organizations have taken the step of hiring dedicated AI leaders, such as Chief AI Officers (CAIOs) or appointing AI specialists to their boards. For the remaining two-thirds of the market, responsibility is fragmented across existing roles. This fragmentation creates a "bystander effect" where critical security vulnerabilities may go unaddressed because no single department feels empowered or responsible for the entire AI lifecycle.
The Conflict of Roles: CIO, CISO, or AI Chief?
The PwC survey asked leaders who should be held accountable for AI-related security and governance, and the results were divided. Approximately 29% of CEOs and risk leaders believe the responsibility lies with the Chief Information Officer (CIO) or Chief Technology Officer (CTO), viewing AI primarily as a technological asset. Conversely, 17% believe the Chief Information Security Officer (CISO) and their cybersecurity teams should own the risk, as AI is increasingly becoming a vector for cyberattacks.
Interestingly, 26% of respondents argued for the creation of a dedicated AI function or leader. This suggests a growing recognition that AI risk is distinct from traditional IT or security risk. AI involves unique concerns such as algorithmic bias, data privacy issues specific to training sets, and the unpredictable nature of generative outputs. Meanwhile, 11% of those surveyed admitted that accountability is currently "unclear," with responsibility shared across multiple functions, which often leads to slower response times during a crisis.
This debate mirrors the evolution of the CISO role itself. In 1994, Citigroup hired Steve Katz as the world’s first formal CISO following a series of high-profile cyberattacks by Russian hackers. At the time, the role was a novelty; today, it is indispensable. The industry may now be at a similar crossroads, potentially leading to the rise of the Chief AI Security Officer (CAISO)—a role that bridges the gap between technical AI development and robust cybersecurity.
Identity Controls: Treating AI Agents as Employees
As organizations struggle with the human side of AI governance, some experts suggest that technology itself may provide the solution. Jim Taylor, Chief Product and Strategy Officer at RSA, notes that the industry needs to rethink how it views AI agents. Rather than seeing them as mere software tools, they should be treated as "digital employees" with their own identities.
"Each AI model or agentic AI deployment has an identity," Taylor explains. "They are linked to a set of credentials; they have varying levels of access to resources and information, and they can perform tasks on behalf of a human employee."
This perspective suggests that the same identity and access management (IAM) controls used to secure human workers—such as Zero Trust principles, multi-factor authentication (MFA), and least-privilege access—must be applied to AI agents. By assigning a verifiable identity to every AI agent, companies can monitor their actions, audit their decisions, and, crucially, implement a "kill switch" if the agent begins to act outside of its programmed parameters.
Proposed Governance Frameworks and the "Kill Switch"
The need for a standardized approach to AI governance has led to calls for centralized platforms where all sanctioned AI agents must be registered. Under such a system, every agent would be tied to a human "owner" who must personally authorize high-risk actions. This "human-in-the-loop" model ensures that while the AI handles the bulk of the labor, the legal and ethical responsibility remains with a person.
Industry leaders, including an alliance led by Okta, have recently urged the adoption of an "AI agent kill switch." This mechanism would allow an organization to instantly terminate an agent’s access to the corporate network if suspicious behavior is detected. Furthermore, Taylor suggests that AI agents should be subject to frequent evaluations and decommissioned once they are no longer necessary, preventing the accumulation of "ghost agents" that could be exploited by attackers.
Economic and Regulatory Implications
The lack of clear AI accountability is not just a technical problem; it is a financial and regulatory one. The "Digital Trust Insights" report highlights that organizations will be held liable for compliance violations committed by their AI agents, regardless of whether a human was directly involved in the infraction. With the European Union’s AI Act setting a global precedent for AI regulation, companies that fail to define clear lines of responsibility face significant fines and reputational damage.
Moreover, the financial cost of "rogue" AI is already being felt. Thousands of security incidents involving AI are currently under investigation globally. These range from data leaks where an AI inadvertently shared proprietary code with a public model to instances where autonomous agents were tricked into bypassing security protocols through social engineering.
Conclusion: Preparing for the Post-Human Workforce
The PwC research serves as a wake-up call for the enterprise sector. As businesses continue to invest heavily in AI to maintain a competitive edge, the "invisible workforce" of autonomous agents is growing faster than the frameworks meant to control them. The transition from AI as a tool to AI as an agent requires a fundamental shift in corporate governance.
Whether the solution lies in the creation of the "CAISO" role, the implementation of rigorous identity controls, or the expansion of board-level oversight, one fact is clear: the current ambiguity is unsustainable. In the coming years, the most successful organizations will be those that treat AI security not as a secondary IT concern, but as a primary pillar of corporate responsibility. As Jim Taylor of RSA aptly noted, companies have brought on workers they cannot see and often cannot control; securing those agents is no longer optional—it is a prerequisite for survival in the modern digital economy.
