A board director of a large, publicly traded company has presented a pressing ethical dilemma to the "Ask an Ethicist" column, questioning the implications of integrating "agentic AI" into their customer relationship management (CRM) system. This query probes the fundamental responsibilities of corporate leadership when digital workers are poised to operate alongside their human counterparts, challenging the perception of this move as a mere technological upgrade and framing it as a profound governance issue with far-reaching ethical consequences. The director’s concern, shared by many boards navigating the rapid advancement of artificial intelligence, centers on whether these intelligent agents are truly designed to enhance customer service or if their deployment harbors a more complex, potentially less transparent agenda.
The scenario presented by the anonymous board director, identified only as "BF," highlights a common contemporary corporate discussion. Management is eager to pilot a new offering from their SaaS CRM provider: agentic AI designed to function as a "digital workforce." These intelligent agents, integrated into customer workflows, are promised to leverage real-time data to autonomously address customer needs, resolve routine issues, and escalate more complex cases to human employees. Crucially, these agents would operate within pre-defined "guardrails" customized for the company. The commercial appeal is significant, with a pay-for-outcomes or usage model replacing traditional software licenses, suggesting a performance-driven integration.
However, BF’s apprehension signals a critical oversight. The board director articulates a need to understand the ethical responsibilities accompanying such a deployment, questioning the oversight model required for an organization with a hybrid human-digital workforce. The core of the dilemma lies in distinguishing between a genuine pursuit of enhanced customer experience and potential, albeit perhaps unintentional, optimizations that could prioritize efficiency over genuine customer satisfaction or employee well-being.
The Governance Shift: From Assistance to Autonomy
Vera Cherepanova, the "Ask an Ethicist" columnist, affirms the significance of BF’s query, emphasizing that this transition from AI as an assistant to AI as an actor necessitates a fundamental re-evaluation of board-level governance. Historically, AI has been employed to augment human capabilities, streamlining tasks and providing analytical support. The emergence of agentic AI, however, marks a pivotal shift. These systems are increasingly empowered to act on behalf of employees, and in some instances, on behalf of the company itself. This autonomy demands a more sophisticated oversight framework capable of discerning varying degrees of AI independence and applying commensurate scrutiny. Cherepanova’s principle of ceteris paribus (all other things being equal) suggests that the greater the autonomy of these AI systems, the more rigorous their governance must become.
The implications of this shift are profound. According to a recent report by Gartner, by 2026, organizations that successfully blend AI-augmented workers with augmented workforces will increase their productivity by 30%. However, this report also cautions that the ethical considerations surrounding AI autonomy, bias, and accountability are paramount to realizing these benefits without unintended negative consequences. The board’s role, therefore, extends beyond approving a technology investment; it encompasses establishing a robust ethical framework for the deployment and continuous monitoring of these increasingly autonomous digital workers.
Beyond Productivity: Unpacking the Customer Experience and Commercial Model
A crucial ethical consideration, as highlighted by Cherepanova, revolves around the true intent behind the deployment of agentic AI. The board, she argues, must honestly interrogate whether these agents are designed to genuinely serve customers better or if their implementation is geared towards creating barriers that might, for instance, make it more difficult for customers to reach human support. This is where the commercial model becomes a critical indicator. If the company is paying for "resolved" outcomes, there is an inherent incentive for the AI system to classify issues as resolved, irrespective of whether the customer perceives their problem as truly solved. This potential misalignment between the company’s financial incentives and the customer’s actual experience can lead to a decline in customer trust and satisfaction, even if key performance indicators (KPIs) related to resolution rates appear to improve.
Consider the example of a telecommunications company implementing such AI. If the AI is incentivized to close tickets quickly, it might offer a standard, pre-programmed solution for a recurring technical issue, even if the customer’s specific situation requires a more nuanced approach. This could lead to a perceived resolution by the system, but a frustrated customer who still faces the underlying problem. Such scenarios, while seemingly efficient from a cost-per-resolution standpoint, erode brand loyalty and can lead to negative reviews and increased customer churn in the long run. Research from the Temkin Group consistently shows that customer experience is a primary driver of customer loyalty and revenue growth, underscoring the importance of genuine customer-centricity over superficial metrics.
The Human Element: Redefining Work and Employee Value
The introduction of digital labor also necessitates a critical examination of its impact on the human workforce. Management often presents a narrative where AI liberates human employees from mundane tasks, allowing them to engage in more creative, meaningful, and enriching work. While this ideal scenario may materialize for some, Cherepanova points out a more common reality: human employees are increasingly tasked with training, managing, improving, and ultimately competing with these AI agents. This raises a vital board question: Is human work genuinely becoming more meaningful, or are employees being relegated to the role of "machine validators" until their eventual phasing out?
A 2023 report by the World Economic Forum, "Future of Jobs," forecasts that by 2027, 23% of jobs will change in response to the adoption of AI and other technologies, with tasks like data entry and administrative support being significantly automated. The report also projects the emergence of new roles, such as AI and machine learning specialists, data analysts, and digital transformation specialists. However, the transition is not always seamless. The ethical imperative for boards is to ensure that the integration of AI does not lead to widespread job displacement without adequate reskilling and upskilling opportunities, nor does it create a two-tiered workforce where human employees feel devalued or perpetually in competition with their digital counterparts. The focus should be on augmenting human capabilities and creating synergistic partnerships, rather than simply replacing human roles.
Escalation: The Ethical Crossroads of AI and Corporate Values
One particularly telling area for board scrutiny is the process of escalation – the mechanism by which an AI agent determines when to cede control to a human employee. While appearing as a technical detail, this juncture is where the ethics of AI intersect directly with a company’s core values. The settings and logic governing these escalations reveal what the organization truly prioritizes.
For instance, a company that values customer satisfaction above all else might configure its AI to escalate any customer interaction that involves a hint of frustration or ambiguity, prioritizing human empathy and problem-solving. Conversely, a company primarily driven by cost reduction might set a high threshold for escalation, allowing the AI to handle a broader range of issues even when human intervention might lead to a more satisfactory outcome for the customer.
The "Ask an Ethicist" column, in a previous response to a cybersecurity professional, touched upon the concept of a "three-cornered relationship" in vulnerability disclosure, where the company, the researcher, and the end-user are all interconnected. Similarly, in the context of agentic AI, the escalation process creates a similar interconnectedness between the AI, the customer, and the human employee. When the AI fails to adequately address a customer’s needs and does not escalate appropriately, the customer’s experience suffers, and the human employee may be left to deal with an escalated situation that has already soured. This underscores the ethical responsibility of the board to ensure that escalation protocols are designed not just for efficiency, but to uphold the company’s commitment to its customers and to ensure the well-being and appropriate utilization of its human workforce.
A Board-Level Imperative, Not a Software Feature
Cherepanova concludes by reiterating that the introduction of a digital workforce should not be framed as a mere software upgrade. It represents one of the most consequential decisions a board director can make, demanding a level of strategic consideration and ethical deliberation commensurate with its impact. The decision to integrate agentic AI is a strategic imperative that touches upon corporate governance, risk management, human resources, and customer relations. Boards must approach this with a comprehensive understanding of the potential benefits and risks, ensuring that the deployment of AI aligns with the company’s values and long-term objectives.
Readers Respond: The Enduring Human Element in Cybersecurity
The preceding "Ask an Ethicist" column addressed a dilemma faced by a cybersecurity professional concerning the fallout of a white-hat disclosure gone awry. The core issue revolved around the ethical justification of a researcher releasing exploit code after feeling mistreated by a company, and whether companies have a reciprocal duty to manage vulnerability disclosures effectively to prevent such situations.
In her response, Cherepanova highlighted that the ethical question extends beyond the researcher’s actions to encompass the company’s responsibility. She posited that companies have an ethical obligation to their customers to handle the human relationship with white-hat researchers in a manner that prevents them from becoming disillusioned and potentially turning "gray," thereby leaving customers vulnerable. This concept of a "three-cornered relationship" – involving the company, the researcher, and the end-user – was central to her analysis.
Cherepanova clarified that while a company is not obligated to meet every researcher’s demand, a credible, fair, timely, and respectful disclosure process is essential. Such a process provides white-hat researchers with a tangible reason to adhere to responsible disclosure norms. This, she argued, is an integral part of a company’s duty of care to its users.
A reader, identified as CG, responded to this previous column by affirming Cherepanova’s perspective: "Thanks, Vera, for reminding us that cybersecurity is ultimately a human and ethical challenge, not just a technical one." This sentiment resonates with the current discussion on agentic AI, underscoring the recurring theme that technological advancements, however sophisticated, are deeply intertwined with human interactions, ethical considerations, and the fundamental responsibilities of corporate leadership.
Broader Implications for Corporate Governance and Ethics
The dilemma posed by "BF" and elaborated by Vera Cherepanova serves as a critical case study for boards across industries. The rise of agentic AI is not a future hypothetical; it is a present reality that demands immediate attention. Companies that embrace this technology without a robust ethical framework risk not only reputational damage and customer attrition but also potential legal and regulatory repercussions as AI governance frameworks evolve globally.
The key takeaway for corporate boards is to move beyond a purely technical evaluation of AI solutions and engage in a deep, ethical assessment. This includes:
- Establishing Clear Accountability: Defining who is responsible when an agentic AI makes an error or causes harm.
- Ensuring Transparency: Being transparent with customers and employees about the capabilities and limitations of AI systems.
- Prioritizing Human Oversight: Maintaining meaningful human oversight and intervention capabilities.
- Developing Ethical AI Guidelines: Creating comprehensive ethical AI guidelines that are integrated into corporate policy and training.
- Regular Ethical Audits: Conducting regular audits of AI systems to identify and mitigate potential biases and unintended consequences.
By treating the deployment of agentic AI as a significant governance and ethical undertaking, boards can navigate this transformative technology responsibly, ensuring that it serves to enhance customer experience, empower human employees, and uphold the integrity of the organization.
