The recent revelation that Meta was testing a "human concierge" for its Muse AI agent, involving contractors handling customer interactions without explicit disclosure, has sent ripples through the corporate world, particularly for organizations relying on AI-enabled services. While Meta has since rolled back the feature and acknowledged the oversight, the incident highlights a critical blind spot in third-party risk management: the tendency to focus solely on vendor and model version changes, often overlooking significant shifts in the human roles supporting these technologies. This oversight can leave companies vulnerable, as the service they approved may evolve in ways that bypass contractual notification clauses and existing risk assessments.
The core issue, as underscored by the Meta situation, is that the factual basis upon which an AI service was initially approved can change without a new model release. For chief compliance officers, audit executives, and third-party risk leads, the pertinent question becomes: does the service being delivered today still align with the conditions that mattered when it received the green light? This extends beyond mere software updates to encompass the evolving responsibilities, access levels, and authorities granted to individuals, both within the vendor organization and its subcontractors, who interact with or manage the AI service.
The Meta Incident: A Catalyst for Re-evaluation
On September 22nd, Reuters reported on Meta’s internal testing of a "human concierge" for its Muse AI agent. Contractors were reportedly engaged to handle some of the phone calls initiated through the AI. This practice, which raised immediate privacy concerns among Meta employees, led to an internal acknowledgment from a Meta vice president that initiating contractor-placed calls without proper disclosure was an "oversight." The feature was subsequently rolled back, and while it was limited to internal testing with Meta employees and not enterprise customers, it serves as a potent case study.
This incident is not isolated. The rapid deployment and integration of AI across various industries mean that the human element supporting these sophisticated systems is constantly in flux. Organizations are increasingly outsourcing functions to AI, but the effectiveness and ethical operation of these systems often depend on human oversight, intervention, and support. When these human roles change in scope or authority, the risk profile of the AI service can shift dramatically, even if the underlying AI model remains the same.
Beyond the Model: The Evolving Landscape of AI Service Delivery
A contractual right to be notified of AI vendor model changes is a valuable safeguard, as noted in recent discussions on bank AI governance. However, a model release is only one avenue through which an approved service can evolve. Consider a scenario where an initial AI service approval was granted based on external reviewers sampling redacted outputs. Six months later, the same role might involve individuals resolving customer cases using full, unredacted records. In this instance, the vendor and the model version have remained constant, but the scope of access has demonstrably expanded, potentially altering the level of authority and the types of sensitive information being handled. Such a shift could necessitate the implementation of different privacy, security, or customer-protection controls, yet a standard model-change notification would not capture this crucial alteration.
The key takeaway is that while tracking every individual joining or leaving a vendor’s workforce is impractical and unnecessary, understanding the role and its evolving responsibilities is paramount. Initial vendor diligence establishes the baseline acceptability of a service. Post-approval, the focus must shift to monitoring whether the facts that underpinned that initial decision remain true.
The "Relationship Owner" Blind Spot and the Need for Concrete Triggers
A significant challenge in third-party risk management is the disconnect between those who observe changes and those responsible for compliance. A Gartner survey of third-party relationship owners revealed that a mere 52% reliably shared observed changes in the scope of a relationship with compliance departments. Relationship owners are often the first to identify alterations in service delivery, but they may lack the expertise to discern which changes are material from a risk and compliance perspective.
Consequently, the directive "tell us if anything changes" is often too vague. To be effective, monitoring instructions need to provide business owners with concrete triggers that signal a need for escalation. This requires a more structured approach to documenting and tracking key facts about the human roles supporting AI services.
Establishing a Robust Framework: Documenting Critical Facts at Approval
The National Institute of Standards and Technology (NIST) AI Risk Management Framework Playbook, specifically under MANAGE 3.1, advocates for regular monitoring of third-party AI risks and emphasizes the importance of documenting technologies, personnel, and other resources. However, it doesn’t specify which personnel-related facts are critical to retain.
For external human roles that materially support an AI service, a robust approval process should mandate the documentation of four key facts:
- The Nature of the Role: A clear definition of the tasks and responsibilities undertaken by the individuals in this role. This should go beyond a generic job title to specify their direct interaction with the AI, the data they process, and the decisions they influence. For instance, is the role performing initial data validation, providing direct customer support, or making final judgments on AI outputs?
- The Level of Access: A precise description of the data and systems to which individuals in this role are granted access. This includes specifying whether access is to raw data, anonymized data, redacted information, or full customer records. Understanding the sensitivity and scope of data access is crucial for privacy and security assessments.
- The Scope of Authority: An outline of the decision-making power vested in individuals in this role. This could range from making recommendations to AI outputs, to having the authority to override AI decisions, or even initiating actions on behalf of customers.
- The Delivery Arrangement: A clear understanding of how the service is delivered, including whether it involves direct vendor employees, subcontractors, or a hybrid model. This also encompasses the geographic location of the personnel performing the role, which can have implications for data sovereignty and regulatory compliance.
Vendors should be required to confirm these facts in writing during the initial approval process. For higher-risk services, it is prudent to agree upon the evidence that will substantiate future checks. This could include updated role descriptions, summaries of access control mechanisms, or relevant assurance reports that specifically cover the service in use. If a vendor is unable to confirm a material fact, this gap should be meticulously documented and communicated to the relevant risk owner. A missing answer should not be construed as confirmation that no changes have occurred.
The "Day One" Test: A Pragmatic Approach to Change Assessment
A critical question to pose for any identified change is: "If this had been true on Day One, could it reasonably have changed the approval decision or the controls attached to it?" Changes that do not meet this threshold can typically be managed as operational matters. However, those that could have influenced the initial approval decision must be escalated to the appropriate risk owners. For example, broader access to sensitive data warrants review by privacy or security teams. New authority to act on behalf of a customer necessitates legal review and an assessment of customer impact. If human support becomes essential or is shifted to a different provider, business continuity and exit plans must be re-evaluated.
This approach allows for a focused review of the affected aspect of the approval, rather than requiring a complete restart of the entire vendor assessment process. Legal and procurement teams play a vital role in aligning vendor notice obligations with these documented facts. Contractual clauses pertaining to incidents, model releases, or named sub-processors may not adequately capture changes in human roles. Therefore, it is essential to agree upon which material changes necessitate notification, moving beyond a vendor’s own definition of a model update.
During periodic reviews, business owners should proactively inquire whether any external role has gained materially broader access or authority, has moved to a different delivery arrangement that impacts risk, or has become significantly more important to service delivery. These inquiries should be substantiated by the agreed-upon evidence. Such critical facts should be integrated into existing third-party risk, privacy, procurement, or operational risk processes, rather than creating a separate, siloed AI governance workflow.
Empowering Internal Audit: Identifying and Testing Mismatches
The Institute of Internal Auditors’ Third-Party Topical Requirement, effective September 15, 2026, establishes a baseline for assessing third-party governance, risk management, and control processes when internal audit functions provide assurance on third-party risk. For AI services selected for review, internal audit should be equipped with two fundamental questions:
- Does the service delivered today still align with the relevant facts documented at the time of approval?
- Where discrepancies exist, did the change reach the appropriate risk owner and result in a documented decision?
Returning to the example of expanded data access, internal audit can investigate when the change occurred, who reviewed the new access parameters, and what decisions were made by the organization. This process does not require reverse-engineering the AI model or obtaining the vendor’s complete workforce roster. Instead, it focuses on the tangible evidence of how the service has evolved in practice.
Meta’s employees had the internal visibility to identify the human concierge test. However, enterprise customers often lack such direct insight. The detailed approval record, therefore, provides organizations with a concrete benchmark against which to test the current state of their AI services. It is this factual alignment, not merely the version number of the AI model, that truly matters for ongoing risk management and compliance. The ability to demonstrate that the service in use continues to meet the foundational facts upon which it was approved is paramount in navigating the complex and rapidly evolving landscape of AI adoption.
