On September 22, 2026, Reuters reported that Meta was actively testing a "human concierge" feature for its new personal AI agent, Muse. This initiative involved contractors handling some of the customer inquiries initially directed to the AI. The report quickly ignited internal concerns among Meta employees regarding privacy. A Meta vice president subsequently acknowledged that initiating contractor-led calls without explicit disclosure represented a "miss" and confirmed that the feature had been temporarily rolled back. While this particular test was confined to internal employees and not yet deployed to enterprise clients, it has illuminated a critical, often overlooked, challenge for companies procuring AI-enabled services: understanding how the operational reality of a service can change, even when the underlying AI model and its version remain static.
For Chief Compliance Officers, Audit Executives, and Third-Party Risk Leads, the fundamental question arising from such situations is whether the AI service currently in operation still aligns with the facts that justified its initial approval. Often, the answer to this question is not dictated by a new model release. Instead, it hinges on shifts in how human resources are deployed and their evolving access and authority within the service’s operational framework.
A comprehensive approach to vendor oversight must extend beyond mere model version tracking to encompass the dynamic nature of human involvement. This is not about meticulously monitoring every individual joining or departing a vendor’s workforce, as staff turnover is an inherent aspect of business operations. Rather, it is about recognizing that changes in the roles and responsibilities of personnel supporting AI services can significantly alter the risk profile of that service.
The critical juncture for risk assessment arises not at the point of initial vendor diligence, which establishes the baseline acceptability of a service, but in the ongoing monitoring phase. The objective then becomes discerning when the factual underpinnings of the initial approval decision are no longer valid.
The Limitations of Model-Change Notifications
While contractual clauses mandating notification of AI model changes are a valuable component of vendor agreements, they represent only one facet of service evolution. Elaine F. Duffus and Aoife May, in a recent Corporate Compliance Insights (CCI) article on bank AI governance, rightly identified model updates as a key vendor term to consider. However, a model release is not the sole determinant of a service’s operational integrity or risk posture.
Consider a scenario where an AI service was initially approved based on external reviewers sampling redacted outputs. Six months later, the vendor’s operational model shifts, and individuals in similar roles are now empowered to resolve customer cases using full, unredacted records. In this instance, the vendor and the AI model version remain unchanged. However, the scope of data access has demonstrably expanded, and potentially, the level of authority vested in these individuals has increased. Such a shift necessitates a re-evaluation of existing privacy, security, and customer protection controls, as the risk landscape has demonstrably altered.
The challenge is compounded by the fact that internal stakeholders, often termed "relationship owners," are frequently the first to observe these changes. A Gartner survey of third-party relationship owners revealed that only 52% reliably shared observed changes in the scope of a relationship with their compliance departments. This disconnect often stems from relationship owners lacking clarity on which types of changes warrant escalation to control functions. Consequently, a blanket instruction to "tell us if anything changes" proves insufficient. Businesses require more concrete triggers for reporting.
Documenting Role-Based Facts for Approval
The National Institute of Standards and Technology (NIST) AI Risk Management Framework Playbook, specifically under MANAGE 3.1, advocates for the regular monitoring of third-party AI risks. It also highlights the utility of documenting technologies, personnel, and other resources to aid in risk management. However, the framework does not precisely delineate which personnel-related facts are essential for ongoing monitoring.
For external human roles that materially support an AI service, a robust approval process should ideally record four key facts:
- The specific nature of the human role: A clear definition of the tasks and responsibilities undertaken by individuals in this role.
- The scope of access to data and systems: A precise outline of the information and platforms these individuals can access.
- The level of authority granted: An understanding of the decisions these individuals are empowered to make.
- The criticality of the role to service delivery: An assessment of how essential this human function is to the overall functioning and performance of the AI service.
Vendors should be contractually obligated to confirm these facts in writing. For services deemed higher risk, the agreement should specify the evidence required for future verification. This might include updated role descriptions, summaries of access control configurations, or relevant assurance reports that cover the specific service being utilized. If a vendor is unable to confirm a material fact, this gap should be documented and communicated to the relevant risk owner. A missing confirmation should not be interpreted as an indication that no changes have occurred.
Evaluating Materiality and Escalation Pathways
The critical question to pose for any observed change is: "If this change had been in effect from the inception of the service, could it have reasonably altered the initial approval decision or the associated controls?" Changes that do not meet this threshold may be managed as routine operational matters. However, those that do warrant escalation to the appropriate risk management team. For instance, broader data access would trigger privacy and security reviews, while new authority to act on behalf of customers would necessitate legal review and an assessment of customer impact. If human support has become indispensable or transitioned to a different provider, a review of business continuity and exit strategies becomes paramount.
Typically, such significant changes can be addressed by revisiting specific aspects of the original approval rather than initiating a complete re-evaluation of the vendor. Legal and procurement departments play a vital role in ensuring that vendor notice obligations align with these critical facts. Contractual clauses concerning incidents, model releases, or named subprocessors might not adequately capture shifts in human roles. Therefore, it is crucial to explicitly define which material changes require vendor notification, moving beyond the vendor’s own definition of a "model update."
During periodic vendor reviews, business owners should be prompted to assess whether any external roles have acquired significantly broader access or authority, if their delivery arrangements have changed in a way that elevates risk, or if they have become substantially more integral to service delivery. Answers should be cross-referenced against the agreed-upon evidence. This oversight function should ideally be integrated into existing third-party risk, privacy, procurement, or operational risk processes, rather than creating a standalone AI governance workflow.
Equipping Internal Audit for AI Service Assurance
The Institute of Internal Auditors’ (IIA) Third-Party Topical Requirement, effective September 15, 2026, establishes a baseline for assessing third-party governance, risk management, and control processes when internal audit functions provide assurance on third-party risks. For AI services selected for review, internal audit can be tasked with two pivotal questions:
- Does the service currently being delivered align with the material facts documented at the time of its approval?
- Where discrepancies exist, was the change properly escalated to the appropriate owner, and was a documented decision made regarding its implications?
Returning to the Meta example, the initial approval might have stipulated that an external role involved sampling redacted outputs. If the current service utilizes individuals in that role to resolve customer cases using full records, internal audit can investigate the timeline of this shift, the individuals who authorized the expanded access, and the organization’s subsequent decisions. This approach allows for meaningful testing without requiring internal audit to reverse-engineer the AI model or obtain the vendor’s complete workforce roster.
While Meta’s employees had internal visibility into their AI service’s testing, enterprise customers often lack such direct insight. The detailed approval record provides organizations with a concrete basis for testing and assurance – verifying that the service in use today continues to meet the factual criteria upon which it was originally approved. This focus on the operational reality, rather than solely the model version, is essential for effective and evolving AI vendor oversight.
