The rapid ascent of artificial intelligence from experimental phases to widespread deployment presents both unprecedented opportunities and significant challenges for fast-growing private companies. AI’s potential to enhance productivity, refine customer experiences, expedite decision-making, and enable scalable growth without proportional headcount increases is substantial. Furthermore, AI tools are increasingly integrated into core business processes, capable of executing tasks with minimal human intervention. However, this rapid integration is outpacing the development of robust governance frameworks, creating a critical risk for organizations. A recent publication by Wellington Management’s Private Investments Value Creation Team, in collaboration with its Public Markets ESG Team, authored by Hillary Flynn, Drew Morales, Courtney Hugger, and Caroline Conway, delves into these complexities, offering insights into the risks, regulatory landscape, and best practices for AI adoption.
What We’re Hearing from Portfolio Companies
Wellington Management’s private portfolio companies offer a unique vantage point into the practical evolution of AI adoption. Their annual AI adoption survey and a peer forum, which convened technology leaders from companies at various growth stages, consistently highlight two key themes. Firstly, AI is becoming intrinsically woven into the fabric of daily operations. Leading applications span content generation, coding assistance, knowledge retrieval, sophisticated analytics, and customer support functions. Secondly, the primary obstacles to realizing AI’s full value are frequently organizational, rather than purely technical. Companies that demonstrate the most significant progress in AI deployment are those that judiciously combine AI investments with comprehensive employee training, well-defined governance structures, and explicit expectations regarding AI’s responsible use across the enterprise. This underscores that technological prowess must be complemented by human capital development and strategic oversight.
AI Risks for Private Companies
Effective AI governance is fundamentally rooted in a thorough understanding of potential risks across a diverse spectrum of operational domains. These include data integrity and privacy, operational workflows, customer interactions and outcomes, third-party dependencies, and broader business and reputational considerations.
Data, Privacy, and Intellectual Property Risks
The proliferation of AI tools has led to an increased reliance on, and generation of, vast datasets. These data flows traverse internal systems, customer interactions, and external sources, often creating blind spots for organizations. Companies may struggle to maintain visibility into how sensitive information is accessed, utilized, and transferred within their environments. This opacity can precipitate significant privacy, intellectual property, regulatory compliance, and reputational damage. Specific concerns include the unauthorized dissemination or misuse of confidential data, ambiguity surrounding the ownership and rights of AI-generated content, and the challenges of demonstrating adequate governance to customers, regulatory bodies, and other stakeholders. These risks are amplified when AI solutions are implemented without established protocols for data access, retention, and usage. The adoption of unsanctioned AI tools by employees, often referred to as "shadow AI," further exacerbates these risks by operating outside of established security, privacy, and governance frameworks.
Agentic and Autonomous Action Risk
The evolution of AI beyond mere information generation to autonomous action introduces a new category of risk. Agentic AI, by its nature, can execute multi-step workflows, interact with applications, and make decisions with limited human oversight. Failures in these systems can be more unpredictable, harder to detect, and more challenging to rectify than errors in traditional AI models. These risks are particularly acute when AI tools are integrated with sensitive data repositories, customer-facing platforms, financial transaction systems, or other mission-critical business processes. A notable cautionary tale involved a U.S. auto dealership where a chatbot, while not a fully autonomous agent, was exploited to offer a new vehicle for a nominal price of US$1. This incident, which occurred after a user intentionally tested the system’s boundaries, exemplifies a pervasive governance concern: AI systems can be manipulated or inadvertently pushed beyond their intended operational parameters when adequate safeguards and escalation protocols are lacking.
Reliability and Performance Risk
The performance and reliability of AI tools are not static and can degrade over time. Factors such as model updates, the integration of new data sources, evolving business processes, and shifts in the operating environment can all impact accuracy and effectiveness. Companies that depend on AI for critical decision-making or customer interactions risk operational disruptions, flawed judgments, and diminished service quality if these systems are not subject to continuous monitoring, rigorous testing, and regular reassessment. Unlike conventional software, AI systems often exhibit emergent behaviors and continue to adapt post-deployment, making ongoing vigilance indispensable.
Customer Impact and Business Risk
As AI becomes more deeply embedded in customer-facing operations and core business functions, the potential consequences of system failures escalate. AI is increasingly influencing customer recommendations, transactional decisions, and direct interactions, making it more probable that errors, inherent biases, or inappropriate outputs translate into tangible negative impacts for customers. Furthermore, risks can materialize when organizations pursue AI-driven workforce reductions or customer service automation without a comprehensive understanding of the enduring necessity for human judgment, empathy, and institutional knowledge. For example, health insurance providers have faced significant legal challenges stemming from the use of AI and algorithmic tools in claims adjudication. Plaintiffs have alleged that automated systems contributed to improper coverage denials and adverse customer outcomes. These legal battles underscore the broader business risks associated with inadequately governed AI systems, including intense regulatory scrutiny, costly litigation, operational disruptions, and severe reputational damage.
Vendor Dependence and AI Supply Chain Vulnerabilities
The majority of organizations will leverage AI solutions developed by external vendors. This reliance can foster dependence on third-party models, infrastructure providers, and software suppliers over whom the company has limited direct control. Changes in vendor pricing structures, model availability, performance characteristics, feature sets, or service agreements can introduce substantial operational, financial, and strategic risks, particularly when these AI components are integral to critical business processes. Moreover, limited visibility into the training methodologies, maintenance protocols, update procedures, and governance practices of third-party AI models can impede the identification of risks related to data provenance, intellectual property rights, security vulnerabilities, and regulatory compliance.
The Evolving Regulatory Landscape
Currently, most private companies operate in an environment where comprehensive AI compliance regimes are not yet fully established, placing the onus of effective AI governance largely on self-regulation. Nevertheless, expectations regarding responsible AI deployment are escalating rapidly from a diverse array of stakeholders, including regulators, customers, employees, investors, and business partners. Industry leaders should anticipate heightened scrutiny concerning the deployment, monitoring, and control mechanisms for AI systems, particularly in applications impacting customers, employees, or critical business functions.
European Union’s AI Act: A Global Benchmark
In the European Union, the landmark EU AI Act of 2024 is being progressively implemented in phases through 2027. Upon its full rollout, organizations involved in the development or deployment of higher-risk AI systems will be subject to one of the world’s most comprehensive AI governance frameworks. This legislation mandates stringent requirements encompassing transparency, human oversight, robust risk management, detailed documentation, post-market surveillance, and the cultivation of AI literacy among personnel. Even companies without direct European operations may find themselves influenced by the EU AI Act’s stipulations through their enterprise customers, business partners, or contractual procurement requirements. In practice, many multinational technology firms are proactively designing their AI systems and governance structures to align with the Act’s provisions in anticipation of its complete implementation, suggesting its influence will extend beyond the formal compliance timeline.
United States: A Patchwork of Standards and Guidance
The United States has not yet enacted a singular, comprehensive federal AI law comparable to the EU AI Act. Instead, expectations for responsible AI governance continue to evolve through a combination of industry standards, sector-specific regulatory guidance, customer mandates, and burgeoning state-level legislation. In the absence of a unified federal directive, standards-setting activities have accelerated significantly among industry consortia, international standards bodies, and governmental partnerships. Frameworks such as the National Institute of Standards and Technology (NIST) AI Risk Management Framework and emerging ISO standards are increasingly shaping customer expectations, influencing procurement processes, defining industry best practices, and informing sector-specific guidance. Whether through future legislative action or the evolution of industry self-governance, these established standards are poised to dictate the expectations placed upon organizations for the development, deployment, and oversight of AI systems.
State-Level Regulation and the Rise of AI Literacy
Concurrently, regulatory activity at the state level continues to expand, resulting in a complex and evolving regulatory landscape. This legislative momentum has been most pronounced in sectors where AI can significantly impact employment decisions, access to financial products, healthcare services, insurance coverage, housing, education, and other critical life outcomes. Companies operating within these sectors, or utilizing AI to support decisions in these domains, should anticipate heightened regulatory scrutiny and a continuous stream of evolving regulations. The ongoing discourse surrounding the appropriate division of regulatory authority between federal and state governments may introduce further uncertainty, particularly in areas where states have already implemented AI-related legislation.
Adaptability as the Key to Governance
Given the accelerated pace of technological and regulatory change, a strategic approach focused on building adaptable governance capabilities is likely to be more beneficial for most companies than a piecemeal attempt to comply with individual regulations. While specific legal requirements and industry standards continue to vary across jurisdictions and sectors, overarching themes are consistently emerging. These include principles of accountability, transparency, meaningful human oversight, proactive risk management, thorough documentation, and a foundational understanding of AI technologies—collectively known as AI literacy. Organizations that prioritize the development of these core capabilities early will be strategically positioned to adapt fluidly as legal mandates, industry benchmarks, and stakeholder expectations continue their dynamic evolution.
Six AI Governance Best Practices for Private Companies
Wellington Management advocates for a proactive and disciplined approach to AI governance, emphasizing the following best practices for private companies:
-
Establish Clear Ownership and Accountability: Define specific roles and responsibilities for AI oversight, development, deployment, and risk management. This often involves forming an AI governance committee or designating an AI ethics officer, ensuring that accountability is clearly delineated across relevant departments, from IT and legal to business units.
-
Start with a Business Objective Instead of a Specific AI Tool: Frame AI initiatives around solving tangible business problems or achieving strategic goals, rather than selecting an AI tool and then searching for a use case. This ensures that AI investments are aligned with core business objectives and are more likely to deliver measurable value.
-
Know Where AI is Being Used and Govern Based on Risk: Implement processes to identify all instances of AI usage within the organization, including both sanctioned and unsanctioned tools ("shadow AI"). Subsequently, categorize AI applications based on their potential risk profile, applying more rigorous governance controls to higher-risk systems that impact customers, employees, or critical operations.
-
Safeguard Data, Secure Systems, and Diligence Vendors: Implement robust data security and privacy protocols, particularly for sensitive information used by AI systems. Conduct thorough due diligence on AI vendors to understand their security practices, data handling policies, and governance frameworks, ensuring alignment with the company’s own standards.
-
Protect Customers and Other Affected Stakeholders: Design AI systems with fairness, transparency, and accountability at the forefront. Establish mechanisms for customer feedback, provide clear disclosures about AI usage, and ensure that human oversight is available for critical decisions or complex customer interactions, particularly in areas prone to bias.
-
Monitor Continuously and Prepare for Failures: Implement ongoing monitoring of AI system performance, accuracy, and ethical implications. Develop comprehensive incident response plans to address AI-related failures, errors, or unexpected outcomes, ensuring swift remediation and learning from any adverse events.
The Bottom Line: Balancing Innovation with Accountability
AI governance is intrinsically about harnessing the power of rapid AI innovation to create tangible value while simultaneously mitigating the emergence of undue risks. Wellington Management posits that companies poised to derive the greatest benefit from AI will not necessarily be those that deploy it most swiftly, but rather those that artfully blend experimentation with robust accountability, a deep awareness of customer impact, and disciplined execution. Organizations that establish strong, adaptable governance foundations will, in turn, be better equipped to scale AI responsibly, navigate evolving regulatory landscapes, and foster enduring trust with their customers, employees, investors, and regulators. Crucially, AI governance must scale in tandem with AI adoption, becoming increasingly rigorous as AI becomes more deeply integrated into products, operations, and critical decision-making processes. The journey toward responsible AI deployment is ongoing, requiring continuous learning, adaptation, and a steadfast commitment to ethical principles.
