Rethinking AI governance in the age of agentic systems
July 2026 | SPECIAL REPORT: HEALTHCARE & LIFE SCIENCES SECTOR
Financier Worldwide Magazine
July 2026 Issue
Life sciences companies, like most sectors, moved swiftly to adapt to the generally unanticipated availability of highly capable artificial intelligence (AI) models across the business enterprise.
Most life science companies adopted AI governance policies during the generative AI wave of 2023 and 2024 to handle the new risks introduced by this transformative technology.
But those policies were built around managing risks associated with ‘predictive’ models, models that generate outputs based on predefined training and limited autonomy. Even as those policies were being finalised, the fast pace of AI development has necessitated reassessment of those policies.
In particular, agentic AI systems introduce capabilities such as autonomous task execution, iterative reasoning and planning, and integration with external tools and datasets, therefore exposing governance gaps that traditional AI policies were not designed to address.
Recent industry surveys in the life sciences sector show that 94 percent of life sciences leaders expect AI agents to be essential across their operations, and these systems are already operating inside enterprise resource planning platforms, quality management systems and laboratory information management systems.
Existing AI frameworks often assume that humans remain the primary decision makers, but when AI agents autonomously act across interconnected systems, accountability, meaningful human oversight and real-time governance becomes more difficult to monitor and maintain. These risks are more prevalent in regulated life sciences environments, where insufficient monitoring can create compliance, data security and patient safety concerns.
Furthermore, current Food and Drug Administration (FDA) AI governance frameworks presume that organisations can continuously validate, monitor and maintain accountable oversight over AI-enabled systems throughout the product lifecycle. To address these risks, life sciences companies must rethink their current AI governance framework.
Regulatory considerations
Organisations that deploy autonomous or highly adaptive AI systems remain responsible for maintaining meaningful human oversight, accountability and lifecycle governance.
California’s AB 316 (effective from 1 January 2026), bars defendants from arguing that an AI system acted on its own as a defence to liability – so a company cannot escape responsibility by claiming it did not control what the AI did.
The EU AI Act, which started its tiered enforcement in 2025, imposes strict requirements on high-risk AI systems, including a mandate that such systems be designed for effective human oversight by natural persons “commensurate with the risks, level of autonomy and context of use”. The European Union (EU) Product Liability Directive includes software and AI as ‘products’, opening the door to potential strict liability claims if an AI system is found to be defective.
Although current FDA guidance does not explicitly address ‘agentic AI’ systems, the agency’s evolving total product lifecycle framework emphasises continuous lifecycle oversight, ongoing validation, risk-based credibility assessment, traceability and post-market monitoring for AI-enabled technologies.
In January 2025, the FDA released its draft guidance – ‘Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products’ – establishing a seven-step, risk-based credibility assessment framework for AI models used across the drug product lifecycle.
The guidance applies to nonclinical, clinical, post-marketing and manufacturing contexts and emphasises that AI model credibility should be continuously evaluated and maintained throughout deployment and lifecycle management. Similarly, the FDA’s guidance on predetermined change control plans anticipates that certain AI-enabled systems may evolve over time and requires predefined processes for monitoring, validating and governing post-deployment modifications.
While the FDA and EU frameworks were not developed specifically for agentic AI systems, they highlight governance assumptions that autonomous systems may challenge, including meaningful human oversight, transparency, traceability and accountability structures.
This tension provides more reasons for life sciences companies to improve their AI governance frameworks to address agentic risk across the full product lifecycle. Without such governance structures, companies may face increasing difficulty demonstrating ongoing compliance with FDA and EU requirements.
Considerations for revised AI governance frameworks
Life science companies should build upon their existing compliance infrastructure, GxP validation requirements, data integrity frameworks and quality management systems while adding layers that address the unique risk profile of autonomous systems.
First, establish a risk-tiered framework for agent deployment. Not all agents carry the same risk. A system that autonomously searches literature and summarises findings poses different risks than an agent that can autonomously generate regulatory correspondence. Governance policies should establish tiers that are tied to the potential consequences of autonomous action with corresponding oversight and validation requirements at each tier. High-risk agents should require human in the loop checkpoints throughout the workflow.
Second, redefine accountability. Every agentic workflow should have an individual responsible for its outputs, with authority to pause, redirect or shut down the agent. In GxP environments, this accountability should be documented with the same standard as any other quality responsibility.
Third, design audit trails. Agentic AI compliance requires comprehensive visibility into what agents do and which tools they invoke. In life sciences, where data integrity is already a core regulatory requirement, this audit trail must be capable of supporting regulatory inspection.
Fourth, integrate agent governance into existing validated systems. Agentic AI deployed in regulated environments must be governed as part of the validated system where it operates. This means formal change control for agent updates, documented risk assessments and performance qualification that addresses the agent’s autonomous decision logic.
Finally, plan for multi-agent architecture. Increasingly, the most powerful (and soon to be the most common) agentic deployments involve multiple agents. Governance policies must address not just individual agents but the aggregate behaviour of multi-agent systems, including how accountability flows when one agent’s output becomes another’s input.
In the absence of comprehensive regulatory standards, life sciences companies should proactively update internal AI governance policies to incorporate controls addressing agent autonomy, continuous monitoring, escalation protocols, validation drift and accountable human oversight.
Failure to do so risks increasing regulatory, compliance, product liability and patient safety exposure as autonomous AI capabilities continue to expand across the product lifecycle.
Amir R. Ghavi and Gary Giampetruzzi are partners and Katelyn Katsuki is an associate at Paul Hastings LLP. Mr Ghavi can be contacted on +1 (212) 318 6725 or by email: amirghavi@paulhastings.com. Mr Giampetruzzi can be contacted on +1 (212) 318 6417 or by email: garygiampetruzzi@paulhastings.com. Ms Katsuki can be contacted on +1 (212) 318 6952 or by email: katelynkatsuki@paulhastings.com.
© Financier Worldwide
BY
Amir R. Ghavi, Gary Giampetruzzi and Katelyn Katsuki
Paul Hastings LLP