FDA’s AI device rules spur debate over evidence gaps

By early 2026, the FDA's Quality Management System Regulation (QMSR) has operationalized Predetermined Change Control Plans (PCCPs) by embedding them within a lifecycle-oriented quality framework that incorporates ISO 13485 standards. This approach shifts quality management from a final compliance checkpoint to an integrated innovation enabler, requiring coordinated oversight of design planning, risk management, data integrity, supplier capabilities, and postmarket learning to support planned iterative changes in AI-enabled medical devices. Crucially, the QMSR emphasizes differentiating exploratory innovation from committed product changes to prevent costly late-stage regulatory rework, aligning with PCCPs’ goal of managing adaptive AI updates post-authorization while maintaining rigorous control over cybersecurity, human factors, and clinical evidence.

The FDA’s statutory authority for PCCPs, granted under the 2022 FDORA legislation, provides a foundational legal framework to manage adaptive AI model updates post-market, addressing long-standing challenges in regulating rapidly evolving software within medical devices. Despite over 1,000 FDA-approved AI-enabled devices as of 2026, none yet incorporate fully adaptive, continually learning AI models, reflecting the cautious and deliberate operationalization of PCCPs. Recent FDA discussion papers and evolving regulatory proposals underscore a commitment to balancing innovation with patient safety by enabling lifecycle management approaches that anticipate and control AI model changes without compromising regulatory rigor.

Effective implementation of PCCPs under the QMSR framework calls for a dual-track operating model that distinctly separates exploratory innovation from controlled product execution. This model facilitates disciplined handoffs and coherent governance, ensuring that evidence architecture and risk management—guided by ISO 14971:2019—are leadership priorities throughout the device lifecycle. Such structured governance records must consistently link decisions, risks, controls, and verification activities, providing the transparent and traceable evidence base necessary to operationalize PCCPs for adaptive AI devices while fostering continuous innovation within a robust regulatory environment.

AI decision support tools are being integrated into clinical trials at a pace that outstrips the development of robust validation frameworks, creating an 'Evidence Latency Problem' where real-world deployment precedes comprehensive evaluation. For example, Tempus AI expanded its Next platform across multiple cancer types based on a multi-center prospective study aligned with draft FDA ML-DSF guidance that remains unfinalized, highlighting the gap between operational use and regulatory validation. This rush risks building evidentiary records that may not satisfy future regulatory standards, as underscored by the Validation Accords initiative seeking to close this critical gap.

Current validation efforts predominantly focus on workflow metrics—such as biomarker testing rates and time to result—rather than rigorously assessing whether AI models generalize reliably across diverse patient populations or improve clinical outcomes. This narrow focus contributes to a significant evidentiary gap: less than 1% of FDA-cleared AI devices have been evaluated for patient-centered outcomes like mortality or morbidity. Moreover, most clinical evidence is industry-led and centered on accuracy metrics, resulting in an evidence base skewed toward optimism rather than meaningful patient benefit.

The non-deterministic nature of adaptive AI models, which continuously retrain and drift over time, challenges traditional regulatory frameworks designed for static devices. Legacy validation approaches, such as Computer System Validation (CSV), are costly and inefficient, adding at least 30% to project costs by applying blanket testing rather than risk-based strategies. Innovative solutions like Sware’s Res_Q platform leverage Agentic AI to embed continuous, automated validation and comprehensive audit trails directly into development pipelines, enabling real-time monitoring of model drift and ensuring compliance while addressing the 'black box' transparency concerns mandated by FDA and EMA.

A critical challenge lies in the lack of broadly accepted consensus standards to verify AI model validation within clinical trials, risking data integrity and regulatory compliance. Validation must be context-specific, as models validated in academic medical centers may not perform equivalently in decentralized or community-based trial settings. Regulatory bodies now emphasize transparency, requiring sponsors to document AI system limitations and performance characteristics in terms understandable to reviewers, underscoring the urgent need for standardized, explainable, and traceable AI validation practices to safeguard patient safety and trial integrity.

Stakeholders such as Radiology Partners have actively petitioned the FDA for clearer regulatory frameworks specifically tailored to AI applications in medical imaging, underscoring a broader demand for transparency and defined pathways amid the rapid evolution of AI-enabled medical devices. This call for clarity is amplified by the FDA’s own initiatives, including their exploration of voluntary Foundation Model Device Master Files to enhance oversight of foundational AI models without direct approvals, reflecting the agency’s recognition of the complex governance challenges posed by third-party AI components that manufacturers cannot fully control.

The FDA is pioneering a shift from traditional validation methods toward competency-based evaluation models that assess generative AI’s safety, clinical scope adherence, and consistent performance across diverse patient populations, signaling a nuanced approach to oversight that balances innovation with patient safety. This evolving framework is complemented by stakeholder feedback efforts, such as the agency’s August 2026 discussion paper on generative AI-enabled medical devices, which seeks to address the unique challenges of risk assessment, premarket evaluation, and postmarket monitoring in an adaptive AI landscape.

Debates around accountability and trust remain central as AI increasingly influences healthcare decisions, with experts like Ravi Nabar of QuAIZen highlighting the necessity of robust validation, documentation, and quality-system controls to govern AI-assisted innovation responsibly. Yet, questions persist about who bears ultimate responsibility when AI contributes to device design or clinical decisions—physicians, health systems, or vendors—complicating governance and underscoring the need for coordinated incident reporting and transparency frameworks that build justified trust among patients and clinicians.

International and institutional stakeholders emphasize that effective AI governance in healthcare requires a holistic approach extending beyond isolated regulatory silos to encompass intersecting laws on digital medical products, data privacy, and clinical use. For instance, Korean hospital leaders advocate for risk-based approval and verification processes to prevent unregulated 'shadow AI,' alongside clear documentation of clinician decision authority and AI incident responses, illustrating the global push to design integrated systems that uphold accountability, ensure continuous compliance, and balance innovation with patient safety.