FDA Proposes Physician-Style Competency Tests for Medical Generative AI

FDA Wants Generative AI Medical Devices to Pass Board-Exam-Style Competency Tests

The FDA is proposing that generative AI products used in healthcare pass "competency-based" evaluations before they can be marketed as medical devices, borrowing the credentialing logic used for human physicians.

The agency published a discussion draft on August 20 outlining the proposed framework and is collecting public comment through October 19, 2026.

Key Takeaways

• Tests target the final user-facing product. The FDA would evaluate the AI application itself, not the underlying foundation model or subcomponents, assessing clinical knowledge, analytic capabilities, safety behavior, and generalizability for its intended use.
• Rigor scales with risk. Products that steer patients toward actions face higher scrutiny than those that only provide information. The framework accounts for the consequences of a user relying on a faulty AI output.
• Real-world evidence would supplement testing. The FDA acknowledged that pre-market tests alone may not be sufficient and indicated it may also collect post-market evidence to verify ongoing performance.
• The FDA is not alone. Multiple academics have proposed physician-style licensure exams for generative AI. The American Medical Association is also evaluating whether AI licenses are the right path forward.

Broader Significance

The FDA's framework is one of the first federal proposals to regulate generative AI at the application layer, evaluating what the AI produces rather than what powers it. This approach could influence how other sectors build AI accountability frameworks as regulators search for mechanisms that work in practice.

Read the full article on Nextgov/FCW