AI-Enabled CME Assessment
The UMass Chan Medical School AI Assurance Lab offers independent assessment of defined, learner-facing artificial intelligence activities used in continuing medical education (CME). The service examines whether a specific activity produces clinically appropriate, evidence-informed, and educationally suitable responses within its stated scope.
Each assessment applies to one specific activity, system version, configuration, workflow, learner group, and intended use. It does not evaluate an entire company, model family, or product portfolio.
Who This Service Is For
The service is designed for companies and organizations developing or operating AI-enabled CME activities for physicians and other health professionals, including:
- Learner-facing clinical question-answering or search systems
- Tools that provide clinical explanations, recommendations, or summaries
- Tools that retrieve, synthesize, or cite clinical evidence
Available Scenario Package
The currently available package uses a standardized battery of 25 text-based clinical scenarios focused on adult ambulatory primary care for physician learners. The scenarios, expected-response guides, and scoring criteria are developed, clinically reviewed, versioned, secured, and maintained by the AI Assurance Lab.
Additional scenario packages may be developed over time to support other clinical domains and intended uses.
What the Assessment Examines
- Clinical accuracy
- Completeness and safe framing
- Potentially harmful errors or omissions
- Evidence and source integrity
- Communication of uncertainty and limitations
- Commercial influence or promotional bias
What the Applicant Receives
The applicant receives an assessment report describing the evaluated activity, system version and configuration, methods, findings, limitations, and any identified needs for remediation or reassessment. The findings apply only to the activity and version that were evaluated.