Assessment Process
The service separates scenario development, system testing, clinical review, methodological oversight, applicant communication, and final reporting. This structure supports consistent evaluation and clear accountability.
Creating and Maintaining a Scenario Package
Each scenario package is developed for a defined domain and intended use, reviewed for methodological consistency and clinical validity, then locked and secured. The package is reusable and receives periodic maintenance.
Figure 1 - Eight-step process for creating and maintaining a scenario package, from defining the domain through periodic maintenance.
Figure 1. Creating and maintaining a scenario package.
Assessing an AI-Enabled CME Activity
A company begins with an initial consultation. Once scope, readiness, system identity, and configuration are confirmed, the team runs the standardized 25-scenario battery, completes independent review and adjudication, and issues a final report.
Figure 2 - Nine-step process for assessing an AI-enabled CME activity, from initial consultation through the final report.
Figure 2. Assessing an AI-enabled CME activity.
Assessment Roles
| Role | Primary Responsibility |
|---|---|
| Clinical Director | Provides clinical leadership and program oversight, approves the clinical framework, resolves material clinical issues, and authorizes final clinical conclusions. |
| Evaluation Methodology Lead | Oversees rubric consistency, scenario coverage, analytic methods, data summaries, and interpretation in support of clinical leadership. |
| Senior Clinical Reviewer | Provides independent secondary clinical review and participates in adjudication of material findings. |
| Program Manager | Coordinates intake, readiness, scheduling, applicant communication, documentation, records, and project milestones. |
| Technical Staff | Confirms system identity and configuration, executes scenarios, captures outputs, and supports access controls. |
| Executive Sponsor | Provides institutional governance, strategic oversight, and escalation support without directing individual clinical ratings. |