Page Menu

Preparing for an Assessment

An assessment can begin after the activity, system configuration, governance structure, applicant materials, and testing access are sufficiently defined. The readiness review confirms that the evaluation can be completed under controlled and reproducible conditions.

Activity and Intended Use

  • A description of the AI-enabled CME activity
  • The intended learners and educational purpose
  • The intended clinical domain and types of learner questions
  • The organization responsible for the educational activity and learner-facing content
  • Known limitations, excluded uses, and situations in which the activity should not be used

System Information

  • System and model name, version, and release date
  • The configuration, system instructions, prompt structure, and learner workflow used for the assessment
  • Retrieval sources, citation functions, and evidence-update processes
  • Guardrails, escalation rules, and other controls that influence learner-facing output
  • A description of material changes made during the development or review period

Governance and Monitoring

  • The clinical oversight structure for the educational activity
  • The role with authority to suspend or discontinue the activity when safety concerns arise
  • The learner-facing disclosure that AI is being used, including relevant limitations
  • The process for monitoring outputs, reviewing incidents, correcting errors, and revalidating material changes
  • The ability of the responsible organization to access records of AI outputs and relevant learner interactions
  • Funding, sponsorship, commercial relationships, and other interests relevant to the activity or its content

Testing Access and Documentation

  • A stable test environment that matches the agreed configuration
  • Credentials and instructions needed to access and operate the activity
  • Permission to execute scenarios and capture outputs and associated metadata
  • A technical contact who can address access or configuration questions
  • Applicable logging, retention, and access-control information

Data Protection

The assessment is designed to be completed without protected health information or personally identifiable information. Applicants should not provide patient information, learner information, or other sensitive data unless specifically requested through an approved secure process. Proprietary information should be clearly identified and limited to what is necessary for the assessment.