Mapping Mastery: Evaluation of a Radiation Oncology Competency-Based Curriculum's Alignment with Current ACGME Milestones

Background The Accreditation Council for Graduate Medical Education (ACGME) has established Radiation Oncology Milestones (ROMs) for use in evaluation of resident performance. Recently, a radiation oncology curriculum, comprised of consensus-defined content domains and entrustable professional activities (EPAs) was developed by key stakeholders in the United States. EPAs can be used to assess resident competency and preparedness for independent practice. This study hypothesized that there would be substantial alignment between ACGME ROMs and the proposed field specific EPAs. Methods This analysis included four participants, three attending physicians in academic training programs and a current resident trainee. Each participant completed an independent blinded analysis between all 52 radiation oncology specific EPAs and the ACGME ROMs. Descriptions of each of the 21 ROMs were obtained from the ACGME website. Each actionable unit of work defined by the EPAs was mapped to each relevant ROM. Participants were instructed to identify all relevant ROMs for each of the EPAs. Additionally, each participant defined what they believed was the primary ROM for each of the EPAs. These results were compiled and analyzed for agreement across the evaluators. Results Each of the EPAs mapped to at least one ROM. The majority of EPAs aligned with multiple ROMs, with each EPA mapping to a median of 6 (range, 1-13) ROMs. Despite heterogeneity when assessing all relevant ROMs, this analysis found strong agreement between participants on the primary ROM fulfilled by each EPA, with 96.2% of the EPAs having unanimous agreement on the identified primary ROM. Of the 21 ROMs, 19 were found to have at least one primary EPA unanimously identified by all four participants. ROMs were mapped to a median of 2 EPAs. Two ROMs (Professionalism 2 & 3) were not consistently mapped to any of the EPAs by the participants. Discussion This analysis demonstrates that all EPAs can be mapped to at least one ROM and that most ROMs directly map to at least one EPA, supporting the initial hypothesis. Two ROMs were not consistently mapped to an EPA. If these ROMs are determined to be critical to radiation oncology resident education by stakeholders, modification of the EPAs should be considered. Alternatively, if deemed not critical, consideration of removal from the ROMs may be warranted. These findings support that the proposed competence-based educational curriculum adequately encompasses the knowledge and skills outlined in the ACGME ROMs.