Transitioning from Faculty-Written Examinations to National Board of Medical Examiners Custom Examinations in Medical Education

Purpose A challenge for medical educators is choosing a method that best evaluates preclinical students’ performance in preparation for Step 1. In previous years, block directors (BDs) of the 2nd year (MS2) neuroscience course at Texas Tech University Health Sciences Center School of Medicine issued faculty-written (FW) examinations during the course. In 2022, BDs replaced FW examinations with National Board of Medical Examiners (NBME) custom examinations. The rationale being that the customized NBME exams would better reflect the national neuroscience curriculum and enhance student preparedness for taking standardized exams. Methods FW examinations (2021) were created by the faculty in the neuroscience course and reviewed by BDs. In contrast, questions that best aligned with the material covered for the 2022 course were selected by BDs using MyNBME SM Services Portal. The custom questions selected are assigned a “difficulty” score by NBME, generating a predicted national average score. At the end of the course, undergraduate medical students in the School of Medicine at Texas Tech University Health Sciences Center completed an online Qualtrics questionnaire to compare the transition of assessment type. Results Participants reported greater satisfaction in their neuroscience education and block organization with NBME examinations. For example, there was a nearly twofold (1.83) increase in the number of students that strongly agreed with the statement “Overall, I am satisfied with the quality of my neuroscience education in this block.” They were also less likely to report the workload as being “much too heavy.” Overall, students expressed a preference for the customized NBME exams as opposed to faculty generated exams (88.1%). Conclusions From the student perspective, building customized assessments through MyNBME SM Services Portal was found to be useful and preferable for evaluating student performance. From block directors’ perspective, it is noted that time is saved assisting faculty in writing valid questions, time defending/justifying FW questions, and time expended generating exams. The only perceived negative regarding the NBME exams is the cost.