INTRODUCTION

Advances in genetics and its clinical applications have been emerging at a tremendous pace as a result of the Human Genome Project, technological developments in DNA sequencing, and growing collections of patient data. There have been substantial gains in understanding rare disease, creation of noninvasive prenatal screening strategies, design of precision medicine treatment approaches, and a resurgence of gene therapy strategies.1 However there has not been a proportional expansion of genetics specialists to deliver these transformative approaches clinically, and training efforts to increase the provider pool are unable to meet clinical need.2,3 It is becoming clear that to fully realize the benefits of genomic medicine, clinicians will need to expand the traditional definition of providers to also include nongenetics specialists and primary care physicians whose practice will be impacted by genomic medicine.4

If we are to encourage meaningful adoption of genomic medicine approaches by nongeneticists, we need to ensure that general medical education properly prepares physicians to implement these strategies in their practice area. Some of these educational efforts will be specialty-specific, but first we need to build a solid foundation in the undergraduate medical education (UME) phase of training. To that end, it is important to consider trends in UME with respect to genetics education. In a recent study, it was determined that 75% of US and Canadian medical schools teach the majority of their genetics content in the first year of study with only 26% teaching genetics at all in the third and fourth year of study.5 This is a significant observation because the first year is typically steeped in foundational basic science with limited connection to clinical content, while the third and fourth years are when most clinical learning takes place. By minimizing the connection to clinical sciences, genetics appears to be a field of science with limited applicability to clinical practice, which can increase barriers to full adoption of genomic medicine into practice. The significant absence of genetics education in the clinical phase of training is a missed opportunity to demonstrate the relevance of the field to nonspecialists, develop basic clinical competencies, and increase the visibility of the field.4,6

To catalyze curriculum development with a focus on clinical genetics education, we must also consider the impact of external assessment mechanisms that can motivate curricular change. Medical school graduates are required to pass the United States Medical Licensing Examinations (USMLE) Steps 1, 2 Clinical Knowledge (CK); Step 2 Clinical Skills (CS); and Step 3, which together demonstrate readiness for unsupervised practice prior to obtaining a medical license in the United States and must reflect competency expectations for the undifferentiated physician.7 Because many aspects of UME share the same goal of developing the competencies needed to independently practice general medicine, medical schools in North America often consider USMLE content in making curricular decisions. To characterize the assessment of these goals with respect to genetics, a team of clinical genetics educators composed of representatives from the Association of Professors of Human and Medical Genetics (APHMG) performed an analysis of the genetics content on a robust sample of recent Step 1, Step 2 CK, and Step 3 examination forms. The complete exam sequence includes 1.5 days of additional testing, including the Step 2 CS exam and the Computer-based Case Simulations portion of the Step 3 exam; material from these portions of the USMLE sequence was not part of the review.

MATERIALS AND METHODS

United States Medical Licensing Examinations

Step 1 focuses on the basic foundation of medical practice, Step 2 CK and CS assess knowledge and skills needed for safe and effective patient care under supervision, and Step 3 measures biomedical and clinical knowledge necessary for independent patient care. Step 1, Step 2 CK, and Step 3 use a multiple choice question testing approach, and Step 2 CS uses an objective structured clinical exam (OSCE). The content of these examinations is overseen by two parent organizations, the National Board of Medical Examiners (NBME) and Federation of State Medical Boards, through their Composite and Management Committees. New content is created by Test Material Development Committees, and all content is periodically reviewed by Interdisciplinary Review Committees. With input from representatives of the NBME, the team from APHMG chose to focus on genetics content in the multiple choice examinations described above.

Review of examination materials

During the visit, the team met with staff members of NBME, reviewed online versions from Step 1, Step 2 CK, and Step 3 of the recent USMLE, and discussed the preliminary findings with staff and a volunteer faculty representative (J.W.) of the USMLE Management Committee. Over the course of two days in November 2018, the team reviewed a total of 2252 items from a sample of Step 1 test forms, Step 2 CK test forms, and Step 3 test forms in the NBME’s facilities. The team of six educators divided into three teams of two so that consensus was reached in each pair on the analysis of each examination item. The questions were reviewed for any genetics content in the questions, answers, or both. Genetics content was defined by a reference to DNA, genes, genome, family history, genetic or hereditary risk, chromosomes, hereditary disorders, congenital malformations, clinical diseases or diagnosis with a known genetic basis, or genetic risk of developing a disorder. The content was also mapped to curriculum recommendations from the peer reviewed APHMG white paper Medical School Core Curriculum in Genetics8 and the USMLE Content Outline.9 The team reviewed whether the genetics content of the items was from the stem of the question or was used only as an incorrect distractor in the option/answer set. Results were also compared with findings from prior visits conducted in 1997 and 2007. Team members signed a nondisclosure agreement, and communication of these results was done with approval of the NBME.

RESULTS

The team identified 13.4%, 10.4%, and 4.4% of Steps 1, 2, and 3 respectively as having genetics content (Fig. 1a). Compared with the previous review10 in 2007, the content for Step 1 was relatively unchanged (13.3%), Step 2 was increased from the previous 7.5%, and Step 3 was modestly decreased from 5.2%. With respect to where in the questions the genetics content was found, when genetics content was identified only within the option set, it was common that knowledge of the specific genetics topic was not required to correctly answer the question. Overall 20.7% of the items with genetics content were in the distractors only. More importantly when broken down by different Step test forms, Step 1 had 11.7% of its genetics content in the distractors only while Steps 2 and 3 had substantially more of their genetics content in the distractors only, at 30.7 % and 30.4% respectively (Fig. 1b). Thus, the genetics content became less pertinent to the questions from Step 1 to 3 with decreasing genetics content and increasing percentages of questions identified as having genetics content in the distractors only, creating a situation where genetics knowledge was no longer required to answer the vast majority of questions correctly.

Fig. 1: Genetics content in USMLE by step exam.
figure 1

(a) Percent of questions with genetics content in either the stem, answers, or both, sorted by Step exam and year of question review. (b) Percent of questions with genetics content separated by stem vs. answers sorted by Step examination reviewed in 2018.

Full size image

Question content was also mapped to the most recent Medical School Core Curriculum in Genetics8 by the APHMG, and a defined list of task and topic areas from the USMLE outlines.9 The APHMG learning objectives (Fig. 2a) and USMLE tasks (Fig. 2b) were closely aligned and organized around six competencies patterned from the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS). The topic areas were broadly extracted from the USMLE content outline. These topic areas do not capture all of the genetics content listed in the USMLE outline as there are many genetic diseases listed as examples throughout. In general, the genetics content mapped across all of these domains but was least represented in areas of professionalism, communication, and systems-based practice. In addition, when mapped to a list of organ systems, content was widely distributed. There were few questions in biostatistics or social sciences categories. The lack of alignment between the USMLE topic list and the APHMG learning objectives led our groups’ coding of test items to occasionally be subject to interpretation, thus those findings are not consistent enough to report.

Fig. 2: Genetics content in usmle step exams by topic area.
figure 2

(a) Number of genetics questions for each collection of step exams subdivided by Association of Professors of Human and Medical Genetics (APHMG) Core Curriculum Learning Objective Areas. (b) Number of genetics questions for each collection of Step exams subdivided by United States Medical Licensing Examination (USMLE) Content Outline Areas.

Full size image

The previous report from the 2007 review highlighted certain topics that were not covered on the exam including imprinting, comparative genomic hybridization, and genetic factors in common disease. From a conceptual basis it was noted in 2018 that there was little content devoted to the genetics of common disease, genetic counseling, appropriate ordering and interpretation of genetic tests, or the proper acquisition and interpretation of a family history.

DISCUSSION

Many medical schools are in the process of initiating curriculum change with integration of basic sciences and clinical skills. Genetics curricula have been particularly affected with decreased class time devoted to genetics and expectations for genetics to be integrated throughout the curriculum.5 With these shifts in focus and education, the distribution of genetics content on the USMLE examinations should also begin to shift. The current distribution with decreasing content from Step 1 through Step 3 reflects traditional curricular approaches with genetics as a basic science course. These observations further exacerbate the potential impression among trainees that genetics content is of limited clinical relevance, even as applications of clinical genetics grow across many specialties. Furthermore, the increasing proportion of questions where genetics is provided as a wrong answer calls into question the applicability of genetics. A limitation to this study was that only a sample of forms was reviewed, so the findings above may not generalize to the overall item pool.

The team recommends that the USMLE program consider increasing the clinical relevance of genetics content within the questions and flattening the distribution of genetics content across the Step exams. This strategy could also incentivize institutions to include additional opportunities to engage with genetics concepts during the clinical phases of training, thus improving the genetic literacy of the undifferentiated medical school graduate. This strategy would represent an increase from the current stated target for genetics content in Step 1 (5–9%)11 and of general principles at large (1–3%) in Steps 212 and 3.13 However, recommending an increase of genetics content need not be at the expense of other disciplines. There are many options for increasing genetics content, particularly in the Step 2 and 3 exams, through use of family history information, genetic testing, and counseling about recurrence risks in the context of cases in a variety of organ systems and practice areas. These issues warrant a further discussion by the USMLE Management Committee to determine the best way to incorporate these critical topics.

Similar to the observations from 2007, there were specific topics in 2018 that should be targeted for content development. There was little to no content devoted to variant interpretation or the terminology utilized in interpretation (pathogenic, variant of uncertain significance, benign), genetic counseling, newborn screening, locus heterogeneity (panel gene testing, technology development, testing and interpretation), cancer treatment, (screening, management based on genetics), or pharmacogenetics. In addition, the family history could be more broadly utilized in many questions to emphasize the importance of this history tool. Notably, some of these topics have emerged as clinically significant since the last revision of the APHMG core curriculum,8 which is currently in the process of being updated. The USMLE Management Committee has reviewed these recommendations, and in some cases dissent on the importance of including certain topics mentioned above. In particular, they questioned the relevance of interpretation of complex tests, genetic approaches to cancer treatment, implications of locus heterogeneity, and pharmacogenomics for the undifferentiated medical graduate. However, it is the position of the team that while implementation of genomic medicine in these areas may fall to specialists, it will be important for nonspecialists to be able to understand the clinical implications of these concepts to avoid potential harms such as overinterpretation of a negative genetic test result or prescription of a drug with a predictable adverse reaction or lack of therapeutic effect.

To fully implement the recommended changes to the USMLE examination, it will be critical to have a cohort of specialists willing to volunteer as item writers for the USMLE Test Material Development Committees. The NBME also encourages the nomination (or self-nomination) of physicians and other health professionals with medical education expertise who may be interested in Test Material Development Committees. These volunteers would be equally valuable in the context of the pathology/genetics committee or the various clinically oriented committees, and once chosen, the NBME provides training, support, and a small honorarium. To be considered, please send the name of the nominee and a CV to VolunteerServices@nbme.org. As a community of geneticists and genetics educators, this service opportunity has the potential for dramatic ripple effects, from motivating the study habits of future US physicians to catalyzing increased representation of genetics in medical school curricula. Together, we can raise the profile of this increasingly essential area of medicine and improve adoption of genomic medicine across a wide range of practice areas.