Main

The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project has begun a shift toward competency-based evaluation of medical education. Although this change is being widely incorporated into residency training, it is just beginning to permeate medical school education. We have not found competency-based milestone templates for medical students specific to an elective in clinical genetics.

Over the past two centuries, assessment of medical trainees has moved from a judgment model based on the approval of superior character to evaluation of performance on exams, and more recently to a focus on outcomes.1 In the new outcome-based model, raters are asked to evaluate trainees in the way they already make day-to-day clinical entrustment decisions rather than against abstract scales.2 In 2011, the ACGME organized specialty groups to develop milestones to guide curriculum development and assessment. The milestones are work-based assessments categorized into six core competencies and rely heavily on observations of clinical faculty.3 Two years later, the ACGME released a set of milestones for residents and fellows in medical genetics.4 Milestones in several other specialties, including emergency medicine, obstetrics/gynecology, and internal medicine, have been adapted at the medical school level.5,6,7 Milestone templates have also been successfully implemented for genetic counseling students.8 We considered that development of milestones and an evaluation-driven process for improvement of the genetics elective would increase student and faculty engagement.

At Johns Hopkins School of Medicine, third- and fourth-year students can choose to schedule a 4-week genetics elective. Typically, five to seven students do so per year. The elective is individually tailored to fit student interests, but it generally includes 1 week of dysmorphology inpatient/consult service, 1 week of metabolic inpatient/consult service, and 2 weeks spent between various genetic outpatient clinics. Each student designs his or her elective experience in conjunction with the rotation director, which ensures that students receive adequate coverage of the major genetic subfields.

The knowledge and skills expected of medical students completing a genetics elective both cross-cut other clerkships and contain unique genetics-specific competencies. Clinical genetics requires specific skills, such as recognizing dysmorphic features, and also poses unique interpersonal challenges, such as delivering news that may affect the risk status and health management of an entire family. After completing a genetics elective, a student should carry a genetics perspective with them to other clinical areas of medicine, ascertain relevant family history in clinical encounters, and consider appropriate genetic testing.9 Pediatric residents completing a 1-month genetics rotation did report increased knowledge and awareness of the importance that medical genetics has in clinical practice.10 Our aim was to create a milestone template that captures both the unique skills and more general capabilities required for a student completing a genetics elective. We also intended to use these milestones to identify gaps in the experience and strengthen the elective.

Materials and Methods

We created a milestone template that captures the unique characteristics of a genetics elective and more widely applicable competencies. We began with the ACGME Level 1 milestones and added and removed what we considered appropriate for the medical student level. The template was piloted among eight medical students and the eight attendings with whom they worked. Students were asked to rank themselves for each milestone as well as to complete a medical student qualitative feedback form about the template and the elective. Each student also selected one faculty member who knew them well to rank them in each milestone and to complete the faculty feedback form. Feedback forms were intended to inquire about both appropriateness and completeness of the milestone template.

Through an iterative process, we incorporated student feedback and modified the milestones to better fit the goals and realities of the elective. We also designed a set of activities to address specific gaps that were identified during the experience. These activities took the form of a “scavenger hunt,” which we based on a similar exercise for the Johns Hopkins University School of Internal Medicine core clerkship in which students actively gather information through exploration of the hospital and various resources. We piloted our hunt with five rotating students. The activities were targeted to ensure coverage of milestones that students felt the elective curriculum did not adequately cover. In line with milestone philosophy of competency, we tried to make these tasks active rather than didactic. For instance, tasks involved presenting a patient to an attending, participating in nutrition counseling, researching a mock case in cancer genetics, and answering insurance questions that a patient may have.

Results

We created the milestone template shown in Figure 1 . Overall, students felt the milestone list was comprehensive, promoted common understanding of attributes they were expected to exhibit, and aided fair evaluation. In response to feedback, we broadened coverage in the milestones of prenatal counseling/reproductive options and genetic laboratory techniques. Students also highlighted several areas in which the elective did not provide enough exposure for them to become competent, corresponding to the following milestones:

  • Recognizes that cancer is the accumulation of genetic mutations, some of which can be inherited

  • Explains insurance coverage as it pertains to genetic and other tests and helps patients sort through financial options

  • Recognizes state, national, public, and private influences on genetic health care management (e.g., US Preventive Services Task Force guidelines, Discretionary Secretary’s Advisory Committee, policies of the Centers for Medicare and Medicaid Services and the Food and Drug Administration, ACGME recommendations)

  • Participates appropriately in team rounds

  • Understands how genetic tests are performed

Figure 1
figure 1

Milestones for medical students completing a medical genetics elective.

Full size image

The lack of exposure in certain areas was largely related to the organization of the elective and of our clinical administrative structure. The elective provides students the opportunity to work in a large variety of clinics and counseling settings and allows the students flexibility in creating their schedules. Overall, this meant that students saw a wide variety of genetics patients. However, electives in the Cancer Center are organized separately and include genetic counseling to varying degrees, so students here reported limited exposure to cancer patients. Additionally, students reported that they had little exposure to insurance issues and actual laboratory testing. Metabolic team rounds were another area where students reported they were not adequately involved. The primary coverage of genetics patients is by pediatric teams and genetics fellows, which meant less opportunity for active involvement of the medical students.

Because we felt that all of these milestones correlated with critical parts of a genetics elective, we opted to increase coverage during the elective rather than remove the milestones. Although longer-term measures may help alleviate some of the structural barriers (e.g., access to cancer patients), we created a scavenger hunt to more immediately add missing domains to the genetics elective ( Figure 2 ). This exercise focused on insurance coverage, genetic laboratory tests, cancer syndromes, and involvement in rounds. There was also a question that emphasized interprofessional interaction, which students felt was a highlight of the elective. This question required students to participate in and reflect on a nutrition counseling session for a metabolic patient. Students who subsequently completed the elective felt that these activities addressed the gaps in the curriculum and that they were able to gain competency relevant to all milestones.

Figure 2
figure 2

Genetics elective scavenger hunt.

Full size image

Faculty evaluators reported that all milestones were appropriate to the medical student level and did not suggest any additions, but they also felt that no one faculty member would be able to observe all domains. They attributed this to the organization of the elective, in which students typically rotate through a variety of clinics and generally spend approximately 3–4 days with a given faculty member. As such, one faculty member suggested that each evaluator should select five milestones that they felt they had observed adequately and then rank the student in these domains only. After we adopted this strategy, all evaluators felt they were able to accurately evaluate students and that the milestone template offered a helpful way to gauge student performance.

In the evaluation of the milestone template, all students reported that this elective would affect their consideration of genetic possibilities in future clinical experiences and that the elective increased their confidence dealing with genetic diagnoses. They perceived usefulness for entering a wide variety of fields. In the words of one student, it “gave me more confidence in the identification and management of genetic disorders. Before this elective, I was apprehensive about working with genetics patients during residency. Now, I am more inclined to seek out opportunities to work with patients with genetic disorders.” One of the elements of the elective that students felt most passionate about was exposure to the unique interpersonal experiences that arise in genetics. Almost every student reported a moving story of interacting with patients with difficult news related to genetic diagnoses. Many noted that this was one of the electives that gave providers the most time to deal with difficult personal issues and that they greatly enjoyed seeing and often participating in these discussions.

Discussion

Developing a milestone template has taken us a step closer to meaningful outcome assessment of the genetics elective and simultaneously allowed us to strengthen the elective. We believe the final milestone template is comprehensive and appropriate for the medical student level.

The organization of our elective, which entails spending time with many attendings and residents, makes single-faculty use of the milestone template challenging. In general, no one person is able to assess a student in all domains. However, both students and evaluators felt that this template was practicable and useful when several domains are evaluated by multiple physicians. This approach also reduces the phenomenon of evaluator fatigue, which has been a major obstacle in implementing milestone templates.11

The milestone template enabled evaluation not only of the students but also of the elective itself. The template provided a systematic way to elicit student and faculty feedback to identify gaps in the curriculum that might have been unnoticed. These included insufficient exposure to cancer genetics, a lack of understanding of genetic testing procedures, and little knowledge about health insurance coverage for genetic services. The scavenger hunt model provided an active and efficient way to address inadequately addressed topics on a rotation. Most students responded positively and enjoyed completing the activities. This strategy of independent learning may be useful in a variety of other clinical areas when gaps in learning are identified.

Overall student feedback to the milestone template and associated survey suggested that the genetics elective was a valuable experience and would have long-lasting effects on their practice in a variety of fields. They especially appreciated the opportunity to be part of interesting and often difficult interpersonal situations that arise in the field of genetics. All students enjoyed the opportunity for interprofessional work, which is at the heart of clinical genetics and is often lacking in medical school electives. As genetics progresses and it becomes increasingly important for all physicians to have adequate understanding of this field, we hope that more students will complete a genetics elective. Making sure that these electives and their related evaluations are of high quality will be paramount.

We hope this is one step toward more effective evaluation and a more comprehensive elective. This project was limited by a small sample size and use in only one program. However, given that genetics electives at many other medical schools may have a similar structure, we hope this milestone template and flexible scavenger hunt method of addressing gaps may be useful more broadly. We share them here in the hope that they will be adapted for use by other schools and further validated in this way. Further work should include rigorous study at a broad range of medical schools, making appropriate modifications, and setting performance standards.

Disclosure

The authors declare no conflict of interest.