Introduction

Bioinformatics databases are available to shape the landscape of curricula in medical school, particularly in the field of genetics. A structured approach for the student that incorporates use of bioinformatics databases has the potential to empower students to adapt to novel clinical situations and patients.1 This adaptability is critical because the sheer quantity of individual genetic conditions makes it impossible to teach them all effectively, but their collective pervasiveness makes it imperative that trainees are able to address them in a meaningful way. Students’ knowledgeable use of bioinformatics databases also achieves competencies in genomic medicine created by the Inter-Society Coordinating Committee for Physician Education in Genomics, a group convened by the National Human Genome Research Institute. Competencies include the ability to “identify sources of information on genetic disorders, such as OMIM (Online Mendelian Inheritance in Man), and GeneReviews” and “be familiar with the available databases and resources relevant to genetic variation, including ongoing clinical trials involving patients with genetic disorders, pharmacogenomics, and patient-oriented Internet resources from reliable organizations.”2

Here we report 5 years of successive data evaluating an educational intervention to integrate the Online Mendelian Inheritance in Man (OMIM®) database into first-year medical school curricula at The Johns Hopkins School of Medicine. The OMIM database—begun and developed at Johns Hopkins—collates curated primary research on the genetic underpinnings of disease and syndromes to provide a foundation for progress.3 Additionally, it provides comprehensive external linking to resources and databases for users to find expanded information, such as molecular information or clinical summaries within the databases Uniprot and GeneReviews.

Prior studies on genetic educational curricula have demonstrated student receptiveness to bioinformatics databases—with subsequent usage, success on assessments, and increased confidence.1,4,5,6 The natural question is if teaching the use of the OMIM database can be a sustainable educational model with reproducible, measurable impact on clinical practice in the development of students’ confidence in fundamental skills for clinical genetics.

Materials and methods

In 2013 OMIM was integrated into the Johns Hopkins School of Medicine Scientific Foundations of Medicine curricula as a brief orientation to OMIM, followed by four nongraded question sets designed to reinforce and expand on students' use and navigation of OMIM to access clinical and molecular knowledge from reliable resources. The question sets individually address sickle cell anemia, phenylketonuria, mitochondrial disease, and nonpolyposis colon cancer. Each question set is immediately administered after the “clinical correlation session” on the same topic, which includes a classroom presentation on a medical condition and an interactive session with an affected patient regarding their experiences. The answer key is distributed and a faculty member reviews the student answers to the question sets for major gaps in understanding and provides individual student and instructor feedback.

To assess the impact of exposure to OMIM-centered curricular exercises on individual students, a custom piloted survey1 is administered to first-year medical students prior to the classroom orientation to OMIM, and again after the four question sets. The survey assesses baseline characteristics of student resource usage to access genetic information, the frequency of searching for genetic information, usage of OMIM specifically, confidence in broad clinical genetic skills, and the value of OMIM to achieve educational goals. Items are either Likert scale or multiple choice, and an open-ended comment section is included. The same survey is administered after students complete the OMIM-based structured exercises to assess impact of the intervention. Survey results were not paired due to anonymization. Long-term impact was assessed through a one-time survey of Likert scale questions distributed in Spring 2017 to all the third and fourth-year medical students at that time, henceforth referred to as “Spring 2017 survey.” All students had completed the OMIM-centered question sets as first-year medical students and the survey measured confidence in their ability to generate a differential diagnosis that includes genetic conditions and find accurate information on the clinical presentation, diagnostic testing, and management of genetic conditions.

For items that utilized a Likert scale we used SPSS Statistics to run a Mann–Whitney U test of two independent samples to compare the students who “Agree” or “Strongly Agree” they feel confident in various categories of genetic skills before and after the educational intervention.7,8 We also compared by running a Mantel–Haenszel chi-squared test to eliminate any effects due to year.9 For items that used an ordinal scale of frequency, we used SPSS Statistics to run Pearson’s chi-squared test of independence to analyze for difference in responses before and after the educational intervention.7,10 A grounded-theory qualitative approach was followed to analyze the free text comments,11 which were independently evaluated by two authors, classifying comments in up to six different thematic categories. Comments were categorized with 84% interrater reliability, and all differences resolved with quorum discussion.

Results

Over 5 years 555 students, of a total student body of 600, completed the survey prior to the OMIM intervention and 362 students completed the survey immediately after, for response rates of 93 and 60%, respectively. Prior to the intervention, 84 students (15%) reported they had never searched for information on genetic conditions and 228 students (41%) reported they searched for information once every few months. Only 67 students (12%) had ever used OMIM, and 493 students (89%) responded they never use OMIM to search for information on genetic conditions. Immediately after engaging with the OMIM intervention, there was a higher frequency of using OMIM as a source of information, χ2 (5, N = 914) = 514.81, p < 0.001; higher frequency of searching for genetic information, χ2 (5, N = 914) = 131.45, p < 0.001; and higher student satisfaction with their education on the use of medical databases to find information on genetic conditions (p < 0.001) (Table 1).

Table 1 Pooled responses from 2013 to 2017

Mann–Whitney U test analysis after the educational sessions revealed an increase in the number of students who agreed or strongly agreed they (1) felt confident in their ability to build a differential diagnosis that includes genetic conditions (3 to 29%, p < 0.001); (2) felt confident in sharing information on genetic conditions and other supportive resources with patients and families (9 to 52%, p < 0.001); and (3) felt confident in their ability to find accurate information on the clinical presentation, diagnostic testing, and management of genetic conditions (20 to 66%, p < 0.001) (Table 2). The increases in reported confidence in measured skills each year have effect sizes greater than 0.26 (ref. 12).

Table 2 Students across all years that Agree and Strongly Agree with the following statements

After experiencing the OMIM curriculum there was no statistically significant change on Mann–Whitney U test analysis in the percentage of students who agreed or strongly agreed increased use of OMIM in the first 2 years of medical school will better prepare students to (1) generate a differential diagnosis that includes genetic conditions (54% before, 62% after, p = 0.546); (2) find accurate information on the clinical presentation, diagnostic testing, and management of genetic conditions (58% before, 68% after, p = 0.115); and (3) share information on genetic conditions and other supportive resources with patients and families (57% before, 67% after, p = 0.148) (Table 2). Mantel–Haenszel chi-squared analysis, which was used to see if responses differed from pre and post across individual years, demonstrated a statistically significant main effect for students who agreed or strongly agreed increased use of OMIM in the first 2 years of medical school will better prepare students to find accurate information on the clinical presentation, diagnostic testing, and management of genetic conditions (p < 0.05); and share information on genetic conditions and other supportive resources with patients and families (p < 0.05). The total percentage of students who chose Disagree or Strongly Disagree that increased use of OMIM in the first 2 years will better prepare students to complete various genetic skills notably increased from 1% before the intervention to 14% after the intervention.

When initially searching for information, pooled across all 5 years, 11 students (2%) reported using OMIM, Genetics Home Reference, or GeneReviews. Most students reported using Google (297, 54%), PubMed (107, 19%) or Wikipedia (65, 12%) as their first source of information on genetic conditions. Immediately after the intervention, 104 students (29%) reported using OMIM as their preferred first source of genetic information (Table 3).

Table 3 First source used by respondents when seeking information on genetic conditions

Review of the content of the optional student free text comments on the utility of OMIM as an educational and clinical tool showed six major thematic categories (totaling greater than 100% as several themes often applied to each comment). Of the 234 total comments, 151 students (65%) endorsed that OMIM was a good resource, 95 (41%) reported it was complex and difficult to navigate, 56 (24%) expressed wanting further education on OMIM, 36 (15%) reported using other resources than OMIM to complete the work, 20 (9%) expressed wanting more clinically oriented information, and 13 students (6%) stated that OMIM was not helpful.

There were 113 respondents (response rate of 47%) to the Spring 2017 survey sent to third and fourth-year medical students to assess retention of confidence in students’ skills. Of the respondents, 49 students had completed 2 years of clinical rotations while 64 students had completed 1 year of clinical rotations. Compared with postintervention pooled results, students' confidence in their ability to include genetic conditions on a differential diagnosis was not changed (29 to 25%, p = 0.409), whereas there was a statistically significant, but modest decrease in students’ confidence in finding accurate information on genetic conditions (66 to 47%, p < 0.001). Compared with preintervention results, confidence in both of these skill areas was maintained well above baseline (Table 2).

Discussion

In this study of an OMIM-based bioinformatics educational intervention, we demonstrate statistically significant increases in students’ reported confidence in their ability to build a differential which includes genetic conditions, ability to provide information for families, and ability to find accurate information on the clinical presentation, diagnostic testing, and management of genetic conditions—with maintenance of reported confidence above baseline up to 2 years out from intervention.

These results demonstrate consistent success over several years using the OMIM database as an educational modality to increase students’ confidence in broad clinical genetics skills with demonstration of persistent skill retention.

The successive yearly data enables comparison of baseline student characteristics as an indirect reflection of evolving trends in the digital age. Baseline student confidence and database usage largely remained unchanged over subsequent years. Consistently the majority of incoming students rely on nonmedical resources, such as Wikipedia and Google (Table 3), which suggests that despite technological advances, students’ premedical exposure, training and usage of bioinformatics databases is not actively evolving. Even after the curricular intervention, approximately half of students still used Google as their first resource (Table 3), similar to the rate of resident physician usage of Google to answer clinical questions.13,14 Frequent use of Google and Wikipedia is additionally demonstrated in other published literature on medical student self-reported survey data,15,16 with students ranking these two resources high in accessibility, understandability, and usefulness.15 In our optional free text comments, 42% of comments had content describing OMIM as complex and difficult to navigate—potentially leading to lower usage compared with Google. The use of Google by our students does not reflect which information source students ultimately chose to utilize, which is an opportunity to shape their choices. Acknowledging and incorporating our students’ searching preferences can engage them in the importance of correctly identifying appropriate scientific and clinical resources.17

Survey of medical genetics course directors shows a significant decrease in teaching of genetic content during students’ clinical years, with 75% of medical schools teaching genetic material exclusively in the first year.18 As a result, retention of skills gained as a first-year medical student is imperative. Studies evaluating medical students a year or more after brief educational interventions are limited; however a decline in effect is often demonstrated.19,20 Strikingly, our students’ increased confidence in their ability to generate a differential diagnosis that includes genetic conditions was maintained 1–2 years after our educational intervention. Although maintained above baseline, there was a decline in our students' confidence in finding accurate information on genetic conditions compared with immediately after the intervention. Reinforcement of content through continued exposure in each year of school as a mechanism of vertical integration may assist with skill maintenance and retention.4,21,22

This study is limited by the drop off in response rate of students participating in the surveys and survey responses are unpaired to preserve student anonymity, preventing direct comparison of progression of an individual's confidence. In addition, less than half of the third and fourth-year medical students responded to the Spring 2017 survey. It is unclear whether students who did not respond use OMIM during their rotations and, thus, we cannot generalize the findings from the upperclassmen. The predominant use of Google obscures what resources students are selecting.

The weakest area in our cohort across all years was students’ confidence in generating a differential diagnosis—as only 105 students (29%) agreed or strongly agreed they are confident in this area. Focusing additional teaching on the OMIM Clinical Synopsis comparison tool could potentially address this knowledge gap.

Instruction in OMIM enabled our students to gain increased confidence in broadly adaptable skills in the clinical application of genetics. With these skills, students can navigate the challenges of their future diverse clinical practice environments and specialties.