INTRODUCTION

Rapid technological advancements in genomic sequencing (GS) in the past two decades have escalated the use of genomic data for diagnosing, predicting, and preventing disease, resulting in an increasing integration of genomic information in clinical decision-making. The shifting landscape of genetic testing has had a substantial impact on the practice of genetic counseling. Results from GS can reside along a spectrum of pathogenicity, from benign to ambiguous to pathogenic, and interpretation of results can change over time as evidence for pathogenicity accumulates. There is also the possibility of identifying findings unrelated to the primary purpose of the testing, and findings that may have peripheral impact upon family members, who are likely to have the limited genetics expertise of the general population.1 For all these reasons, genomic information is challenging to convey effectively. Despite this, GS is increasingly offered in a diversity of clinical settings, contributing to a greater demand for genomic medicine services across a variety of health-care specialties.2 With the expansion of GS, genetic counselors (GCs) are challenged to scale services to meet the needs of a wide array of providers, patients, and their families.

Innovations in other technologies, including smartphones, artificial intelligence, and digital communication, are increasingly playing an important role in health systems.3,4,5 While genomic technology is driving the proliferation of inexpensive and accessible genetic tests, communication technology has also had a substantial impact on the practice of genetic counseling. Some technological solutions aim to bridge the barrier of access to counseling services. These include telehealth platforms, decision support tools, and genetic counseling aids such as educational videos and interactive web-based learning.6 Artificial intelligence solutions like chatbots are emerging as tools to help patients navigate genetic testing.7 There are also self-service platforms designed for consumers available through commercial laboratory websites that include self-guided educational modules, results delivery, and the option to speak with a GC.8,9,10 Other models like My46, a web-based tool that allows individuals to manage their genetic results, and Genomics ADvISER, an online, interactive decision aid to help patients in selecting secondary findings, have been born out of research initiatives to facilitate education and support patients outside of clinic walls.11,12

While these digital tools may help to improve access by streamlining the genetic counseling workflow, most of the patient-facing educational content is designed to address patients’ pretest educational needs or tailored to consumer-driven genetic testing. Current technological solutions are not designed to enhance the delivery of genomic results in routine clinical care, nor are they designed to increase understanding of the results and associated medical recommendations. Furthermore, there is a paucity of research on alternative delivery models and technological solutions for delivering genomic medicine and counseling services in diverse populations. Large scale, national research programs, such as the All of Us Research Program,13 the eMERGE Network,14 and the Clinical Sequencing Evidence-Generating Research (CSER) consortium,15 are exploring methods for implementing genomic medicine across diverse populations and settings. Outcomes from these studies will help to inform best practices for delivering genomic medicine services.

We created the Genomic Understanding, Information and Awareness (GUÍA, Spanish for “guide”) application, a novel digital platform designed to facilitate the delivery of GS results and related clinical information to participants and families of diverse backgrounds enrolled in the NYCKidSeq Project. GUÍA allows GCs to walk patients through their genomic test results in a personalized, highly visual, and narrative manner. GUÍA was developed based on the perspectives and input of providers, patients, and community stakeholders as part of the NYCKidSeq Project. NYCKidSeq is jointly funded by the National Human Genome Research Institute and the National Institute on Minority Health and Health Disparities, and is one of seven national clinical sites that are part of the CSER consortium. The development of GUÍA occurred in five overlapping phases, including a formative study (phase 1), assembly of an expert working group and content development (phase 2), stakeholder and community member input (phase 3), design of the user interface (phase 4), and web application development (phase 5) (Fig. 1). GUÍA was then piloted with 18 participants enrolled in the NYCKidSeq pilot phase.

Fig. 1: GUÍA development timeline.
figure 1

GUÍA development occurred in five overlapping phases over a 16-month period. Development phases included a formative qualitative study of parents (N = 22) whose child had undergone genomic testing (phase 1), content development guided by an expert working group (phase 2), recurring feedback from stakeholders and community members (N = 11) (phase 3), design of the user interface (phase 4), and development and testing of the web application (phase 5).

Full size image

MATERIALS AND METHODS

Formative study

Setting and study population

We recruited parents, 20 mothers and 2 mother–father dyads, of 22 index children who had undergone exome sequencing, targeted gene sequencing, or microarray clinical genetic testing within the previous 12 months to participate in an in-depth interview. Participants were recruited at two major health systems in New York City: Mount Sinai Health System and Montefiore Medical Center. We used a stratified purposive sampling approach to recruit a diverse group of participants: five Black/African American (AA), ten Hispanic/Latinx (H/L), five White/European American (EA), and two who self-reported as more than one race or ethnicity (MR). We also stratified the sample to ensure representation of different testing results: ten negative, seven uncertain, and five positive results.

Study design

In-depth interviews were conducted using narrative and focused interviewing techniques to highlight parents’ storied experiences paired with systematic probing for breadth and depth.16,17 A semistructured interview guide addressed the following domains: perceived purpose of testing; expectations of results; the results disclosure process and sequelae; and emotional responses. Interviews were administered in English or Spanish, and were recorded, transcribed, and translated into English. Parents received transportation costs and a $50 gift card for participating.

Analysis

Analysis was conducted by a multidisciplinary team including GCs, a medical geneticist, and qualitative methods experts. First, individuals conducted case-based analyses to identify “repeating ideas” as a first step in codebook development.17 Then, team members independently applied the codebook to an interview to identify areas of (dis)agreement across coders, to develop code definitions and clarify labeling of emergent themes. Groups of two coders (1 GC + 1 non-GC) then applied the codebook to a single interview until consensus was achieved on current and new code applications. Using grounded theory’s constant comparative method,18,19 this process was repeated until all interviews were coded by the 2-person groups. Full-team meetings were held to resolve issues and to discuss higher order theme development across the entire data set.19 Findings were communicated to the Expert Working Group and the Genomics Community Board to inform GUÍA content development.

Expert Working Group

The Expert Working Group (EWG) was formed with transdisciplinary expertise in the development of educational and medical genetics content. The group included four GCs, three medical geneticists, a pediatric cardiologist, a population geneticist, and a sociologist. The EWG led the development of the overall design of GUÍA and the development of the content.

Genomics Community Board

Members of the GUÍA EWG met regularly with the Mount Sinai Genomics Community Board (GCB). The GCB is made up of community leaders, patients, and clinicians predominantly from Harlem and the Bronx who self-report as Black, African, African American, Afro-Latinx, or Afro-Caribbean.20 The GCB contributed to the conceptual design of GUÍA, and reviewed development plans, prototypes, and final versions of visuals and text throughout the development process.

User interface and content design of GUÍA

The EWG developed the design of the interface to ensure the application encompassed relevant genetic counseling components. The EWG consulted with experts in data visualization and user interface (UI) design to draft the design specifications that included personalization (displaying name and preferred gender pronoun); text, illustrations, and hyperlinks on web pages; ability to display designated pages, data points, and illustrations based upon result type; tiered complexity of information; easy navigation; bilingual capability; option to export as a PDF and print full content; and compatibility for desktop and tablet. Using Zeplin21 collaborative design software, the EWG and a UI designer collaborated to produce a final set of wireframes (page schematics) and visual designs, which were used as the basis for the personalized result templates.

A bilingual version of each wireframe was generated that displayed both English (small font) and Spanish text (large font). The only text not translated to Spanish was the navigation button, as this was not a priority for the research study. We aimed to ensure that the Spanish text was understandable to patients who speak different dialects of Spanish, therefore, the translation team included individuals representing five Spanish dialects: Spanish, Mexican, Cuban, Dominican, and Puerto Rican. Discrepancies in the translated text were discussed and resolved.

Development of the GUÍA web portal

The EWG, software development team, and UI designer developed a specification for GUÍA development, and established the display logic and the data flow. The back-end database contains a web form through which the GC inputs patient specific information to populate the patient-facing front-end application. The database includes a library of template text including over 5,000 variables and approximately 50 illustrations that are displayed based on patient-specific results. A team of software engineers built the application over a 3-month period, followed by a 4-week user acceptance testing period, during which the EWG performed extensive UI testing to evaluate UI interactions before the platform went live.

Pilot testing and evaluation of GUÍA

Setting and study population

GUÍA was piloted in clinical settings with 18 parents (described as parent-participants) of children enrolled in the pilot phase of the NYCKidSeq clinical trial, which is described in detail elsewhere.22 One parent-participant was enrolled in the formative study and the NYCKidSeq pilot phase.

Prior to the results disclosure session, GCs built a participant’s case in GUÍA by inputting case details, such as the child’s genomic test results, condition-specific details, recommendations for medical management, and patient resources. Case details were input as both structured and free text. Structured text can be displayed in both English and Spanish. Spanish free text was generated using Google Translate, and Spanish-speaking staff reviewed the translations for accuracy. GCs used GUÍA via an iPad during the in-person results disclosure session. Families did not have access to GUÍA outside of the counseling session. Families were provided with a PDF printout of the full GUÍA content in addition to the laboratory report. Feedback was solicited from four GCs regarding their experience using GUÍA during the pilot phase.

Study design and analysis

We developed a brief, structured feedback guide to explore parent-participants’ reactions to GUÍA. The guide contained 14 questions designed to capture general and specific reactions to GUÍA content and design elements. Research coordinators interviewed parents in English or Spanish directly following disclosure of their child’s genomic results using GUÍA. Feedback interviews were audio-recorded and parent-participants were provided with a $40 gift card. A separate research coordinator reviewed the audio recordings and extracted structured and narrative details of the parent-participants’ responses. These responses were reviewed for themes relating to the GUÍA interface, including layout, language, images, health literacy, and navigation.

RESULTS

Phase 1: formative study

The formative study comprised 22 interviews with parents of children who had received genomic results within the previous 12 months, sampling for diversity in both child race/ethnicity and type of genomic result. Seven core themes were identified, which are listed in Table 1 with participant quotes exemplifying each theme.

Table 1 Themes related to the development of GUÍA identified through the formative study.
Full size table

One core theme is that parents often feel overwhelmed by the amount of information and details provided during a result disclosure session. There is considerable variability in what information parents want about genetics, genetic testing, and the results, and in how much depth. To accommodate a variety of educational preferences, we developed GUÍA to present information in increasing levels of complexity and on different web pages, so that patients can modulate how many and in which order they access pages. For instance, for positive results, the overall, “big picture” details of the result are presented on the top page, and further genetic and condition details are displayed on separate tabs (Supplementary Fig. 1; Supplemental Table 1).

Parents’ primary concern was improving health outcomes for their child. Consequently, parents were more interested in understanding how genetic results affected their child’s care than understanding technical details. We therefore designed GUÍA to prominently display follow-up care recommendations on the Next Steps page. Parents expressed confusion about the labels of genetic test results. For example, negative results were sometimes described as “normal” and were confused with the child’s symptoms. GUÍA clearly defines the result category (positive, uncertain, negative) on both the Learn about Genome Sequencing page and the Result Summary page.

Another theme identified was that parents who received uncertain results adapted to the uncertainty when the provider explained that genomic medicine is a developing science and knowledge of the genetic cause of disease will increase over time. We included language addressing this in GUÍA. Additionally, drawing complex concepts and showing illustrations to parents were appreciated. Parents expressed the desire to take home drawings made by a provider to revisit with family members. Thus, illustrations are integrated throughout the application, including on the Education pages, the Genetic Results pages, and the Inheritance pages.

Phase 2: content development

The EWG first identified discrete components of a typical pediatric genetic counseling result disclosure session, as shown in the schema in Supplementary Fig. 1. GUÍA addresses the following components: genetic education, primary (related to the primary indication for testing) and secondary (unrelated to the primary indication for testing) results disclosure, clinical implications, inheritance and family implications, and resources. Contracting and psychosocial counseling are addressed through the interpersonal interaction between the counselor and the patient/family.

To facilitate readability and comprehension, all text was written in the active voice and at the lowest possible reading level. We calculated the reading grade level by averaging the Flesch–Kincaid grade level of four randomly identified cases representing the three primary results categories (positive, negative, and, uncertain) and a positive secondary finding result. While our goal reading level was fifth grade, as recommended by the Joint Commission for patient education materials,23 the inclusion of genetic terminology meant that we could only decrease the reading level to an average ninth grade level.

Phase 3: stakeholder and community member input

Throughout the development of GUÍA, we met with the GCB to discuss how a web-based application could be enhanced for the provision of genetic counseling for historically underserved populations, and to receive input on the content and design. Examples of specific implications for the application include using illustrations inspired by biology textbooks to help explain complex concepts, ensuring images were culturally appropriate and resonated with the target audience, defining results categories clearly, simplifying pages by removing superfluous text, and including referrals to appropriate support services (see Supplemental Table 2 for GCB feedback).

Phase 4–5: web application design and development

We designed GUÍA to have a user-friendly interface with a visual design that is meant to make information communicated during the result disclosure easier to understand. GUÍA enables GCs to guide patients through the distinct components of a traditional result disclosure session while catering to patients’ diverse learning styles. Using GUÍA can be a counselor- or patient-driven experience and its modular design permits users to control the order in which information is accessed. If desired, the user can follow a proposed linear flow by using the navigation arrows at the bottom of each page, or they can navigate to different pages of the application from the Home page or the navigation panel. The user can toggle between Spanish/English or English only text by clicking the language button. Illustrations accompanied by plain language text are incorporated throughout the application to convey key concepts, such as inheritance patterns, and demonstrate aspects specific to the patient’s condition, such as affected organ systems.

The GUÍA sitemap (Supplementary Fig. 1) displays the structure of the application, which consists of 9 distinct pages with subtabs to access educational modules, primary and secondary results, inheritance information, and resource links. A high-level summary page provides key takeaways for the patient including results, next steps, and links to additional resources. Figure 2 shows the GUÍA Home page (a) in English, and the Result Summary page (b) and Family page (c) in Spanish/English for a positive genetic result. The Condition and Genetic Details subtabs on the Result Summary page are personalized for the patient and populated with relevant information garnered from the genetic test report and from the GC’s research into the patient’s genetic condition. At the end of the counseling session, the entire GUÍA experience or individual pages can be converted to a PDF file format for the patient’s use outside of the session.

Fig. 2: Images of the GUÍA digital platform.
figure 2

Images show the GUÍA Home page displayed in English only (a), the Result Summary page for a positive result displayed in Spanish/English (b), and the Family page for a positive result displayed in Spanish/English (c).

Full size image

Evaluating GUÍA during NYCKidSeq pilot phase

Eighteen NYCKidSeq pilot phase parent-participants completed feedback interviews. All 18 were female; mean age was 44 years (range 28–56 years). Two (11%) were AA, 10 (56%) were H/L, 5 (28%) were EA, and 1 (6%) was MA (Table 2). Six (33%) interviews were conducted in Spanish. Six (33%) parent-participants’ children received positive genetic results, 3 (17%) received negative results, and 9 (50%) received uncertain results.

Table 2 Sociodemographic characteristics of parent-participants enrolled in the pilot phase of the NYCKidSeq study.
Full size table

During the result disclosure session, GUÍA was used by both the parent-participant and counselor to guide the delivery of information. After the result (positive, negative, or uncertain) was disclosed by the GC, parent-participants were asked which component they would like to focus on next. Of the 18 feedback participants, ten sessions were reported as GC-driven, seven as both GC- and parent-driven, and one was not indicated. Of note, none of the sessions were reported as parent-driven, as all parent-participants when given the option to explore GUÍA on their own deferred to the GC to walk them through the tool.

In the feedback interview, parent-participants’ general reactions to GUÍA were overwhelmingly positive (Table 3). Specifically, parent-participants felt that GUÍA made receiving information about the result manageable, and they appreciated being able to read along as the counselor discussed the results. Spanish-speaking parent-participants valued reading text in their preferred language. All parent-participants (N = 10) who had previous experience with genetic testing agreed that using GUÍA improved result disclosure. Seventeen (94%) parent-participants said the content was clear and all parent-participants stated that the amount of information contained in GUÍA was the right amount. Parent-participants appreciated that they could control the flow and amount of information provided to them.

Table 3 NYCKidSeq parent-participants reactions to GUÍA and feedback on the content and design.
Full size table

Seventeen (94%) parent-participants felt that the user interface was easy to navigate. One parent-participant suggested including a “back” button on every page to better enable the user to navigate between pages of the application. While the majority of parent-participants felt that the number of illustrations included in GUÍA was the right amount and that the illustrations helped them understand complex genetic concepts (15 [83%], 17 [94%], respectively), one parent-participant stated that including additional illustrations would be helpful for individuals with limited understanding of genetics.

GCs report spending approximately 15 minutes inputting data for each case. When asked to describe their experience using GUÍA compared with traditional counseling tools (e.g., flipbooks), GCs highlighted the advantage of having information displayed in both Spanish and English when counseling a Spanish-speaking family using a telephone interpreter. They also expressed that GUÍA provided GCs the ability to personalize patients’ results, saved time in developing patient educational material, and made it easier to communicate results with other providers. Limitations of GUÍA were noted as being restricted to GS tests, children could disrupt the session by their interest in the iPad, patients/families do not have access to an online version of the tool, and GUÍA does not support annotation on pages.

DISCUSSION

We describe the design, development process, and evaluation of GUÍA, a novel web-based platform for communicating GS results. Digital health tools like GUÍA have the potential to reduce disparities in access to genomic services by broadening the reach of genetics specialists in diverse and underserved settings and simplifying genomic medicine delivery for nongenetics providers.24 GUÍA allows highly technical information to be communicated in an understandable, personalized, and digestible manner. It enables patients to actively engage with their results and control the speed and depth of information delivery. Participants in the NYCKidSeq pilot phase responded positively to the use of GUÍA in their result disclosure session, and all parents with prior genetic testing experience expressed that receiving genomic results with GUÍA was superior. These findings provide preliminary evidence of families’ satisfaction with the integration of this novel application into genetic counseling. The utility of GUÍA to improve parental outcomes will be thoroughly investigated through the NYCKidSeq clinical trial where participants across a variety of clinical settings will be randomized to genetic counseling using GUÍA versus traditional genetic counseling.22

The clinical utility of genomic information is inherently linked to understanding of results and adherence to medical recommendations. Traditionally, genetic specialists communicate findings from genetic testing; however, there is a relative scarcity of medical geneticists and GCs in the United States and worldwide.25,26 Proposed methods to scale genomic medicine services include training additional genetics professionals and training nongenetics professionals in genomic communication.27 Technological solutions can also be leveraged to increase access to genomic medicine services.6,24 Research in this space has explored different modalities for scaling genetic counseling services, such as using web-based decisional aids and educational modules for patients to review prior to their appointment.28 These efforts have primarily focused on pretest genetic counseling, likely due to the amount of education typically occurring during the initial appointment, and are not designed to improve patient understanding of their genetic results. Findings from this preliminary evaluation suggest that GUÍA may be well positioned to support the growth of applied GS in that it facilitated result disclosure and was favorably reviewed by participants and GCs alike.

Currently, the clinical utility of GS tests is limited in historically underserved populations, mainly due both to reduced access to GS tests and an underrepresentation of non-European ancestry populations in genomic databases.29,30,31,32 Large-scale research efforts are actively addressing this disparity by expanding genomic databases to better represent diverse populations.15,33,34,35 The fruits of these efforts will increase the frequency with which GS tests provide conclusive results to patients of diverse ancestral backgrounds, and this improvement in clinical utility will further propel the application of GS. To ensure this improvement in accuracy promulgates health equity, equal attention must also be paid to accessibility to and communication of genomic health information.36 Digital communication tools that are culturally aware and multilingual will be needed to best serve diverse communities. Stakeholders, including genetics and medical professionals, patients and their families, and community members, contributed to the development of GUÍA, helping to assure cultural appropriateness and understandability for patients, and functionality for providers. Testing GUÍA with the diverse participant population enrolled in NYCKidSeq will provide insights into the value of the application in delivering genomic medicine services. Furthermore, as the use of GS increases, nongenetics providers are increasingly required and challenged to interpret and discuss GS results with their patients.37 Future research could explore the impact of GUÍA on nongenetic provider outcomes and methods for implementing GUÍA to support the disclosure of results in nongenetics clinics.

There were some limitations in this study. The pilot evaluation of GUÍA described here was carried out in a small, all-female cohort, which might limit the generalizability of the findings. Since the cohort consisted of participants in an ongoing study, their reflections on GUÍA may have been influenced by pretest counseling, familiarity with the genetic testing process, and repeated interactions with study staff. This study demonstrates the benefits of the development of technological solutions for communicating genomic information by medical professionals; however, this nonstandard software development environment imposes pragmatic limitations on feature development, portability, and long-term sustainability. For now, GUÍA is only available in English and Spanish, with limited utility to families with a different primary language. In a diverse city like New York, where over 34% of citizens speak a different language in their home, this may curtail the utility of the tool.

Conclusion

To realize the full benefit of genomic medicine, patients and families must understand their genomic results and make informed decisions utilizing this information. Genomic information must be rendered accessible to all individuals regardless of health literacy and English language fluency. GUÍA was developed by a multidisciplinary group of stakeholders as a bilingual communication tool to support a large and varied population beginning with the diverse NYCKidSeq participants. Preliminary evidence suggests that GUÍA will be well-received and valued by patients, research participants, families, and providers.