Introduction

Next-generation sequencing of an individual’s genome or exome with analysis of the data for high-penetrance disease-causing variants (genome-wide sequencing or GWS) is increasingly used for the clinical diagnosis of genetic disease.1,2 Diagnostic rates observed in reported patient series range from 25% to more than 60%, depending on how patients were selected for testing, the point in the diagnostic process at which the testing was performed, and how the sequencing and analysis were done. GWS has been shown to provide a higher diagnostic yield as a first-tier test than conventional clinical assessments and genetic testing in head-to-head comparisons in heterogeneous groups of pediatric patients with suspected genetic disorders.3,4,5,6,7 Some of the highest diagnostic rates reported to date have been in acutely ill infants tested in neonatal intensive care unit (NICU) or pediatric intensive care unit (PICU) settings.8,9,10

Making a specific genetic diagnosis in the newborn period is especially important because precise diagnosis can have critical implications for an infant’s health and wellbeing for her entire life.9,10,11 Most severe Mendelian diseases and pathogenic copy-number changes are manifest in infancy, and congenital anomalies and genetic disorders are the most frequent cause of death in infancy.12

However, there are thousands of individually rare genetic conditions and the presentation in babies may be nonspecific or differ from that observed in older patients. In addition, the general fragility of very small or premature infants may produce secondary multisystem involvement that obscures the primary pathology. These factors often make precise diagnosis of a rare genetic disease difficult or impossible in the newborn period without knowledge of the underlying pathogenic variant(s).

Diagnosis of a specific genetic disease in acutely ill infants may, therefore, be a particularly valuable clinical application of GWS. It has been suggested that rapid diagnostic GWS of acutely ill infants could be the breakthrough application that leverages widespread adoption of genomic medicine throughout the healthcare system.13

However, the complexity and cost of performing this test in critically ill newborns may be especially high because genome rather than exome sequencing, GWS on both parents as well as the affected child, and a very rapid turnaround time may be necessary.13 Moreover, the financial cost of the testing itself is insufficient to capture the full impact of clinical GWS. This is well illustrated by a recent study that compared children with suspected genetic disease who had initial chromosomal microarray analysis and subsequent diagnostic genome sequencing.14 The volumes and costs of medical interventions prompted by negative chromosomal microarray results and diagnostic GWS results were quantitatively similar but qualitatively different. Negative chromosomal microarray analysis sometimes led to additional laboratory investigations in pursuit of a diagnosis, while obtaining a diagnosis by genome sequencing often led to more visits to medical specialists or allied health professionals, reflecting changes in clinical management.

The clinical value of diagnostic GWS in patients suspected of having a rare genetic disease must consider the downstream effects of making a genetic diagnosis, including the impact on care of the patient, emotional suffering for the patient and family, informed reproductive decision-making, societal costs and savings, and other important aspects of diagnostic GWS.15 Moreover, the value that the healthcare system places on achieving a diagnosis through GWS and the value families place on achieving such a diagnosis may be substantially different.16

Rapid diagnostic GWS in acutely ill infants also highlights a number of ethical issues related to the care of acutely ill newborn infants and to GWS more generally.17,18,19 The central ethical tenet of clinical pediatrics is that the best interests of the child are paramount, but determining the best interests of a severely ill infant may be challenging. For example, some of the benefits attributed to diagnostic GWS result from avoidance of high-intensity treatment and risky medical or surgical interventions in favor of palliative “comfort care” for infants who have uncontrollable suffering or whose prognosis is dismal.9,10,11,13,20 Is it in a baby’s best interests for his parents to find out that he has an untreatable genetic condition that has been fatal within the first few months of life in all previously reported cases? What if there are only three such cases reported in the entire world literature? What if the only available therapy is experimental and the child is likely to be severely impaired even if the therapy succeeds? Would the fact that the family has already lost two babies to the same disease be relevant? Such ethical concerns about GWS in newborn infants trouble the families of affected infants, healthcare providers, and insurers.19

Although the benefits of GWS in acutely ill infants do appear promising, the clinical value of this (or any other) diagnostic test should be rigorously demonstrated before it is accepted as a standard of care.21 Until the necessary evidence is available, it is impossible to say whether GWS is worth the associated costs and risks, and its adoption as a routine diagnostic service by publicly funded healthcare systems and private health insurers is likely to remain limited and inconsistent. This white paper was developed by the Paediatric Task Team of the Global Alliance for Genomics and Health’s Regulatory and Ethics Work Stream to address the question of how we can determine the clinical value of diagnostic GWS in acutely ill newborns.

Assessing the clinical value of GWS

The ACCE (Analytical validity, Clinical validity, Clinical utility, and associated Ethical, legal, and social implications) framework is the most widely used standard for evaluation of genetic tests for clinical use.22,23 Analytical validity is a measure of the accuracy with which a test determines a genotype of interest. It is a purely technical assessment that is independent of whether or not the genotype is likely to be pathogenic. In contrast, clinical validity is a measure of a test’s ability to diagnose a specific phenotype, such as a genetic disease, accurately. Clinical utility, a measure of the net benefit of the test in terms of clinical care, depends on factors like the availability of effective management if a particular disease is diagnosed as well as on the economic costs, accessibility, and operational impact of testing on the delivery of health services.

It has been argued that the clinical utility of a genetic test should also include consideration of ethical, legal, and social issues related to the diagnosis, prevention, or treatment of the disease that is being tested. Even this broad definition of clinical utility may not be fully inclusive of the overall costs and benefits of genetic testing: elements of “personal utility” may also need to be considered.15,24,25,26,27 As a practical matter, however, personal utility and social consequences are difficult to measure and have contributed little to the funding decisions healthcare systems and insurers have made regarding genetic testing to date.

The ACCE approach and others that organize or assess available evidence regarding the appropriateness of implementing particular genetic tests have recently been reviewed in detail by a committee of the US National Academy of Sciences.28 This expert committee pointed out that the ACCE framework was developed for single-gene tests and is difficult to apply to GWS. There are several reasons for this, including:

  • Every person’s genome differs by about 4 to 5 million nucleotides from the standard “reference genome”.29,30 As a consequence, the one or two variants that are actually causal for a Mendelian disease in an affected individual must be distinguished from the noise of millions of variants that are unrelated to the disease.

  • Clinical and bioinformatics analysis of exome-sequencing data provides a survey for disease-causing variants of all protein-coding genes simultaneously. Over 3400 different genes are currently known to be associated with Mendelian disease,31 and it would be extremely difficult to demonstrate analytical validity for any possible variant of every one of these genes. This problem is even greater for genome sequencing, which also detects structural variants that may involve multiple genes and noncoding regulatory regions.

  • The variants that can be detected by diagnostic GWS are associated with more than 5000 different phenotypes,31 and additional Mendelian conditions and disease-associated genes are being discovered almost every day. Demonstrating clinical validity for the genetic variants associated with each of these diseases would be very challenging because most of the conditions are rare and our knowledge of their penetrance, phenotypic spectra, and natural history is incomplete.21

  • Determining the clinical utility of GWS for each of these thousands of different genetic disorders on a gene-by-gene or disease-by-disease basis would be even more difficult. Clinical utility is dependent, among other things, on the availability and efficacy of medical interventions, which are nonspecific and of unproven benefit for most rare genetic conditions.21,32

All of these factors apply to assessment of the clinical value of diagnostic GWS in critically ill infants. This assessment is also colored by ethical issues related to the testing of such patients9,11,20,33 and further complicated by the technical and analytical complexity (and consequent cost) of using diagnostic GWS in a NICU or PICU setting. These considerations might constitute an insurmountable barrier to assessing the clinical value and appropriateness of diagnostic GWS if it were an enormous panel of single-gene tests. It is not: it is a broad scan for high-penetrance genetic variants that can cause disease. Analysis of diagnostic GWS as a single test, rather than as thousands of different tests done at once, simplifies consideration of all aspects of clinical value.

Performing genome-wide surveys for pathogenic genetic alterations in patients suspected of having serious genetic disorders has been widely used clinically for more than 50 years—initially as cytogenetic analysis of solid-stained chromosomes, then as banded chromosomal analysis, and more recently as chromosomal microarray analysis. Treating diagnostic GWS as a genome-wide scan for high-penetrance disease-causing variants rather than as a large panel of single-gene tests provides a practical approach to assessing its clinical value in critically ill infants.

Some practical solutions

Empirical studies of variants identified by clinical GWS using an orthogonal method, most often Sanger sequencing, have demonstrated very high rates of confirmation, implying excellent analytical validity, or at least specificity34,35,36 of the test. More generally, analytical validity can be established for GWS by using reference DNA specimens, such as those available through the Genome In A Bottle Consortium.37 Each of these specimens includes millions of presumably benign polymorphic single-nucleotide or structural variants that have been characterized in detail by several analytical technologies. Although comparison of GWS results with these reference standards does not establish sensitivity or specificity for every individual disease gene, such an analysis does provide a robust assessment of analytical validity across the entire genome.

The clinical validity of GWS can be evaluated directly for a few rare diseases against standard clinical tests for these disorders, either in samples with known pathogenic variants or prospectively for samples submitted for clinical testing. For this assessment to be relevant to the question at hand, the diseases chosen would have to be ones that present in infants hospitalized in a NICU or PICU. Extrapolating the findings to genetic disease in general would require an assumption that the small subset actually evaluated is representative of all genetic diseases that present in infancy. This assumption is unlikely to be correct, given our current limited knowledge of the penetrance and phenotypic variability of most genetic conditions, and thus the clinical validity for diagnostic GWS obtained in this way would have to be considered a maximal estimate.

An holistic approach to assessing the clinical value of GWS in acutely ill newborn infants

Petrikin and his associates33 have speculated that rapid diagnostic WGS in critically ill infants may substantially reduce mortality. If this were true, a single important outcome (mortality) could be used to determine the clinical value of diagnostic GWS in infants in an intensive care setting (Fig. 1). It is important to note, however, that there are two different ways that making a precise diagnosis in a critically ill infant might influence the child’s survival. Definitive diagnosis of an untreatable condition that carries a very poor prognosis might enable intensive medical support to be withdrawn and comfort care instituted, in accordance with the family’s wishes. This would accelerate early mortality among newborns who would otherwise die later, after futile efforts to alter the inevitable course of their disease (red portion of the GWS curve in Fig. 1). The second possible effect of diagnosing a specific genetic defect in an acutely ill newborn infant is that it would permit effective precision treatments to be provided and access to promising experimental treatments for some of these infants much earlier in the course of their disease. These interventions may be lifesaving, reducing overall mortality among patients who receive diagnostic WGS and dramatically improving the quality of life in surviving children (purple portion of the curve with GWS in Fig. 1). Examples from our own experience of lifesaving therapeutic interventions that were instituted because a specific genetic diagnosis was made include bone marrow transplantation, administration of riboflavin (vitamin B2) in very large doses,38 and repurposing an available drug for treatment of overexpression of a cell signaling pathway.39

Fig. 1: Hypothetical 180-day mortality of severely ill infants with suspected genetic disease without diagnostic genome-wide sequencing (GWS) (green curve) or with the benefit of diagnostic GWS (red and purple curves).
figure 1

Note that diagnostic GWS is postulated to have two opposing effects on mortality: In infants diagnosed as having a condition that is lethal early in infancy, diagnostic GWS permits substitution of comfort care for intensive support, which results in earlier death (red portion of the curve). Some other infants with potentially lethal conditions are diagnosed as having a treatable condition, and the institution of precision therapy provides a reduction in mortality (purple portion of the curve). Figure redrawn after Petrikin et al., 2015 (ref. 33)

Full size image

An overall reduction in infant mortality would only occur if the prolongation of life in healthier infants who receive precision therapy outweighs the accelerated mortality that occurs in infants who are diagnosed with lethal conditions and changed from intensive care to comfort care early in life (Fig. 1). These two different kinds of interventions influence infant mortality in opposite directions, and might exert opposing effects on costs, as well. Earlier transition to comfort care and consequent earlier death in infants with a grim prognosis would be expected to produce substantial healthcare cost savings. Provision of a lifesaving treatment to an acutely ill newborn infant might either reduce or increase healthcare costs, depending on the effectiveness and expense of the treatment and on the residual symptoms and limitations that persist after treatment is begun.

Death can only be used as the primary outcome for a health technology assessment when the background mortality is very high over a relatively short period of time, as is the case for newborn infants hospitalized in NICUs or PICUs. A statistically significant improvement in, say, 6-month survival in the GWS arm of a randomized diagnostic trial designed to test this possibility against conventional diagnostic evaluation might provide compelling evidence of clinical value, given the financial, systemic, and personal costs of caring for acutely ill infants with serious genetic diseases. Such studies would necessarily include patients who have a range of clinical problems and who are found to have a variety of specific genetic conditions, each of which has its own prognosis, characteristic clinical course, potential therapeutic interventions, and effects on quality of life for the affected individual and family. Using mortality as an outcome circumvents this clinical heterogeneity, which confounds the disease-specific outcome metrics used to assess clinical validity and clinical utility of single-gene tests.40,41,42,43

The use of mortality as a measure of the clinical value of GWS also has important limitations. To begin with, even if mortality could be used to compare diagnostic GWS with conventional genetic testing, mortality is unlikely to be useful for other important clinical comparisons—e.g., whether genome sequencing is better than exome sequencing or a turnaround time of 48 h is better than 2 weeks, despite the higher costs. Moreover, there are few circumstances outside of an intensive care unit in which mortality would be frequent enough to be used as an outcome measure in a study of the clinical value of genetic or genomic testing.

Are there alternative outcomes that could be used for this purpose? Improvements in treatment or earlier decisions to pursue comfort care after a precise diagnosis might be reflected in an easily measurable metric like NICU or PICU length of stay. An obvious advantage of using length of stay as an outcome measure is that it is amenable to conventional economic analysis because NICU or PICU care is so costly.38 A major disadvantage is that length of stay in an intensive care unit considers only a small portion of the life, benefits achieved, and costs incurred by surviving patients and their families.

Other outcomes that might be considered as part of the assessment are quality-adjusted life years (QALYs), a measure of survival time adjusted by the quality of that life,44,45 or disability-adjusted life years (DALYs), a measure of duration of survival adjusted for severity of disability.46,47 Unfortunately, both QALYs and DALYs are difficult to estimate for infants and are unknown for the vast majority of rare genetic diseases.48 Moreover, estimating overall QALYs or DALYs for a group of critically ill infants when almost every baby has an entirely different diagnosis (and the group selected for study differs from every other group of similar size) is challenging. The use of QALYs or DALYs in infants also raises important ethical issues regarding how to value the duration and quality of life in young children in comparison with adults and in individuals with qualitatively different kinds of disabilities and diseases (e.g., a serious cardiac malformation versus severe intellectual disability).49

Comparisons have been made in other clinical contexts between diagnostic GWS and current standard-of-care genetic testing using cost per case diagnosed as the outcome.7,8,38,43,50,51 For example, the current standard of care in the evaluation of children with intellectual disability is to perform chromosomal microarray analysis52 in combination with additional genetic tests suggested by the clinical features.53,54 Comparing GWS to conventional genetic testing on a per-case diagnosed basis has the advantage of permitting straightforward comparisons of diagnostic yields and testing costs. In addition, it can be done as two simultaneous undertakings in a single group of patients, providing a perfect control that mitigates the effect of clinical and genetic heterogeneity.

It is important to note, however, that determining the diagnostic yield for GWS is not the same thing as establishing its clinical value or even its clinical validity. This is because most of the thousands of different rare genetic diseases that have been described cannot be diagnosed by clinical means alone, i.e., there is no “gold standard” method of establishing the presence (or absence) of the disease without a genetic test. For most rare genetic diseases, demonstration of the presence of a pathogenic variant that has (or of pathogenic variants that have) previously been shown to produce a phenotype consistent with the clinical features found in a patient is necessary to establish a diagnosis. In other words, the diagnosis depends on genotype–phenotype correlation, which requires the use of clinical expertise by specialist physicians.55 Such clinical judgment is necessarily subjective, so it is critically important, in a comparison of the clinical value of GWS with conventional genetic testing, that all diagnoses are made in a consistent fashion and without knowledge of the test used to identify the disease-associated genotype. The use of a multidisciplinary consensus panel may be useful in this process.38,56,57

There are other issues that might also confound a head-to-head comparison of diagnostic rates achieved by GWS and conventional genetic testing in critically ill infants. These include the fact that different diagnoses may be made in the two arms of the study, with GWS being more sensitive for most variants that cause Mendelian disease and conventional testing being more sensitive for some others (e.g., trinucleotide repeat expansions). In addition, GWS analysis pipelines identify more than one Mendelian diagnosis in about 5% of cases, sometimes producing a “blended” phenotype, sometimes with each variant producing its own separate characteristic phenotype, and sometimes with a secondary finding that is completely unrelated to the indication(s) for testing.58,59

Nevertheless, comparing the benefits and costs of diagnostic GWS with the current standard of care per case of serious genetic disease identified holds promise as a practical means of assessing clinical value in a NICU or PICU setting, and in other contexts as well. For a comparative study to provide the robust evidence that is needed, both costs and benefits will have to be to be collected in a rigorous prospective manner, and the results will need to be analyzed comprehensively.

Conclusions

On the basis of the foregoing considerations, we conclude that:

  • Diagnostic genome-wide sequencing shows great promise for improving the management of acutely ill infants in a NICU or PICU setting.

  • Before diagnostic GWS can be considered to be a standard of care, the test must first be shown to have analytical validity and clear clinical value.

  • Evaluating diagnostic GWS as a comprehensive scan for major genetic disease provides a practical approach to assessing its clinical value in acutely ill infants.

  • The analytical validity of diagnostic GWS can be assessed using standard reference specimens and data.

  • Total mortality over a defined period of time might serve as a compelling summary measure to assess the clinical value of diagnostic GWS in comparison with conventional genetic testing in acutely ill infants. This will only be possible if lifesaving treatments enabled by precision diagnosis are frequent enough. Mortality is unlikely to be useful as an outcome measure in comparing incremental changes in diagnostic GWS, such as different sequencing technologies or improvements in bioinformatics software.

  • Summary outcomes, such as hospital length of stay, QALYs, or DALYs, which have been used for cost-effectiveness studies in other contexts, are incomplete or difficult to obtain as metrics for comprehensive assessment of diagnostic GWS in infants in an intensive care setting.

  • Comparing diagnostic GWS with the current standard of care per case of serious genetic disease diagnosed can provide a practical means of assessing clinical value. Such comparisons should be designed and carried out in a scientifically rigorous manner through collaboration of clinical investigators, health services researchers, and health economists. These studies are needed to determine if clinical GWS should be supported by healthcare systems and private insurers as the standard of care for diagnosis of genetic disease in seriously ill newborn infants.