Introduction

Genetic testing is increasingly offered to individuals with a clinical diagnosis of autism spectrum disorder (ASD) to identity variants associated with specific phenotypes or comorbidities. Integrating genetic testing into ASD assessment can enable more accurate and earlier diagnosis and identify recurrence risk in families.1, 2, 3 More routine use of genetic testing may also alter medical management and increase referrals to specialists. It is not yet known whether expanding genetic testing will lead to increased use of services that may be offset by earlier diagnosis and intervention.

Assessing the value of alternative genetic testing approaches in ASD requires consideration of diagnostic yield. While chromosomal microarray analysis (CMA) is being adopted as a first-tier test for ASD,2, 4 it explains only 5–10% of ASD.5 Massively parallel high-throughput clinical genome and exome sequencing (CGES) detects a broader range of variants associated with ASD.6, 7, 8, 9 Whole-exome sequencing (WES) primarily targets the protein-coding portion of the genome, about 1% of the genome, and detects single nucleotide variants and smaller insertion and deletion (indel) mutations, but has limited power to detect copy number variations (CNV) and more complex structural variations that contribute to ASD.8, 10, 11, 12, 13, 14 By definition, whole-genome sequencing (WGS) covers the complete genome with the potential to capture all classes of variation.10, 11, 13, 14, 15 CMA, WES, and WGS can generate findings unrelated to the purpose of the test, i.e. secondary or incidental findings, that predict risk for other health conditions or that have unknown clinical significance.16

Because of its hypothesized higher diagnostic yield, potential for closer medical management of primary findings, and perceived ability to eliminate the need for multiple genetic tests, demand for CGES is increasing.17 Yet its cost-effectiveness and how best to translate CGES from research to practice are not yet clear.18, 19 Recent reviews highlight the paucity of economic evaluations comparing WES and WGS with standard care.10, 20, 21 While laboratory costs of sequencing have been decreased dramatically,22, 23 reliable and comprehensive cost estimates that capture the full workflow do not exist. WGS, and to a lesser extent WES, generate copious data requiring substantial storage capacity and bioinformatics capability to identify clinically meaningful variants and personnel to interpret these variants.10, 24 Economic evaluations of CGES that assess the incremental costs of CGES in terms of benefits to patients require accurate estimations of all costs. The study objectives were to estimate the precise costs associated with CMA, WES, and WGS tests in children with ASD using a microcosting approach and to explore the incremental costs and diagnostic yields of CMA, WES, and WGS in alternative hypothetical clinical testing scenarios for children with ASD.

Materials and methods

Study design

Using a bottom-up microcosting approach, the opportunity costs per sample excluding mark-ups, fees, and charges for CMA, WES, and WGS tests for patients with ASD were estimated from an institutional perspective based on the diagnostic laboratory practices at The Hospital for Sick Children (SickKids), Canada. The cost per sample was determined for each year of a 5-year program and included the full array of laboratory-related inputs from blood draw to results reporting. Total program costs to service an ASD population were estimated over 5 years. A target population approach focusing on costs encountered as part of the referral and diagnostic pathway for children with ASD was selected in contrast to a centralized clinic approach in which genetic test costing would be undertaken for a heterogeneous group of children with mixed diagnoses and complex etiologies.

Microcost item identification

Major cost categories include labor, small and large equipment, supplies, and follow-up testing (Table 1). Bioinformatics is added for WES and WGS, reflecting a computing category. Bioinformatics included multiple subcategories. The labor cost associated with the bioinformatics analyst’s time to perform sample logistics and data processing was estimated. Storage of sequenced data and computation tasks were also costed. Computation tasks utilized 72 compute nodes housed at SickKids, each with 40 compute cores and 256 GB of RAM. Equipment and labor costs associated with purchasing and maintaining computing nodes were estimated but bioinformatics software costs were not included. Periodic validation, quality control, and pipeline updating and testing were not included.

Table 1 Categories of resource use for CMA, WES, and WGS tests
Full size table

Each of the subcategories was broken down into individual items according to laboratory workflow procedures. Volume of use and price data were obtained for 38 (CMA), 56 (WGS), and 61 (WES) separate microcost items. Inputs for volumes and prices for each item were obtained from laboratory staff or industry representatives, or were extracted from published or gray literature.25 Item costs were calculated by multiplying volume of use by unit price. For labor, time in minutes for each task was multiplied by wages. Benefits of 18% were added based on SickKids policy. Price data are reported in 2015 Canadian dollars (CAD) (CAD$ 1.00=US$ 0.78; 2015 average rate). Prices collected prior to 2015 were adjusted for inflation. Additional details are available.26

Model assumptions

Future item costs were discounted at 3%. Small and large equipment costs were depreciated using straight-line depreciation. An opportunity cost of 3% was added to the cost of large equipment, such as array or sequencing machines, and their maintenance contracts. An overhead cost of 23% was added to labor, large and small equipment, and bioinformatics. The sample cost for ASD patients was determined by allocating a fraction of the total equipment use for all clinical indications in the institution. The total number of CMA tests conducted in 2013–2014 at SickKids for all indications was 3,948 (D.J. Stavropoulos, personal communication). Based on the prevalence of ASD,27 it was assumed that 300 genetic tests per year would be run for ASD. It was further assumed that the annual number of WES and WGS tests for all indications could vary from 500 to 1000 per sequencer and was assumed to be 500 (100% of all tests) in the reference case. A list of analytic assumptions is presented in Supplementary Table S1 online.

Microcost item valuation

CMA costs were based on the Affymetrix (Santa Clara, CA, USA) GeneChip 3000Dx platform. A 1-year maintenance contract constituting 10% of the platform price was included. Two bundles of equipment are needed to process the total institutional volume. Follow-up testing with fluorescence in situ hybridization (FISH) for the proband and real-time polymerase chain reaction (qPCR) for two parents was assumed to occur in 10% and 5% of cases, respectively. CMA microcost items are in Supplementary Table S2.

WES costs were based on the Illumina (San Diego, CA, USA) HiSeq 2500 sequencing platform. Equipment included the sequencing platform, maintenance contract, Agilent Technologies (Santa Clara, CA, USA) Bioanalyzer and TapeStation instruments, and small equipment. Follow-up with Sanger sequencing was assumed for the proband and two parents in 50% of cases (C.R. Marshall, personal communication).28 Clinical interpretation and reporting costs included the classification and reporting of primary and secondary variants and depended on the number of primary variants followed up to determine clinical significance. For ASD, this varied from zero to four with an average of two variants per case assumed. It was assumed that secondary variants were found in 4% of cases. WES microcost items are shown in Supplementary Table S3.

To increase generalizability, two WGS sequencing platforms were considered: Illumina HiSeq 2500 and Illumina HiSeq X. Although these platforms are currently used mainly for research, all inputs for volume and price assumed a clinical application. However, costs related to test and pipeline development were excluded.

The Illumina HiSeq X requires a greater initial investment but has lower reagent prices. The price per HiSeq X instrument assumed an initial purchase of a minimum of five sequencers. Costs were calculated for sample processing on a single sequencer. Labor costs for bioinformatics management and analysis, and maintenance and infrastructure costs of file storage and computation were estimated for the HiSeq 2500 and HiSeq X instruments. As with WES, the target number of primary variants per sample ranged from zero to four, with an average of two. Microcosts for both platforms are in Supplementary Tables S4–S5.

Sensitivity analysis

Prices and volumes of use for many items may vary by institution and jurisdiction. Plausible ranges for each item’s volume of use and price were established based on consultation with experts. To assess parameter uncertainty, a probabilistic sensitivity analysis was conducted, where ranges were used to define probability distributions for each parameter (Supplementary Tables S2–S5). A truncated normal distribution was assumed for price or volume inputs with upper and lower bounds. The estimate corresponded to the mean of the normal distribution and lower and upper bounds corresponded to a 99.7% confidence interval (i.e., upper and lower bounds were assumed to lie within three standard deviations from the mean). Volume of use for FISH, qPCR, and Sanger sequencing were quantified as the proportion of cases in which follow-up testing was done. For such inputs, a beta distribution was assumed. Since no evidence existed for any specific form of correlation, all input distributions were assumed to be independent. To propagate variance in the model, 10,000 values were drawn from each input’s distribution. Point estimates of inputs with fixed values, i.e., inputs without ranges, were repeated 10,000 times. Simulations were run in R.29 Ninety-five percent confidence intervals (CI) were derived for mean cost per sample for each test strategy.

Deterministic sensitivity analysis was conducted for selected parameters that were highly uncertain or expected to vary substantially between institutions. Overhead cost was varied from 10 to 30%. The number of WES or WGS tests for all indications was varied from 500 to 1,000 to examine how cost per ASD sample changed with increasing test volume. A third deterministic sensitivity analysis was conducted to vary the number of primary variants from zero to four.

Cost–consequence analysis

An exploratory cost–consequence analysis examined the incremental costs per unit increase in diagnostic yield for CGES compared to standard care. Incremental costs and diagnostic yields were calculated for three hypothetical scenarios potentially reflecting clinical practice: (i) substitution of CMA alone with a combination of CMA and WES for all patients (CMA+WES versus CMA), (ii) substitution of CMA with WGS (WGS versus CMA), and (iii) substitution of a combination of CMA and WES with WGS (WGS versus CMA+WES). The rationale for combining CMA and WES is to detect CNVs and single nucleotide variants. CMA can reliably identify CNVs, while WES is limited in CNV detection.28, 30 WGS can identify both large and small variants.9, 13, 14 Since WES can be viewed as a complement to CMA, WES was not directly compared with CMA. These scenarios reflect how one technology or combination of technologies might substitute for another. Due to limited data on diagnostic yields for various configurations, these scenarios do not consider serial testing, in which only patients who test negative on a first-tier test (e.g., CMA) proceed to CGES.

To calculate incremental diagnostic yields for clinical testing scenarios, diagnostic yields were sought from recently published studies reporting diagnostic yields for CMA, WES, or WGS in ASD. Tammimies et al.5 conducted CMA on 258 children diagnosed with ASD and estimated the diagnostic yield at 9.3%, where a positive result indicated variants that were pathogenic or likely pathogenic according to the American College of Medical Genetics guidelines.31 They also examined CMA plus WES on 95 children diagnosed with ASD with a resulting diagnostic yield of 15.8%. Yuen et al.9 performed WGS on 85 quartet families with two ASD-affected siblings and reported a diagnostic yield of 42.4%; however, this yield included variants of uncertain clinical significance. Based on Jiang et al.15 it was assumed that a clinical WGS could detect 10% more single nucleotide variants than CMA+WES, resulting in a hypothetical clinical WGS diagnostic yield of 17.4% (R. Yuen, personal communication). This calculation does not take into account noncoding variants or CNVs detected by WGS beyond those detected by CMA.14 Therefore, 42.4% was utilized as a best case yield for WGS in ASD.

Due to uncertainty in diagnostic yield estimates, a one-way sensitivity analysis was conducted where the diagnostic yield for each test was varied for each scenario. For CMA and CMA+WES, the lower and upper values were the 2.75% and 97.5% confidence bounds. Since clinical diagnostic yields are not yet available for WGS, the lower bound was the estimated diagnostic yield of CMA+WES, 15.8%, and the upper bound was a best case yield of 42.4%.

Results

Sample costs

The mean total sample costs for CMA, WES, WGS (HiSeq 2500), and WGS (HiSeq X) for Year 1 of a 5-year ASD testing service are shown in Table 2. The total cost of CMA per ASD sample was $744 (95% CI: 714, 773). The largest cost components were supplies and labor, accounting for 58.5 and 19.1% of the total, respectively. The total annual cost of WES was $1,655 (95% CI: 1,611; 1,699). Supplies and large equipment were the most expensive categories at 39.8% and 23.3%, respectively. WGS on HiSeq 2500 cost $5,519 (95% CI: 5,244; 5,785) per ASD sample, with supplies constituting 73.7% of total cost. WGS conducted on HiSeq X cost $2,851 (95% CI: 2,750; 2,956) per ASD sample. The difference in total mean cost between the HiSeq 2500 and the HiSeq X platforms was largely attributable to the greater cost of supplies and labor for HiSeq 2500. The costs of computing and labor for WGS were higher than WES due to greater bioinformatics and clinical interpretation demands.

Table 2 Estimated annual cost per ASD sample for CMA, WES, and WGS
Full size table

Program costs

The total institutional program cost to offer CMA for ASD diagnosis over 5 years was $1.05 million (95% CI: 1.01, 1.09) compared to $2.31 million (95% CI: 2.25, 2.37) for WES, $7.78 million (95% CI: 7.39, 8.15) for WGS HiSeq 2500, and $3.98 million (95% CI: 3.84, 4.13) for WGS HiSeq X based on 300 ASD cases per year. Figure 1 shows total program costs for each major category for each test. Supplies were the largest cost component for all three tests.

Figure 1
figure 1

Present value of ASD program costs over 5 years for CMA, WES, and WGS. Estimates are given in 2015 Canadian dollars (CAD). Program costs are based on 300 ASD cases annually. ASD, autism spectrum disorder; CMA, chromosomal microarray analysis; WES, whole-exome sequencing; WGS, whole-genome sequencing.

Full size image

Sensitivity analysis

Owing to economies of scale, when test volume for all indications increased from 500 to 1,000, sample costs decreased by 14.7% for WES, by 4.4% for WGS HiSeq 2500, and by 12.8% for WGS HiSeq X (Figure 2). The smaller reduction for WGS on HiSeq 2500 was due to equipment cost constituting a lesser proportion of total cost relative to supplies. Increasing overhead cost to 30% increased sample cost by only 1.7% for CMA to 3.0% for WES. The increase in sample cost when four variants were detected per sample was 9.6% for WES, 2.9% for WGS HiSeq 2500, and 5.5% for WGS HiSeq X.

Figure 2
figure 2

Deterministic sensitivity analysis of the effect of increasing the number of tests per year for all indications from 500 to 1,000 on cost per ASD sample. (a) WES; (b) WGS (HiSeq 2500); (c) WGS (HiSeq X). Estimates are given in 2015 Canadian dollars (CAD) for year 1 of a 5-year program. ASD, autism spectrum disorder; WES, whole-exome sequencing; WGS, whole-genome sequencing.

Full size image

Cost–consequence analysis

The incremental costs and diagnostic yields for three clinical testing scenarios for the exploratory cost–consequence analysis are shown in Table 3. The ratio of incremental cost to incremental diagnostic yield reflects the additional cost for every additional patient with a positive finding. These results ranged from $25,459 for CMA+WES versus CMA to $195,056 for WGS HiSeq 2500 versus CMA +WES. The tornado diagram (Supplementary Figure S1) shows substantial variation in the ratio depending on the diagnostic yield. For the CMA+WES versus CMA scenario, the ratio varied from $10,745 to $71,948. If the WGS diagnostic yield was 42.4%, the cost per additional patient with a positive finding decreased substantially. If WGS replaced CMA, the ratio decreased to $6,367 for HiSeq X and $14,428 for HiSeq 2500. For WGS versus CMA+WES, the incremental cost per additional patient with positive finding was $1,702 for the HiSeq X and $11,733 for HiSeq 2500.

Table 3 Estimated total annual incremental cost per ASD sample, estimated incremental diagnostic yield, and estimated incremental cost per additional patient with a positive finding for hypothetical clinical scenarios
Full size table

Discussion

Comprehensive cost estimates for emerging sequencing technologies are essential for making informed acquisition and deployment decisions. In this study, WGS using older technology (HiSeq 2500) was the most expensive, costing almost three times more than WES and seven times more than CMA. The newer HiSeq X reduced the cost of WGS test by approximately 50%. Although equipment costs are higher with this platform, automation enhances the number of samples that can be processed simultaneously and labor time is reduced because alignment and variant calling is more streamlined and uses specially designed HiSeq analysis software. Overall, supplies, followed by equipment and labor, constituted the largest proportions of total cost for all three tests. WGS displayed the highest supply costs due to greater consumption of costly sequencing reagents.

The optimal positioning of CMA, WES, and WGS in the diagnostic pathway for ASD and other pediatric conditions is being informed by ongoing variant discovery research.28 CMA is useful for detecting microdeletions and duplications not detectable by WES, although these can be detected by WGS.20 Variant discovery and linkage to phenotypes is proceeding at a rapid rate, creating pressure to introduce WGS into practice.32 The MSSNG Project is an example of a promising collaboration between advocacy, research, and data experts to create an open source database to catalyze identification of ASD subtypes to improve diagnostics and contribute to a deeper understanding of ASD to provide more effective treatments.14 Discovery research is overlapping with early stages of clinical implementation, necessitating frequent updates to microcosting and diagnostic yield estimates.

The question of which test or combination of tests should replace older technology is informed by translation research. In the present analysis, alternative scenarios are presented as complete substitutions, e.g., CMA plus WES replacing CMA alone, or WGS replacing CMA. This approach would be very costly, as an incremental cost of over $25,000 for every additional patient with a positive finding would be incurred if CMA were to be wholly replaced by CMA+WES or by WGS with our current knowledge of diagnostic yield. In reality the testing pathway is likely to be more complex, where, for example, only syndromic patients with a negative first-line test (CMA) go on to receive WES or WGS.5 An approach that reserves the more expensive technology for patients who initially test negative but demonstrate an ASD phenotype is expected to demonstrate a lower incremental cost per additional positive finding. Another more cost-effective option may be to target newer sequencing technologies to high-risk infant siblings of children already diagnosed with ASD, in whom a higher diagnostic yield is expected.33

Practice variation in genetic test ordering between clinicians makes it difficult to determine the potential for savings through avoidance of older generation genetic tests. As CGES becomes more established, test ordering protocols that prevent superfluous testing will be required. CGES may also lead to cascade testing in family members, further increasing costs. Rigorous criteria limiting family member testing to validation of phenotypically deleterious variants will be needed.

The estimated CMA cost was comparable to published reports. Trakadis and Shevell reported a CMA cost of approximately $682 (2010 CAD) for children with global development delay.34 Woodworth et al. estimated the cost of CMA in idiopathic learning disability to be £44235 (CAD$924 using the average 2006 UK pound/CAD exchange rate of 2.09).36 Regier et al. reported a CMA cost of $710 (2007/2008 CAD) for intellectual disability in children.37

Published cost estimates for WES or WGS are limited.10, 20 Wright et al. reported approximate costs of £6,000 (CAD$9,660, 2013) for WGS and £200–500 (CAD$322–805, 2013) for WES.38 Neither study provided a breakdown of items or the platform used. Monroe et al. estimated the cost of trio-WES at US$3,972 in patients with intellectual disability39 (CAD$4,409, 2014).36 That estimate included patient registration and blood draw, DNA isolation, sample preparation, exome enrichment, sequencing with Illumina HiSeq 2500, interpretation, reporting of results, data storage, and infrastructure. They calculated potential savings if standard genetic and metabolic testing was replaced with WES at US$3,547 (CAD$3,937) per patient receiving a diagnosis and US$1,727 (CAD$1,917) for patients not receiving a diagnosis.

While examining test costs as well as institutional program costs are necessary prerequisites to full economic evaluations, studies that examine costs to a health region or jurisdiction are also necessary, especially if the workflow is segmented. For example, regional centralization for certain steps, such as the sequencing, computing, and data storage, may increase efficiency and reduce costs to the health-care system compared to relying on individual institutional providers.23 As demand for CGES grows, health regions may form partnerships to offer a CGES service to their population. While introducing a CGES service may involve substantial start-up costs, savings could be realized through large-scale purchasing contracts, although this may entail overhead and administrative costs as well as transaction fees.

As WGS is currently used mainly in research, these costs were calculated as expected costs in a clinical setting. However, no estimate of the costs for updating of the annotation pipeline and maintenance were available. Training of technical and lab personnel and implementation costs were also not included. However, many steps are shared between WES and WGS during sample collection, library preparation, and sequencing. All stages involved in the workflow of CMA, WES, and two different WGS platforms were accounted for using microcosting. The cost estimates were based on only one institution and the same experts were used for uncertain volume and price estimates. However, there was no evidence for any specific form of correlation between responses and independence was assumed. For most of the price parameters, a range was chosen to reflect potential price and currency fluctuations and was within the variation for other parameters reported by experts. Uncertainty associated with parameter estimates was captured in the sensitivity analysis, which demonstrated robustness of the results to changes in assumptions. A 5-year time horizon was chosen based on a projected shelf-life for the sequencing equipment, and because procurement decisions for large equipment can be based on a 5-year budget plan. In reality, the life cycle for sequencers may be shorter due to rapid evolution of high-throughput sequencing technologies. A shorter life cycle would result in higher costs due to a shorter period of amortization. Although the estimates in this study are for an ASD patient population, the microcosting model was deliberately constructed to be flexible and easily adapted to other patient populations by varying select inputs such as the number of primary variants targeted and the total volume of testing in the institution. Another limitation was the limited diagnostic yield information available from the various testing scenarios and the reliance on a hypothetical yield for WGS. While increased knowledge of pathogenic ASD variants as well as secondary variants will improve the clinical utility of CGES, a longer list of reportable variants will add to the cost of validation, interpretation, and reporting. Therefore, the cost–consequence analysis should be considered speculative and caution must be exercised when interpreting the ratios.

This study is the first to estimate the cost of WGS using a bottom-up microcosting approach. While costs for WES and WGS remain high, newer WGS platforms demonstrate increased value compared to older technology through reductions in material and equipment costs and increased automation. Their cost-effectiveness is expected to grow as more is learned about the genetic architecture and phenotypic expression of newly discovered variants and variants of previously unknown significance. Additional research is required to determine which populations would benefit from CMA alone, and which from serial testing. These decisions will need to consider recent WGS studies revealing that approximately 50% of informative siblings with ASD carry unique penetrant mutations (some de novo, some inherited), suggesting WGS of each individual would be preferable to locus-specific segregation testing in family members.9, 14 CMA, WES, and WGS should be used in a manner that maximizes clinical utility without incurring high opportunity costs that might result in reduction of other services or limited access. Prospective economic evaluations are also needed to assess the impact of CGES on the pathway of care for children with ASD and to weigh the costs of the care pathway against ultimate improvements in health outcomes as a result of testing. The estimates generated in this study can be used in future such health technology assessments that more fully investigate the cost-effectiveness of CGES in the care pathway of ASD. It is essential that programs of health services and policy research that perform such studies be executed in tandem with translation of CGES into clinical practices to generate evidence to inform institutional and provincial health policy decision makers.40