Main

Venous thromboembolism (VTE), deep vein thrombosis (DVT) with or without pulmonary embolism (PE), occurs in 40–58 per 10,000 pediatric tertiary care hospitalizations, and the incidence is climbing (1,2,3). Many factors are thought to contribute to the rising incidence of this complication. These include the survival of children with previously fatal chronic illnesses, increasing awareness of VTE by pediatric practitioners, and the highly prevalent use of central venous access devices (CVAD) for management of critically and/or chronically ill children (4,5).

Unfortunately, despite the increasing awareness of pediatric VTE, the diagnosis in children remains reliant on a high index of suspicion on the part of clinicians (4). Signs and symptoms of acute VTE in children may include swelling and pain in an extremity, sometimes associated with dusky discoloration (plethora), loss of central venous access patency, superior vena cava syndrome, or respiratory compromise secondary to PE (4,6,7,8,9,10). Because these may be otherwise nonspecific signs in critically ill children who are at greatest risk for VTE, the diagnosis is often delayed or not considered (4). Prompt recognition is important, as early institution of appropriate therapy may decrease the likelihood of mortality and chronically debilitating complications of VTE (11,12,13).

Similarly, for adult patients with suspected VTE, the signs and symptoms are relatively nonspecific (14). However, diagnosis has been facilitated through the development of clinical probability tools (CPT), which have led to diagnostic algorithms which assist in accurate and timely diagnosis of VTE (14,15,16,17,18). The diagnostic algorithm has been further refined by combining CPT scores with the results of quantitative D-dimer assays, which have also shown high sensitivity, but only moderate specificity for adult VTE (18,19,20). Unfortunately, application of these adult CPTs to children has revealed poor reliability for childhood VTE diagnosis (5,21). This is likely related to the differing epidemiologic features of adult vs. pediatric VTE, for which the adult CPTs have been optimized. For example, in adults, VTE are often idiopathic occurrences whereas in children they are most commonly identified as a complication of a chronic comorbid condition that requires central venous access for management. Recently, a pediatric-specific clinical risk-factor algorithm has been proposed for childhood PE to optimize the appropriate utilization of computed tomography pulmonary angiography (22). However, no algorithm exists for childhood DVT or overall VTE. A retrospective single institution study of the sensitivity and specificity of D-dimer in children revealed promising results (23). However, recent studies have revealed suboptimal D-dimer assay performance for diagnosis of PE in children (21,22).

For this study, we hypothesized that it is feasible to utilize specific clinical variables associated with pediatric VTE to develop a pediatric-specific CPT. Wells et al. began development of their adult CPT for DVT by studying outpatients referred for evaluation of suspected DVT (14,16). Thus, we took a similar approach in the present study. However, because the majority of childhood VTE are identified in children who are hospitalized in a tertiary care center (3), we performed a retrospective pilot study examining the characteristics of children objectively diagnosed with VTE and/or imaged for suspected VTE at a major tertiary care children’s hospital.

Results

Suspected VTE Cases—Training Cohort

For the training cohort, our radiology keyword search produced 2,839 records, 392 (13.8%) of which met eligibility criteria. Records were excluded for age >18 y: 449 (15.8%); follow-up evaluation for preexisting VTE: 245 (8.6%); arterial thrombosis: 139 (4.9%); intracranial sinovenous thrombosis: 196 (6.9%); intracardiac thrombus: 14 (0.5%); tumor “thrombus”: 14 (0.5%); duplicate record: 250 (8.8%); study performed for reasons other than suspected VTE with absence of VTE mentioned as a pertinent negative: 436 (15.4%); and nonqualifying study: 704 (24.8%). Examples of “nonqualifying studies” included the use of a keyword in reference to intentional, therapeutic embolization or a non-VTE “clot” (e.g., soft-tissue hematoma “clot”). Complete medical records were not available for 3 subjects, leaving a final sample of 389 subjects. Of these, 91 (23.4%) were diagnosed with VTE and 298 (76.6%) without. Subject selection, including reasons for exclusion, is shown in Figure 1a .

Figure 1
figure 1

Study subject selection. (a) Derivation of training and (b) validation study cohorts.

PowerPoint slide

Full size image

Demographics and Characteristics of Suspected VTE Cases

Overall, the children with VTE were similar to those without VTE with respect to age, BMI, race, and imaging for DVT vs. PE ( Table 1 ). However, children with VTE were more likely to be male than those without VTE (68.1% vs. 41.9%; P < 0.001). Although overlapping presentations prevent a statistical comparison, children with VTE appeared to have a higher proportion of upper extremity, head, and/or neck signs and symptoms than did those without VTE (29.7% vs. 16.8%), consistent with previously reported pediatric VTE anatomic distribution (1,3,6).

Table 1 Demographics and characteristics of study subjects
Full size table

Data Availability

As described in Methods, 32 variables were collected from the comprehensive chart review for each of these 389 subjects, for a possible total of 12,448 data points. Of these, 1,065 (8.6%) fields were counted as “missing data” due to the variable not appearing in the medical record upon chart review ( Table 2 ). Similar quantities of data were missing from the VTE and non-VTE cohorts (209 (7.2%) vs. 856 (9.0%)).

Table 2 Candidate comorbid diseases and signs/symptoms of VTEa
Full size table

Univariate Analysis of Signs and Symptoms

Univariate analyses were performed after eliminating any subjects with missing data for the variable of interest ( Tables 2 and 3 ). Only 7 of the 32 signs and symptoms included in this analysis were significantly associated with the probability of VTE diagnosis: male gender (odds ratio (OR) = 2.96; 95% CI: 1.80–4.87; P < 0.001), CVAD utilization (OR = 2.53; 95% CI: 1.56–4.11; P < 0.001), loss of CVAD patency (OR = 4.11; 95% CI: 1.84–9.17; P = 0.001), active cancer (OR = 2.35; 95% CI: 1.19–4.63; P = 0.014), pitting edema (OR = 1.92; 95% CI: 1.02–3.61; P = 0.043), and dusky discoloration (plethora) of the extremity (OR = 2.09; 95% CI: 1.00–4.39; P = 0.050). Conversely, documentation of an alternative diagnosis by the clinician was predictive of negative VTE imaging (OR = 0.42; 95% CI: 0.24–0.76; P = 0.004). The remaining 25 parameters were not significantly associated with the probability of VTE ( Table 2 ).

Table 3 Univariate and multivariable ORs for VTE
Full size table

In order to simplify multivariate CPT model building, we grouped similar variables into new categorical variables without losing univariate significance. These new variables are shown in Table 3 : “asymmetric extremity” (any combination of swelling, edema, and/or discoloration; OR = 1.76; 95% CI: 1.05–2.97; P = 0.033) and “CVAD” (utilization and/or dysfunction; OR = 2.51; 95% CI: 1.53–4.14; P < 0.001). Other grouping strategies (e.g., “any chronic disease”) were not statistically relevant (data not shown).

The well-established bimodal age distribution of pediatric VTE was observed in this cohort ( Table 1 ) (1,3,6). Thus, because age was not significantly associated with VTE when considered as a continuous variable, we further considered age as a categorical variable. However, there was no statistically relevant relationship when age was categorized accordingly (infants < 1 y (28.8% had VTE), children 1–12 y (22.9% had VTE), and adolescents 13–17.99 y (21.5% had VTE); P = 0.429). Contraception was not a statistically significant univariate variable in the adolescent female subpopulation (OR = 1.38; 95% CI: 0.39–4.84; P = 0.613).

Multivariable Analysis

Multivariable logistic regression modeling was performed on the 326 subjects with complete data for all of the following variables: gender (male), asymmetric extremity, CVAD, active cancer, and alternative diagnosis ( Table 3 ). Thus, 10 (11.0%) VTE subjects and 53 (17.8%) non-VTE subjects were eliminated from the multivariable analysis. From this model, the OR for VTE was 2.96 for male patients (95% CI: 1.68–5.22; P < 0.001), after adjusting for asymmetric extremity, CVAD, active cancer and alternative diagnosis. The adjusted OR for CVAD (1.90; 95% CI: 1.07–3.39) was also statistically significant (P = 0.029). While the adjusted ORs for asymmetric extremity and active cancer were not significant, they were left in the final model because of their significance in univariate analysis, trend toward relevance in the multivariable model, and strong evidence in the literature that they are associated with pediatric VTE (1,3,7,8,9,10,24). As expected, a documented alternative diagnosis was associated with a significantly lower adjusted OR (0.33; 95% CI: 0.16–0.66; P = 0.002) for VTE in the multivariate model. The Hosmer-Lemeshow goodness-of-fit test for this model was adequate (P = 0.157).

Pilot Pretest Probability Tool

A pilot CPT algorithm was developed from the multivariable analysis where the pretest probability of VTE was estimated using the following equation, in which a 1 or 0 is substituted for the presence or absence (respectively) of each sign or symptom:

Validation Cohort and Pretest Probability Performance

The radiology keyword search to identify the validation cohort produced 1,239 records, 149 (12.0%) of which met eligibility criteria. Reasons for record exclusion are summarized in Figure 1b . Of these, 41 (27.5%) were diagnosed with VTE and 108 (72.5%) without ( Table 1 ). The demographic characteristics of the validation cohort were similar to those of the training cohort with respect to age (including bimodal distribution), gender, race, and evaluation for DVT vs. PE (P = 0.756, 0.930, 0.194, and 0.927, respectively). Discreet variables such as gender, cancer diagnosis, and current or recent CVAD use were 100% available for the validation cohort, which was collected exclusively in the electronic medical record (eMR) era. Some asymmetric extremity and CVAD patency data were not able to be determined from the chart review ( Table 2 ). Thus, 509 (42.7%) of 1,192 possible data fields were missing from the validation cohort. Similar quantities of data were missing from the VTE and non-VTE cohorts (133 (40.5%) vs. 376 (43.5%)). When these signs/symptoms were not specifically documented, they were assumed to be absent, in an “intention to diagnose” fashion. The pilot CPT performed similarly for the training cohort (n = 389) and the validation cohort as demonstrated by the receiver operating characteristic (ROC) curves shown in Figure 2 . The ROC AUC values were 0.73 (95% CI: 0.67–0.79; P < 0.001) for the training cohort and 0.64 (95% CI: 0.54–0.73; P = 0.011) for the validation cohort, indicating in both cases that the CPT was better able than chance to detect VTE. The validation cohort data fit the original multivariable logistic regression model adequately (Hosmer-Lemeshow goodness-of-fit test P = 0.344).

Figure 2
figure 2

Pilot venous thromboembolism (VTE) clinical probability tool (CPT) receiver operating characteristic curves. The pilot CPT algorithm was significantly reliable for discriminating the pretest probability of VTE in both the training (solid line) and validation (dashed line) cohorts with area under the curve (AUC) values of 0.73 (95% CI: 0.67–0.79; P < 0.001) and 0.64 (95% CI: 0.54–0.73; P = 0.011), respectively. The 45° diagonal represents the line of nondiscrimination (equivalent to a coin toss).

PowerPoint slide

Full size image

Discussion

In this retrospective single institution cohort study, we have taken the first steps toward development of a pediatric-specific CPT for VTE. In our training cohort, boys and children with CVAD had a significantly higher pretest probability of VTE, while children with a documented alternative diagnosis are less likely to have VTE. Condensing the most relevant signs and symptoms down to five variables resulted in the development of a simplified algorithm that would be amenable to clinical applications. However, the technique used to develop our model, built on retrospective data, requires further development and validation in a prospective cohort prior to clinical application. Importantly, a validated, easy-to-use CPT would significantly improve our ability to rapidly and cost efficiently screen children for VTE. Thus, when indicated, confirmatory testing could then be obtained and therapy started promptly, potentially resulting in a significant reduction of VTE-related morbidity and mortality (11,12,13).

This project not only achieved our objective to begin development of a VTE screening tool, but our pilot CPT was significantly able to predict the likelihood of VTE in an independent cohort. This suggests that, despite the inherent limitations of our retrospective study design, a robust VTE probability tool is feasible for pediatric applications. One reason for the observed difference in model performance in this study may be a transition from paper charts (training cohort) to an electronic medical record (validation cohort) which may have affected the quality of the available data (e.g., due to the use of template fields and drop-down menus in the electronic era). For example, there was an increased frequency of missing (i.e., indeterminate data) for the asymmetric extremity variables (entire leg swelling, calf swelling, pitting edema, and discolored extremity) in the eMR era, whereas the amount of indeterminate data that is discretely entered into eMRs (e.g., problem lists or surgical histories) decreased (e.g., cancer and CVAD utilization). Over 90% of the sought-after variables were included in the patient charts for the training cohort. Subjects with missing variables were eliminated from the model building step to avoid the introduction of bias for or against the potential importance of a variable. It is reasonable to assume that clinicians are more likely to leave a pertinent negative finding undocumented, whereas a pertinent positive finding is more likely to be documented. In contrast, a greater amount of data were missing from the validation cohort and when missing, were assumed to be absent, in an “intention to diagnose” fashion. Thus, the utility of these signs and symptoms for predicting the odds of VTE may have been biased in this study, and some of the eliminated variables may be relevant in a prospectively collected dataset with greater data completeness and integrity. Despite these issues, the pilot CPT derived from this feasibility study demonstrated moderate performance in the training cohort (AUC = 0.73) with slightly lower performance (AUC = 0.64) in the validation cohort.

Our pilot CPT was derived from a cohort of patients assessed for VTE by a variety of different imaging techniques, which almost certainly have varying sensitivity and specificity, in part depending on the anatomic location of the suspected thrombus, which may have resulted in misclassification of some cases (25). However, this method was necessary to (i) represent the multimodal approach to pediatric VTE diagnosis and (ii) identify an adequate number of cases to power our analyses (26). We did not include blinded adjudication of radiology results for this retrospective feasibility study but, with the high interobserver agreement for these imaging modalities (see Methods), we can be fairly confident that VTE were objectively identified (27,28,29,30). Because D-dimers have not yet been incorporated into the standard of care for children with suspected VTE they were rarely available, thus we were unable to include D-dimer values in our analysis. In adult studies, D-dimer is typically utilized to further stratify risk and guide the diagnostic evaluation after CPT score assignment (19). Thus, D-dimer will be an important variable for inclusion in prospective pediatric studies of this topic. However, because D-dimer is commonly elevated in children for non-VTE reasons (i.e., infections and other inflammatory illnesses), it may be necessary to adjust D-dimer results for biomarkers of inflammation or to consider the development of other VTE biomarkers (31).

Published interobserver agreement for VTE symptoms using structured surveys in adult subjects is “substantial” (κ-coefficients = 0.6–0.86 for DVT and ~0.62 for PE) (14,32,33,34). Interrater agreement is considered “moderate” at κ-coefficients of 0.4–0.6, “substantial” at 0.6–0.8, and “almost perfect” at >0.8 (34). We would expect that the unstructured, retrospective clinical history and exam findings in our study would be subject to greater variability, which may have influenced our results, especially in the validation cohort. Interobserver agreement for clinical signs of pediatric VTE has not previously been assessed, but is “moderate” to “almost perfect” in other areas of pediatrics: abscess diagnosis (κ = 0.48) (35), appendicitis signs and symptoms (κ = 0.49–0.54) (36), physical exam findings in children with abdominal pain (κ = 0.54) (37), and blunt head trauma signs and symptoms (κ = 0.83–0.93) (38). Thus, the evidence strongly suggests that a structured CPT for pediatric VTE is feasible. Even though our pilot CPT has only modest performance characteristics, as illustrated above, this is also the case for many other CPTs and is also true for the adult VTE CPT. Nonetheless, these tools have been successfully utilized to improve upon accurate and timely diagnosis of these conditions and/or to improve healthcare resource utilization. For example, a universal VTE screening strategy utilizing ultrasound in high-risk children might detect nearly all VTE but with a very high negative study rate, generating substantial costs. While a CPT screening strategy may also increase the number of ultrasounds performed, these would be in patients already predicted to have a high likelihood of VTE, potentially reducing the overall number of studies while improving the detection of this life-threatening condition. Thus, when prospectively validated VTE CPTs become available for children, an important step will be to perform cost-utility analyses to ensure that they are providing the desired advantages at reasonable costs to the healthcare system.

In this CPT model, males were nearly threefold more likely to have VTE than females. Gender is a discreet demographic variable captured on admission of all patients, thus the quality of this variable is robust. Recent large pediatric VTE epidemiology studies have demonstrated a slight male predominance (3,39). In our smaller, validation cohort, the male predominance was smaller and not statistically significant, thus our training cohort may have overestimated the relevance of this variable. Alternatively, because the male predominance of pediatric VTE is generally only noted in very large epidemiologic studies, our validation cohort may have been too small to be sensitive to this variable.

The presence of a CVAD in the symptomatic anatomic region and/or loss of CVAD patency were similarly associated with a higher pretest probability of VTE (nearly twofold). This is not a surprising finding, given that CVADs are associated with over 50% of all pediatric VTE and consequently are considered to be the single most important risk factor for VTE by many experts (8,10,40,41).

Although cancer and the aggregate “asymmetric extremity” variable were no longer significant after multivariable adjustment, these symptoms were still included in the final model because of their reported relevance throughout the pediatric VTE literature. Cancer is consistently amongst the three most common chronic illnesses observed in association with pediatric VTE (1,3,24). While extremity symptoms have not been directly studied in the epidemiology of pediatric VTE, they are considered by many experts to be a hallmark of the disease (7,8,9,10). Because this study included nonextremity VTE, it is not unexpected that this variable lost some relevance in the final model. However, recent epidemiologic studies have demonstrated that extremity VTE are more common than central or neck VTE (1,3). Furthermore, our study may have been underpowered to detect the statistical relevance of this variable, which approached significance after multivariable adjustment (P = 0.052). Therefore, these variables were included in our pilot model and should be studied carefully in future prospective cohort studies. However, if they are not statistically relevant in a robust, prospectively collected dataset, they should be excluded from the model prior to clinical application.

As expected, documented clinical suspicion for an alternate diagnosis was correlated with lower odds for VTE. In contrast to other missing variables, the logical assumption is that listing other diagnostic possibilities indicated that the clinician felt strongly that there were other diagnostic possibilities that should be considered. Whereas omitting other possibilities would indicate that the practitioner felt that VTE was so highly probable that time was not taken to document other remote possibilities. Furthermore, imaging studies may never be ordered for patients with a highly probable alternative diagnosis, resulting in their exclusion from our study sample and underestimation of the negative predictive value of this variable. This is likely the most difficult variable to properly assign in a retrospective study and certainly deserves careful attention in prospective research that includes an image-independent control group.

For this feasibility study, we included both DVT and PE. However, for adult VTE, separate CPTs have been developed due to variation in both epidemiology and symptomatology of these conditions (15,18,42). Thus, it is possible that separate pediatric models may also be required to achieve desired performance characteristics. Similarly, the epidemiology of neonatal VTE may be different enough from VTE in older children to justify development of a neonatal-specific model. In the future, it may be necessary to develop separate models for noncatheter related (idiopathic) VTE or cancer vs. noncancer related VTE (or other disease groups) in order to provide tools to subspecialty clinicians that are specific to their patient groups. Published studies of disease-specific VTE epidemiology in pediatrics are lacking, thus these types of studies are needed and may elucidate appropriate signs and symptoms for inclusion in these types of CPT algorithms.

In summary, despite the limitations of this retrospective study we have demonstrated that development of an accurate and simple CPT for pediatric VTE screening is feasible. Such a tool could be applied clinically to improve the timely detection of childhood VTE, resulting in more efficient healthcare resource utilization by better targeting which children should have imaging studies performed and potentially improving VTE outcomes. Similarly, such a tool would be useful in the pediatric VTE research community to more efficiently screen for VTE amongst prospective study cohorts. However, prior to implementation, this tool needs to undergo additional development and validation in a prospective cohort study with the capability to gather all relevant data in an accurate and complete manner, which is expected to increase the sensitivity and specificity of the algorithm to a clinically useful level. Relevant VTE biomarkers, such as D-dimer, should also be incorporated into these studies to determine their utility to optimize the diagnostic screening process. The availability of pediatric VTE evidence continues to lag significantly behind that available for adult VTE; development of a pediatric VTE CPT will make scientifically rigorous study of this disease much more feasible.

Methods

Ethics

This retrospective cohort study was approved by the Nationwide Children’s Institutional Review Board (IRB09-00018). The requirement for informed consent was waived according to 45 CFR 46.116(d) of the US Code of Federal Regulations.

Study Subjects

Nationwide Children’s Hospital (Columbus, OH) radiology records from diagnostic ultrasound, computed tomography, magnetic resonance imaging, nuclear medicine, and interventional radiology studies were searched for keywords associated with a diagnosis or description of VTE. Keywords included “DVT,” “clot,” “thrombus,” “thrombi,” “thrombosis,” “embolus,” “emboli,” “embolism,” “leg swelling,” and “arm swelling.” The presence of a single keyword was necessary for record inclusion. A training cohort was derived from radiology studies performed from 1 January 2007 to 31 December 2008 (inclusive). Subsequently, a validation cohort was derived from studies performed in the 1 July 2011–31 December 2011 (inclusive) era.

The resulting radiology records were reviewed by one of the investigators (B.A.K.) to identify eligible cases for inclusion in the study cohort. Eligibility was defined as age ≤18 y and performance of the study for either suspected VTE or incidental discovery of VTE. Cases were excluded if the study was performed for arterial thromboembolic disease (PE was not excluded), intracranial sinovenous thrombosis (because the presenting neurologic signs and symptoms are substantially different enough from other VTE that including them in a single model was not considered feasible), intracardiac thrombi, intravascular tumor “thrombus,” or reevaluation of prior VTE. All other types of (suspected) VTE were considered eligible, including: lower and upper extremity DVT (including extension into central chest or abdominal-pelvic veins), superior or inferior vena cava thrombosis, jugular or other neck venous thrombosis, central abdominal (mesenteric, renal, portal) venous thrombosis, and PE. Eligible subjects were segregated into VTE and non-VTE cohorts according to the reading radiologists’ impression of the presence or absence of objective evidence of thrombosis or filling defect (in the case of PE), as recorded in the radiology report. There is ample evidence that interobserver agreement for imaging diagnosis of VTE (κ-coefficient of 0.77–0.99 for DVT and ~0.73 for PE) is high (27,28,29,30). Thus, for the purposes of this retrospective pilot study, the radiology exams were not adjudicated by a blinded reviewer.

Data Collection

All of the eligible cases underwent systematic chart review by trained clinical research assistants to determine the presence or absence of signs and symptoms related to VTE that were defined a priori as follows (6,15): The adult CPT parameters were considered, including: active cancer, paralysis, paresis, or plaster immobilization, bedridden for ≥3 d, major surgery within 12 wk, localized tenderness along the course of a deep vein, entire leg swelling, calf swelling, pitting edema confined to a symptomatic leg, collateral superficial veins, previous DVT, and documentation of a suspected alternative diagnosis (15). In some situations, special considerations for pediatric application of these adult signs/symptoms were needed. For instance, infants are physiologically nonambulatory. Thus, bedridden, paralysis, and paresis was defined as pathologically limited activity. Iatrogenic, pharmacologically induced paralysis, which is commonly employed in critically ill, ventilated children, was counted as paralysis. A nonambulatory infant without any pathologic limitation or medical restraint was not “bedridden.” Assigning the presence or absence of an alternative diagnosis is particularly challenging in a retrospective study. Thus, an alternative diagnosis was considered present only if a documenting practitioner listed one or more differential diagnoses in addition to VTE.

These parameters were supplemented with pediatric-specific VTE variables: presence of a CVAD in the symptomatic venous system (currently or within the past year), loss of CVAD patency, congenital heart disease, kidney disease, prematurity, sepsis, systemic lupus erythematosus, sickle cell disease, pain in the symptomatic extremity, chest pain, dusky discoloration (plethora) of the symptomatic extremity, head or neck swelling suggestive of superior vena cava syndrome, and respiratory compromise (respiratory distress and/or a new or increasing oxygen requirement) (4,6,7,8,9,10). Additional parameters of general interest including age, gender, ethnicity/race, BMI (for children ≥2 y of age), and hormonal contraceptive within 3 mo of assessment (for adolescent females) were also considered. Documentation of all variables was only considered prior to the day and time of the imaging study that qualified the subject for the study. To validate the collected data, five percent of the charts were also reviewed by an additional research assistant or an investigator (B.A.K.) in a blinded fashion, which yielded >95% raw agreement.

For the validation cohort, only those parameters included in the final model were abstracted from the medical records.

Statistics

Demographics and characteristics of suspected VTE cases were compared between patients with VTE and those without VTE. Wilcoxon rank-sum tests were used to compare continuous variables and χ2 tests or Fisher’s exact tests were used for categorical variables. Backward stepwise modeling methods were used to estimate the relative significance of potential predictors for VTE for this group of subjects. Model fit was determined by the Hosmer-Lemeshow test and discrimination by the AUC for the ROC curve. These analyses were performed using SAS version 9.2 (SAS Institute, Cary, NC) and Stata/SE 10.0 (StataCorp, College Station, TX). Subjects with missing variables were excluded from the univariate analyses utilized to determine which parameters to include in the multivariate analysis. The pilot CPT algorithm, generated from the multivariate analysis was applied to the validation cohort, without excluding subjects with missing data in an “intention to diagnose” fashion. ROC curves were generated to compare the performance of the pilot CPT in the training and validation cohorts using SPSS Statistics version 21 (IBM, Armonk, NY).

Statement of Financial Support

This project was supported by the George & Elizabeth Kelly Foundation (Lewis Center, OH) and grants 239409 and 285712 from The Research Institute at Nationwide Children’s (Columbus, OH) both to B.A.K., and award numbers UL1TR001070 and UL1TR000090 from the National Center for Advancing Translational Sciences, National Institutes of Health (Bethesda, MD).

Disclosure

There is no conflict of interest to disclose.