The second iteration of the Autism Brain Imaging Data Exchange (ABIDE II) aims to enhance the scope of brain connectomics research in Autism Spectrum Disorder (ASD). Consistent with the initial ABIDE effort (ABIDE I), that released 1112 datasets in 2012, this new multisite open-data resource is an aggregate of resting state functional magnetic resonance imaging (MRI) and corresponding structural MRI and phenotypic datasets. ABIDE II includes datasets from an additional 487 individuals with ASD and 557 controls previously collected across 16 international institutions. The combination of ABIDE I and ABIDE II provides investigators with 2156 unique cross-sectional datasets allowing selection of samples for discovery and/or replication. This sample size can also facilitate the identification of neurobiological subgroups, as well as preliminary examinations of sex differences in ASD. Additionally, ABIDE II includes a range of psychiatric variables to inform our understanding of the neural correlates of co-occurring psychopathology; 284 diffusion imaging datasets are also included. It is anticipated that these enhancements will contribute to unraveling key sources of ASD heterogeneity.
Machine-accessible metadata file describing the reported data (ISA-tab format)
Background & Summary
Multiple sources of evidence have substantiated models of abnormal neural connectivity in autism spectrum disorder (ASD)
The initial ABIDE I effort established the feasibility of aggregating multisite data without prior harmonization, leading to more than 55 peer-reviewed studies in the 48 months since inception. Despite its success, ABIDE I is limited in regard to sample characterization and sample size. Specifically, despite containing more than 1,000 datasets, ABIDE I was not sufficiently large to furnish optimally sized discovery and replication subsamples. By combining the ABIDE I and ABIDE II data resources, investigators can select larger samples for discovery and replication, depending on their investigative endeavors. Replication samples are needed to minimize false positives and avoid settling for ‘approximate replications’ 19 —a practice that has plagued biological psychiatry 19 and neuroscience more broadly 17 . Additionally, as recently demonstrated, the utility of datasets for prediction increases with sample size—even if heterogeneous data sources are used to amass large samples 23 .
Along with increased sample size, ABIDE II provides greater phenotypic characterization than was available across the ABIDE I data collections to better address two key sources of heterogeneity. The first is psychopathology co-occurring with ASD, which has been largely overlooked in the imaging literature 15,16,24 . Accordingly, ABIDE II actively encouraged investigators to provide phenotypic information regarding co-occurring illness, if assessed. The second source of heterogeneity is driven by sex-related differences. These have been generally ignored in the ASD imaging literature due to the markedly higher prevalence of males with ASD and the tendency of single sites to exclude or minimally represent females. The ABIDE II sample has increased the number of available datasets from females with ASD from 65 in ABIDE I to 138 when ABIDE I+II are combined. We believe these enhancements will allow investigators to more directly investigate pathophysiology specific to ASD, to potentially identify neurobiological subgroups and facilitate the identification of protective and risk factors.
Finally, beyond its focus on intrinsic functional connectivity and other indices of intrinsic brain function, ABIDE II now includes a subset of datasets (N=284) with diffusion-weighted images. In order to facilitate immediate access and use of ABIDE II, the methods utilized to generate this resource, the resulting currently available data and their technical validation are described below.
Criteria for data contributions
We solicited investigators willing and able to openly share their previously collected awake R-fMRI data of individuals with ASD and controls, along with corresponding high-resolution anatomical images and phenotypic information. Contributions have been sought from all charter ABIDE I members and invitations are extended to any other investigators involved in ASD neuroimaging. The present work includes information regarding all contributions received prior to June 24, 2016. Contributions will continue to be accepted up to December 2016.
Contributors are encouraged to share at least 20 unique datasets per diagnostic group (i.e., ASD and controls). Data collections of only individuals with ASD are also accepted as they can be utilized for data-driven explorations addressing heterogeneity e.g., refs 25,26. Consistent with prior FCP/INDI efforts 27 , investigators are also encouraged to contribute nearly all MRI datasets, without a priori quality criteria (see Technical Validation for quality assessment (QA) measures incorporated into ABIDE II).
The availability of minimal phenotypic information essential for data analyses and sample characterization (i.e., diagnostic classification, age, sex) is required for contribution. To enhance phenotypic characterization, sharing of additional measures commonly used in ASD research, information on psychiatric comorbidity, medication status, cognition and/or language are highly encouraged. Similarly, to enhance the breath of investigations about the ASD connectome, whenever available, contributions of corresponding diffusion images for each individual are welcome for aggregation.
Finally, prior to data contribution, sites are required to confirm that their local Institutional Review Board (IRB) or ethics committee have approved both the initial data collection and the retrospective sharing of a fully de-identified version of the datasets (i.e., after removal of the 18 protected health information identifiers including facial information from structural images as identified by the Health Insurance Portable and Accountability Act [HIPAA]).
Of note, two institutions provided longitudinal MRI scans from subsets of individuals’ datasets (n=23 ASD and n=15 controls) previously contributed to ABIDE I. Given the relevance of developmental changes
Data preparation and aggregation
Prior to contribution, each institution is asked to rename all data by replacing local subject identification numbers with FCP/INDI identifiers. They are also asked to remove personally identifying information (PHI) including those from images (e.g., NIFTI headers and face information from any high-resolution images) using the FCP/INDI anonymization script available in http://fcon_1000.projects.nitrc.org/. Once data are fully anonymized at each site, they are submitted to the coordinating centers (Nathan Kline Institute and New York University) for review and harmonization within and across sites. Specifically, MRI data are visually inspected and edited as needed to ensure complete removal of facial information. Additionally, to further protect personal privacy, images of ears are removed from high-resolution images. Regarding phenotypic datasets, each entry is also reviewed to identify and correct missing data, any impossible entry values (e.g., beyond published maxima and minima), and extreme outliers (relative to each sample). To ensure uniformity across sites, all entries are recorded as needed and organized in a common template along with a legend of code keys. As a final step in preparation for release, both donating and coordinating sites jointly prepare a narrative for each data collection, documenting information on the methods utilized, funding sources, the investigators involved, whether any link with other databases (e.g., National Database for Autism Research—NDAR 33 ) exists, along with publications related to the contributed datasets. Before open release, each donating site reviews their reorganized phenotypic records, five random images per imaging modality and their collection-specific narrative for final approval.
The current ABIDE II dataset encompasses 17 collections of unique independent datasets (i.e., from individuals whose data were not previously shared in ABIDE I) yielding 487 datasets classified as ASD and 557 as controls (Fig. 1a, Table 1). These represent previously collected datasets across 16 sites, including nine charter ABIDE I institutions and seven new members (See Supplementary Table 1 for information on each institution). A simple naming convention is used to label each data collection: <ABIDEII> -<institution acronym name>_<collection number>(e.g., ABIDEII-NYU_1). When a collection in ABIDE II is a continuation of one initiated in ABIDE I, we employ the same collection number used in ABIDE I (or 1 if none was used, e.g., SDSU_1, KKI_1). For new collections, a unique consecutive number is assigned (e.g., BNI_1, KUL_3). Accompanying the primary cross-sectional aggregate, two longitudinal collections are also aggregated in ABIDE II. These include MRI datasets collected as follow-ups to the MRI and phenotypic data released in ABIDE I (N total=38 unique IDs). These pilot longitudinal collections are identified as <ABIDEII> -<institution acronym name>_<Long> (Table 1).
All ABIDE II datasets can be accessed, after establishing a login and user password, through FCP/INDI at the Neuroimaging Informatics Tools and Resources Clearinghouse (NITRC; http://fcon_1000.projects.nitrc.org/indi/abide/). The datasets are organized by data collection and stored in.tar files, each containing imaging and phenotypic data.
All phenotypic data are stored in comma separated value (.csv) files. A legend describing each phenotypic variable source is available at the website http://fcon_1000.projects.nitrc.org/indi/abide/abide_II.html. Phenotypic files are organized by data collection; a phenotypic composite file including all variables across all collections is also available. Counts of phenotypic variables available for each collection and distributions of selected key variables for each diagnostic group are provided in Supplementary Tables 2 and 3. Below, we briefly describe the main demographics and key phenotypic variables provided in the 17 cross-sectional ABIDE II data collections (Figs 1 and 2).
A dummy variable indicates diagnostic group (1 and 2 for ASD and controls, respectively). Given the retrospective nature of this data aggregate, assessment protocols used to identify ASD and controls varied across institutions. They are documented in each data collection narrative. Briefly, ASD classification was determined by either 1) combining clinical judgment with ‘gold standard’ diagnostic instruments—Autism Diagnostic Observation Scale 34,35 and/or Autism Diagnostic Interview-Revised 36 [ADOS, ADI-R]; (n=12 data collections; 368 ASD datasets) or 2) by using these ‘gold standard’ diagnostic instruments only (n=4 collections; 92 ASD datasets), with one exception. Specifically, in EMC_1 (n=27 datasets), which was selected from the longitudinal Generation R sample 37 , the ASD classification was based on prior medical records documenting ASD among those individuals meeting screening cutoffs in at least one of two distinct ASD questionnaires or for whom the mother reported a diagnosis of ASD. Regarding controls (N=557, available for 15 collections), all datasets are characterized by absence of ASD diagnosis and absence of history of any other major neurodevelopmental disorders for the vast majority of the datasets (N=546; 98%). This was determined using semi-structured/unstructured in-person interviews (N=7 data collections; 353 datasets), or parent/self- (if adults) reports/questionnaires (N=8; 193 datasets). The remaining 11 control datasets (OHSU_1 data collection) are from individuals assigned a ‘rule out’ psychiatric disorder, but without ASD or Attention-Deficit/Hyperactivity Disorder (ADHD) diagnoses.
Other specific inclusion/exclusion criteria used for selecting controls (e.g., IQ range, first degree relative with ASD) or ASD (e.g., absence of reported seizure and genetic syndromes) varied across collections. Each collection narrative on the ABIDE II website provides details regarding these criteria.
Across collections, age at time of scanning ranges from 5 to 64 years; four of the collections focused specifically on adults—with one of these collections specifically enrolling on older adults (BNI_1)—and eight enrolling only children and/or adolescents. The remaining five data collections include children, teens and young adults, which allows for cross-sectional age-related explorations (Fig. 1c; Supplementary Table 3). All but four collections include data from both sexes (Fig. 1b). Reflecting the higher prevalence of males in ASD 38 , 15% of the ASD datasets consist of females versus 31% of the control datasets (Supplementary Table 3).
Full scale intelligence quotient (FIQ) and/or verbal and/or performance IQ standard scores are provided. Across collections, although variation exists with respect to the minimum FIQ, 97% of the datasets have FIQ above 80 (Fig. 1d). For both groups, mean FIQ is above average, albeit significantly higher in controls versus ASD (Mann Whitney U=86.5; P<0.0001; Supplementary Table 3).
Categorical handedness codes for right, left or mixed handedness are available across all collections. Additionally, handedness strength scores are available for eight collections, enabling dimensional characterization of handedness (n=244 ASD and n=327 controls). Across collections, right-handedness is more frequent in both diagnostic groups (84 and 90% for ASD and controls, respectively), though a significantly higher prevalence of non-right-handedness (either left or mixed handedness) occurs in ASD relative to controls (χ1 2=10.6, P=0.01; Supplementary Table 3).
ASD core measures
Scores from the ADOS and ADI-R are available (Supplementary Table 3). Only nine collections share ADOS-2 calibrated severity scores 34 (CSS; N=9 sites; 228 ASD datasets) recently designed to adjust for differences in age, intellectual abilities and language skills across ADOS modules 39,40 . As illustrated in Fig. 1f, CSS distribution are similar across most sites. ADOS-G 41 scaled total scores are available for 15 collections (n=280 ASD datasets). Additionally, data from parent or self-report questionnaires commonly used in the field to quantify severity on multiple ASD domains collected across both diagnostic groups are also available. The Social Responsiveness Scale 42 is the most common (n=378 ASD, n=407 controls; Fig. 1f) followed by the Repetitive Behavior Scale Revised 43,44 (n=217 ASD, n=208 controls; Supplementary Table 3).
Comorbid psychopathology in ASD
Information on psychopathology accompanying ASD is provided either as 1) categorical diagnostic labels (or its absence, if assessed) with corresponding diagnostic code based on the International Classification of Diseases-9th edition
(N=9 data collections; 281 ASD datasets) and/or as severity scores in one or multiple psychopathology dimensions across available for 11 collections (see Supplementary Table 3 for a list of measures used) (Fig. 2). Categorical comorbid psychiatric diagnoses were determined based on clinicians’ assessments in seven data collections, parent-questionnaires in one data collection (UCD_1) and self-report in another (KUL_3). Consistent with the clinical literature
For each of the 17 ABIDE II cross-sectional collections, for each unique ID#, at least one structural MRI (sMRI), one corresponding R-fMRI dataset are available (except for one individual in the IP collection for which only MRI is available); corresponding diffusion MRI (dMRI) datasets are available for six collections. One data collection (SDSU) provided field map-corrected version of its R-fMRI and DTI data. The two pilot longitudinal collections include sMRI and R-fMRI datasets collected at two time points (1–2 years apart) for 23 individuals with ASD and 15 controls.
Consistent with its popularity in the imaging community and prior usage in FCP/INDI efforts, the NIFTI file format was selected for storage of the ABIDE II MRI datasets. With the exception of a single collection (IP_1, 1.5 Tesla), all MRI data were acquired using 3 Tesla scanners. Table 1 lists the specific MRI scanners and head coils utilized for each collection, along with the number of individuals available for each MRI modality within diagnostic groups (i.e., ASD and controls). Specific MRI sequence parameters for the various data collections are summarized in Tables 2, 3, 4 and detailed on the ABIDE II website. Across collections, R-fMRI acquisition durations varied from five to eight minutes (6:21±0.04 min) per individual; in all but four collections, individuals were verbally asked to keep their eyes open. For 12 collections, exposure to scan simulators prior to scanning was also used for habituation, as documented in the narratives.
Consistent with the established FCP/INDI policy, all data contributed to ABIDE II was made available to users regardless of data quality 27 . The rationale of this decision includes the lack of consensus on optimal quality criteria in regards to specific measures or their combinations and cutoffs. Additionally, depending on the study goal, the availability of scans with a range of quality can facilitate the development of artifact correction techniques 18 . For initiatives focusing on clinical populations like ABIDE II, the inclusion of datasets with artifacts such as motion are valuable, as they enable investigators to determine impact of such real-world confounds on reliability and reproducibility.
To facilitate quality assessment of the ABIDE II collections and selection of datasets for analyses by individual users, we used the Preprocessed Connectome Project quality assurance protocol 49 (http://preprocessed-connectomes-project.github.io). These encompass quantitative metrics commonly used in the imaging literature for assessing data quality, particularly for multisite projects, e.g., ref. 50. They include spatial metrics of scanner performance such as contrast to noise ratio 50 artifactual voxel detection 51 as well as temporal metrics including those quantifying head motion 52 ; all metrics are summarized in Table 5 and all are available in the data release. As expected by design, within- and between-site variation exists across quality metrics (see Figs 3 and 4 and Supplementary Fig. 1 for examples of spatial and temporal metrics in sMRI, R-fMRI and DTI). It is important to note that the field remains without consensus standards for the usage of QA measures. Additionally, differences in some measures across collections may reflect purposeful tradeoffs in the design of an imaging protocol, which may not be readily obvious at times. As such, caution should be taken in over-interpretation of between-collection differences in QA measures. At a minimum, the various QA measures provided can be used to find outlier datasets for a given site; though, potentially they may be used to provide insights into the impact of differences in acquisition protocols on quality measures as well.
As data aggregation followed independent data collections across multiple sites, various sources of heterogeneity exist between collections. They can range from inclusion/exclusion criteria, recruitment/sampling strategies, MRI scanner types, data acquisition parameters and instructions (e.g., eyes open versus closed). Users must be aware of such factors when designing their research questions and selecting data for analyses accordingly. Care should be taken when attempting to draw comparisons across ABIDE I and ABIDE II, as they are independently created aggregate datasets, bringing with them both commonalities and differences. Nine institutions participated in both initiatives with either related collections in regard to both phenotypic and imaging protocols (e.g., NYU_1 in ABIDE II is a continuation of NYU in ABIDE I) or collections acquired through independent protocols (e.g., KUL_3 in ABIDE II). We suggest consideration of the commonalities and differences among contributions when attempting to combine datasets from the two ABIDE initiatives. The narratives included in the ABIDE II website should facilitate this process—see Supplementary Fig. 2 for the collections distributed among the ABIDE initiatives. As a general rule, for aggregate data analyses datasets should be selected to ensure that the number ASD and TDC data are balanced at each collection, unbalanced designs (e.g., all typical participants selected from one collection, all ASD selected from another) should be avoided.
The impact of known and unknown sources of heterogeneity between collections should also be taken in account at the analytical level. First, we encourage the use of standardization at individual- and group-level analyses e.g., refs 53,
The need for careful consideration of variation in acquisition parameters also applies to the use of the quality assurance (QA) metrics available for the ABIDE-II sample. Some QA measures may be more or less comparable across data collections. Mean FD is an example of a measure commonly used for QA in resting state fMRI studies, albeit without significant considerations on the impact of the specific acquisition protocol employed. Motion-induced fluctuations in the BOLD signal are primarily due to spin history effects and partial voluming, which are proportional to the amount of tissue displacement between subsequent excitations. From this perspective, one might expect that factors capable of impacting spin history effects or partial voluming, would in turn impact meanFD. Importantly, these relationships may not necessarily be linear or additive. As such, some caution is suggested when interpreting systematic differences in meanFD, or related motion metrics (e.g., DVARS), across collections. Users may also employ this and other shared multisite datasets e.g., refs 60,61 to explore the impact of possible differences related to acquisition parameters, such as TR and other, on motion metrics. MeanFD computed in DTI data should not be used for comparisons between different collections with different MRI protocols. Mean FD in DTI is the result of the combination of both eddy current effects and head motion. As a result, meanFD can be used to compare and select data within collections obtained with the same scanning protocols and equipment.
Finally, to facilitate replications among studies using ABIDE data, we encourage users to provide the ID list utilized for their published manuscripts in the manuscript section of the ABIDE website (http://fcon_1000.projects.nitrc.org/indi/abide/manuscripts.html). Users are also requested to acknowledge the primary funding source for ABIDE II (NIMH 5R21MH107045) in any manuscripts using the ABIDE II data.
How to cite this article: Di Martino, A. et al. Enhancing studies of the connectome in autism using the autism brain imaging data exchange II. Sci. Data 4:170010 doi: 10.1038/sdata.2017.10 (2017).
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Di Martino, A. Functional Connectomes Project International Neuroimaging Data-Sharing Initiative http://dx.doi.org/10.15387/FCP_INDI.ABIDE2 (2016)
We would like to thank the numerous contributors at each donating institution (see Supplementary Table 1 and http://fcon_1000.projects.nitrc.org/indi/abide/) as well as the www.nitrc.org team for providing the data-sharing platform. We are particularly thankful to Tanmay Nath for support in aspects of website organization and assistance in some aspects of MRI data review along with Dorothea Floris, Yuta Aoki, Lindsay Alexander and Erica Ho. Support for ABIDE II coordination and data aggregation was provided by NIMH (521MH107045) to A.D.M. and gifts from Joseph P. Healey, Phyllis Green and Randolph Cowen to M.P.M. (M.P.M. is a Randolph Cowen and Phyllis Green Scholar). Support for data collection at each site was provided by NIH (MH084961-GU_1; MH094409-IU_1; NS048527, MH085328, MH078160-KKI_1; MH102660, MH087770, MH084126, MH081218, HD065282-NYU_1; MH102660-NYU_2; MH095888-ONRC_2; MH096773, MH091238, MH096773-03S1, MH086654-OHSU_1; MH081023, MH097972-SDSU_1; MH099250-01-UCD_1; HD055784-UCLA_1; MH092697, MH080826, MH60450, DC008553, NS34783-USM_1; HD065280-01-UCLA_Long; MH081191, MH67924, HD55748-UPSM_Long), Autism Speaks (KKI_1, 04593; UPSM_Long), the Simons Foundation (307280; EMC_1, OHSU_1), IDDRC (HD040677-07; GU_1), State of Arizona Alzheimer’s Consortium, BNI and Department of Defense (AR140105; BNI_1), Dutch ZonMw TOP grant (91211021; EMC_1), European Community’s 7th Framwork Programme (FP7/2008-2013, 212652; EMC_1), Physical Sciences Division, SURFsara, the Municipal Health Service Rotterdam area, the Rotterdam Homecare Foundation, and the Stichting Trombosedienst & Artsenlaboratorium Rijnmond STAR-MDC (EMC_1), Institut Pasteur, CNRS, INSERM, AP-HP, University Paris 7 Diderot, the BioPsy Labex, the DHU PROTECT, the Bettencourt-Schueller Foundation, the FondaMental Foundation, and the ANR SynDiv (IP_1), Branco Weiss fellowship of the Society in Science ETH Zürich, Marguerite-Marie Delacroix Foundation, Flanders Fund for Scientific Research (FWO project 1521313N, G.0401.12;1206013N; KUL_3), the Stavros Niarchos Foundation, The Leon Levy Foundation, an endowment provided by Phyllis Green and Randolph Cowen, and Goldman Sachs Gives on behalf of Ram Sundaram (NYU_1), DeStefano Family Foundation, Oregon Clinical and Translational Institute, and Medical Research Foundation (OHSU_1), National Children’s Research Centre Our Lady’s Children’s Hospital (TCD_1), University of Utah Multidisciplinary Research Seed Grant, NRSA Predoctoral Fellowship (F32 DC010143; USM_1), Ben B. and Iris M. Margolis Foundation (USM_1), and UCLA Autism Center for Excellence (UCLA_Long).