A large collection of real-world pediatric sleep studies

Despite being crucial to health and quality of life, sleep—especially pediatric sleep—is not yet well understood. This is exacerbated by lack of access to sufficient pediatric sleep data with clinical annotation. In order to accelerate research on pediatric sleep and its connection to health, we create the Nationwide Children’s Hospital (NCH) Sleep DataBank and publish it at Physionet and the National Sleep Research Resource (NSRR), which is a large sleep data common with physiological data, clinical data, and tools for analyses. The NCH Sleep DataBank consists of 3,984 polysomnography studies and over 5.6 million clinical observations on 3,673 unique patients between 2017 and 2019 at NCH. The novelties of this dataset include: (1) large-scale sleep dataset suitable for discovering new insights via data mining, (2) explicit focus on pediatric patients, (3) gathered in a real-world clinical setting, and (4) the accompanying rich set of clinical data. The NCH Sleep DataBank is a valuable resource for advancing automatic sleep scoring and real-time sleep disorder prediction, among many other potential scientific discoveries.

• Patient subtyping. There is increasing evidence that many sleep disorders (e.g. insomnia 16 ) are heterogeneous and have different subtypes. Identifying them can help us understand the disorder better and develop a more tailored course of treatment for different groups of patients. • Treatment (e.g. medications and procedures) efficacy analysis.

Methods
Sleep study data acquisition. The NCH Sleep DataBank contains sleep studies acquired under standard care at NCH between Dec. 16,2017 and Dec. 31, 2019 using Natus Sleepworks versions 8 and 9 17,18 . Physiological data collected during an overnight sleep study contain: • Electroencephalogram (EEG) to identify sleep stages, • Electromyelogram (EMG) of chin activity to help identify the decreased tone seen during REM sleep, • Leg EMG to measure leg movements, • Electrooculogram (EOG) to identify characteristic eye movements seen during REM sleep, • Electrocardiogram (ECG) to monitor cardiac rate and rhythm, • Nasal and oral sensors to measure airflow, • Thoracic and abdominal belts to measure chest and abdominal movements during breathing, which is helpful in demonstrating increased or decreased respiratory effort, • Pulse oximetry to measure blood oxygen saturation, • End-tidal carbon dioxide (CO 2 ) measurement of exhaled air to indirectly measure blood CO 2 to assess for hypoventilation.
Sleep studies were annotated in real time by technicians at the time of the study, and then were staged and scored by a second technician after the study was completed. Technicians annotated studies using a combination of free-form text entries and selections within Natus Sleepworks. Technicians tried to identify all events of interest, however each technician may have their own style of text annotation. Due to the variability in sleep stages in children, NCH does not use automatic scoring of sleep stages. All sleep stages were manually scored by a technician and then verified or changed by a physician board certified in sleep medicine.
Sleep studies were manually downloaded and converted to EDF + format between May 2019 and Feb. 2020 using Natus Sleepworks version 9. Any gaps in the time-series data were padded with zeros as part of the conversion. The specific acquisition equipment, setup, and montage all followed standard care protocol at NCH. While changes may have been made to some studies, the NCH protocol for PSG is in accordance with the rules and technical specifications recommended by the American Academy of Sleep Medicine 10,11 . Standard channel names are used and documented in the header of the EDF files, allowing inference of the montage. Patient cohort. The NCH Sleep DataBank consists of 3,984 sleep studies performed on 3,673 unique patients. Of them, 3,400 patients have one sleep study in the dataset, 238 have two studies, and 35 patients have more than two studies, with a maximum of 5 sleep studies for one patient. In terms of gender distribution, 2,068 patients were male, and 1,604 were female, with one unknown. Table 1 shows the distribution of the unique patients' races, where the majority of the patients were White, and about a fifth were Black or African American. In regards to ethnicity, 186 patients were Hispanic or Latino, 3,446 patients were Not Hispanic or Latino, and 41 had ethnicity of Other, Unknown, or No Information.
The majority of patients (2,412) in the dataset were less than 10 years old at the time of the sleep study, as seen in Fig. 1. Figure 2 summarizes the length of care at NCH before and after the first sleep study. The length of www.nature.com/scientificdata www.nature.com/scientificdata/ care prior to first sleep study was calculated as the time between the patient's earliest EHR entry (i.e. diagnosis, encounter, medication, measurement, procedure) and their first sleep study. If the patient's earliest EHR entry was after the first sleep study, length of care is defined as 0. The length of follow up was calculated as the time between the patient's first sleep study and their last recorded EHR entry. Patients had a median of 289 days of follow-up after their first sleep study, and 74% (2,718) had follow-up between 90 days and 2 years. Patient data linkage. Sleep study recordings and associated reports at NCH are stored in a database that is independent from the EHR, using Natus Sleepworks as a front end. It was therefore necessary to link patient information in two places. The first link was between the header information in the EDF + files and the patient data entered in Natus Sleepworks. The second link was between the patient information in Natus and the EHR.
A spreadsheet listing all sleep studies was exported from Natus Sleepworks. This listing included the date and time of each sleep study and patient information such as first and last name, date of birth and medical record number (MRN) for most sleep studies. Sleep studies were then downloaded from Natus in mini-batches, and exported to EDF + . Sleep study specific header information in the EDF + files were used to match these files to the Natus spreadsheet export. When ambiguity was present, or when MRNs were not present in Natus, we removed the EDF + file from our dataset. We then used each patient's last name, date of birth, and MRNs extracted from Natus to retrieve patient records from the EHR. When matches could not be confidently made to the EHR, the sleep studies were removed from the dataset.
Data de-identification and IRB exemption. Each unique patient was given a random identifier (STUDY_PAT_ID), and each sleep study was given a separate random identifier (SLEEP_STUDY_ID). A single patient may have multiple sleep studies in the dataset, and therefore have multiple associated SLEEP_STUDY_ IDs, but only one STUDY_PAT_ID. Sleep studies were then renamed (STUDY_PAT_ID)_(SLEEP_STUDY_ID). edf.
All EDF + headers were de-identified by replacing the first 256 bytes of the EDF + file with a standard de-identified header. As such, users are advised to ignore all header information in the EDF files (such as patien-tID, recordID, startdate, duration), but instead rely on the metadata in the accompanying .csv files to interpret the PSG results. Annotation channels were read from EDF + using Python MNE 19 and written to text. All EDF + files were converted to EDF by removing the annotation channel using Luna (https://zzz.bwh.harvard.edu/luna). Annotation text files were then de-identified by replacing any word that was not in a whitelist  www.nature.com/scientificdata www.nature.com/scientificdata/ with "XXX". This process affected 10,888 annotations, which is about 0.22% of the total number of annotations (5,046,370). The whitelist was a combination of 162 common phrases found in the annotations obtained by manual inspection, and a larger whitelist used by the de-identification program Philter 20 . The Philter whitelist contains approximately 195,000 tokens of medical terms and codes and common medical abbreviations, in addition to 20,000 most common English words, and excludes the most common Social Security and Census names. The tab delimited, de-identified annotations were then renamed to match the EDF filenames.
To protect every patient's privacy, random date shifts were applied to all data: for each patient (i.e. patients with the same STUDY_PAT_ID), one random date shift of +/− 180 days was chosen and applied to all data that are linked to the patient.
Finally, we considered the risk of re-identification through rare diagnoses. Say a malicious user of this dataset is interested in re-identifying a specific patient, and the attacker has some information about the patient such as their sex and race, as well as the fact that the patient has been diagnosed for a very rare genetic disease at NCH. If this diagnostic information is visible within the NCH Sleep DataBank, then the malicious user can likely figure out who this patient is, e.g. by searching for a female Asian child born between 2012 and 2016 with this very rare disease. Therefore, we redacted rare diagnosis codes from DIAGNOSIS.csv through the following procedure as an extra precaution.
The EHR in NCH and the diagnoses table in NCH Sleep DataBank (DIAGNOSIS.csv) contain several variables. One is DX_CODE (diagnosis code), which holds the International Classification of Diseases (ICD) code for each diagnosis. On Oct. 1, 2015, hospitals in the United States, including NCH, have switched from using ICD 9 (the 9th revision of ICD) codes to ICD 10 (the 10th revision of ICD) codes in EHR. Another relevant variable is DX_SOURCE_TYPE (diagnosis source type), which indicates whether the diagnosis was given at admission, presented as a part of the patient's previous medical history, etc. We were interested in the ones labeled "Final Dx", i.e. the final diagnoses the clinicians gave after relevant examinations and tests.
Using these variables, we defined rare diagnosis codes as ICD 10 or ICD 9 codes that were given as Final Dx to less than 10 unique patients in the entire NCH patient population (not limited to the NCH Sleep DataBank patients) during a given time period. Specifically, we queried (1) for every ICD 9 code, the number of unique NCH patients given the diagnosis as Final Dx between Jan. 1, 2000 and Sep. 30, 2015, and (2) for every ICD 10 code, the number of unique NCH patients given the diagnosis as Final Dx between Oct. 1, 2015 and Dec. 31, 2020. If a code had less than 10 unique patients in either ICD 9 or ICD 10 lists, it was deemed a rare diagnosis code for our purpose. We did not consider diagnoses before 2000 since the earliest diagnosis in NCH Sleep DataBank was from 2001.
Then, in every row of DIAGNOSIS.csv where a rare diagnosis code appeared, we changed the entries in DX_CODE, DX_NAME (diagnosis name), DX_ALT_CODE (corresponding ICD 10 codes for records before Oct 2015, and ICD 9 codes for those after Oct 2015), CLASS_OF_PROBLEM ("Stage 1", "Chronic", "Acute", "Present upon Admission"), CHRONIC_YN (Indication of chronic disease) to the phrase "redacted". This process affected a total of 6,460 rows and 834 unique patients in DIAGNOSIS.csv.
As this project concerns analysis on de-identified data, the project did not fit the definition of Human Subjects Research as defined by the United States Department of Health and Human Services and Food and Drug Administration. Therefore, this study received NCH Institutional Review Board (IRB) exemption with HIPAA waiver. The protocol that concerns the de-identification and processing of the data, which requires www.nature.com/scientificdata www.nature.com/scientificdata/ handling identified data, and the collection and publication of data and summary statistics, was approved under "STUDY00000505: Preparation of sleep study data" on September 22, 2019.
The NCH Sleep DataBank consists of two folders: Sleep_Data and Health_Data. Sleep_Data contains annotated PSG recordings, while Health_Data contains patient demographic and clinical data extracted from the EHR. Inside Sleep_Data, PSG sleep studies are provided in the EDF format, and annotations are provided in a separate tab-delimited file. Sleep studies and their matched annotations share the same file name (STUDY_PAT_ ID)_(SLEEP_STUDY_ID) but different extensions (.edf, .tsv). Clinical data in Health_Data are in .csv files, and they are linked to the files in Sleep_Data through the same STUDY_PAT_ID. Variables follow EHR conventions, and descriptions can be found in the file Sleep_Study_Data_File_Format.pdf in Health_Data.
Sleep studies. The 3,984 sleep study files (.edf) contain PSG recordings taken in clinical setting at NCH. An example plot of the signals can be seen in Fig. 3. Almost half (1,972) of the files have 26 channels, a quarter (1,012) have 29, a fifth (820) have 25, and the rest have 28, 24, 40, 27, 9, or 56 channels, in decreasing order of frequency. The most commonly appearing channel names are summarized in Table 2. The channel PATIENT EVENT was not used and can be excluded from analyses. We note again that all EDF headers were replaced with a standard de-identified version as part of the de-identification process.
The total length of recording in the NCH Sleep DataBank amounts to 40,884 hours, where the minimum length of study is 3 minutes, the maximum is 16.5 hours, and the mean is 10.3 hours. 94.85% of the files contain between 8 and 12 hours of recordings, and the patients slept for a subset of those times. Users of the dataset should take into account that the majority of the recordings (3,204) are collected with a sampling frequency of 256 Hz, but 581 studies were sampled in 400 Hz, and the rest (199) in 512 Hz.
Sleep study annotations. The 3,984 annotation files (.tsv) contain a total of 5,046,370 annotations. The minimum number of annotations contained in a sleep study is 5, while the maximum is 6,047, and the mean value is 1,267. Each annotation has the following information, where an example is given in Table 3.
• onset: The start time of the event since the beginning of the study in seconds.
• duration: The length of the event in seconds.
• description: The description of the event, which may be sleep stage label or free-form text entry by the NCH technician, or standard sleep event label by Natus Sleepworks.
35,821 unique descriptions appear in NCH Sleep DataBank. In particular, sleep stages are found in annotations with a duration of 30 seconds, where the descriptions include "Sleep stage W", "Sleep stage N1", "Sleep stage N2", "Sleep stage N3", "Sleep stage R", or "Sleep stage?". In sleep stage classification, W indicates awake, R stands for REM sleep, and N1, N2, N3 are non-REM stages 1, 2, 3, respectively. The annotation "Sleep stage?" typically occurs after "Lights On", and physiological data acquired during that time can usually be ignored, as it indicates that the study has ended. Of the total number of annotations, 79.48% were related to sleep staging: 6.88% (347,294) are "Sleep stage?", 13.19% (665,676) are "Sleep stage W", 2.54% (128,410) are "Sleep stage N1", 27.41% (1,383,765) are "Sleep stage N2", 17.35% (875,486) are "Sleep stage N3", and 12.11% (611,320) are "Sleep stage R". This is equivalent to 30,539 hours of data with sleep stage labels. The mean length of such data per study is 7.7 hours, and 96.63% (3,850) of the studies contain between 6 and 10 hours of sleep data with stage labels.
Free text annotations by the NCH technician typically describe events in the room, movements, and other patient activities, and will often have a duration of 0 seconds. Additionally, hypopneas, apneas, seizures, and other patient events may be mentioned in the free text annotations. On the other hand, standard sleep event annotations are selected in, or automatically applied by Natus Sleepworks 17,18 , and are likely to have varying durations other than 0 or 30 seconds.
While there may be some variation, the general format for sleep studies is as follows: Sleep staging begins at the annotation "Lights Off " and ends at "Lights On". Descriptive annotations will typically precede sleep stage scoring at irregular intervals prior to "Lights Off ". Sleep stages are annotated in 30 second epochs, beginning at "Lights Off "; however not all studies include this annotation.
Clinical data. The NCH Sleep DataBank includes patient demographics and longitudinal clinical data such as encounters, medication, measurements, diagnoses, and procedures. The number of observations and variables for each file are listed in Table 4. More details about the variables can be found in Sleep_Study_Data_File_Format.pdf in the same folder. Note that the age of the patient at the time of sleep study is calculated in SLEEP_STUDY.csv. Measurements include body mass index, body mass index percentile, or blood pressure. Table 5 lists 20 diagnoses that are given to the highest number of unique patients in the NCH Sleep DataBank according to DIAGNOSIS.csv. Only diagnoses indicated as Final Dx in DX_SOURCE_TYPE were considered for this analysis. Any DX_CODEs recorded in ICD 9 code were converted to the corresponding ICD 10 codes, according to the ICD 10 codes provided under the variable DX_ALT_CODE in DIAGNOSIS.csv. 17 unique ICD 9 diagnoses (across 75 rows) that did not have corresponding ICD 10 codes were disregarded from further consideration. We leveraged the hierarchical structure of ICD 10 codes to get a broad overview of the patient www.nature.com/scientificdata www.nature.com/scientificdata/ population. For example, ICD 10 code "G47.33 Obstructive sleep apnea (adult) (pediatric)" fall under the more general ICD 10 code "G47.3 Sleep apnea" which in turn is under the even more general ICD 10 code "G47 Sleep disorders. " Therefore, two patients with "G47.33 Obstructive sleep apnea (adult) (pediatric)" and "G47.61 Periodic limb movement disorder", respectively, counted as two patients diagnosed with "G47 Sleep disorders" in Table 5. Note that we started by considering all diagnoses in the EHR data, not just the diagnoses resulting from the specific sleep studies included the NCH Sleep DataBank. www.nature.com/scientificdata www.nature.com/scientificdata/ technical Validation Validation of de-identification procedure. After EDF files were de-identified, we performed several validation steps to confirm that the data matched the original EDF + export. We loaded all channels from both the de-identified EDF file and the original EDF + export and confirmed that all signal channels matched. Finally, all files included in the data set have been read by Python MNE through this validation procedure and any files with read errors were not included in the data set.     www.nature.com/scientificdata www.nature.com/scientificdata/ Validation of data maps. We identified and tested three separate points in our data pipeline: (1) mapping of sleep study from Natus Sleepworks to the de-identified EDF file, (2) mapping of clinical data from EHR to the de-identified CSV files, and (3) the linkage between the sleep study and the clinical data.

Channel name
The first was the mapping between the de-identified EDF file and the original sleep data file accessible via Natus Sleepworks. We first chose four random sleep studies (about 0.1% of the dataset), and a random 30-second segment from each study. Then we confirmed that the sleep data viewed on Natus Sleepworks (Fig. 3 top) matched data visualized from the corresponding EDF file in the published dataset (Fig. 3 bottom).
The second mapping was between the de-identified clinical data and the EHR. We extracted from the dataset all clinical data associated with the four random patients chosen in the first verification step, and confirmed that they are identical to the medical records viewed from the physician interface of the EPIC electronic medical record.
The last mapping we verified was SLEEP_STUDY_ID, the random identifier linking the sleep studies to the patient data. We verified this by matching the sleep study, which is represented by SLEEP_STUDY_ID, with its corresponding encounter in the patient data, which is represented by STUDY_ENC_ID. If an encounter had procedure codes and departmental codes associated with sleep study, had the same randomly assigned STUDY_ PAT_ID as the sleep study, and the same starting date and time (within a window of +/− one hour) as the sleep study start time obtained from Natus Sleepworks, we considered it a match. We were able to match 3,964 sleep studies to encounter codes in the patient data using this method, therefore providing validation of a mapping between the sleep studies and patient data and consistency of date shifting. This information is provided in the file SLEEP_ENC_ID.csv.

Sleep stage classification for PSG data validation.
We developed a baseline sleep stage classifier and included it in the codebase to demonstrate the technical quality as well as a potential utility of the dataset, especially the PSG data. This simple algorithm predicts the sleep stages (W, N1, N2, N3, R) based on 30 seconds of 7 EEG channels (F4-M1, O2-M1, C4-M1, O1-M2, F3-M2, C3-M2, CZ-O1) after they are down sampled to 128 Hz.
Wavelet transform is a powerful method that can flexibly represent the time-frequency content of a signal. As such, it is particularly useful in analyzing non-stationary signals, and have previously been used for EEG-based sleep stage classification [21][22][23][24] . After applying multi-resolution Daubechies wavelet transform 25 to each EEG channel, we computed summary statistics such as min, max, mean, and standard deviation of the coefficients, resulting in 84 features. A random forest classifier with 100 decision trees was then trained on these features using 67% of the dataset, and tested on the rest. Table 6 reports the 3-fold stratified cross validation results on 3,928 sleep studies that had the 7 EEG channels, in addition to the results on some subgroups (0 to 1 year old, 1 to 2 years old, and 18 + patients). Fitting the classifier with default parameters from Scikit-learn 26 took 1 hour on Intel Xeon Gold 3.60 GHz CPU in parallel; subgroups took less than 2 minutes each. This quick and straightforward algorithm, without any denoising or parameter tuning, achieves a classification accuracy of over 80% on the 222 adult sleep studies, suggesting high www.nature.com/scientificdata www.nature.com/scientificdata/ quality of the PSG recordings. Moreover, the difference in classification results between age groups supports the importance of having a dataset dedicated to pediatric sleep.

Prader-Willi syndrome (PWS) patient analysis for eHR data validation. The availability of EHR
allows the study of clinically meaningful patient subpopulations in the NCH Sleep DataBank. As a use case, we examine the sleep patterns of PWS patients within this dataset. To provide context, PWS is a rare genetic disorder that is estimated to affect 1 out of 10,000 to 30,000 people, and many researchers and clinicians are interested in sleep abnormalities and sleep-disordered breathing of PWS patients [27][28][29][30][31] . We construct two PSG cohorts, where Cohort 1 includes the PSGs of PWS patients, and Cohort 2 consists of PSGs of obese but non-PWS patients. To control for the effect of OSA, both cohorts only consider PSGs during which patients were diagnosed OSA.
To construct the PSG cohorts, we first searched for all STUDY_ENC_IDs in DIAGNOSIS.csv during which a patient was given a final diagnosis of OSA. Then, we only kept the encounter IDs that were also present in SLEEP_ENC_ID.csv, as we have matched them with SLEEP_STUDY_IDs in an earlier validation step. This process identified 860 PSGs (763 unique patients) with OSA diagnoses. Among these, 16 PSGs (12 unique patients) were designated Cohort 1, since they were associated with STUDY_PAT_IDs that had a final diagnosis of PWS in the EHR. For reference, the NCH Sleep DataBank has a total of 34 unique patients who had final diagnosis of PWS in the EHR. On the other hand, 370 PSGs (311 unique patients) were associated with STUDY_PAT_IDs with obesity diagnoses but not PWS, and selected Cohort 2.
For every PSG in Cohort 1 and Cohort 2, we tallied the number of each sleep stage (W, N1, N2, N3, R) annotation, and extracted the following sleep characteristics: total length of sleep (sleep time) by counting 30 seconds of sleep for each sleep stage annotation, and distribution of sleep stages, e.g., W constitutes 20% of the sleep time. Table 7 describes summary statistics of the two cohorts' sleep characteristics. In summary, the ease-of-navigation of the EHR data makes it possible to conduct disease-specific data mining using NCH Sleep DataBank, e.g. extraction of sleep characteristics such as apnea-hypopnea index (AHI), and refined statistical analysis that accounts for potential confounding variables such as BMI and age.

Usage Notes
The NCH Sleep DataBank can potentially be used to study many problems related to pediatric sleep, including but not limited to: • Automatic sleep scoring (sleep stage classification): Sleep scoring divides sleep into two stages, rapid eye movement (REM), and non-REM, then further divides the latter into shallow sleep (stages N1 and N2) and deep sleep (stage N3) [9][10][11] , in addition to wake (Stage W). In typical pediatric clinical settings, this is a time-consuming and tedious process done by a technician. Many computational algorithms have shown promise for automatic sleep scoring in adults 12 , which encourage exploration on automatic sleep scoring for infants and children. Algorithms that combine PSG modalities beyond EEG or ECG 13 especially warrant more investigation. • Automatic sleep disorder (e.g. obstructive apnea) detection: Large sets of PSG signals published with expert annotations can be leveraged to develop computational algorithms in sleep disorder detection, unleashing the potential of eventual real-time systems that read these signals and detect sleep disorders at their onsets 14,15 . OSA detection is particularly important, as OSA is associated with various cardiovascular, respiratory, and neurocognitive deficits and morbidity among infants and children 2,3 . • Diagnosis prediction: Statistical models that predict or measure the risk of diagnoses using other variables (e.g. other diagnoses, demographic, features from PSG, encounters, measurement values) can be constructed and validated to create hypotheses for further experiment.  www.nature.com/scientificdata www.nature.com/scientificdata/ • Identifying patient subgroups: Given the demographics and medical history, patients can be divided into clinically meaningful subgroups before further analysis, as demonstrated in this paper for PWS. Additionally, data-driven approaches may be developed to reveal clusters within the patient population, which could affect their symptoms or best courses of treatment, e.g. as suggested for insomnia 16 . • Treatment efficacy analysis: Retrospective studies using the accompanying longitudinal clinical data (e.g. medications and procedures) can be used to analyze efficacy of different treatments options.

Code availability
The code that was used to analyze patient data, read EDF files, run baseline sleep stage classifier, and generate figures and tables in this paper is published at https://github.com/liboyue/sleep_study.