NCH Sleep DataBank: A Large Collection of Real-world Pediatric Sleep Studies
Harlin Lee, Boyue Li, Shelly DeForte, Mark Splaingard, Yungui Huang, Yuejie Chi, Simon Lin
NNCH Sleep DataBank: A Large Collection of Real-worldPediatric Sleep Studies
Harlin Lee, Boyue Li, Shelly DeForte, Mark Splaingard, Yungui Huang, Yuejie Chi, Simon Lin March 1, 2021
Abstract
Despite being crucial to health and quality of life, sleep—especially pediatric sleep—is notyet well understood. This is exacerbated by lack of access to sufficient pediatric sleep datawith clinical annotation. In order to accelerate research on pediatric sleep and its connectionto health, we create the Nationwide Children’s Hospital (NCH) Sleep DataBank and publishit at the National Sleep Research Resource (NSRR), which is a large sleep data common withphysiological data, clinical data, and tools for analyses. The NCH Sleep DataBank consistsof 3,984 polysomnography studies and over 5.6 million clinical observations on 3,673 uniquepatients between 2017 and 2019 at NCH. The novelties of this dataset include: 1) large-scalesleep dataset suitable for discovering new insights via data mining, 2) explicit focus on pediatricpatients, 3) gathered in a real-world clinical setting, and 4) the accompanying rich set of clinicaldata. The NCH Sleep DataBank is a valuable resource for advancing automatic sleep scoringand real-time sleep disorder prediction, among many other potential scientific discoveries.
Background & Summary
Sleep is an active process associated with physiological changes that involve multiple organ systems,and is vital for the maturation and daily functioning of infants, children and adolescents. Conse-quently, disruption of the complex interplay between sleep and other physiological processes canlead to significant medical consequences [1]. Sleep disorders, like obstructive sleep apnea (OSA)[2, 3], can lead to derangements in function that contribute to significant morbidity and even mor-tality. Sleep can also be disrupted by many organ-specific diseases like asthma, sickle cell disease,renal failure, or depression that alter the course of a particular medical condition and result in apoorer quality of life.Sleep disturbances in children are classified as behavioral insomnias of children, sleep-relatedbreathing disorders, parasomnias, sleep-related movement disorders, circadian rhythm disordersor hypersomnias [4]. These sleep disorders may be associated with excessive daytime sleepiness(rare in young children), hyperactivity–impaired attention, poor school performance from impairedconcentration and vigilance, and behavior problems including irritability.
1. Department of Electrical and Computer Engineering, Carnegie Mellon University, 5000 Forbes Avenue, Pitts-burgh PA 15213, USA.2. Nationwide Children’s Hospital, 700 Children’s Drive, Columbus OH 43205, USA.*Corresponding authors: Yuejie Chi ([email protected]), Simon Lin ([email protected]) a r X i v : . [ ee ss . SP ] F e b leep problems suffer from under-reporting by parents and under-diagnosis by primary carephysicians, but are conservatively estimated to occur in approximately 25% of healthy childrenyounger than 5 years and in up to 80% of children with special health care needs. Estimates ofprevalence of sleep disorders in children vary more widely for behavioral sleep problems like insomniathan organic sleep problems like OSA.While some childhood sleep disorders need only medical history to be properly diagnosed andmanaged, some infants and children require an analysis of the child actually sleeping, called anovernight sleep study or polysomnography (PSG), to accurately diagnose their sleep-related con-dition. During an overnight PSG, the sleeping child’s physiological signals are recorded under thedirect supervision of specially trained sleep technicians, who attach monitoring sensors to specialcomputer software and adjust them during the night. The technician also provides observationsabout the child’s sleep that are invaluable in making an accurate diagnosis. Video monitoring is alsoincorporated into the PSG, allowing review of movements necessary to diagnose nocturnal seizures,which occur in about 20% of children with epilepsy.The physiological data collected during a PSG provide a picture of clinically useful informationabout different sleep stages, sleep disruption, respiratory status during different sleep stages, legmovements, and changes in cardiac rate and rhythm during sleep. For instance, episodes of OSAmay consist of decreased airflow in spite of normal respiratory effort in thoracic and abdominalbelts, changes in electroencephalogram (EEG) pattern called arousals, cardiac deceleration, andoxygen desaturation. These findings may be mild during non-random eye movement (non-REM)sleep but profound during REM sleep.Computational algorithms that learn from large amounts of data have seen remarkable suc-cess in healthcare, particularly with the proliferation of electronic health records (EHR) and im-proved sensors. Regrettably, without a curated and comprehensive dataset of substantial sizeand accessibility, pediatric sleep has not been able to fully benefit from such opportunities yet.As a first step, this data descriptor introduces the Nationwide Children’s Hospital (NCH) SleepDataBank, which has 3,984 pediatric sleep studies on 3,673 unique patients conducted at NCHbetween 2017 and 2019, along with the patients’ longitudinal clinical data. They were gath-ered in the real-world clinical setting at NCH as opposed to, for example, a controlled clinicaltrial. The published PSG contain the patient’s physiological signals as well as the technician’sassessment of the sleep stages and descriptions of additional irregularities [5]. The accompany-ing 5.6 million records of clinical data are extracted from the EHR, and are separated into en-counters, medications, measurements (e.g. body mass index), diagnoses, and procedures. Thedataset is deposited in the National Sleep Research Resource (NSRR) [6] and can be requestedfrom https://sleepdata.org/datasets/nchsdb . Accompanying code in Python to assist usersin interacting with the dataset is published at https://github.com/liboyue/sleep_study .We expect the NCH Sleep DataBank to be used to study many problems related to pediatricsleep, including but not limited to:• Automatic sleep stage classification, especially algorithms that combine modalities beyondEEG or ECG [7, 8, 9, 10, 11],• Automatic real-time sleep disorder (e.g. OSA) detection [12, 13],• Diagnosis prediction,• Identifying patient subgroups that could affect their symptoms or best courses of treatment,e.g. as suggested for insomnia [14], 2 Treatment (e.g. medications and procedures) efficacy analysis. Methods
Sleep study data acquisition
The NCH Sleep DataBank contains sleep studies acquired under standard care at NCH betweenDec. 16, 2017 and Dec. 31, 2019 using Natus Sleepworks versions 8 and 9 [15, 16]. Physiologicaldata collected during an overnight sleep study contain:• Electroencephalogram (EEG) to identify sleep stages,• Electromyelogram (EMG) of chin activity to help identify the decreased tone seen duringREM sleep,• Leg EMG to measure leg movements,• Electrooculogram (EOG) to identify characteristic eye movements seen during REM sleep,• Electrocardiogram (ECG) to monitor cardiac rate and rhythm,• Nasal and oral sensors to measure airflow,• Thoracic and abdominal belts to measure chest and abdominal movements during breathing,which is helpful in demonstrating increased or decreased respiratory effort,• Pulse oximetry to measure blood oxygen saturation,• End-tidal carbon dioxide (CO2) measurement of exhaled air to indirectly measure blood CO2to assess for hypoventilation.Sleep studies were annotated in real time by technicians at the time of the study, and then werestaged and scored by a second technician after the study was completed. Technicians annotatedstudies using a combination of free-form text entries and selections within Natus Sleepworks. Tech-nicians tried to identify all events of interest, however each technician may have their own style oftext annotation. Due to the variability in sleep stages in children, NCH does not use automaticscoring of sleep stages. All sleep stages were manually scored by a technician and then verified orchanged by a physician board certified in sleep medicine.Sleep studies were manually downloaded and converted to EDF+ format between May 2019and Feb. 2020 using Natus Sleepworks version 9. Any gaps in the time-series data were paddedwith zeros as part of the conversion. The specific acquisition equipment, setup, and montage arevariable between studies. Standard channel names are used and documented in the header of theEDF files, allowing inference of the montage.As this project concerns analysis on de-identified data, the project did not fit the definitionof Human Subjects Research as defined by the United States Department of Health and HumanServices and Food and Drug Administration. Therefore, this study received NCH InstitutionalReview Board (IRB) exemption, and the protocol that concerns the de-identification and processingof the data, which requires handling identified data, and the collection and publication of summarystatistics, was approved under “STUDY00000505: Preparation of sleep study data.”3 atient cohort
The NCH Sleep DataBank consists of 3,984 sleep studies performed on 3,673 unique patients. Ofthem, 3,400 patients have one sleep study in the dataset, 238 have two studies, and 35 patientshave more than two studies, with a maximum of 5 sleep studies for one patient. In terms of genderdistribution, 2,068 patients were male, and 1,604 were female, with one unknown. Table 1 showsthe distribution of the unique patients’ races, where the majority of the patients were White, andabout a fifth were Black or African American. In regards to ethnicity, 186 patients were Hispanicor Latino, 3,446 patients were Not Hispanic or Latino, and 41 had ethnicity of Other, Unknown, orNo Information. Race description Count PercentageWhite 2,433 66.24%Black or African American 738 20.09%Multiple races 277 7.54%Asian 93 2.53%Others and unknown 132 3.59%Total 3,673 100%Table 1: The distribution of 3,673 unique patients’ races.The youngest patient at the time of the sleep study was 6 days old, while the oldest patient atthe time of the sleep study was 58 years old. The majority of patients (2,412) in the dataset wereless than 10 years old at the time of the sleep study, as seen in Figure 1. Figure 2 summarizesthe length of care at NCH before and after the first sleep study. The length of care prior to firstsleep study was calculated as the time between the patient’s earliest EHR entry (i.e. diagnosis,encounter, medication, measurement, procedure) and their first sleep study. If the patient’s earliestEHR entry was after the first sleep study, length of care is defined as 0. The length of follow up wascalculated as the time between the patient’s first sleep study and their last recorded EHR entry.Patients had a median of 289 days of follow-up after their first sleep study, and 74% (2,718) hadfollow-up between 90 days and 2 years.
Patient data linkage
Sleep study recordings and associated reports at NCH are stored in a database that is independentfrom the EHR, using Natus Sleepworks as a front end. It was therefore necessary to link patientinformation in two places. The first link was between the header information in the EDF+ files andthe patient data entered in Natus Sleepworks. The second link was between the patient informationin Natus and the EHR.A spreadsheet listing all sleep studies was exported from Natus Sleepworks. This listing includedthe date and time of each sleep study and patient information such as first and last name, date ofbirth and medical record number (MRN) for most sleep studies. Sleep studies were then downloadedfrom Natus in mini-batches, and exported to EDF+. Sleep study specific header information in theEDF+ files were used to match these files to the Natus spreadsheet export. When ambiguity waspresent, or when MRNs were not present in Natus, we removed the EDF+ file from our dataset.We then used each patient’s last name, date of birth, and MRNs extracted from Natus to retrieve4
Age at time of sleep study (years) N u m b e r o f s l ee p s t ud i e s Figure 1: Age at the time of sleep study, where patients that are more than years old are notshown.patient records from the EHR. When matches could not be confidently made to the EHR, the sleepstudies were removed from the dataset. Data de-identification
Each unique patient was given a random identifier (STUDY_PAT_ID), and each sleep study wasgiven a separate random identifier (SLEEP_STUDY_ID). A single patient may have multiple sleepstudies in the dataset, and therefore have multiple associated SLEEP_STUDY_IDs. Sleep studieswere then renamed (STUDY_PAT_ID)_(SLEEP_STUDY_ID).edf.All EDF+ headers were de-identified by replacing the first 256 bytes of the EDF+ file with astandard de-identified header. Annotation channels were read from EDF+ using Python MNE [17]and written to text. All EDF+ files were converted to EDF by removing the annotation channelusing Luna [18]. Annotation text files were then de-identified by replacing any word that was notin a whitelist with ‘XXX’. The whitelist was a combination of 162 common phrases found in theannotations obtained by manual inspection, and a larger whitelist used by the de-identificationprogram Philter [19]. The Philter whitelist contains approximately 195,000 tokens of medical termsand codes and common medical abbreviations, in addition to 20,000 most common English words,and excludes the most common Social Security and Census names. The tab delimited, de-identifiedannotations were then renamed to match the EDF filenames.The STUDY_PAT_ID and a random date shift of +/- 180 days were used to adjust all identifiedpatient data pulled from the EHR, as well as the sleep study dates recorded in Natus.5 − − − − Length of care before first sleep study (YEARS) N u m b e r o f p a t i e n t s Follow up after first sleep study (DAYS) N u m b e r o f p a t i e n t s Figure 2: Length of care at NCH before and after first sleep study, where each patient has twoentries: one negative for length of care prior to first sleep study (in years), and one positive forfollow up after first sleep study (in days). One entry above 1200 days and 6 entries below -25 yearsare not shown.
Data Records
The NCH Sleep DataBank consists of two folders: Sleep_Data and Health_Data. Sleep_Datacontains annotated PSG recordings, while Health_Data contains patient demographic and clinicaldata extracted from the EHR. Inside Sleep_Data, PSG sleep studies are provided in the EDFformat, and annotations are provided in a separate tab-delimited file. Sleep studies and theirmatched annotations share the same file name (STUDY_PAT_ID)_(SLEEP_STUDY_ID) butdifferent extensions (.edf, .tsv). Clinical data in Health_Data are in .csv files, and they are linkedto the files in Sleep_Data through the same STUDY_PAT_ID. Variables follow EHR conventions,and descriptions can be found in the file Sleep_Study_Data_File_Format.pdf in Health_Data.
Sleep studies
The 3,984 sleep study files (.edf) contain PSG recordings taken in clinical setting at NCH. Anexample plot of the signals can be seen in Figure 3. Almost half (1,972) of the files have 26channels, a quarter (1,012) have 29, a fifth (820) have 25, and the rest have 28, 24, 40, 27, 9, or56 channels, in decreasing order of frequency. The most commonly appearing channel names aresummarized in Table 2. The channel PATIENT EVENT was not used and can be excluded fromanalyses.The total length of recording in the NCH Sleep DataBank amounts to 40,884 hours, where theminimum length of study is 3 minutes, the maximum is 16.5 hours, and the mean is 10.3 hours.64.85% of the files contain between 8 and 12 hours of recordings, and the patients slept for a subsetof those times. Users of the dataset should take into account that the majority of the recordings(3,204) are collected with a sampling frequency of 256Hz, but 581 studies were sampled in 400Hz,and the rest (199) in 512Hz.
Sleep study annotations
The 3,984 annotation files (.tsv) contain a total of 5,046,370 annotations, among which there are35,821 unique descriptions. The minimum number of annotations contained in a sleep study is5, while the maximum is 6,047, and the mean value is 1,267. Each annotation has the followinginformation, where an example is given in Table 3.• onset: The start time of the event since the beginning of the study in seconds.• duration: The length of the event in seconds.• description: The description of the event, which may be sleep stage label or free-form textentry by the NCH technician, or standard sleep event label by Natus Sleepworks.In particular, sleep stages are found in annotations with a duration of 30 seconds, where thedescriptions include ‘Sleep stage W’, ‘Sleep stage N1’, ‘Sleep stage N2’, ‘Sleep stage N3’, ‘Sleepstage R’, or ‘Sleep stage ?’. The annotation ‘Sleep stage ?’ typically occurs after ‘Lights On’,and physiological data acquired during that time can usually be ignored, as it indicates that thestudy has ended. Of the total number of annotations, 79.48% were related to sleep staging: 6.88%(347,294) are ‘Sleep stage ?’, 13.19% (665,676) are ‘Sleep stage W’, 2.54% (128,410) are ‘Sleep stageN1’, 27.41% (1,383,765) are ‘Sleep stage N2’, 17.35% (875,486) are ‘Sleep stage N3’, and 12.11%(611,320) are ‘Sleep stage R’. This is equivalent to 30,539 hours of data with sleep stage labels. Themean length of such data per study is 7.7 hours, and 96.63% (3,850) of the studies contain between6 and 10 hours of sleep data with stage labels.Besides sleep stage labels, the most common events include: Oxygen Desaturation, OximeterEvent, EEG Arousal, Obstructive Hypopnea, Limb Movement, Gain/Filter Change, Move, BodyPosition: (Left, Right, Supine, Prone, Upright), Obstructive Apnea, Hypopnea, Central Apnea,and Mixed Apnea.Free text annotations by the NCH technician typically describe events in the room, movements,and other patient activities, and will often have a duration of 0 seconds. Additionally, hypopneas,apneas, seizures, and other patient events may be mentioned in the free text annotations. On theother hand, standard sleep event annotations are selected in, or automatically applied by NatusSleepworks [15, 16], and are likely to have varying durations other than 0 or 30 seconds.While there may be some variation, the general format for sleep studies is as follows: Sleepstaging begins at the annotation ‘Lights Off’ and ends at ‘Lights On’. Descriptive annotationswill typically precede sleep stage scoring at irregular intervals prior to ‘Lights Off’. Sleep stagesare annotated in 30 second epochs, beginning at ‘Lights Off’; however not all studies include thisannotation.
Clinical data
The NCH Sleep DataBank includes patient demographics and longitudinal clinical data such asencounters, medication, measurements, diagnoses, and procedures. The number of observations7hannel name EDF file count PercentageEEG C3-M2 3,971 99.67%EEG O1-M2 3,971 99.67%EEG O2-M1 3,971 99.67%EEG CZ-O1 3,971 99.67%RATE 3,970 99.65%ETCO2 3,970 99.65%CAPNO 3,970 99.65%RESP RATE 3,970 99.65%SPO2 (2,819) or OSAT (1,152) 3,970 99.65%EEG F3-M2 3,969 99.62%RESP THORACIC (2,821) or RESP CHEST (1,148) 3,969 99.62%RESP ABDOMINAL (2,821) or RESP ABDOMEN (1,148) 3,969 99.62%SNORE 3,968 99.60%EEG C4-M1 3,962 99.45%EEG F4-M1 3,960 99.40%C-FLOW 3,943 98.97%EOG LOC-M2 3,933 98.72%EOG ROC-M1 3,931 98.67%EMG CHIN1-CHIN2 3,782 94.93%PRESSURE 2,824 70.88%EMG LLEG-RLEG 2,820 70.78%ECG EKG2-EKG 2,820 70.78%RESP AIRFLOW 2,820 70.78%TIDAL VOL 2,818 70.73%RESP PTAF 2,817 70.71%PATIENT EVENT 2,722 68.32%TCCO2 1,417 35.57%SNORE_DR 1,148 28.82%XFLOW 1,148 28.82%EMG LLEG+-LLEG- 1,146 28.77%EMG RLEG+-RLEG- 1,146 28.77%ECG LA-RA 1,146 28.77%FLOW_DR 1,146 28.77%RESP FLOW 1,146 28.77%C-PRESSURE 1,146 28.77%EEG CHIN1-CHIN2 136 3.41%Table 2: List of 33 most common channels and their frequencies in 3,984 EDF files. Other 101channels appear in less than 1% of the files. 8nset duration description15985.234375 0.0 Chewing motion15990.93359375 30.0 Sleep stage W16002.09375 0.0 Movement16002.34375 1.21875 Limb MovementTable 3: Example annotations from a .tsv file. ‘Chewing motion’ and ‘Movement’ are free textentries by the NCH technicion, while ‘Limb Movement’ is a standard sleep event labeled by NatusSleepworks.and variables for each file are listed in Table 4. More details about the variables can be found inSleep_Study_Data_File_Format.pdf in the same folder. Note that the age of the patient at thetime of sleep study is calculated in SLEEP_STUDY.csv. Measurements include body mass index,body mass index percentile, or blood pressure.Table 5 lists 20 diagnoses that are given to the highest number of unique patients in theNCH Sleep DataBank according to DIAGNOSIS.csv. Only final diagnoses as indicated by theDX_SOURCE_TYPE and DX_ENC_TYPE variables were considered for this analysis. AnyDX_CODEs recorded in ICD 9 code were converted to the corresponding ICD 10 codes, accordingto the ICD 10 codes already provided under the variable DX_ALT_CODE in DIAGNOSIS.csv.17 unique ICD 9 diagnoses (across 75 rows) that did not have corresponding ICD 10 codes weredisregarded from further consideration. In order to get a broad overview of the patient population,the ICD codes were abstracted to a more general level before their frequencies was counted, e.g.diagnoses “G47.33 Obstructive sleep apnea (adult) (pediatric)” and “G47.61 Periodic limb move-ment disorder” both counted under “G47 Sleep disorders.” Note that we started by considering alldiagnoses in the EHR data, not just the diagnoses resulting from the specific sleep studies includedthe NCH Sleep DataBank.
Technical Validation
Validation of de-identification procedure
After EDF files were de-identified, we performed several validation steps to confirm that the datamatched the original EDF+ export. We loaded all channels from both the de-identified EDF fileand the original EDF+ export and confirmed that all signal channels matched. We then confirmedthat no more than 15 percent of the annotations had been changed between the files due to thede-identification step. Finally, all files included in the data set have been read by Python MNEthrough this validation procedure and any files with read errors were not included in the data set.
Validation of data maps
We identified and tested three separate points in our data pipeline: 1) mapping of sleep studyfrom Natus Sleepworks to the de-identified EDF file, 2) mapping of clinical data from EHR to thede-identified CSV files, and 3) the linkage between the sleep study and the clinical data.The first was the mapping between the de-identified EDF file and the original sleep data fileaccessible via Natus Sleepworks. We first chose a random sleep study, and then a 30-second segment9ile name Variable names RowsDEMOGRAPHIC.csv study pat ID, birth date, pcori gender cd,pcori race cd, pcori hispanic cd, gender de-scr, race descr, ethnicity descr, language de-scr, peds gest age num weeks, peds gest agenum days 3,673SLEEP_STUDY.csv study pat ID, sleep study ID, sleep study startdatetime, sleep study duration datetime, ageat sleep study days 3,984SLEEP_ENC_ID.csv study pat ID, sleep study ID, study enc ID 3,964ENCOUNTER.csv study enc ID, study pat ID, encounter date,visit start datetime, visit end datetime, adtarrival datetime, ed departure datetime, en-counter type, visit type cd, visit type de-scr, ICU visit Y/N, prov ID, prov type, deptID, dept specialty, admit source, hosp admitsource, discharge disposition, discharge desti-nation, drg code, drg name, visit reason 495,138MEDICATION.csv study med ID, study enc ID, study pat ID,med start datetime, med end datetime, medorder datetime, med taken datetime, medsource type, quantity, days supply, frequency,effective drug dose, eff drug dose source value,drug dose unit, refills, RxNorm code, RxNormterm type, medication descr, generic drug de-scr, drug order status, drug action, route,route source value, prescribing prov ID, pharmclass, pharm subclass, thera class, thera sub-class 3,035,986MEASUREMENT.csv study meas ID, study pat ID, study enc ID,meas recorded datetime, meas type, measvalue number, meas value text, meas source,study prov ID 332,569DIAGNOSIS.csv study dx ID, study enc ID, study pat ID, dxstart datetime, dx end datetime, dx sourcetype, dx enc type, dx code type, dx code, dxname, dx alt code, class of problem, chronicY/N, prov ID 1,513,853PROCEDURE.csv study proc ID, study pat ID, study enc ID,procedure datetime, study prov ID, proc IDNCH, proc code, proc code type, proc descr 283,599PROCEDURE_SURG_HX.csv study surghx ID, study pat ID, proc noteddate, proc start time, proc end time, proccode, cpt code, proc descr 10,190Table 4: The variable names and number of observations for each patient data file in Health_Data.More details about the variables can be found in Sleep_Study_Data_File_Format.pdf in the samefolder. 10iagnosis ICD 10 code Patients, N Sleep disorders G47 3,379Sleep apnea G47.3 2,558Sleep disorder, unspecified G47.9 1,163Other sleep disorders G47.8 914Circadian rhythm sleep disorders G47.2 566Insomnia G47.0 388Hypersomnia G47.1 257Sleep related movement disorders G47.6 180Parasomnia G47.5 165Narcolepsy and cataplexy G47.4 47Abnormalities of breathing R06 2,776Encounter for immunization Z23 1,720Chronic diseases of tonsils and adenoids J35 1,686Encounter for general examination without com-plaint, suspected or reported diagnosis Z00 1,587Acute upper respiratory infections of multiple andunspecified sites J06 1,537Body mass index (BMI) Z68 1,417Suppurative and unspecified otitis media H66 1,378Symptoms and signs concerning food and fluid intake R63 1,369Acute pharyngitis J02 1,260Other symptoms and signs involving the circulatoryand respiratory system R09 1,256Other functional intestinal disorders K59 1,185Cough R05 1,176Lack of expected normal physiological developmentin childhood and adults R62 1,097Encounter for follow-up examination after completedtreatment for conditions other than malignant neo-plasm Z09 1,068Nausea and vomiting R11 1,051Fever of other and unknown origin R50 1,043Specific developmental disorders of speech and lan-guage F80 1,002Asthma J45 991Gastro-esophageal reflux disease K21 982Table 5: 20 diagnoses that are given to the highest number of unique patients in the NCH SleepDataBank according to DIAGNOSIS.csv. Note that the diagnoses were abstracted to a higher levelbefore being counted. For example, patients with diagnosis “G47.33 Obstructive sleep apnea (adult)(pediatric)” were counted under G47 and G47.3.11rom that study. Figure 3 shows that the sleep data viewed on Natus Sleepworks (top) and visualizedfrom the corresponding EDF file in the published dataset (bottom) match.The second mapping was between the de-identified clinical data and the EHR. We extracted fromthe dataset all clinical data associated with the random patient chosen in the first verification step,and confirmed that they are identical to the medical records viewed from the physician interface ofthe EPIC electronic medical record.The last mapping we verified was SLEEP_STUDY_ID, the random identifier linking the sleepstudies to the patient data. We verified this by matching the sleep study, which is represented bySLEEP_STUDY_ID, with its corresponding encounter in the patient data, which is represented bySTUDY_ENC_ID. If an encounter had procedure codes and departmental codes associated withsleep study, had the same randomly assigned STUDY_PAT_ID as the sleep study, and the samestarting date and time (within a window of +/- one hour) as the sleep study start time obtainedfrom Natus Sleepworks, we considered it a match. We were able to match 3,964 sleep studies toencounter codes in the patient data using this method, therefore providing validation of a mappingbetween the sleep studies and patient data and consistency of date shifting. This information isprovided in the file SLEEP_ENC_ID.csv.
Sleep stage classification for PSG data validation
We developed a baseline sleep stage classifier and included it in the codebase to demonstrate thetechnical quality as well as a potential utility of the dataset, especially the PSG data. This simplealgorithm predicts the sleep stages (W, N1, N2, N3, R) based on 30 seconds of 7 EEG channels(F4-M1, O2-M1, C4-M1, O1-M2, F3-M2, C3-M2, CZ-O1) after they are down sampled to 128Hz.Wavelet transform is a powerful method that can flexibly represent the time-frequency contentof a signal. As such, it is particularly useful in analyzing non-stationary signals, and have previouslybeen used for EEG-based sleep stage classification [20, 21, 22, 23]. After applying multi-resolutionDaubechies wavelet transform [24] to each EEG channel, we computed summary statistics such asmin, max, mean, and standard deviation of the coefficients, resulting in 84 features. A randomforest classifier with 100 decision trees was then trained on these features using 67% of the dataset,and tested on the rest.Table 6 reports the 3-fold stratified cross validation results on 3,928 sleep studies that had the7 EEG channels, in addition to the results on some subgroups (0 to 1 year old, 1 to 2 years old,and 18+ patients). Fitting the classifier with default parameters from Scikit-learn [25] took 1 houron Intel Xeon Gold 3.60GHz CPU in parallel; subgroups took less than 2 minutes each. This quickand straightforward algorithm, without any denoising or parameter tuning, achieves a classificationaccuracy of over 80% on the 222 adult sleep studies, suggesting high quality of the PSG recordings.Moreover, the difference in classification results between age groups supports the importance ofhaving a dataset dedicated to pediatric sleep.
Prader-Willi syndrome (PWS) patient analysis for EHR data validation
The availability of EHR allows the study of clinically meaningful patient subpopulations in theNCH Sleep DataBank. As a use case, we examine the sleep patterns of PWS patients within thisdataset. To provide context, PWS is a rare genetic disorder that is estimated to affect 1 out of10,000 to 30,000 people, and many researchers and clinicians are interested in sleep abnormalitiesand sleep-disordered breathing of PWS patients [26, 27, 28, 29, 30]. We construct two PSG cohorts,12igure 3: Visual verification that a randomly chosen 30-second segment of sleep data on NatusSleepworks (top) matches the sleep data in the corresponding EDF file (bottom), especially at theregion of interest marked by red box. Natus Sleepworks may denoise or auto-scale some signals forthe viewer. 13utomated score sleep stageW N1 N2 N3 R M a nu a l s c o r e s l ee p s t ag e , N W (661,645)
0. 34.0 1.5 1.4N1 (127,602) 23.9 (a) All age groups. 3,928 sleep studies and 3,644,305 samples. Overallaccuracy is 64.4%.
Automated score sleep stageW N1 N2 N3 R M a nu a l s c o r e s l ee p s t ag e , N W (52,979) (b) 18 years and older. 222 sleep studies and 196,135 samples. Overallaccuracy is 81.1%.
Automated score sleep stageW N1 N2 N3 R M a nu a l s c o r e s l ee p s t ag e , N W (63,041)
0. 2.4 2.8 11.4N1 (4,579) 28.7 (c) 0-1 year olds. 242 sleep studies and 230,824 samples. Overallaccuracy is 76.6%.
Table 6: Sleep stage classification results of our baseline algorithm applied to different age groups.One sample is a 30-second epoch of sleep. Cell (row i , column j ) of the normalized confusion matrixindicates the percentage (%) of samples in stage i (manually scored by NCH technician) that werepredicted to be in stage j (by our automated algorithm). Each row adds to 100%. Bolded diagonalentries are the percentages of samples in each stage that were correctly classified. Overall accuracyis the total number of correctly classified samples divided by the total number of samples in %.All numbers reported are averaged over 3-fold stratified cross validation trials and rounded to onedecimal point. Standard deviation was <1% for all entries except one and not shown here.14ohort 1 Cohort 2PSG, N
16 370Unique patients, N
12 311Age, mean ± s.d. (years) 10.5 ± ± ± s.d. (hours) 8.0 ± ± ± s.d. (%) 14.4 ± ± ± s.d. (%) 4.1 ± ± ± s.d. (%) 45.2 ± ± ± s.d. (%) 20.5 ± ± ± s.d. (%) 15.8 ± ± ± s.d. (%) 49.3 ± ± ± s.d. (%) 69.8 ± ± sage Notes The NCH Sleep DataBank is available for download from National Sleep Research Resource (NSRR)at https://sleepdata.org/datasets/nchsdb for non-commercial use.
Potential uses
The NCH Sleep DataBank can potentially be used to study many problems related to pediatricsleep, including but not limited to:• Automatic sleep scoring (sleep stage classification): Sleep scoring divides sleep into two stages,rapid eye movement (REM), and non-REM, then further divides the latter into shallow sleep(stages N1 and N2) and deep sleep (stage N3) [7, 8, 9], in addition to wake (Stage W). Intypical pediatric clinical settings, this is a time-consuming and tedious process done by atechnician. Many computational algorithms have shown promise for automatic sleep scoringin adults [10], which encourage exploration on automatic sleep scoring for infants and children.Algorithms that combine PSG modalities beyond EEG or ECG [11] especially warrant moreinvestigation.• Automatic sleep disorder (e.g. obstructive apnea) detection: Large sets of PSG signals pub-lished with expert annotations can be leveraged to develop computational algorithms in sleepdisorder detection, unleashing the potential of eventual real-time systems that read thesesignals and detect sleep disorders at their onsets [12, 13]. OSA detection is particularly im-portant, as OSA is associated with various cardiovascular, respiratory, and neurocognitivedeficits and morbidity among infants and children [2, 3].• Diagnosis prediction: Statistical models that predict or measure the risk of diagnoses usingother variables (e.g. other diagnoses, demographic, features from PSG, encounters, measure-ment values) can be constructed and validated to create hypotheses for further experiment.• Identifying patient subgroups: Given the demographics and medical history, patients canbe divided into clinically meaningful subgroups before further analysis, as demonstrated inthis paper for PWS. Additionally, data-driven approaches may be developed to reveal clusterswithin the patient population, which could affect their symptoms or best courses of treatment,e.g. as suggested for insomnia [14].• Treatment efficacy analysis: Retrospective studies using the accompanying longitudinal clin-ical data (e.g. medications and procedures) can be used to analyze efficacy of different treat-ments options.
Code availability
The code that was used to analyze patient data, read EDF files, run baseline sleep stage classifier,and generate figures and tables in this paper is published at https://github.com/liboyue/sleep_study . 16 cknowledgements
Research reported in this publication was supported by the National Institute Of Biomedical Imag-ing And Bioengineering of the National Institutes of Health under Award Number R01EB025018.The content is solely the responsibility of the authors and does not necessarily represent the officialviews of the National Institutes of Health. The authors thank Tim Held for data identification,Melody Kitzmiller for data query, Dan Digby for data pipelines, Rajesh Ganta for data valida-tion, Rahul Ragesh, Ramachandra Mannava, and Jacob Hoffman for help with sleep stage classifierdevelopment, and Daniel Mobley and Michael Rueschman for uploading the data to NSRR.
Author contributions
Y.C. and S.L. designed and supervised the study. S.D., B.L., Y.H. and H.L. prepared the dataset.M.S. provided clinical interpretations. H.L., B.L., and S.D. conducted data analysis and technicalvalidation. H.L., Y.C., S.D., M.S., Y.H., B.L., and S.L. drafted the manuscript.
Competing interests
The authors declare no competing interests.
References [1] Splaingard, M. L. & May, A. Sleep Disturbances. In McInerny, T. K. et al. (eds.)
AmericanAcademy of Pediatrics Textbook of Pediatric Care (American Academy of Pediatrics, Elk GroveVillage, IL, 2016).[2] Lumeng, J. C. & Chervin, R. D. Epidemiology of pediatric obstructive sleep apnea.
Proc. Am.Thorac. Soc. , 242–252 (2008).[3] Beebe, D. W. et al. Neuropsychological effects of pediatric obstructive sleep apnea.
J. Int.Neuropsychol. Soc. , 962 (2004).[4] American Academy of Sleep Medicine. International Classification of Sleep Disorders (Amer-ican Academy of Sleep Medicine, Darien, IL, 2014), 3rd edn.[5] Kushida, C. A. et al.
Practice parameters for the indications for polysomnography and relatedprocedures: an update for 2005.
Sleep , 499–523 (2005).[6] Zhang, G.-Q. et al. The national sleep research resource: towards a sleep data commons.
Journal of the American Medical Informatics Association , 1351–1358 (2018).[7] Grigg-Damberger, M. et al. The visual scoring of sleep and arousal in infants and children.
J.Clin. Sleep Med. , 201–240 (2007).[8] Berry, R. B. et al. The AASM Manual for the scoring of Sleep and Associated Events: Rules,Terminology and Technical Specifications. Version 2.4. (American Academy of Sleep Medicine,Darien, IL, 2017). 179] Berry, R. B. et al. The AASM Manual for the scoring of Sleep and Associated Events: Rules,Terminology and Technical Specifications. Version 2.5. (American Academy of Sleep Medicine,Darien, IL, 2018).[10] Fiorillo, L. et al. Automated sleep scoring: A review of the latest approaches.
Sleep Med. Rev. , 101204 (2019).[11] Yan, R. et al. Multi-modality of polysomnography signals’ fusion for automatic sleep scoring.
Biomed. Signal Process. Control , 14–23 (2019).[12] Mendonca, F., Mostafa, S. S., Ravelo-García, A. G., Morgado-Dias, F. & Penzel, T. A reviewof obstructive sleep apnea detection approaches. IEEE J. Biomed. Health Inform. , 825–837(2018).[13] Xie, B. & Minn, H. Real-time sleep apnea detection by classifier combination. IEEE Trans.Inf. Technol. Biomed. , 469–477 (2012).[14] Benjamins, J. S. et al. Insomnia heterogeneity: characteristics to consider for data-drivenmultivariate subtyping.
Sleep Med. Rev. , 71–81 (2017).[15] SleepWorks 8 Reference Manual (Natus Medical Incorporated, Ontario, Canada, 2017).[16]
SleepWorks 9 Reference Manual (Natus Medical Incorporated, Ontario, Canada, 2017).[17] Gramfort, A. et al.
MEG and EEG data analysis with MNE-Python.
Front. Neurosci. , 267(2013). URL .[18] Purcell, S. Luna: software for the analysis of sleep signal data (2019). URL http://zzz.bwh.harvard.edu/luna/ .[19] Norgeot, B. et al.
Protected Health Information filter (Philter): accurately and securely de-identifying free-text clinical notes.
NPJ Digit. Med. , 1–8 (2020).[20] Ebrahimi, F., Mikaeili, M., Estrada, E. & Nazeran, H. Automatic sleep stage classificationbased on EEG signals by using neural networks and wavelet packet coefficients. In ,1151–1154 (IEEE, 2008).[21] Fraiwan, L., Lweesy, K., Khasawneh, N., Wenz, H. & Dickhaus, H. Automated sleep stageidentification system based on time–frequency analysis of a single EEG channel and randomforest classifier. Comput. Methods Programs Biomed. , 10–19 (2012).[22] Hassan, A. R. & Bhuiyan, M. I. H. A decision support system for automatic sleep stagingfrom EEG signals using tunable Q-factor wavelet transform and spectral features.
J. Neurosci.Methods , 107–118 (2016).[23] Şen, B., Peker, M., Çavuşoğlu, A. & Çelebi, F. V. A comparative study on classification ofsleep stage based on EEG signals using feature selection and classification algorithms.
J. Med.Syst. , 18 (2014).[24] Daubechies, I. Orthonormal bases of compactly supported wavelets. Comm. Pure Appl. Math. , 909–996 (1988). 1825] Pedregosa, F. et al. Scikit-learn: Machine learning in Python.
J. Mach. Learn. Res. ,2825–2830 (2011).[26] Vela-Bueno, A. et al. Sleep in the prader-willi syndrome: clinical and polygraphic findings.
Arch. Neurol. , 294–296 (1984).[27] Hertz, G., Cataletto, M., Feinsilver, S. H. & Angulo, M. Sleep and breathing patterns inpatients with prader willi syndrome (pws): effects of age and gender. Sleep , 366–371(1993).[28] Nixon, G. M. & Brouillette, R. T. Sleep and breathing in prader-willi syndrome. Pediatr.Pulmonol. , 209–217 (2002).[29] Meyer, S. L. et al. Outcomes of adenotonsillectomy in patients with prader-willi syndrome.
Arch. Otolaryngol. Head Neck Surg. , 1047–1051 (2012).[30] Pavone, M. et al.
Sleep disordered breathing in patients with prader–willi syndrome: A mul-ticenter study.
Pediatr. Pulmonol.50