Abstract
Bilirubin-induced brain injury in the neonatal period has detrimental effects on neurodevelopment that persist into childhood and adulthood, contributing to childhood developmental disorders. Unconjugated bilirubin is a potent antioxidant that may be useful for protecting against oxidative injuries, but it becomes a potent neurotoxin once it crosses the blood brain barrier. Because bilirubin toxicity involves a myriad of pathological mechanisms, can damage most types of brain cells, and affects brain circuits or loops that influence cognition, learning, behavior, sensory, and language, the clinical effects of bilirubin-induced neurotoxicity are likely to be manifold. One possible effect that several experts have identified is bilirubin-induced neurological dysfunction (subtle kernicterus). However, the underlying biological mechanisms or pathways by which subtle kernicterus could lead to developmental disorders has not been elucidated previously. Our aim in this review is to describe a spectrum of developmental disorders that may reflect subtle kernicterus and outline plausible biological mechanisms for this possible association. We review existing evidence that support or refute the association between unconjugated hyperbilirubinemia and developmental disorders, and limitations associated with these studies.
Similar content being viewed by others
Introduction
Unconjugated hyperbilirubinemia (UHB) is one of the most common problems during the first 2 weeks after birth among premature and term infants and warrants timely evaluation and management to prevent bilirubin-induced brain injury. Studies amply document the neurotoxic effects of unbound unconjugated bilirubin (UB, bilirubin not bound to protein).1,2,3,4 Bilirubin-induced neurotoxicity (kernicterus) occurs when UB crosses the blood–brain barrier. Bilirubin has a predilection for specific brain areas including the basal ganglia, cerebellum, brainstem nuclei, peripheral and central auditory pathways, and hippocampus.3,4 In these areas, bilirubin may compromise multiple developmental processes including neurogenesis, myelination, and synaptogenesis during early human brain development.2,4 Thus, the characteristic clinical manifestations of kernicterus that are routinely described and are consistent with neuropathological findings include athetoid cerebral palsy, paralysis of upward gaze, and hearing disorders.4,5,6 However, these may represent “the tip of the iceberg.” In fact, several experts have suggested that subtle kernicterus or bilirubin-induced neurological dysfunction (BIND) without the frank characteristic clinical findings of kernicterus may be more common (Fig. 1).7,8,9,10 The BIND spectrum may encompass multiple overlapping and comorbid neurodevelopmental disorders (NDDs), including cognitive delay, attention deficit hyperactivity disorder (ADHD), specific learning disorder (SLD), autism spectrum disorder (ASD), and/or language disorder (LD).5,7,9,10,11,12,13 Evidence for a link between the BIND and NDDs comes from observational studies. However, plausible biological mechanisms for this link have not been fully articulated, and the quality of evidence has not been carefully evaluated. The current review aims to fill these gaps in the literature.
Pathogenesis of bilirubin-induced cognitive delay, ADHD, ASD, specific learning disorder, and language disorder
Bilirubin-induced cerebral cortex injury
There is both in vitro and in vivo evidence demonstrating that UHB may result in damage to cortical neurons. There is also some evidence to suggest that the cortex is relatively less susceptible as compared to other regions such as cochlear nucleus or cerebellum.2,14 Although the bilirubin-induced injury to cortex may not be readily apparent by gross histopathology or light microscopy, electron microscopy, and immunohistochemistry analyses have both demonstrated abnormalities indicative of damage to cortical neurons.4,14 The effects of bilirubin on cortical neurons include a reduction in neurite extension and dendritic and axonal arborization as well as increased cell death by apoptosis. This would contribute to dysfunction within various corticocortical and cortico-subcortical circuits. Moreover, a reduction in dendritic arborization is a hallmark feature of many NDDs.15 Furthermore, based on in vitro and in vivo models, Brites et al. have reported that moderate-to-severe UHB may impair the development and maturation of neurons, glial cells, and oligodendrocytes and have a significant impact on crucial brain functions such as cognition.2,12,16,17,18
Bilirubin-induced basal ganglia injury
Basal ganglia refers to a group of subcortical nuclei that can be broadly categorized as input nuclei (striatum), output nuclei (globus pallidus interna and substantia nigra pars reticulate), and intrinsic nuclei (globus pallidus externa, subthalamic nucleus, and substantia nigra pars compacta). The input nuclei receive incoming information from different sources including cortex and thalamus, while output nuclei send information to the thalamus. Neuropathologic studies have consistently shown that bilirubin induces injury to the globus pallidus (both the internal and external segments) and subthalamus.3,4,19,20 These structures play a critical role in regulating cognition and behavior via five parallel and segregated cortico-striatal-pallidal-thalamic-cortical circuits.21 The prototypic connections of all circuits begin in the frontal lobes and project from there as follows: frontal lobes → striatum (caudate, putamen, ventral striatum) → globus pallidus → specific thalamic nuclei → cortex (Fig. 2). There are both direct and indirect pathways within each circuit. The direct pathway (unbroken line, Fig. 2) directly links striatum to globus pallidus interna whereas the indirect pathway (dashed lines, Fig. 2) arrives at the globus pallidus interna through the following path: striatum → globus pallidus externa → subthalamus nucleus → globus pallidus interna. The spontaneous firing of the globus pallidus interna inhibits the thalamus tonically and prevents the thalamus from activating the cortex. Activation of the direct pathway inhibits the activity of the thalamus, thereby releasing (or decreasing) tonic inhibition on the cerebral cortex and increasing behavior.21 By contrast, activation of the indirect pathway increases the tonic inhibitory activity of the thalamus on the cortex thereby suppressing behavior.21 Furthermore, a hyper-direct pathway (thick lines, Fig. 2) connects the cortex to subthalamus. Activating a hyper-direct pathway results in more activity in the subthalamus, which then leads to tonic inhibition of the thalamus via activation of globus pallidus interna. Therefore, bilirubin-induced injury involving the subthalamus may result in impulse and stimulus bound behavior commonly seen in children with ADHD.22,23 The cortico-striatal-pallidal-thalamic-cortical circuits are also critical for a range of non-hippocampal learning and memory functions, including the acquisition of motor skills and perceptual-motor learning (e.g., writing), stimulus-response learning, reward-based learning and grammar and thus may contribute to a variety of symptoms/disorders ranging from clumsiness (impaired motor incoordination) to specific language impairment.24,25,26 Thus depending on where damage or dysfunction is incurred within the cortico-striatal-pallidal-thalamic-cortical circuits, it would be possible to experience inhibition or disinhibition of cognitive, motor, and behavioral functions, leading to a range of cognitive and behavioral symptoms ranging from decreased psychomotor processing speed to hyperkinetic/stimulus-bound behavior.
In addition to well-described segregated corticostriatal-pallidal-thalamic-cortical circuits, specific integrative networks function with parallel circuitry. This interaction between circuits allows flow of information and integration between limbic/emotional/motivational and cognitive/motor circuitries critical for generating and executing goal-directed behavior as well as modifying existing behavior and learning new skills. As globus pallidus and subthalamus are an integral part of these integrative networks, bilirubin toxicity can potentially affect the learning of novel cognitive skills and manifest as a SLD. Globus pallidus-cortical interactions are also crucial for the function of working memory, which predicts performance in real-word cognitive tasks.11,27,28 Working memory depends on cortical storage of multiple representations, plans, and ideas in the prefrontal-parietal lobe circuits and prevention of intrusions or distractions mediated by globus pallidus-cortical interactions.27,28 Working memory, which is the ability to carry and utilize information in the mind that is no longer present in the environment, is required for many forms of learning, including mathematics.29,30,31
In summary, depending on the area of basal ganglia involvement from bilirubin injury, clinical features of ADHD, SLD, and executive dysfunction may manifest.
Bilirubin-induced cerebellar injury
Bilirubin-induced cerebellar injury has been described in neuropathological reports.3,4 The cerebellum is heavily interconnected with both the cerebral cortex and the basal ganglia via the cerebro-ponto-cerebellar-thalamo-cortical-pathways as well as the other cerebello-cortical pathways and numerous reciprocal basal ganglia cerebellar connections.25 The reciprocal connections between cerebellum and basal ganglia are important because injury to either one can contribute to similar neurodevelopmental impairments.25 As described for basal ganglia, there is growing evidence for cerebellum involvement in cognition and behavior. The cerebellum is important for optimal neurodevelopment, and cerebellar injury during the perinatal period can have long-lasting effects on cognition and behavior.32,33,34 For example, cerebellar dysfunction has been implicated in ADHD and dyslexia.33,34,35,36,37 The impairment of the cerebellar component of cerebro-cerebellar circuit (cortex→pons→cerebellum→thalamus →cortex) has also been linked to disturbances in executive function and working memory.38,39 Cerebellar dysfunction has also been linked to linguistic deficits including disruption of language pragmatics, verbal fluency, phonological and semantic word retrieval, expressive and receptive syntax, and impairment in reading and writing.40
Based on new insights in cerebellar function, specifically anatomical and functional interactions between the cerebellum and cerebrum, and known neuropathological findings of bilirubin-induced cerebellar injury, the underlying putative mechanism for the possible association between UHB and ASD was first described by Amin et al.9 Cerebellar abnormalities are commonly observed among children with ASD. The autopsies of autistic children have shown cerebellar hypoplasia involving both vermis and cerebellar hemispheres with significantly decreased number of purkinje and granule cells of cerebellum.9 Various anatomical, neuroimaging, and electrophysiological studies suggest that the cerebellum is actively involved in spatial attention and sensory function. Consistent with these studies, bilirubin-induced cerebellar injury could disrupt multisensory cerebellar-cerebral feedback loop, contributing to the clinical characteristics of ASD.9
Bilirubin-induced hippocampal injury
Bilirubin-induced injury is associated with greater reduction in dendritic and axonal arborization in the hippocampus than in cortex.2 Hippocampus is critical for declarative memory and injury to hippocampus profoundly impair recent memory critical for learning new task (or information) or learning how to navigate one’s environment.41 Learning and memory require rapid and persistent changes (also referred as synaptic plasticity) in hippocampal neuronal circuits.42 Hippocampal cells “learn” to encode the key features of experience and this rapid and persistent neuronal encoding is a crucial step toward the formation of long-term memory. Bilirubin-induced injury to the hippocampus during the neonatal period may adversely influence synaptic plasticity and therefore may lead to memory deficits with profound implications on cognition and learning abilities necessary for academic performance.4,43,44,45
Bilirubin-induced auditory nervous system injury
The auditory neural system consists of an intricate set of neural connections interposed between hierarchically arranged nuclei. The ascending pathways carrying sound information from the cochlea to the auditory cortex are negatively altered from injury to auditory structures such as auditory nerve and cochlear nuclei. Furthermore, auditory nervous system development is intricately related to subsequent language development and auditory sensory deprivation during critical periods of brain development has been demonstrated to increase the risk of LD.13,46,47,48 Neuropathological studies and neonatal clinical studies have consistently shown that bilirubin has specific predilection to brainstem auditory structures, including cochlear nuclei, superior olivary complex, lateral lemniscus, and inferior colliculus which are components of central auditory nervous system.4,49 Therefore, bilirubin-induced auditory nervous system dysfunction during the critical period of brain development may adversely influence subsequent language development.13
Clinical studies of the association between UHB and NDDs
Global cognitive functioning
Premature infants have an increased risk for cognitive delay reported as high as 30%; however, the underlying modifying risk factors are unknown.50,51,52,53,54,55 Premature infants have a higher prevalence of UHB and the available evidence suggest that premature infants may be more susceptible to neurotoxic effects of bilirubin than term infants.4,49,56 Although the association between UHB and cognitive delay is biologically plausible, data regarding the association between UHB and cognitive delay, that is, lower measured intelligence quotient (IQ) relative to same-age peers, are conflicting. The National Institute of Child Health and Human Development phototherapy trial, which involved preterm and term infants, failed to demonstrate any significant difference in mean IQ at 6 years of age despite lower mean peak total serum bilirubin (TSB) in the phototherapy group versus control group.57,58 However, in subgroup analysis of white children with birth weight <2000 g (n = 155), children exposed to phototherapy for 96 h had significantly higher verbal IQ score than the control group (104.2 vs. 99.2).57 In another subgroup analysis of 224 control infants (not exposed to phototherapy) with birth weight <2000 g, of which 24% had received exchange transfusion, the authors reported no association between UHB, indexed by peak TSB, and IQ at school age.59 However, 42% of the sample was lost to follow-up at 6 years of age, a major limitation reported in these studies.57,59 Several other observational studies, mainly involving late preterm and term infants or low birth weight (LBW) infants, have also failed to demonstrate an association between UHB and IQ.60,61,62,63,64,65,66,67,68 However, in a larger Helsinki study, infants with TSB > 20 mg/dL or who received exchange transfusion (n = 244) had significantly lower performance IQ compared to control infants (n = 44) when evaluated at 9 years of age.69 Several other studies involving preterm and/or term infants had findings that suggest hemolysis or infection-associated UHB may lead to cognitive delay.70,71,72,73,74,75,76 Among late preterm and term infants with TSB ≥ 25 mg/dL, 9 infants with hemolysis (direct antiglobulin test [DAT] positive) had significantly lower IQ than infants without hemolysis.71 The adjusted absolute differences between the 2 groups were −18.3 (95% CI, −26.6 to −10.1) for verbal IQ, −12.0 (95% CI, −21.3 to −2.8) for performance (non-verbal) IQ, and −17.8 (95% CI, −26.8 to −8.8) for full-scale IQ when evaluated at 5 years of age.71 In a separate study involving 56 infants ≥36 weeks gestational age (GA) with peak TSB ≥ 25 mg/dL who participated in the Collaborative Perinatal Project, 19 DAT positive infants had significantly lower IQ at 7 years of age in comparison to 37 DAT negative infants.74 In addition, in a Norwegian study involving 39 male infants born at 28–42 weeks GA, 7 infants with a positive DAT and UHB for ≥5 days had lower mean IQ scores than the national average at 18 years of age.76 Male predisposition for cognitive delay has also been reported among full-term neonates with UHB and negative DAT.77 Seidman et al. reported that the risk for IQ <85 was significantly higher (p = 0.014) among full-term male but not female subjects with TSB > 20 mg/dL (odds ratio, 2.96; 95% CI, 1.29–6.79).77 It is unclear why hemolytic jaundice and male gender are risk factors for cognitive delay among children with UHB. Despite the association between UHB, as indexed by TSB, and cognitive delay being controversial, limited studies suggest that UB could be critical in evaluating the association between UHB and cognitive delay.78,79 In an observational study involving 74 Neonatal Intensive Care Unit graduates, there was a significant association between bilirubin albumin binding measure, an indirect measure of UB, but not peak TSB, with scores on the Kaufman Mental Processing Composite at 9–11 years of age.79 Similarly, Odell et al. found no significant association between peak TSB and cognitive dysfunction; however, they reported a significant association between salicylate saturation index, an indirect measure of UB, and abnormal cognition at 4–7 years of age.78
Attention deficit hyperactivity disorder
ADHD is the most common neurobehavioral disorder, affecting ~8% of children in the United States.80,81 Most studies have reported two-to-three fold increased risk of ADHD in very premature infants, yet the underlying modifying risk factors are unknown.52,54,80,82,83,84 As conveyed in sections “Bilirubin-induced basal ganglia injury” and “Bilirubin-induced cerebellar injury”, the association between UHB and ADHD has a great deal of biological plausibility due to bilirubin-induced basal ganglia and cerebellar dysfunction.11 Sparse evidence from animal and human neonatal studies also suggests that UHB may be associated with ADHD.85,86,87 In a large population-based cohort study (n = 94,914), Wei et al. reported that neonatal jaundice was associated with a significant higher risk for ADHD (adjusted hazard ratio [HR] 2.53; 95% CI: 2.23–2.98) after controlling for gender, urbanization, age at follow-up, newborn respiratory conditions, infections, preterm birth, LBW, other birth conditions, and G6PD deficiency.85 ADHD was diagnosed by a psychiatrist according to the Diagnostic and Statistical Manual for Mental Disorders–Fourth Edition criteria. Although, the GA and the birth weight of subjects were not reported, the adjusted HR for the preterm/LBW population (HR 2.83, 95% CI: 1.55–5.19) was higher than non-preterm/non-LBW population (HR 2.51, 95% CI: 2.20–2.86) after controlling for confounders. The study also demonstrated that subsequent risk of ADHD increases with a longer follow-up period in children with jaundice (Adjusted HR 2.41 at ≤6 years versus adjusted HR 2.76 at ≥6 years). Phototherapy, an index of severity of jaundice, was associated with a higher risk for ADHD than no phototherapy (adjusted HR 1.25; 95% CI:1.07–1.47), suggesting that the degree of jaundice may be critical for evaluating the association with ADHD. In a large population-based cohort study, Jangaard et al. also reported a significant increase in the risk of ADHD (adjusted relative risk 1.9; 95% CI:1.1–3.3) among ≥35 weeks GA healthy infants with TSB ≥19 mg/dL.87 Limitation of the study was that only 71.2% of subjects were followed ≥4 years of age. There has only been one study demonstrating a lack of association between UHB and ADHD in infants with GA ≥34 weeks and birth weight ≥2000 g. This retrospective study linked laboratory, demographic, and outpatient visit databases of infants from 1995 to 2004 in the Northern California Kaiser Permanente Medical Care Program and investigated the association between child’s maximum reported TSB levels in the first 30 days after birth and ever having an outpatient clinic visit at least once at or after 3 years of age with a diagnosis of ADHD assessed using the International Classification of Diseases, 9th Revision codes.88 The study had limitations as the diagnosis of ADHD is difficult in preschool age children.11,81
Specific learning disorder
Premature infants are at greater risk for SLD; however, underlying, modifying neonatal risk factors are not known.89,90,91,92 A recent meta-analysis of 14 studies on academic achievement reported that very preterm and/or very LBW children scored 0.60 standard deviations (SD) lower on mathematics tests, 0.48 SD lower on reading tests, and 0.76 SD lower on spelling tests than term-born peers.82 In addition to strong biological plausibility, the limited existing evidence suggests a possible association of UHB with SLD. Using a teacher questionnaire, Michelsson et al. reported that the frequency of poor total school marks and amount of extra help by class teacher, special education teacher, and speech-language pathologist at 9 years was significantly higher among infants with TSB > 20 mg/dL or those who had exchange transfusion for UHB (n = 256) compared to control subjects (n = 72).69 A similar group of term infants with TSB > 20 mg/dL were followed for their neurobehavioral outcome at 9 years of age and educational outcome at 30 years of age.93 Neurobehavioral disability was seen at 9 years of age among 57 out of 128 children with UHB compared to 12 out of 82 children in the control group without UHB (odds ratio 4.6, 95% CI: 2.2–10.1). The UHB group was sub-grouped as affected UHB group (with neurodisability, n = 57) and unaffected UHB group (without neurodisability, n = 71) and compared with the unaffected control group (n = 70). There were no significant differences in peak TSB or duration of UHB between the two UHB subgroups.93 The affected UHB group performed significantly poorly on the cognitive test and on tests of writing and reading compared to the unaffected UHB group and the unaffected control group at 9 years of age.93 The frequency of remedial instruction until 9 years of age was significantly higher in affected UHB group (86%) compared to both the unaffected UHB group (44%) and the unaffected control group (28%).93 In addition, the affected UHB group scored significantly higher (i.e., more inattention and hyperactivity/impulsivity symptoms) on the ADHD symptom rating scales than the unaffected control group.93 At 30 years of age, the affected UHB group had a history of significantly lower mean school grades, and smaller potential to graduate from secondary or tertiary education compared to the other two groups.93 These results indicate long-lasting effects of UHB on learning skills, which may place these children at a disadvantage when seeking future employment.
Autism spectrum disorder
Several observational studies have evaluated the association between UHB and ASD.87,94,95 Our published meta-analysis of 13 reported observational studies strongly suggests that UHB is associated with ASD (OR: 1.43, 95% CI: 1.22–1.67).9 Additionally, Maimburg et al. and Sugie et al. reported a dose–response relationship between UHB and ASD further supporting the likely association.96,97 Since the publication of the meta-analysis, additional observational studies have reported the association between UHB and ASD in a diverse group of population.98,99,100 However, most observational studies were retrospective and included infants >34 weeks GA, which is a subject population in which TSB concentration has high sensitivity but low specificity for BIND.49,101,102 A longitudinal study in premature infants is in progress to evaluate the association between UHB and ASD as a function of UB.
Language disorder
Premature infants are at higher risk for LD with the reported prevalence in school age children born prematurely being ~20%.103,104,105 There is considerable evidence that the auditory system is the most sensitive neural system to overt bilirubin toxicity in preterm and term infants.4,49 Several studies have demonstrated that UHB may be associated with transient auditory and/or brainstem dysfunction, auditory neuropathy spectrum disorder, or sensori-neural hearing loss (SNHL), which leads to auditory sensory deprivation.6,49,106,107,108,109,110,111,112,113,114,115,116,117 Auditory development is complexly related to language development and auditory sensory deprivation during early sensitive periods of brain development has been shown to increase risk for LD.46,47,48,118,119,120,121,122 Additionally, neonatal or early auditory dysfunction as evaluated by ABR has been associated with LD at later ages.46,47,123 Therefore, UHB may be associated with LD as a corollary. In addition, as noted above, damage to basal ganglia and/or cerebellum may also contribute to developmental LDs and thus may explain the incidence of LDs in children with otherwise normal hearing. Previous kernicterus reports have strongly suggested that LD is a common manifestation of bilirubin-induced neurotoxicity.5,6,7 In preterm infants, there was no association between UHB and LD when using TSB as a biochemical measure;13 however, our recent pilot study in 24–33 weeks GA infants strongly suggested that UB may be critical in evaluating the association between UHB and LD (unpublished). Also, literature from the pre-phototherapy era carries some evidence supporting the usefulness of UB. In a longitudinal observational study, Johnson et al. reported that 30–41 weeks GA infants exposed to TSB in the range of 20 mg/dL demonstrated a high incidence of deficit in central auditory perception and language functions at 4 and 7 years of age despite normal cognition and hearing.7 The audiological and language deficits, as indicated by deficient performance on norm-referenced receptive and expressive language tests were far more evident at 7 years (33%) than at 4 years (15%) due to increased brain maturation, i.e., the gap between developmentally expected, and actual language skills was greater among older children. More importantly, the authors reported a highly significant association of low bilirubin binding reserve (evaluated using HABA binding method for indirect estimation of UB) and not peak TSB with abnormal language functions at 7 years of age after controlling for known confounders such as prematurity, perinatal, and neonatal clinical factors, family history of LD, maternal education, and socio-economic status. Thus, the current limited literature suggests that UHB may be associated with LD and that the optimal age to evaluate this association may be at ≥7 years of age when language skills evolve from sentence level to discourse level capacities such as in narrative production.
Discussion
In summary, because of the myriad of pathological mechanisms involved in bilirubin toxicity; susceptibility of neuronal, oligodendrocytes, and glial cells to bilirubin toxicity; and involvement of brain areas that are an integral part of various circuits or loops that influence cognition, learning, behavior, sensory, and language, the clinical effects of bilirubin-induced neurotoxicity extend far beyond kernicterus. Our review demonstrates that multiple observational studies show an association between UHB and NDDs, providing evidence for a BIND spectrum. Most of these studies involved late preterm and term infants and there is scarce literature concerning premature infants. Furthermore, most studies have focused on a single NDD. However, the complex pathogenesis and multitude of pathways involved with bilirubin-induced neurotoxicity indicate that the occurrence of developmental disorders of BIND is unlikely to be an all-or-none phenomenon. A child may have one or more developmental disorders of the BIND spectrum, depending on the timing, degree, and duration of unbound unconjugated hyperbilirubinemia, susceptibility of specific brain area at the time of UHB, co-morbidity that can influence local neuronal and oligodendrocyte susceptibility, and potential for repair and inherent brain plasticity during early brain development. Current literature suggests that future well-designed and adequately powered prospective longitudinal studies should not only evaluate the independent association of UHB with each NDD, but also evaluate a possible comorbidity of multiple NDDs in the BIND spectrum using appropriate statistical analyses and power. Such studies will help define subtle kernicterus or BIND spectrum and the relative incidence of each NDDs that may be associated with bilirubin-induced neurotoxicity. The NDDs included in BIND spectrum often coexist among school aged children, indicating a possibility of common etiology for these disorders when present simultaneously. Furthermore, most studies have used TSB as the primary biochemical measure of UHB to evaluate the association with NDDs of BIND. However, because recent studies have demonstrated that UB but not TSB is associated with early clinical manifestations of bilirubin-induced neurotoxicity, UB will be critical in evaluating the association between UHB and later clinical manifestations of bilirubin-induced neurotoxicity, specifically NDDs of BIND.111,112,113,114,115,117,124,125,126 In the absence of UB, lack of association between UHB (indexed by TSB) and NDDs of BIND should be viewed as inadequate evidence for refuting the association. A carefully planned prospective longitudinal study is urgently needed to study BIND or subtle kernicterus at an appropriate age as a function of neonatal unbound unconjugated hyperbilirubinemia.
References
Calligaris, S. D. et al. Cytotoxicity is predicted by unbound and not total bilirubin concentration. Pediatr. Res. 62, 576–580 (2007).
Brites, D. Bilirubin injury to neurons and glial cells: new players, novel targets, and newer insights. Semin. Perinatol. 35, 114–120 (2011).
Ahdab-Barmada, M. & Moossy, J. The neuropathology of kernicterus in the premature neonate: diagnostic problems. J. Neuropathol. Exp. Neurol. 43, 45–56 (1984).
Volpe, J. J. Neurology of the Newborn 5th edn, 619–651 (W B Saunders Company, Philadelphia, 2008).
Hyman, C. B. et al. CNS abnormalities after neonatal hemolytic disease or hyperbilirubinemia. A prospective study of 405 patients. Am. J. Dis. Child. 117, 395–405 (1969).
Perlstein, M. The late clinical syndrome of posticteric encephalopathy. Pediatr. Clin. North Am. 7, 665–674 (1960).
Johnson, L. & Bhutani, V. K. The clinical syndrome of bilirubin-induced neurologic dysfunction. Semin. Perinatol. 35, 101–113 (2011).
Shapiro, S. M. Definition of the clinical spectrum of kernicterus and bilirubin-induced neurologic dysfunction (BIND). J. Perinatol. 25, 54–59 (2005).
Amin, S. B., Smith, T. & Wang, H. Is neonatal jaundice associated with autism spectrum disorders: a systematic review. J. Autism Dev. Disord. 41, 1455–1463 (2011).
Wusthoff, C. J. & Loe, I. M. Impact of bilirubin-induced neurologic dysfunction on neurodevelopmental outcomes. Semin. Fetal Neonatal Med. 20, 52–57 (2015).
Koziol, L. F., Budding, D. E. & Chidekel, D. Hyperbilirubinemia: subcortical mechanisms of cognitive and behavioral dysfunction. Pediatr. Neurol. 48, 3–13 (2013).
Brites, D. & Fernandes, A. Bilirubin-induced neural impairment: a special focus on myelination, age-related windows of susceptibility and associated co-morbidities. Semin. Fetal Neonatal Med. 20, 14–19 (2015).
Amin, S. B., Prinzing, D. & Myers, G. Hyperbilirubinemia and language delay in premature infants. Pediatrics 123, 327–331 (2009).
Jew, J. Y. & Sandquist, D. CNS changes in hyperbilirubinemia. Functional implications. Arch. Neurol. 36, 149–154 (1979).
Kaufmann, W. E. & Moser, H. W. Dendritic anomalies in disorders associated with mental retardation. Cereb. Cortex. 10, 981–991 (2000).
Brites, D. The evolving landscape of neurotoxicity by unconjugated bilirubin: role of glial cells and inflammation. Front. Pharmacol. 3, 88 (2012).
Brites, D. et al. Biological risks for neurological abnormalities associated with hyperbilirubinemia. J. Perinatol. 29(Suppl 1), S8–13 (2009).
Brito, M. A. et al. Bilirubin injury to neurons: contribution of oxidative stress and rescue by glycoursodeoxycholic acid. Neurotoxicology 29, 259–269 (2008).
Johnston, M. V. & Hoon, A. H. Jr. Possible mechanisms in infants for selective basal ganglia damage from asphyxia, kernicterus, or mitochondrial encephalopathies. J. Child Neurol. 15, 588–591 (2000).
Turkel, S. B. Autopsy findings associated with neonatal hyperbilirubinemia. Clin. Perinatol. 17, 381–396 (1990).
Alexander, G. E., DeLong, M. R. & Strick, P. L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev. Neurosci. 9, 357–381 (1986).
Frank, M. J. Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw. 19, 1120–1136 (2006).
Aron, A. R. & Poldrack, R. A. Cortical and subcortical contributions to Stop signal response inhibition: role of the subthalamic nucleus. J. Neurosci. 26, 2424–2433 (2006).
Packard, M. G. & Knowlton, B. J. Learning and memory functions of the Basal Ganglia. Annu Rev. Neurosci. 25, 563–593 (2002).
Bostan, A. C., Dum, R. P. & Strick, P. L. Cerebellar networks with the cerebral cortex and basal ganglia. Trends Cogn. Sci. 17, 241–254 (2013).
Ullman, M. T. The declarative/procedural model of lexicon and grammar. J. Psycholinguist. Res. 30, 37–69 (2001).
Frank, M. J., Loughry, B. & O’Reilly, R. C. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect Behav. Neurosci. 1, 137–160 (2001).
McNab, F. & Klingberg, T. Prefrontal cortex and basal ganglia control access to working memory. Nat. Neurosci. 11, 103–107 (2008).
Geary, D. C. Mathematics and learning disabilities. J. Learn Disabil. 37, 4–15 (2004).
Geary, D. C. Consequences, characteristics, and causes of mathematical learning disabilities and persistent low achievement in mathematics. J. Dev. Behav. Pediatr. 32, 250–263 (2011).
Geary, D. C., Hoard, M. K., Byrd-Craven, J., Nugent, L. & Numtee, C. Cognitive mechanisms underlying achievement deficits in children with mathematical learning disability. Child Dev. 78, 1343–1359 (2007).
Stoodley, C. J. & Limperopoulos, C. Structure-function relationships in the developing cerebellum: evidence from early-life cerebellar injury and neurodevelopmental disorders. Semin. Fetal Neonatal Med. 21, 356–364 (2016).
Stoodley, C. J. The cerebellum and neurodevelopmental disorders. Cerebellum 15, 34–37 (2016).
Steinlin, M. Cerebellar disorders in childhood: cognitive problems. Cerebellum 7, 607–610 (2008).
Stoodley, C. J. & Stein, J. F. Cerebellar function in developmental dyslexia. Cerebellum 12, 267–276 (2013).
Baillieux, H. et al. Developmental dyslexia and widespread activation across the cerebellar hemispheres. Brain Lang. 108, 122–132 (2009).
Krain, A. L. & Castellanos, F. X. Brain development and ADHD. Clin. Psychol. Rev. 26, 433–444 (2006).
Schmahmann, J. D. Disorders of the cerebellum: ataxia, dysmetria of thought, and the cerebellar cognitive affective syndrome. J. Neuropsychiatry Clin. Neurosci. 16, 367–378 (2004).
Koziol, L. F. et al. Consensus paper: the cerebellum’s role in movement and cognition. Cerebellum 13, 151–177 (2014).
Bodranghien, F. et al. Consensus paper: revisiting the symptoms and signs of cerebellar syndrome. Cerebellum 15, 369–391 (2016).
Eichenbaum, H. A cortical-hippocampal system for declarative memory. Nat. Rev. Neurosci. 1, 41–50 (2000).
Shapiro, M. Plasticity, hippocampal place cells, and cognitive maps. Arch. Neurol. 58, 874–881 (2001).
Rose, S. A., Feldman, J. F., Jankowski, J. J. & Van Rossem, R. The structure of memory in infants and toddlers: an SEM study with full-terms and preterms. Dev. Sci. 14, 83–91 (2011).
van Praag, H., Black, I. B. & Staubli, U. V. Neonatal vs. adult unilateral hippocampal lesions: differential alterations in contralateral hippocampal theta rhythm. Brain Res. 768, 233–241 (1997).
Burgess, N., Maguire, E. A. & O’Keefe, J. The human hippocampus and spatial and episodic memory. Neuron 35, 625–641 (2002).
Espy, K. A., Molfese, D. L., Molfese, V. J. & Modglin, A. Development of auditory event-related potentials in young children and relations to word-level reading abilities at age 8 years. Ann. Dyslexia 54, 9–38 (2004).
Fuess, V. L., Bento, R. F. & da Silveira, J. A. Delay in maturation of the auditory pathway and its relationship to language acquisition disorders. Ear Nose Throat J. 81, 706–710 (2002).
Moore, D. R. Postnatal development of the mammalian central auditory system and the neural consequences of auditory deprivation. Acta Otolaryngol. Suppl. 421, 19–30 (1985).
Amin, S. B. Clinical assessment of bilirubin-induced neurotoxicity in premature infants. Semin Perinatol. 28, 340–347 (2004).
Marlow, N., Wolke, D., Bracewell, M. A., Samara, M. & Group, E. P. S. Neurologic and developmental disability at six years of age after extremely preterm birth. N. Engl. J. Med. 352, 9–19 (2005).
Linsell, L., Malouf, R., Morris, J., Kurinczuk, J. J. & Marlow, N. Prognostic factors for poor cognitive development in children born very preterm or with very low birth weight: a systematic review. JAMA Pediatr. 169, 1162–1172 (2015).
Bhutta, A. T., Cleves, M. A., Casey, P. H., Cradock, M. M. & Anand, K. J. Cognitive and behavioral outcomes of school-aged children who were born preterm: a meta-analysis. JAMA 288, 728–737 (2002).
Poulsen, G. et al. Gestational age and cognitive ability in early childhood: a population-based cohort study. Paediatr. Perinat. Epidemiol. 27, 371–379 (2013).
Johnson, S. Cognitive and behavioural outcomes following very preterm birth. Semin. Fetal Neonatal Med. 12, 363–373 (2007).
Davis, D. W. Cognitive outcomes in school-age children born prematurely. Neonatal Netw. 22, 27–38 (2003).
Amit, Y. & Brenner, T. Age-dependent sensitivity of cultured rat glial cells to bilirubin toxicity. Exp. Neurol. 121, 248–255 (1993).
Scheidt, P. C., Bryla, D. A., Nelson, K. B., Hirtz, D. G. & Hoffman, H. J. Phototherapy for neonatal hyperbilirubinemia: six-year follow-up of the National Institute of Child Health and Human Development clinical trial. Pediatrics 85, 455–463 (1990).
Brown, A. K., Kim, M. H., Wu, P. Y. & Bryla, D. A. Efficacy of phototherapy in prevention and management of neonatal hyperbilirubinemia. Pediatrics 75(2 Pt 2), 393–400 (1985).
Scheidt, P. C. et al. Intelligence at six years in relation to neonatal bilirubin levels: follow-up of the National Institute of Child Health and Human Development Clinical Trial of Phototherapy. Pediatrics 87, 797–805 (1991).
Johnston, W. H. et al. Erythroblastosis fetalis and hyperbilirubinemia. A five-year follow-up with neurological, psychological, and audiological evaluation. Pediatrics 39, 88–92 (1967).
Vuchovich, D. M., Haimowitz, N., Bowers, N. D., Cosbey, J. & Hsia, D. Y. The influence of serum bilirubin levels upon the ultimate development of low birthweight infants. J. Ment. Defic. Res. 9, 51–60 (1965).
Stewart, R. R., Walker, W. & Savage, R. D. A developmental study of cognitive and personality characteristics associated with haemolytic disease of the newborn. Dev. Med. Child Neurol. 12, 16–26 (1970).
Culley, P. E., Powell, J. E., Waterhouse, J. A. & Wood, B. S. Sequelae of neonatal jaundice. Arch. Dis. Child. 45, 712 (1970).
Ebbesen, F., Ehrenstein, V., Traeger, M. & Nielsen, G. L. Neonatal non-hemolytic hyperbilirubinemia: a prevalence study of adult neuropsychiatric disability and cognitive function in 463 male Danish conscripts. Arch. Dis. Child. 95, 583–587 (2010).
Rubin, R. A., Balow, B. & Fisch, R. O. Neonatal serum bilirubin levels related to cognitive development at ages 4 through 7 years. J. Pediatr. 94, 601–604 (1979).
Upadhyay, Y. A longitudinal study of full-term neonates with hyperbilirubinemia to four years of age. Johns. Hopkins Med J. 128, 273–277 (1971).
Bengtsson, B. & Verneholt, J. A follow-up study of hyperbilirubinaemia in healthy, full-term infants without iso-immunization. Acta Paediatr. Scand. 63, 70–80 (1974).
Rosta, J. et al. Neonatal pathologic jaundice: seven to nine years follow-up. Acta Paediatr. Acad. Sci. Hung. 12, 317–321 (1971).
Michelsson, K., Lindahl, E., Helenius, M. & Parre, M. Five and nine year check-up of 314 children with neonatal hyperbilirubinemia. Early Child Dev. Care. 30, 167–180 (1988).
Gerver, J. M. & Day, R. Intelligence quotient of children who have recovered from erythroblastosis fetalis. J. Pediatr. 36, 342–348 (1950).
Newman, T. B. et al. Outcomes among newborns with total serum bilirubin levels of 25 mg per deciliter or more. N. Engl. J. Med. 354, 1889–1900 (2006).
Naeye, R. L. Amniotic fluid infections, neonatal hyperbilirubinemia, and psychomotor impairment. Pediatrics 62, 497–503 (1978).
Ozmert, E. et al. Long-term follow-up of indirect hyperbilirubinemia in full-term Turkish infants. Acta Paediatr. 85, 1440–1444 (1996).
Kuzniewicz, M. & Newman, T. B. Interaction of hemolysis and hyperbilirubinemia on neurodevelopmental outcomes in the collaborative perinatal project. Pediatrics 123, 1045–1050 (2009).
Day, R. & Haines, M. S. Intelligence quotients of children recovered from erythroblastosis fetalis since the introduction of exchange transfusion. Pediatrics 13, 333–338 (1954).
Nilsen, S. T., Finne, P. H., Bergsjo, P. & Stamnes, O. Males with neonatal hyperbilirubinemia examined at 18 years of age. Acta Paediatr. Scand. 73, 176–180 (1984).
Seidman, D. S. et al. Neonatal hyperbilirubinemia and physical and cognitive performance at 17 years of age. Pediatrics 88, 828–833 (1991).
Odell, G. B., Storey, G. N. & Rosenberg, L. A. Studies in kernicterus. 3. The saturation of serum proteins with bilirubin during neonatal life and its relationship to brain damage at five years. J. Pediatr. 76, 12–21 (1970).
Hansen, R. L., Hughes, G. G. & Ahlfors, C. E. Neonatal bilirubin exposure and psychoeducational outcome. J. Dev. Behav. Pediatr. 12, 287–293 (1991).
Visser, S. N. et al. Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003-2011. J. Am. Acad. Child Adolesc. Psychiatry 53, 34–46 e2 (2014).
Subcommittee on Attention-Deficit/Hyperactivity Disorder et al. ADHD: clinical practice guideline for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics 128, 1007–1022 (2011).
Aarnoudse-Moens, C. S., Weisglas-Kuperus, N., van Goudoever, J. B. & Oosterlaan, J. Meta-analysis of neurobehavioral outcomes in very preterm and/or very low birth weight children. Pediatrics 124, 717–728 (2009).
Lindstrom, K., Lindblad, F. & Hjern, A. Preterm birth and attention-deficit/hyperactivity disorder in schoolchildren. Pediatrics 127, 858–865 (2011).
Scott, M. N. et al. Behavior disorders in extremely preterm/extremely low birth weight children in kindergarten. J. Dev. Behav. Pediatr. 33, 202–213 (2012).
Wei, C. C. et al. Neonatal jaundice and increased risk of attention-deficit hyperactivity disorder: a population-based cohort study. J. Child Psychol. Psychiatry 56, 460–467 (2015).
Stanford, J. A. et al. Hyperactivity in the Gunn rat model of neonatal jaundice: age-related attenuation and emergence of gait deficits. Pediatr. Res. 77, 434–439 (2015).
Jangaard, K. A., Fell, D. B., Dodds, L. & Allen, A. C. Outcomes in a population of healthy term and near-term infants with serum bilirubin levels of >or=325 micromol/L (>or=19 mg/dL) who were born in Nova Scotia, Canada, between 1994 and 2000. Pediatrics 122, 119–124 (2008).
Kuzniewicz, M., Escobar, G. J. & Newman, T. B. No association between hyperbilirubinemia and attention-deficit disorder. Pediatrics 123, e367–e368 (2009).
Johnson, S. et al. Academic attainment and special educational needs in extremely preterm children at 11 years of age: the EPICure study. Arch. Dis. Child Fetal Neonatal Ed. 94, F283–F289 (2009).
Taylor, H. G., Espy, K. A. & Anderson, P. J. Mathematics deficiencies in children with very low birth weight or very preterm birth. Dev. Disabil. Res Rev. 15, 52–59 (2009).
Kovachy, V. N., Adams, J. N., Tamaresis, J. S. & Feldman, H. M. Reading abilities in school-aged preterm children: a review and meta-analysis. Dev. Med. Child Neurol. 57, 410–419 (2015).
Grunau, R. E., Whitfield, M. F. & Davis, C. Pattern of learning disabilities in children with extremely low birth weight and broadly average intelligence. Arch. Pediatr. Adolesc. Med. 156, 615–620 (2002).
Hokkanen, L., Launes, J. & Michelsson, K. Adult neurobehavioral outcome of hyperbilirubinemia in full term neonates-a 30 year prospective follow-up study. PeerJ 2, e294 (2014).
Croen, L. A., Yoshida, C. K., Odouli, R. & Newman, T. B. Neonatal hyperbilirubinemia and risk of autism spectrum disorders. Pediatrics 115, e135–e138 (2005).
Maimburg, R. D., Bech, B. H., Vaeth, M., Moller-Madsen, B. & Olsen, J. Neonatal jaundice, autism, and other disorders of psychological development. Pediatrics 126, 872–878 (2010).
Maimburg, R. D. et al. Neonatal jaundice: a risk factor for infantile autism? Paediatr. Perinat. Epidemiol. 22, 562–568 (2008).
Sugie, Y., Sugie, H., Fukuda, T. & Ito, M. Neonatal factors in infants with autistic disorder and typically developing infants. Autism 9, 487–494 (2005).
Froehlich-Santino, W. et al. Prenatal and perinatal risk factors in a twin study of autism spectrum disorders. J. Psychiatr. Res. 54, 100–108 (2014).
Lozada, L. E. et al. Association of autism spectrum disorders with neonatal hyperbilirubinemia. Glob. Pediatr. Health 2, 2333794X15596518 (2015).
Duan, G., Yao, M., Ma, Y. & Zhang, W. Perinatal and background risk factors for childhood autism in central China. Psychiatry Res. 220, 410–417 (2014).
Wennberg, R. P., Ahlfors, C. E., Bhutani, V. K., Johnson, L. H. & Shapiro, S. M. Toward understanding kernicterus: a challenge to improve the management of jaundiced newborns. Pediatrics 117, 474–485 (2006).
Amin, S. B. & Wang, H. Bilirubin albumin binding and unbound unconjugated hyperbilirubinemia in premature infants. J. Pediatr. 192, 47–52 (2018).
Woods, P. L., Rieger, I., Wocadlo, C. & Gordon, A. Predicting the outcome of specific language impairment at five years of age through early developmental assessment in preterm infants. Early Hum. Dev. 90, 613–619 (2014).
Barre, N., Morgan, A., Doyle, L. W. & Anderson, P. J. Language abilities in children who were very preterm and/or very low birth weight: a meta-analysis. J. Pediatr. 158, 766–774.e1 (2011).
van Noort-van der Spek, I. L., Franken, M. C. & Weisglas-Kuperus, N. Language functions in preterm-born children: a systematic review and meta-analysis. Pediatrics 129, 745–754 (2012).
Chisin, R., Perlman, M. & Sohmer, H. Cochlear and brain stem responses in hearing loss following neonatal hyperbilirubinemia. Ann. Otol. Rhinol. Laryngol. 88(3 Pt 1), 352–357 (1979).
Kaga, K., Kitazumi, E. & Kodama, K. Auditory brain stem responses of kernicterus infants. Int. J. Pediatr. Otorhinolaryngol. 1, 255–264 (1979).
Saluja, S., Agarwal, A., Kler, N. & Amin, S. Auditory neuropathy spectrum disorder in late preterm and term infants with severe jaundice. Int. J. Pediatr. Otorhinolaryngol. 74, 1292–1297 (2010).
Lenhardt, M. L., McArtor, R. & Bryant, B. Effects of neonatal hyperbilirubinemia on the brainstem electric response. J. Pediatr. 104, 281–284 (1984).
Akman, I. et al. Auditory neuropathy in hyperbilirubinemia: is there a correlation between serum bilirubin, neuron-specific enolase levels and auditory neuropathy? Int J. Audiol. 43, 516–522 (2004).
Ahlfors, C. E., Amin, S. B. & Parker, A. E. Unbound bilirubin predicts abnormal automated auditory brainstem response in a diverse newborn population. J. Perinatol. 29, 305–309 (2009).
Amin, S. B. et al. Bilirubin and serial auditory brainstem responses in premature infants. Pediatrics 107, 664–670 (2001).
Nakamura, H. et al. Auditory nerve and brainstem responses in newborn infants with hyperbilirubinemia. Pediatrics 75, 703–708 (1985).
Funato, M., Tamai, H., Shimada, S. & Nakamura, H. Vigintiphobia, unbound bilirubin, and auditory brainstem responses. Pediatrics 93, 50–53 (1994).
Amin, S. B., et al. Chronic auditory toxicity in late preterm and term infants with significant hyperbilirubinemia. Pediatrics. 140, e20164009 (2017).
Amin, S. B. et al. Auditory toxicity in late preterm and term neonates with severe jaundice. Dev. Med. Child Neurol. 59, 297–303 (2017).
Amin, S. B., Wang, H., Laroia, N. & Orlando, M. Unbound bilirubin and auditory neuropathy spectrum disorder in late preterm and term infants with severe jaundice. J. Pediatr. 173, 84–89 (2016).
Abrams, D. A., Nicol, T., Zecker, S. G. & Kraus, N. Auditory brainstem timing predicts cerebral asymmetry for speech. J. Neurosci. 26, 11131–11137 (2006).
Mason, S. M. & Mellor, D. H. Brain-stem, middle latency and late cortical evoked potentials in children with speech and language disorders. Electroencephalogr. Clin. Neurophysiol. 59, 297–309 (1984).
Kral, A., Tillein, J., Heid, S., Hartmann, R. & Klinke, R. Postnatal cortical development in congenital auditory deprivation. Cereb. Cortex 15, 552–562 (2005).
Tallal, P., Stark, R. E. & Mellits, D. The relationship between auditory temporal analysis and receptive language development: evidence from studies of developmental language disorder. Neuropsychologia 23, 527–534 (1985).
Yoshinaga-Itano, C. Benefits of early intervention for children with hearing loss. Otolaryngol. Clin. North Am. 32, 1089–1102 (1999).
Amin, S. B., Vogler-Elias, D., Orlando, M. & Wang, H. Auditory neural myelination is associated with early childhood language development in premature infants. Early Hum. Dev. 90, 673–678 (2014).
Amin, S., Orlando, M. & Wang, H. Unbound bilirubin and auditory neuropathy spectrum disorder in premature infants. Pediatric Academic Society Meeting, Boston, 2012. p. 752525.
Amin, S. B. et al. Auditory toxicity in late preterm and term neonates with severe jaundice. Dev. Med. Child Neurol. 59, 297–303 (2016).
Amin, S. B. & Wang, H. Unbound unconjugated hyperbilirubinemia is associated with central apnea in premature infants. J. Pediatr. 166, 571–575 (2015).
Acknowledgements
The work was supported by R-21 NIH grant HD078744.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Amin, S.B., Smith, T. & Timler, G. Developmental influence of unconjugated hyperbilirubinemia and neurobehavioral disorders. Pediatr Res 85, 191–197 (2019). https://doi.org/10.1038/s41390-018-0216-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41390-018-0216-4
This article is cited by
-
Models of bilirubin neurological damage: lessons learned and new challenges
Pediatric Research (2023)
-
Vascular network expansion, integrity of blood–brain interfaces, and cerebrospinal fluid cytokine concentration during postnatal development in the normal and jaundiced rat
Fluids and Barriers of the CNS (2022)
-
Normalizing hyperactivity of the Gunn rat with bilirubin-induced neurological disorders via ketanserin
Pediatric Research (2022)
-
A Gunn rat model of preterm hyperbilirubinemia
Pediatric Research (2020)