Diagnostic Accuracy of Transcranial Sonography of the Substantia Nigra in Parkinson’s disease: A Systematic Review and Meta-analysis

A large number of articles have reported substantia nigra hyperechogenicity in Parkinson’s disease (PD) and have assessed the diagnostic accuracy of transcranial sonography (TCS); however, the conclusions are discrepant. Consequently, this systematic review and meta-analysis aims to consolidate the available observational studies and provide a comprehensive evaluation of the clinical utility of TCS in PD. Totally, 31 studies containing 4,386 participants from 13 countries were included. A random effects model was utilized to pool the effect sizes. Meta-regression and sensitivity analysis were performed to explore potential heterogeneity. Overall diagnostic accuracy of TCS in differentiating PD from normal controls was quite high, with a pooled sensitivity of 0.83 (95% CI: 0.81–0.85) and a pooled specificity of 0.87 (95% CI: 0.85–0.88). The positive likelihood ratio, the negative likelihood ratio and diagnostic odds ratio were calculated 6.94 (95% CI: 5.09–9.48), 0.19 (95% CI: 0.16–0.23), and 42.89 (95% CI: 30.03–61.25) respectively. Our systematic review of the literature and meta-analysis suggest that TCS has high diagnostic accuracy in the diagnosis of PD when compared to healthy control.

Scientific RepoRts | 6:20863 | DOI: 10.1038/srep20863 Methods Search strategy. A systematic and comprehensive literature search using Pubmed, ISI Web of Science, EMBASE, Cochrane Library databases, and CNKI (a Chinese database), from 1966 until March 2015, was conducted for all the existing literatures regarding the diagnostic accuracy of TCS in the diagnosis of PD. The Medical Subjective Heading (MeSH) terms or keywords "transcranial sonography" and "Parkinson's disease" were used. Subsequently, only studies published in English or Chinese were evaluated. Repeat articles were manually deleted. If an article did not present complete data, a request for raw data was sent to the original authors via e-mail. In addition, an earnest attempt to acquire unpublished data was made but no studies were appropriate for inclusion. This work was performed by two independent authors (Li and He).
Eligibility and Exclusion criteria. Two authors carefully read and evaluated all of the articles independently. Studies were included in the current review if they met the following criteria: 1) Cross-sectional study that evaluated the ability of TCS of the SN to distinguish PD patients from healthy controls; 2) Cross-sectional study that compared SN echogenicity between patients with PD, essential tremor, or other movement disorders. Review articles, conference reports, letters, editorial comments, opinions, preface, and articles not published in English or Chinese were excluded. Other exclusion criteria for the current systematic review were: 1) articles focused on therapy and management of PD; 2) articles on Parkinsonism or other diseases, but not idiopathic PD; 3) studies that did not contain a healthy control group; 4) studies investigating the pathogenesis of SN echogenicity; 5) epidemiological studies of TCS in community dwelling elders. Two independent investigators evaluated the eligibility of all included studies.
Data extraction, Quality assessment and Statistical analysis. All relevant data of the 31 studies, including: the first author, the year when the study was carried out, diagnostic criteria of PD, ultrasound device, number of true positives, false negatives, true negatives, and false positives were extracted in a unified form. Any divergence in this procedure was resolved by discussion. The revised version of the Quality Assessment of studies of Diagnostic Accuracy Studies (QUADAS-2), with 4 key domains containing 11 items 6 , was used to assess the quality of all included studies. Each domain facilitates assessment of the risk of bias and applicability of the primary investigation. Two authors performed the quality assessment independently, with disagreements resolved by discussion or appealing to a third author.
The statistical software Meta-Disc, version 1.4 for windows (XI Cochrane Colloquium, Barcelona, Spain) and STATA, version 12.0 (Stata Corporation, College Station, TX, USA) were used in the present study. To explore potential heterogeneity arising from the threshold effect, we computed Spearman correlation coefficients between sensitivity and 1-specificity. For any possible non-threshold heterogeneity, we applied the chi-square-based Q test and the inconsistency index I 2 . A significant Q test (I 2 value > 50%) identifies a moderate or high degree of heterogeneity 7 . Subsequently, a random-effect model (DerSimonian Laird method) was used to calculate the pooled sensitivity, specificity, diagnostic odds ratio (DOR), and other related indexes. Otherwise, the Mantel-Haenszel fixed effect model was utilized. In order to assess the source of heterogeneity, we used subgroup analysis according to different threshold variables when heterogeneity arose from the threshold effect, and sensitivity analysis was chosen for non-threshold heterogeneity. Furthermore, meta-regression was implemented to investigate the source of heterogeneity within the included studies. We produced Deeks' funnel plot to test the potential publication bias in our study, with a p value < 0.1 suggesting significance 8 .

Results
Characteristics and quality of the included studies. The inclusion and exclusion criteria for article selection are illustrated in Fig. 1. Ultimately, 31 studies 4,5,9-37 containing 1,926 idiopathic PD patients and 2,460 healthy controls from 13 countries, were included in our meta-analysis. The main characteristics of the included studies are summarized in Table 1.

Meta-regression analysis. Meta-regression analysis was utilized to investigate potential reasons for
inter-study heterogeneity based on geographical location (Europe, Asia or America), sample size (< 50 or ≥ 50), age of PD patients (< 65 or ≥ 65), ultrasound equipment (< 2.5 MHz or ≥ 2.5 MHz), and QUADAS-2 scores (< 10 or ≥ 10). However, none of the above covariates were found to be significant sources of heterogeneity, as all p values were > 0.05. Sensitivity analyses. Sensitivity analyses were performed to explore the possible heterogeneity and verify the consistency of the results from our meta-analysis by applying the leave-one-out method in which the first of the K studies is left out on repeat meta-analysis of the resulting subgroup containing K− 1 studies. This analysis is repeated for the next K studies until all distinct meta-analyses are performed, each leaving out one study. Overall, no substantial alterations of the results were found in our investigation, with the pooled sensitivity ranging from 0.82 (95% CI: 0.80-0.84) with omission of the study by Maria Sierra 2013 5 to 0.84 (95% CI: 0.82-0.85) with omission of the study by Yu-Wen 2007 22 , and the pooled specificity ranging from 0.86 (95% CI: 0.85-0.88) by removing the study by SinemTunc 2015 9 to 0.89 (95% CI: 0.88-0.90) by removing the study by Philipp Mahlknecht 2013 20 . These sensitivity analyses indicate statistically consistent results with a high level of overall accuracy using TCS in the diagnosis of PD. Moreover, among the included studies, no single study was found to be the source of heterogeneity.
Evaluation of publication bias. Deeks' funnel plots were produced to explore the potential presence of publication bias. Based on the symmetric shape of the funnel plot of pooled DOR (Fig. 5) and the Deeks' test non-significant value (p = 0.29), there is no potential publication bias in the current meta-analysis.
Discussion. The results of our meta-analysis, which included 1,926 PD patients and 2,460 healthy controls from 13 countries, demonstrated a high clinical utility of TCS in the diagnosis of PD, with a pooled sensitivity (83%) and specificity (87%). The AUC (0.9306) and DOR (42.89) further indicate an excellent overall accuracy. In addition, a PLR value of 6.94 (95% CI: 5.09-9.48), which is more clinically meaningful for our measures of diagnostic accuracy 38 , suggests that patients with SN hyperechogenicity have a moderate increase in the likelihood of having PD.
For all meta-analyses, heterogeneity is a potential problem when interpreting the results. One major source of heterogeneity is the threshold effect in which different cut-offs are used in the studies included in a meta-analysis. The Spearman correlation coefficient in our study indicates that there is no threshold effect related heterogeneity. Furthermore, meta-regression analysis to find other possible sources of heterogeneity, including geographical location (Europe, Asia or America), sample size (< 50 or ≥ 50), age of PD patients (< 65 or ≥ 65), ultrasound equipment (< 2.5 MHz or ≥ 2.5 MHz), and QUADAS-2 scores (< 10 or ≥ 10), revealed that none of the variables were substantial sources of heterogeneity. Therefore, we subsequently performed sensitivity analyses to explore the possibility of significant overall inter-study heterogeneity and to verify the consistency of our results. No obvious alterations were detected, indicating no conceivable source of heterogeneity and statistically consistent results.
In recent years, applications of TCS in the clinical differentiation of PD patients from the healthy population have shown great value. Investigations into the differential diagnosis of PD from atypical parkinsonian syndrome (APS), essential tremor (ET), restless leg syndrome (RLS), or other neurological diseases utilizing TCS suggest that normal SN echogenicity was correlated with multiple system atrophy (MSA) 39 and ET 4,17,26,27 . Furthermore, SN hypoechogenicity was detected in patients with RLS 21 . More interestingly, abnormal SN hyperechogenic areas were also discovered in 67% of amyotrophic lateral sclerosis (ALS) patients 16 , a disease that might be related to impairment of the nigrostriatal system based on neuroimaging data 40,41 . Additionally, lenticular nucleus hyperechogenicity in combination with third-ventricle dilatation of more than 10 mm by TCS helps differentiate progressive supranuclear palsy (PSP) from PD 39 . Moreover, the combination of TCS and olfactory test 42 or MIBG myocardial scintigraphy 10 has been identified to improve the differential diagnostic power for identifying PD. All of these investigations demonstrated that the clinical application of TCS may not only help identify PD patients, but also differentiate PD patients from other movement disorders, which suggests great value for TCS in routine clinical practice.
The origin of SN hyperechogenicity, assessed by animal and postmortem studies, has been shown to be related to midbrain iron deposition 43 . Furthermore, the levels of H-and L-ferritins 44 , iron metabolizing protein 45 , plasma ferroxidase activity 46 , and serum CRP 47 were abnormal in PD patients with SN hyperechogenicity, which further bolsters the concept that SN hyperechogenicity is related to alterations in iron metabolism in PD. Other sources of SN hyperechogenicity include microglia activation 48 and gliosis 49 , which were found in brain tissue with SN echogenicity after correction for iron and neuromelanin contents. The LRRK2 gene, an autosomal-dominant PD gene, participates in the regulation of neuroinflammation 50 and microglia activation 51 , and has been found to correlate with SN echogenicity as well. Specifically, carriers of the LRRK2 mutation with no clinical manifestation of PD have a similar proportion of SN hyperechogenicity when compared with idiopathic PD patients 5 . Other PD related gene mutation loci, such as PINK1 52 , GBA 53 have been also reported to correlate with diverse echogenicity. In the previous research 54 , we explored the potential correlation between SN hyperechogenicity with  3 first reported a specific high echogenic area within the SN of PD patients over 20 years ago, midbrain echo-features of PD patients have been confirmed and further investigated by numerous groups. However, the utility of TCS in the clinical diagnosis of PD is not universally accepted for several reasons.  When a physician wants to utilize a clinical tool, the first parameters examined are the sensitivity and specificity. Unfortunately, different groups report inconsistent results 4,5 due to small sample sizes, and this leads to varied sensitivity and specificity values which precludes the application of TCS for the diagnosis of PD. Therefore, we sought to perform a comprehensive study to evaluate the diagnostic accuracy of TCS. Our study, containing 1,926 PD patients and 2,460 healthy controls from 13 countries, revealed a high pooled sensitivity and specificity, which strongly indicates that TCS could be applied as a clinical tool for the diagnosis of PD patients from healthy controls. Nevertheless, some technical shortcomings must be acknowledged.
One inevitable problem that a sonographer may confront is transcranial insonability. In European populations, 4-15% of participates were found to have an insufficient temporal window 5,9,16,17,24,25 . However, the value rises to 15-60% in Asian populations 10,11,21,22,28,34 . This high recording failure rate in TCS application would mostly affect patients of advanced age with female gender 56 or patients with a small temporal window seen in Asian populations. Recently, high-resolution ultrasound systems with standardized settings or with automated segmentation technique were reported to reduce inter-observer and intra-observer variability 57 , which may help improve TCS image quality and decrease the incidence of insufficient temporal window. Moreover, a novel approach using transcranial B-mode sonography, a 3-D ultrasound platform, was shown to be technically feasible and less dependent on sonographer experience or good bone windows 58 . These innovations and developments in ultrasound systems may effectively improve the application value and diagnostic accuracy of TCS.
To our knowledge, this is the first systematic review and meta-analysis assessing the overall diagnostic accuracy of TCS in PD. A thorough literature search and careful data extraction were performed to avoid any bias. Nevertheless, limitations still exist in our study. First, although we carefully explored the heterogeneity by  meta-regression and sensitivity analyses, notable heterogeneity was still observed, which can be due to random variation between individual studies 59 . Second, failure to acquire unpublished data or studies not published in English or Chinese for language limitation may affect the validity of our results.
In conclusion, our systematic review and meta-analysis suggest that TCS has high diagnostic accuracy in the diagnosis of PD patients from the healthy population. As a non-invasive, non-radioactive and convenient neuroimaging technique, application of TCS in routine clinical practice is of great value in the diagnosis of PD. However, large cohorts of high-quality prospective studies are still required to further confirm the value of TCS in the diagnosis of PD.