Subphenotypes of Cardiac Arrest Patients Admitted to Intensive Care Unit: a latent profile analysis of a large critical care database

Article metrics

  • 194 Accesses

Abstract

Cardiac arrest (CA) may occur due to a variety of causes with heterogeneity in their clinical presentation and outcomes. This study aimed to identify clinical patterns or subphenotypes of CA patients admitted to the intensive care unit (ICU). The clinical and laboratory data of CA patients in a large electronic healthcare database were analyzed by latent profile analysis (LPA) to identify whether subphenotypes existed. Multivariable Logistic regression was used to assess whether mortality outcome was different between subphenotypes. A total of 1,352 CA patients fulfilled the eligibility criteria were included. The LPA identified three distinct subphenotypes: Profile 1 (13%) was characterized by evidence of significant neurological injury (low GCS). Profile 2 (15%) was characterized by multiple organ dysfunction with evidence of coagulopathy (prolonged aPTT and INR, decreased platelet count), hepatic injury (high bilirubin), circulatory shock (low mean blood pressure and elevated serum lactate); Profile 3 was the largest proportion (72%) of all CA patients without substantial derangement in major organ function. Profile 2 was associated with a significantly higher risk of death (OR: 2.09; 95% CI: 1.30 to 3.38) whilst the mortality rates of Profiles 3 was not significantly different from Profile 1 in multivariable model. LPA using routinely collected clinical data could identify three distinct subphenotypes of CA; those with multiple organ failure were associated with a significantly higher risk of mortality than other subphenotypes. LPA profiling may help researchers to identify the most appropriate subphenotypes of CA patients for testing effectiveness of a new intervention in a clinical trial.

Introduction

Cardiac arrest (CA) is associated with substantial morbidity and mortality, and most will require intensive care unit (ICU) admission for post-resuscitation care. Evidence suggests that mortality after CA requiring ICU admission remains unacceptably high (50–60%)1,2; although for those who survive to hospital discharge, a good neurological outcome is common3,4. Despite advances in post-resuscitation care, the improvement in survival outcome after CA has been relatively small5. Therapeutic interventions after CA including therapeutic hypothermia, inhaled Xenon, and neuromuscular blockade have been trialed without much success despite promising results in the earlier animal studies6,7,8,9. One of the many possible reasons for these CA trials to confirm the benefits of these interventions could be due to the problem of selecting the most likely patients who would respond to such interventions. Given the high mortality rate of CA patients, it is possible that most interventions would be deemed futile for the sickest group of patients even when such interventions may offer some benefits to the less severely ill CA patients.

The concept of precision medicine is to customize healthcare, with medical decisions, treatments, practices, or products being tailored to the individual patients. The concept has been well explored in oncology that outcomes have been improved by individualized the chemotherapeutic treatment based on genomic testing10. Given ICU patients are usually heterogeneous in many ways, it is conceivable that individualized treatment may improve their outcomes. Currently many critically ill patients are admitted to the ICU with a board diagnostic syndrome such as sepsis, acute respiratory distress syndrome (ARDS) and acute kidney injury with heterogeneous underlying causes, clinical characteristics, outcomes and also possibly response to treatment. Identifying the subphenotypes of ICU patients within a certain diagnostic group may represent a small step towards precision medicine. Recently, some sophisticated machine learning methods, such as latent profile/class analysis, have been employed to identify subphenotypes of ARDS, and suggested that these subphenotypes have different baseline clinical characteristics and response to fluid strategies11,12. Similarly, CA patients also exhibit significant heterogeneity, and identification of subphenotypes may help to stratify patients who are most likely to benefit from potential therapies in a clinical trial.

To the best of our knowledge, no studies has been published about how we should classify CA patients into different subphenotypes and whether these subphenotypes are associated with different outcomes. We hypothesized that using latent profile analysis of routinely collected clinical data, we can identify subphenotypes of CA patients that are associated with different clinical outcomes. In this large database cohort study, we aimed to assess how many CA subphenotypes might exist, and describe how their clinical characteristics and outcomes were different.

Materials and Methods

Setting

A large US-based critical care database named Medical Information Mart for Intensive Care (MIMIC-III) was used for this study. The description of MIMIC-III is available elsewhere13. Briefly, the MIMIC-III database integrated de-identified, comprehensive clinical data of the patients admitted to the ICUs of Beth Israel Deaconess Medical Center in Boston, Massachusetts, from June 1st, 2001 to October 31st, 2012. There were 53,423 distinct hospital admissions for adult patients (aged 16 years or above) admitted to the ICUs during the study period. Since this study was an analysis of the third party anonymized publicly available database with pre-existing institutional review board (IRB) approval, further IRB approval from our institution was exempted. This study was reported according to the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement14.

Participants

ICU admissions with the diagnosis of cardiac arrest (ICD-9 code: 427.5) were identified for potential eligibility. For a patient with multiple ICU admissions, only the first admission was included in the analysis (e.g. each subject had a unique patient ID, by which duplicated cases could be excluded). Exclusion criteria included: (1) patients younger than 18 years old; (2) hospital stay longer than 200 days; and (3) elective admissions.

Demographical and laboratory variables

The following variables were extracted from the MIMIC- III database for the first day of ICU admission: age at the time of hospital admission, gender, admission type, ethnicity, type of ICU, urine output, Sequential Organ Failure Assessment (SOFA) score, use of vasopressors (including epinephrine, norepinephrine, dopamine and dobutamine), the lowest Glasgow coma score (GCS) and use of renal replacement therapy (RRT). SOFA score used in this study was calculated within the first 24 hours after ICU admission. If a variable was measured more than once in the first 24 hours, the value associated with the greatest severity of illness was used. For example, the lowest value of mean blood pressure (BP) and GCS reported in the first 24 hours were used in the study.

Laboratory variables such as lactate, activated partial thrombin time (aPTT), international normalized ratio (INR), sodium, potassium, creatinine, total bilirubin, platelet, hematocrit and bicarbonate were recorded for the first 24 hours after ICU admission. If there were multiple records of a test, the one associated with the greatest severity of illness was obtained. Vitals signs including mean BP, heart rate, respiratory rate, and body temperature were also extracted. The primary outcome of interest was hospital mortality, defined by the survival status of patients at hospital discharge. Secondary outcomes included length of stay (LOS) in ICU and hospital. Missing variables were common in the MIMIC-III database (Fig. 1), and multiple imputations were performed to replace any missing data. We prespecified that variables with more than 50% missing values were excluded from modeling LPA. Multiple imputation was performed by the following steps15,16:

  1. (1)

    Fit the data with appropriate model. The variables to be imputed were used as response variable and other relevant variables were used as predictors. We used predictive mean matching for continuous variables and Classification and regression trees for categorical variables.

  2. (2)

    Estimate missing data point using the fitted model in step 1.

  3. (3)

    Repeat the steps 1 and 2 for 5 times for each missing data point.

Figure 1
figure1

The percentage of missing values for each variable used for latent profile analysis.

Latent profile analysis

Latent profile analysis (LPA) is a kind of unsupervised machine learning technique that aims to recover hidden groups or patterns from observed data. It is similar to clustering techniques but is more flexible. Specifically, LPA is based on explicit modeling of the underlying data, allowing for the identified subgroups to be uncertain17. In our study, the observed data were laboratory tests and vital signs of CA patients recorded during the first 24 hours of ICU admission and the hidden groups were latent subphenotypes of CA. Variables included for LPA modeling is primarily based on domain knowledge and literature review, as well as the availability in the database. The key to successful LPA modelling was to choose the number of profiles. In this study, the number of profiles were determined by Bayesian information criteria (BIC), entropy and bootstrap likelihood ratio tests. Specifically, BIC was used to compare the models with different numbers of profiles and/or specifying different parameterizations. Lower values of the BIC are indicative of a better model fit18. Entropy ranges from 0 to 1 with a higher value indicates higher classification utility. The Vuong-Lo-Mendell-Rubin Likelihood ratio test (LRT) was used to assess the number of mixture components in a specific finite mixture model parameterization, and p values were reported for the comparison of n-class model with (n-1)-class model19. A p-value of 0.05 was used to judge the statistical significance for the likelihood ratio test. Furthermore, because the number of patients should be sizable in each latent class, we pre-specified that the patient proportion in each subphenotype should not be less than 5% in any of the other latent classes20. The clinical interpretation was also considered when determining the number of latent classes.

Statistical analysis

Continuous variables were expressed as the mean (standard deviation) or median (interquartile range) as appropriate, and were compared between the different subphenotypes of CA using analysis of variance (ANOVA)21. The CBCgrps package was employed for the statistical description and bivariate inference22. Clinical outcomes such as the mortality, length of stay (LOS) in ICU and hospital were compared between latent subphenotypes.

Multivariable logistic regression was used to assess whether mortality outcome was different between different subphenotypes after adjusting for important covariates, including the SOFA score, age, ethnicity, type of ICU, mean BP and time era of the patients’ admission (patients admitted from 2008–2012 versus those enrolled before 2008). All statistical analyses were performed using R package (version 3.4.3) and Mplus (version 7.4). A p-value less than 0.05 was considered to be statistically significant.

Ethics approval and consent to participate

This study was an analysis of the third party anonymized publicly available database with pre-existing institutional review board (IRB) approval.

Results

Patient selection

We initially identified 52,963 ICU admissions from the MIMIC-III database. After application of exclusion criteria and removal of multiple ICU admissions, a total of 1,352 CA patients were included for analysis (Fig. 2).

Figure 2
figure2

Flow chart of patient selection

Choose the best number of latent profiles

LPA models with different number of profiles were compared for their fit. The BIC and AIC values decreased rapidly from a 2-profile model to the 3-profile model (dropped by 2000) with only marginal gain when further increase the number of profiles from 4 to 5 (dropped by 1000). However, the entropy remained relatively stable beginning from 3-profile model. When the number of profiles increased to 4 or more, the number of patients in some subphenotypes was less than 5% than in other subphenotypes. The likelihood ratio test showed that a 4-profile model was not significantly better than the 3-profile model. Taken all profile selection criteria together, the 3-profile model was considered as the best model (Table 1).

Table 1 Choose the best number of profiles for day 1.

Clinical characteristics of the subphenotypes of CA

Profile 3 had the largest proportion of all CA patients (72%) and was considered as the baseline subphenotype to compare with other subphenotypes. Profile 2 (15%) was characterized by multiple organ dysfunction with evidence of coagulopathy (with prolonged aPTT and INR, decreased platelet count), hepatic injury (with a high bilirubin), renal failure (low urine output and high creatinine) and circulatory shock (with low mean BP and elevated serum lactate). Profile 1 (13%) was characterized by neurological injury (with a low GCS). Figure 3 shows the Z-score (i.e. centered at population mean and scaled by its standard deviation) of each observed variable, stratified by the latent profiles.

Figure 3
figure3

Clinical characteristics of the four latent profiles. Z-score was displayed in the y-axis, which was the value centered by the population mean and scaled by standard deviation. The use of Z-score facilitated the comparisons between variables measured at different scales. Abbreviations: aPTT: activated partial thrombin time; INR: international normalized ratio; HR: heart rate; RR: respiratory rate; WBC: white blood cell count; UO: urine output; BP: blood pressure; GCS: Glasgow Coma Scale; SpO2: oxygen saturation of pulse oximetry.

Clinical outcomes of the subphenotypes of CA

The hospital mortality rate for the whole cohort was 54%. Profile 2 had the highest hospital mortality rate (76%), followed by Profile 1 (63%) and Profile 3 (48%). The baseline subphenotype (Profile 3) had the lowest mortality rate (48%). Profile 3 had the longest length of stay in both ICU (median: 4; IQR: 2 to 8 days) and hospital (median: 9; IQR: 4 to 17 days). There was no significant difference between profiles in age and gender. Patients from Profile 2 were less likely to be admitted to the coronary care unit (CCU), and Profile 3 was more likely to be admitted to the CCU. Because Profile 2 patients showed a high incidence of circulatory shock, the use of vasoactive agents were, as expected, more prevalent compared to the other three profiles (Table 2).

Table 2 Baseline characteristics and outcomes by profiles on day 1.

Multivariable Logistic regression showed that hospital mortality was significantly different between the three latent profiles (Table 3). As compared to the Profile 1 subphenotyope, Profile 2 was significantly associated with a higher risk of death (OR: 2.09, 95% CI: 1.30 to 3.38). Profiles 3 were not significantly different from Profile 1. In addition to the LPA profiles, each 10-year increment in age was associated with a 4% increase in risk of death (OR: 1.04, 95% CI: 1.02 to 1.06; p = 0.001), and each point of SOFA score increment was associated with an 8% increase in risk of death (OR: 1.08; 95% CI: 1.04 to 1.12; p < 0.001). Admission period and ethnicity were not significantly associated with mortality.

Table 3 Multivariable logistic regression model for profile on day 1.

Discussion

Using routinely collected clinical data in a large electronic database, this study could identify three subphenotypes of CA patients. The three subphenotypes were: Profile 1 (13%) characterized by neurological injury with a low GCS with the first 24 hours of ICU admission; Profile 2 (15%) characterized by multiple organ dysfunction; Profile 3 (72%) was characterized by a lowest mortality and considered as the baseline subphenotype. More importantly, the mortality rates were also different between LPA profiles, especially between Profile 3 (48%) and Profile 2 (76%). These results have some clinical relevance and require further discussion.

An important feature of Profile 2 was circulatory shock manifested by low BP, elevated lactate, decreased bicarbonate and metabolic acidosis. Recent study showed that metabolic acidosis after cardiac arrest was frequently caused by refractory shock and was associated with a high mortality23. The post-cardiac arrest syndrome comprises of four important components including anoxic brain injury, post cardiac arrest myocardial dysfunction, systemic ischemia/reperfusion response, and persistent underlying precipitating pathology24,25. Our results suggest that these four components do not affect all CA patients equally. For example, the Profile 1 in our study was characterized by predominantly isolated neurological injury, while Profile 2 might be characterized by systemic ischemia/reperfusion response involving multiple organs. Renal dysfunction is an important component in profile 2. It is reported that approximately one in three CA patients may develop acute kidney injury, which has not been consistently found to be associated with mortality outcome26,27,28. The enrollment period appeared to be different among the latent profiles. While profile 3 patients were more likely to be recruited before 2008, profile 1 patients were more likely to be enrolled after 2008 (p < 0.001). Probably, profile 1 patients were those with severe neurological injury and could not survive to ICU admission in older days. With the development of organ supportive techniques such as extracorporeal membrane oxygenation (ECMO), more CA patients can survive the acute phase of cardiac arrest and being treated in ICU.

Our study showed that the clinical patterns, characteristics and outcomes of CA were heterogeneous. Hence, it would be prudent to enroll CA patients into any clinical trials on CA patients based on their LPA profile to maximize the power of the trials. For example, it will be preferable to enroll only patients fitting into Profile 1 for intervention trials aiming at improving neurological outcomes (e.g. therapeutic hypothermia), and only patients fitting into Profile 2 for testing interventions aiming at improving circulatory perfusion and renal outcomes. Enrolling patients fitting into Profile 2 into trials that test interventions to improve neurological outcomes would potentially reduce the power of the studies, with a higher risk of having false negative results. The analysis notes that patients in profile 3 had the longest length of stay, which is attributable to the fact that patients in the other profiles had higher mortality and thus the length of stay was shorter.

This study has some strengths and weaknesses. This large electronic healthcare database study utilized a large number of clinical variables to separate the CA patients by LPA which has not been done before. The obvious disadvantages are inherent to the nature of a retrospective design. First, the missing data was a problem in the database. Using only cases with complete data in all variables would reduce the sample size substantially. We used multiple imputations to preserve power while accounting for the uncertainty induced by the imputation process15. Second, only routinely collected clinical variables were included in the LPA model. Some important clinical information such as ischemic time before restoration of spontaneous circulation, and novel neurological biomarkers including neuron-specific enolase, S100b, liver-specific miR-122-5p and myelin basic protein have not been analyzed in the current study29,30. Because these biomarkers are not routinely used in clinical practice, and their utility to improve any future LPA modelling of CA patients remains uncertain. For a classification system to be adopted for clinical trial purposes, it is important to use clinical and laboratory variables that are routinely collected in most healthcare institutions. Third, our results do not provide any link between any interventions and its effectiveness on mortality outcome. For example, we observed that Profile 2 was associated with the highest mortality rate. Whether any interventions can improve the outcomes of this group of patients remains uncertain, but this merits further assessment by adequately-powered randomized controlled trials. Finally, the current study did not allow to distinguish between intra-hospital cardiac arrest (IHCA) and out-of-hospital cardiac arrest (OHCA). It has been observed that OHCA and IHCA can be quite different in clinical presentations and outcomes31,32.

Conclusion

In summary, LPA using routinely collected clinical data could identify three distinct subphenotypes or clinical patterns of CA; those with multiple organ failure were associated with a significantly higher risk of mortality than the baseline subphenotype or neurological injury alone. LPA profiling may help researchers to identify the most appropriate subphenotypes of CA patients for testing effectiveness of a new intervention in a clinical trial.

Data Availability

Data were fully available in the MIMIC website.

References

  1. 1.

    Langhelle, A. et al. In-hospital factors associated with improved outcome after out-of-hospital cardiac arrest. A comparison between four regions in Norway. Resuscitation 56, 247–263 (2003).

  2. 2.

    Nolan, J. P. et al. Outcome following admission to UK intensive care units after cardiac arrest: a secondary analysis of the ICNARC Case Mix Programme Database. Anaesthesia 62, 1207–1216 (2007).

  3. 3.

    Lilja, G. et al. Cognitive function in survivors of out-of-hospital cardiac arrest after target temperature management at 33 °C versus 36 °C. Circulation 131, 1340–1349 (2015).

  4. 4.

    Moulaert, V. R. M. P., Verbunt, J. A., van Heugten, C. M. & Wade, D. T. Cognitive impairments in survivors of out-of-hospital cardiac arrest: a systematic review. Resuscitation 80, 297–305 (2009).

  5. 5.

    Sasson, C., Rogers, M. A. M., Dahl, J. & Kellermann, A. L. Predictors of survival from out-of-hospital cardiac arrest: a systematic review and meta-analysis. Circ Cardiovasc Qual Outcomes 3, 63–81 (2010).

  6. 6.

    Ahn, S. et al. Sodium bicarbonate on severe metabolic acidosis during prolonged cardiopulmonary resuscitation: a double-blind, randomized, placebo-controlled pilot study. J Thorac Dis 10, 2295–2302 (2018).

  7. 7.

    Hakim, S. M., Ammar, M. A. & Reyad, M. S. Effect of therapeutic hypothermia on survival and neurological outcome in adults suffering cardiac arrest: a systematic review and meta-analysis. Minerva Anestesiol 84, 720–730 (2018).

  8. 8.

    Arola, O. et al. Inhaled Xenon Attenuates Myocardial Damage in Comatose Survivors of Out-of-Hospital Cardiac Arrest: The Xe-Hypotheca Trial. J. Am. Coll. Cardiol. 70, 2652–2660 (2017).

  9. 9.

    Stöckl, M. et al. Continuous versus intermittent neuromuscular blockade in patients during targeted temperature management after resuscitation from cardiac arrest-A randomized, double blinded, double dummy, clinical trial. Resuscitation 120, 14–19 (2017).

  10. 10.

    Alyass, A., Turcotte, M. & Meyre, D. From big data analysis to personalized medicine for all: challenges and opportunities. BMC Med Genomics 8, 33 (2015).

  11. 11.

    Calfee, C. S. et al. Subphenotypes in acute respiratory distress syndrome: latent class analysis of data from two randomised controlled trials. Lancet Respir Med 2, 611–620 (2014).

  12. 12.

    Famous, K. R. et al. Acute Respiratory Distress Syndrome Subphenotypes Respond Differently to Randomized Fluid Management Strategy. Am. J. Respir. Crit. Care Med. 195, 331–338 (2017).

  13. 13.

    Johnson, A. E. W. et al. MIMIC-III, a freely accessible critical care database. Sci Data 3, 160035 (2016).

  14. 14.

    Benchimol, E. I. et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 12, e1001885 (2015).

  15. 15.

    Zhang, Z. Multiple imputation with multivariate imputation by chained equation (MICE) package. Ann Transl Med 4, 30 (2016).

  16. 16.

    van Buuren, S. & Groothuis-Oudshoorn, K. mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software 45, 1–67 (2011).

  17. 17.

    Berlin, K. S., Williams, N. A. & Parra, G. R. An introduction to latent variable mixture modeling (part 1): overview and cross-sectional latent class and latent profile analyses. J Pediatr Psychol 39, 174–187 (2014).

  18. 18.

    Pastor, D. A., Barron, K. E., Miller, B. J. & Davis, S. L. A latent profile analysis of college students’ achievement goal orientation. Contemporary Educational Psychology 32, 8–47 (2007).

  19. 19.

    Lo, Y. Testing the number of components in a normal mixture. Biometrika 88, 767–778 (2001).

  20. 20.

    Nasserinejad, K., van Rosmalen, J., de Kort, W. & Lesaffre, E. Comparison of Criteria for Choosing the Number of Classes in Bayesian Finite Mixture Models. PLoS One 12, e0168838 (2017).

  21. 21.

    Zhang, Z. Univariate description and bivariate statistical inference: the first step delving into data. Ann Transl Med 4, 91–91 (2016).

  22. 22.

    Zhang, Z., Gayle, A. A., Wang, J., Zhang, H. & Cardinal-Fernández, P. Comparing baseline characteristics between groups: an introduction to the CBCgrps package. Ann Transl Med 5, 484–484 (2017).

  23. 23.

    Jamme, M. et al. Severe metabolic acidosis after out-of-hospital cardiac arrest: risk factors and association with outcome. Ann Intensive Care 8, 62 (2018).

  24. 24.

    Hassager, C., Nagao, K. & Hildick-Smith, D. Out-of-hospital cardiac arrest: in-hospital intervention strategies. Lancet 391, 989–998 (2018).

  25. 25.

    Stub, D., Bernard, S., Duffy, S. J. & Kaye, D. M. Post cardiac arrest syndrome: a review of therapeutic strategies. Circulation 123, 1428–1435 (2011).

  26. 26.

    Yanta, J. et al. Renal dysfunction is common following resuscitation from out-of-hospital cardiac arrest. Resuscitation 84, 1371–1374 (2013).

  27. 27.

    Al-Dury, N. et al. Characteristics and outcome among 14,933 adult cases of in-hospital cardiac arrest: A nationwide study with the emphasis on gender and age. Am J Emerg Med 35, 1839–1844 (2017).

  28. 28.

    Nobile, L. et al. The impact of extracerebral organ failure on outcome of patients after cardiac arrest: an observational study from the ICON database. Crit Care 20, 368 (2016).

  29. 29.

    Fink, E. L. et al. Serum biomarkers of brain injury to classify outcome after pediatric cardiac arrest*. Crit. Care Med. 42, 664–674 (2014).

  30. 30.

    Gilje, P. et al. The association between plasma miR-122-5p release pattern at admission and all-cause mortality or shock after out-of-hospital cardiac arrest. Biomarkers 1–25, https://doi.org/10.1080/1354750X.2018.1499804 (2018).

  31. 31.

    Kitamura, T. et al. The profile of Japanese Association for Acute Medicine - out-of-hospital cardiac arrest registry in 2014-2015. Acute Med Surg 5, 249–258 (2018).

  32. 32.

    Patel, N. J. et al. Temporal trends of survival and utilization of mechanical circulatory support devices in patients with in-hospital cardiac arrest secondary to ventricular tachycardia/ventricular fibrillation. Catheter Cardiovasc Interv 135, e146 (2019).

Download references

Acknowledgements

Z.Z. received funding from The public welfare research project of Zhejiang province (LGF18H150005), the National Natural Science Foundation of China (Grant No. 81901929) and Scientific research project of Zhejiang Education Commission (Y201737841). K.M.H. is funded by the Raine Medical Research Foundation and WA Health through the Raine Clinical Research Fellowship.

Author information

Z.Z. and Y.H. conceived the idea. Z.Z. performed statistical analysis, data management, and drafted the manuscript. M.Y. helped interpret the result and reviewed the paper. K.M.H. interpreted the results and reviewed the manuscript.

Correspondence to Zhongheng Zhang.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.