Pre-Operative Delta-MELD is an Independent Predictor of Higher Mortality following Liver Transplantation

Abstract

Clinical decompensation immediately prior to liver transplantation may affect post-liver transplant (LT) outcomes. Using the serial Model for End-Stage Liver Disease (MELD) scores recorded in the United Network for Organ Sharing national registry (2010–2017), we analyzed post-LT mortality among adult LT recipients based on the degree of fluctuation in MELD score during the 30-day period prior to LT surgery. Delta-MELD (D-MELD) was defined as recipient MELD score at LT minus lowest MELD score within the preceding 30 days. Impact of D-MELD as a continuous and categorical variable (D-MELD 0–4, 5–10, >10) on early, 30-day post-LT mortality was assessed. Overall, a total of 12,785 LT recipients were analyzed, of which 8,862 (67.9%) had a pre-operative D-MELD 0–4; 2,574 (20.1%) with a D-MELD 5–10; and 1,529 (12.0%) with a D-MELD > 10. One-point incremental increase in pre-operative D-MELD (adjusted HR, 1.07, 95% CI: 1.04–1.10) was associated with higher 30-day post-LT mortality. Moreover, pre-operative D-MELD > 10 was associated with nearly a two-fold increased risk for 30-day post-LT mortality (adjusted HR, 1.89, 95% CI: 1.30–2.77) compared to D-MELD 0–4. The increased risk of pre-LT mortality associated with severity of clinical decompensation assessed by the magnitude of pre-operative D-MELD persists in the early post-LT period.

Introduction

In patients with liver failure in the setting of cirrhosis, acute-on-chronic liver failure (ACLF) is a clinical entity characterized by sudden onset and rapid progression of underlying hepatic dysfunction which may be accompanied by multi-system organ failure with high risk of short-term mortality (50–90%) without prompt supportive measures and liver transplant (LT) surgery1,2,3,4,5. ACLF often impacts the Model for End-Stage Liver Disease (MELD) score, a validated objective predictor for early pre-LT mortality and the primary allocation criteria for liver transplantation in the United States (US)6,7,8.

Although a ≥5 points rise in MELD score over 30 days in the setting of ACLF has been shown to predict pre-LT waitlist mortality9,10, the data are conflicting in terms of post-LT survival, in part due to widely varying time frames that have been evaluated9,11,12,13,14,15. Using the Organ Procurement Transplant Network/United Network for Organ Sharing (OPTN/UNOS) registry, Northup et al. in 2004 reported that a preoperative rise in MELD score of ≥5 within 30 days of liver transplantation was not predictive of post-LT survival14. Another study analyzing 69,643 waitlist registrants found that a rise in MELD score of >30% during a seven-day period was associated with an increase in waitlist mortality but lacked association with post-LT mortality16. In a single center analysis, Gyori et al. demonstrated that patients with a rise in MELD score of >10 over their time on the waitlist tended to experience worse 1-, 3-, and 5-year post-LT survival11, which was later replicated in a large Eurotransplant registry analysis12. However, the relationship between sudden changes in MELD score immediately prior to LT surgery and outcomes was not examined.

Utilizing the serial laboratory MELD scores from the OPTN/UNOS database, we aim to assess in a standardized manner the impact of sudden rise in recipient MELD score prior to LT surgery on early post-transplant mortality. We studied Delta-MELD (D-MELD) defined as recipient MELD score at the time of LT surgery minus lowest MELD score within the preceding 30 days.

Methods and Materials

Data source

Our study used data from OPTN/UNOS registry, which includes national data on all organ donations, waitlist registrants and LT recipients in the US. All patient and transplant center identifiers were excluded. This study was exempt from Institutional Review Board (Stanford University Research Compliance Office) review at our institution, as the data set used in the analysis was completely de-identified.

Patients

Using serial MELD scores reported within the OPTN/UNOS registry, we analyzed post-LT mortality among adult (age ≥ 18) LT recipients from January 1, 2010 to December 31, 2017 who experienced a sudden pre-operative rise in their MELD score prior to LT surgery. We utilized an objective parameter, the D-MELD which is the absolute difference in recipient MELD score at the time of LT surgery and lowest MELD score (MELDi) tested and reported during the 30 days prior to LT surgery. All LT recipients had a calculated or laboratory MELD score ≥ 15 at the time of LT surgery. LT recipients without a reported MELDi within 30 days preceding LT were excluded. In addition, LT recipients with concomitant hepatocellular carcinoma, allocation MELD exception score higher than their calculated MELD score at LT, Status 1A, living donor liver transplantation and simultaneous multiple organ transplantation were excluded.

Definitions

MELD scores were calculated utilizing serial MELD score parameters including serum sodium (mmol/L), serum creatinine (mg/dL), serum bilirubin (mg/dL), and international normalized ratio for prothrombin time, which were provided by OPTN/UNOS. The standard UNOS formula which incorporated sodium as of January 1, 201617 was used to determine serial MELD scores18.

$$\begin{array}{rcl}MEL{D}_{(i)} & = & {0.957}\times Log(creatinine\,mg/dL)\\ & & +\,{0.378}\times Log(bilirubin\,mg/dL)\\ & & +\,{1.120}\times Log+{0.643}\\ MELD & = & MEL{D}_{(i)}+{1.32}\,\ast \,(137-Na)\\ & & \mbox{--}\,[{0.033}\,\ast \,MEL{D}_{(i)}\,\ast \,(137-Na)]\end{array}$$

The minimum serum creatinine value was set to 1.0 mg/dL and the maximum serum creatinine value was set to 4.0 mg/dL. Serum creatinine value of 4.0 mg/dL was assigned to subjects if they underwent two or more dialysis treatments or received 24 hours of continuous veno-venous hemodialysis within 7-days prior to serum creatinine test. Data on the date of initiation of dialysis or continuous veno-venous hemodialysis were unavailable. The minimum serum sodium value was set to 125 mmol/L and the maximum serum sodium value was set to 137 mmol/L.

Variables

Clinical characteristics at time of LT surgery included demographic data (age, gender, race/ethnicity), etiology of liver disease, ascites, hepatic encephalopathy (HE), renal failure on dialysis, change in creatinine (mg/dL), change in bilirubin (mg/dL), change in INR, initial MELD at the time of waitlist addition and MELD score at LT were included. Donor characteristics included liver donor risk index19 and utilization of donation after cardiac death (DCD) donors.

Outcome

The primary outcome was to evaluate the impact of sudden rise in the magnitude of pre-operative D-MELD on early post-LT mortality. Mortality was assessed at 7-day, 30-day and 1-year following liver transplantation. D-MELD was analyzed as both a categorical (D-MELD 0–4, D-MELD 5–10, D-MELD > 10) and continuous variable (1-point increment).

Statistical analysis

Cox proportional hazards regression models were fitted for each variable separately to determine association with post-LT mortality as the dependent variable. Separate models were constructed to assess post-LT mortality at 7-day, 30-day and 1-year intervals. In these models, patients were censored on the date of last follow-up if death had not occurred, and all observations were censored at 7-day, 30-day or 1-year interval depending on the time-specific model. Forward stepwise regression was performed with entry and exit criteria set to P = 0.10. Variables with biological plausibility for association with survival were included even if P value was above the P value threshold. Demographic and clinical characteristics were compared using Chi-square test for categorical variables and Kruskal-Wallis for quantitative variables. Cumulative incidence and 95% confidence intervals (CI) for post-LT mortality estimates were calculated while accounting for competing risk of graft failure. Log-rank was used to test differences between multiple cohorts. In a pre-planned sensitivity analysis, we analyzed recipients’ MELD scores capped and uncapped above 40. Statistical significance was met with a P value < 0.05. All statistical analyses were performed using the Stata 14 statistical package (StataCorp, College Station, TX).

Results

Clinical characteristics

From January 1, 2010 to December 31, 2017, there were 12,785 adult LT recipients analyzed that met the inclusion criteria as shown in Fig. 1. Demographics and clinical characteristics of the total cohort are demonstrated in Table 1. The majority of patients were Caucasian males with a median age of 55 (IQR, 48–61). Chronic hepatitis C virus (HCV)-related cirrhosis and alcoholic cirrhosis constituted nearly half of all LT recipients and more than one-quarter of recipients were on renal dialysis at the time of LT surgery. Less than 5% of patients received a DCD liver graft. Recipients had a median MELD score at LT of 31 (IQR, 24–39) and median D-MELD of 2 (IQR, 0–6). Separate analyses were performed with recipients’ MELD scores capped at 40 and uncapped above 40. No statistical differences in outcomes were observed between the analyses. Therefore, the results reported utilize recipients’ capped MELD scores in accordance with the current UNOS allocation scoring system. Of the 12,785 recipients included in this study, 8,862 (67.9%) recipients had a pre-operative D-MELD 0–4, 2,574 (20.1) recipients had a D-MELD 5–10, and 1,529 recipients had a D-MELD > 10 within 30 days prior to LT surgery. Recipients with a D-MELD > 10 had a higher percentage of Hispanics, HCV-related cirrhosis, renal dialysis dependency, severe HE or ascites at the time of LT surgery (Table 1). Recipients with a D-MELD > 10 utilized a lower proportion of DCD donors. No statistical difference (P > 0.05) was seen in median age, gender or median LDRI score among the three cohorts. Differences in mean change in MELD score parameters including sodium, creatinine, bilirubin, and INR are tabulated in Table 2.

Figure 1
figure1

Study identification, inclusion, and exclusion criteria utilized in constructing D-MELD cohorts.

Table 1 Clinical characteristics of liver transplant recipients categorized by D-MELD.
Table 2 Study cohort MELD scores and differences in MELD score parameters.

Mortality

Post-LT cumulative mortality incidence rates at 7-day, 30-day and 1-year interval among recipients are depicted in the Supplementary Table. Recipients with a D-MELD 5–10 had significantly higher mortality rates than those with a lower D-MELD between 0–4 (P < 0.05). In addition, both LT recipients with a D-MELD 5–10 and a D-MELD > 10 demonstrated 1-year mortality rates exceeding 10%. Differences in post-LT mortality rates between these three cohorts are further illustrated in Fig. 2 with >10 D-MELD LT recipients demonstrating the highest post-LT mortality rates. Separate univariate and multivariate survival models analyzed post-LT mortality at 7-day, 30-day and 1-year. Univariate analyses demonstrated that D-MELD as a continuous variable with 1-point increments was associated with higher mortality within 1-year of LT surgery (Table 3). Multivariate analyses were adjusted for recipient age, gender, ethnicity, etiology of liver disease, LDRI, DCD donor, renal dialysis, ascites, severe HE, MELD score at the time of LT, mean change in MELD score parameters and OPTN/UNOS Region. After adjusting for these covariates, D-MELD as a continuous variable was still associated with higher post-LT mortality in each survival model at 7-day, 30-day and 1-year. Using similar survival models, D-MELD was analyzed as a categorical variable on post-LT mortality. After adjusting for covariates including MELD score at the time of LT surgery, recipients with a D-MELD 5–10 and a D-MELD > 10 had a significantly higher 30-day and 1-year post-LT mortality (Table 3). Table 4 demonstrates significant predictive risk for 30-day post-LT mortality. At 30-day assessment, D-MELD > 10 was the strongest predictor for post-LT mortality (D-MELD 0–4, reference; D-MELD 5–10, HR, 1.34 95% CI: 1.03–1.76; D-MELD > 10, HR 1.89, 95% CI: 1.30–2.77; P = 0.001). Other significant predictors for 30-day post-LT mortality included utilization of DCD donors (HR, 1.67; 95% CI: 1.09–2.54, P = 0.018), increasing LDRI (HR 1.40, 95% CI: 1.21–1.60, P < 0.001), and renal dialysis (HR, 1.37, 95% CI: 1.08–1.75, P = 0.010). Graft failure was analyzed as a post-LT outcome separate from mortality, however no association was noted between higher D-MELD and acute graft failure within 1-year of liver transplant surgery.

Figure 2
figure2

Cumulative incidence post-transplant mortality curves in cohorts with pre-operative D-MELD < 5 (0–4), D-MELD 5–10 and D-MELD > 10 taking the competing risk of graft failure and re-transplantation into account.

Table 3 Univariate and multivariate Cox regression models analyzing D-MELD as a continuous and categorical variable on early post-transplant mortality.
Table 4 Multivariate Cox regression analyses to predict risk factors for 30-day post-transplant mortality.

Discussion

In this large national cohort, we demonstrate a sudden rise in pre-operative D-MELD is associated with a significantly lower early post-LT survival. Sudden increase in the MELD score ranging between 5–10 to as high as >10 were associated with a significantly higher 30-day and 1-year post-LT mortality after adjusting for confounding factors, including absolute MELD score at transplant. These observations may play an important role in the organ allocation decision-making process to optimize the post-LT outcomes.

While the primary finding of an association between sudden changes in MELD and lower post-LT survival has been hypothesized in the transplant community, it has to our knowledge not previously been shown in a large cohort study. In addition, this primary finding is somewhat in contrast to previously published work in this area. In particular, while Northup et al. in 2004 reported that pre-operative rise in D-MELD was associated with early post-LT mortality, this association was not present after adjusting for absolute MELD. However, the study was limited by the fact that it was conducted in the pre-MELD and pre-sodium-MELD eras. Our analysis also excluded patients with HCC, acute liver failure including those listed as Status 1A and a MELD < 15 at time of LT surgery, allowing for an accurate assessment for ACLF. In two more recent analyses, Gyori et al. did not specifically tackle this question, but instead focused on changes in MELD over the entire waitlist period rather than a sudden change (e.g., preceding 30 days) in MELD preceding LT surgery11,12 and did not adjust for several known risk factors known to adversely impact LT survival such as absolute MELD at transplant, LDRI, and potential complications of decompensated cirrhosis (HE, severe ascites, hemodialysis).

The data presented here demonstrate a robust association between sudden changes in D-MELD within 30 days of liver transplantation and short-term post-LT mortality beyond that which is explained solely by the absolute MELD. Higher D-MELD, reflecting a sudden ACLF within the 30 days preceding LT surgery, and increased post- LT mortality may be directly related through several possible mechanisms. ACLF may result from either the primary hepatic decompensation from worsening liver disease or as a manifestation of an underlying disease process such as severe infection or multi-organ failure. The poor outcomes in LT recipients associated with a sudden and significant rise in pre-LT D-MELD versus relatively low D-MELD may reflect the consequences of ongoing underlying pathology that continues to progress post-operatively without affecting absolute pre-LT MELD score. Infectious complications are a major cause of early post-LT morbidity and mortality20. Known pre-operative risk factors for infection following liver transplantation include worsening renal function, long-term hemodialysis, and rising bilirubin levels, which are all captured by high D-MELD21,22. In addition, LT recipients hospitalized in the intensive care unit requiring vasopressor or ventilator support immediately prior to LT surgery are at a higher risk for post-LT mortality within one-year23,24. Therefore, developing management strategies focused on identifying risk factors pre-emptively and preventing ACLF will be key areas for improvement. More importantly, the same strategies can help improve organ allocation and resource utilization by identifying high-risk candidates prior to liver transplantation25. However, the apparent increased mortality associated with increased D-MELD over that of the absolute pre-transplant MELD would need to be weighed against a concurrent possible increase in waitlist mortality to better risk-stratify patients who are acutely decompensating. In our analysis, donor quality as measured by donation after cardiac death and LDRI score were the only controllable variables affecting early post-transplant mortality and should be heavily considered in this patient population.

Our study was limited by its retrospective design. In collaboration with OPTN/UNOS, we developed a customized dataset and were able to evaluate serial MELD scores reported by transplant centers to update MELD scores of waitlisted patients. However, any dynamic changes in the MELD scores of waitlisted patients that were not reported by liver transplant programs were not evaluated in this study. Therefore, it is plausible that a decline in pre-LT MELD score and D-MELD in a subset of LT recipients was not captured and we therefore did not evaluate the impact of a decline in pre-operative MELD score or negative D-MELD on early post-transplant mortality. In addition, we did not analyze the potential risk factors associated with higher pre-operative D-MELD because we were unable to quantitatively assess the variables known to impact ACLF such as infection, cardiovascular deterioration, vasopressor support, and portal hypertension complications including variceal bleeding, spontaneous bacterial peritonitis, hepatorenal syndrome, and hepatopulmonary syndrome. Other perioperative variables were also not captured in the UNOS/OPTN dataset, such as infection, surgical complication, pulmonary or cardiovascular compromise. In addition, we were unable to determine pharmacologic therapy including antiviral therapy and associated virologic response to therapy. Furthermore, the cause of death in LT recipients was not analyzed due to a significant percentage of missing data (>30%). All prior studies using the OPTN/UNOS database studied only recipient’s MELD score at the time of registration and transplant. However, we studied serial MELD scores in LT recipients utilizing a customized OPTN/UNOS dataset.

Using a national transplant registry, we demonstrate that a higher pre-operative D-MELD is associated with significantly increased risk of early post-LT mortality. Moreover, LT recipients with a pre-operative D-MELD > 10 were noted to have a two-fold increase in 30-day post-LT mortality, even after adjusting for MELD score at transplant. We suggest that an increased risk of pre-LT mortality associated with ACLF may not be captured by absolute MELD at the time of LT surgery. Therefore, D-MELD can identify waitlist patients awaiting liver transplantation who are at increased of early post-LT mortality. To this end, further studies are needed to confirm our findings and utilize these findings in the assessment of and allocation of organs for prospective LT recipients.

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. 1.

    Hernaez, R., Sola, E., Moreau, R. & Gines, P. Acute-on-chronic liver failure: an update. Gut 66, 541–553, https://doi.org/10.1136/gutjnl-2016-312670 (2017).

  2. 2.

    Duseja, A. et al. Non-hepatic insults are common acute precipitants in patients with acute on chronic liver failure (ACLF). Dig Dis Sci 55, 3188–3192, https://doi.org/10.1007/s10620-010-1377-0 (2010).

  3. 3.

    Juneja, D., Gopal, P. B., Kapoor, D., Raya, R. & Sathyanarayanan, M. Profile and outcome of patients with liver cirrhosis requiring mechanical ventilation. J Intensive Care Med 27, 373–378, https://doi.org/10.1177/0885066611400277 (2012).

  4. 4.

    Katoonizadeh, A. et al. Early features of acute-on-chronic alcoholic liver failure: a prospective cohort study. Gut 59, 1561–1569, https://doi.org/10.1136/gut.2009.189639 (2010).

  5. 5.

    Garg, H. et al. Clinical profile and predictors of mortality in patients of acute-on-chronic liver failure. Dig Liver Dis 44, 166–171, https://doi.org/10.1016/j.dld.2011.08.029 (2012).

  6. 6.

    Kamath, P. S. et al. A model to predict survival in patients with end-stage liver disease. Hepatology 33, 464–470, https://doi.org/10.1053/jhep.2001.22172 (2001).

  7. 7.

    Wiesner, R. et al. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology 124, 91–96, https://doi.org/10.1053/gast.2003.50016 (2003).

  8. 8.

    Wiesner, R. H. et al. MELD and PELD: application of survival models to liver allocation. Liver Transpl 7, 567–580, https://doi.org/10.1053/jlts.2001.25879 (2001).

  9. 9.

    Merion, R. M. et al. Longitudinal assessment of mortality risk among candidates for liver transplantation. Liver Transpl 9, 12–18, https://doi.org/10.1053/jlts.2003.50009 (2003).

  10. 10.

    Bambha, K. et al. Predicting survival among patients listed for liver transplantation: an assessment of serial MELD measurements. American journal of transplantation: official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 4, 1798–1804, https://doi.org/10.1111/j.1600-6143.2004.00550.x (2004).

  11. 11.

    Gyori, G. P. et al. Dynamic changes in MELD score not only predict survival on the waiting list but also overall survival after liver transplantation. Transpl Int 25, 935–940, https://doi.org/10.1111/j.1432-2277.2012.01519.x (2012).

  12. 12.

    Gyori, G. P. et al. Impact of dynamic changes in MELD score on survival after liver transplantation - a Eurotransplant registry analysis. Liver Int 36, 1011–1017, https://doi.org/10.1111/liv.13075 (2016).

  13. 13.

    Huo, T. I. et al. Evaluation of the increase in model for end-stage liver disease (DeltaMELD) score over time as a prognostic predictor in patients with advanced cirrhosis: risk factor analysis and comparison with initial MELD and Child-Turcotte-Pugh score. J Hepatol 42, 826–832, https://doi.org/10.1016/j.jhep.2005.01.019 (2005).

  14. 14.

    Northup, P. G. & Berg, C. L. Preoperative delta-MELD score does not independently predict mortality after liver transplantation. American journal of transplantation: official journal of the American Society of Transplantation and the American Society of Transplant Surgeons 4, 1643–1649, https://doi.org/10.1111/j.1600-6143.2004.00593.x (2004).

  15. 15.

    Kahaleh, M. et al. Does cholecystectomy protect from recurrent gallstone pancreatitis after biliary sphincterotomy? A prospective study. Gastrointestinal Endoscopy 65, AB223–AB223, https://doi.org/10.1016/j.gie.2007.03.474 (2007).

  16. 16.

    Massie, A. B. et al. Higher Mortality in registrants with sudden model for end-stage liver disease increase: Disadvantaged by the current allocation policy. Liver Transpl 21, 683–689, https://doi.org/10.1002/lt.24102 (2015).

  17. 17.

    Changes to OPTN Bylaws and Policies from actions at OPTN/UNOS Exectuive Committee Meetings July 2015-November 2015, https://optn.transplant.hrsa.gov/media/1575/policynotice_20151101.pdf.

  18. 18.

    Schafer, M., Heinrich, S. & Clavien, P. A. Acute pancreatitis: Evidence-based indications for surgery - The current concept. Vol. 143 (Springer 2005).

  19. 19.

    Cai, D. et al. Identification of disubstituted sulfonamide compounds as specific inhibitors of hepatitis B virus covalently closed circular DNA formation. Antimicrob Agents Chemother 56, 4277–4288, https://doi.org/10.1128/aac.00473-12 (2012).

  20. 20.

    Fishman, J. A. Infection in solid-organ transplant recipients. N Engl J Med 357, 2601–2614, https://doi.org/10.1056/NEJMra064928 (2007).

  21. 21.

    Kim, S. I. Bacterial infection after liver transplantation. World J Gastroenterol 20, 6211–6220, https://doi.org/10.3748/wjg.v20.i20.6211 (2014).

  22. 22.

    Sun, H. Y., Cacciarelli, T. V., Wagener, M. M. & Singh, N. Impact of the duration of posttransplant renal replacement therapy on bacterial infections in liver transplant recipients. Liver Transpl 17, 1212–1217, https://doi.org/10.1002/lt.22373 (2011).

  23. 23.

    Sibulesky, L. et al. Outcomes following liver transplantation in intensive care unit patients. World J Hepatol 5, 26–32, https://doi.org/10.4254/wjh.v5.i1.26 (2013).

  24. 24.

    Bittermann, T., Makar, G. & Goldberg, D. S. Early post-transplant survival: Interaction of MELD score and hospitalization status. J Hepatol 63, 601–608, https://doi.org/10.1016/j.jhep.2015.03.034 (2015).

  25. 25.

    Sato, A. et al. Bundled strategies against infection after liver transplantation: Lessons from multidrug-resistant Pseudomonas aeruginosa. Liver Transpl 22, 436–445, https://doi.org/10.1002/lt.24407 (2016).

Download references

Acknowledgements

This work was supported in part by Health Resources and Services Administration contract 234-2005-37011C. The content is the responsibility of the authors alone and does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government. No financial support was provided for this study.

Author information

G.C. Study concept and design; acquisition of data; analysis and interpretation of data; drafting and critical revision of the manuscript; and study supervision. A.A.L., B.B.D., C.G., D.K., A.E.T., B.J.M., S.K.S. and S.N. Analysis and interpretation of data; and drafting and critical revision of the manuscript. A.A. Study concept and design; analysis and interpretation of data; drafting and critical revision of the manuscript; and study supervision.

Correspondence to Aijaz Ahmed.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Cumulative incidence post-transplant mortality rates among D-MELD cohorts.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.