Pneumococcal Disease and the Effectiveness of the PPV23 Vaccine in Adults: A Two-Stage Bayesian Meta-Analysis of Observational and RCT Reports

The efficacy of PPV-23 vaccine on outcomes of pneumococcal disease in adults still remains controversial due mainly to the lack of consistency between the results obtained from observational studies(OSs) and those obtained from randomized controlled trials(RCTs). As a consequence, the complexity in the structure of evidence available, in turn, generates a challenge for combining disparate pieces of evidence quantitatively. In this regard, we used a hierarchical Bayesian inference-based evidence synthesis of RCTs and observational data using a two-stage approach (in addition to a traditional random-effects meta-analysis) to examine the effectiveness of PPV-23 in adults. To this end, 21 studies were included involving 826109 adult participants. By a two-stage Bayesian meta-analysis, which was directly used for combining studies of different designs, the overall log OR (95% credible interval) for IPDs was −0.1048 (−0.3920,−0.0250), indicating a significant protective effect of the vaccination against IPDs. No significant effect of PPV-23 was found on all-cause pneumonia, pneumococcal pneumonia, and death from pneumonia, which confirmed the results obtained by a traditional method followed by stratified and sensitivity analyses. The estimated overall log OR (95% credible interval) was −0.0002 (−0.0241,0.0142), −0.0002 (−0.0110,0.0122), and −6.3912 × 10−5 (−0.0219,0.0131), respectively. The PPV-23 vaccine might be effective in preventing the most severe invasive forms of pneumococcal diseases, but not effective in preventing other clinical outcomes, in the adult population of 18 years and older.

SCIeNTIFIC RePoRTS | (2018) 8:11051 | DOI: 10.1038/s41598-018-29280-2 mellitus, and asplenia/sickle cell disease. Furthermore, the immunocompromised people including HIV/AIDS or blood borne dyscrasias patients with cochlear implants or chronic cerebrospinal fluid leak, residents of nursing homes, and patients in long-term care facilities are high-risk populations who are recommended to receive the polysaccharide vaccine. However, the protection level, either in the elderly or at high-risk populations, remains a controversial issue 5 .
The 23-valent pneumococcal polysaccharide vaccine (PPV-23) includes 23 capsular serotypes (1, 2, 3, 4, 5, 6B, 7 F, 8, 9 N, 9 V, 10 A, 11 A, 12 F, 14, 15B, 17 F, 18 C, 19 A, 19 F, 20, 22 F, 23 F, and 33 F) which account for 72% 6 to 95% 7 of IPDs, depending on the geographic location 8 . Typically, IPDs affect sterile body sites including cerebrospinal fluid, pleural fluid, and blood stream. This is while vaccination strategies are aimed to give appropriate protection. On the other hand, the adult pneumococcal vaccination policy is very complex and cannot be fixed; it needs to be adaptive and flexible 9,10 . Vaccination with PPV-23 has some limitations, including (1) inability to induce immunological memory at any age-it activates only the T cell-independent antibody responses; (2) being poorly immunogenic in infants; (3) no effect on pneumococcal carriage; (4) decline in the proportion of vaccine-covered serotypes since the introduction of PCVs in children; (5) the unproven efficacy of combination use of PPV-23 and PCV-13; and (6) being costly if all eligible individuals are vaccinated 10,11 .
Although some evidence support protective efficacy of PPVs against IPDs 8,12 , their effectiveness in preventing pneumonias, mortality, and other pneumococcal infections are still controversial 13 . Our knowledge about the efficacy of PPV-23 in reducing the risk of IPDs in immunocompetent older adults comes from a number of case-control and cohort studies [14][15][16][17][18][19][20][21][22][23] . One reason why the efficacy of the vaccine remains controversial may be the lack of consistency between the results obtained from the observational studies (OSs) and those obtained from controlled trials. Empirical studies have shown that insufficient quality of clinical trials including inadequate allocation concealment or failure to blind patients, caregivers or those assessing outcomes, can lead to biases that may threaten the validity of the results and exaggerate treatment effects 24 . If multiple evidence from different sources is combined into a single pooled effect estimate, it may improve the assessment of clinical effectiveness 25 . However, the complexity in the structure of evidence available, in turn, generates a challenge for combining disparate pieces of evidence quantitatively. The Cochrane Collaboration advices to consider randomized controlled trials (RCTs) and OSs separately and not to pool the different study types into a single pooled effect estimate 26 . However, HTA agencies, such as the National Institute for Health and Care Excellence (NICE) in United Kingdom, do not restrict evidence synthesis to RCTs and often demand the identification of all the relevant sources of evidence 27 . Unique statistical approaches to combine randomized and non-randomized studies in clinical research are increasingly being published in recent years 25 . Of these, Bayesian hierarchical models (BHMs) have been the most popular method for combining disparate sources of evidence 25 .
In the present study, first data from all types of study design were combined into a single random-effects meta-analysis, and then stratified the analysis to explore the source of heterogeneity between the studies. Second, a powerful strategy called fully Bayesian hierarchical modelling in two stages 28 was used to incorporate data from various study designs into a single meta-analysis. It contrasts with common BHM where a single step is applied to estimate all parameters simultaneously. Therefore, a full Bayesian analysis was presented at the level of each study and then the results were summarized by the posteriors resulting from Markov Chain Monte Carlo Simulation, MCMC (a powerful method when the study-specific data structures are complex) 28 . The aim was to obtain a more precise estimation of the efficacy of PPV-23 in protecting against different pneumococcal outcomes, including all-cause pneumonia, pneumococcal pneumonia, death from pneumonia, and bacteremia, septicemia or invasive pneumococcal diseases (IPDs) in the adult population aged 18 years and older, and thus enabling clinicians to draw firmer conclusions.

Results
Selected studies. The process of identifying eligible studies is clarified in Fig. 1. In the current study, 128 out of 615 references were considered to be potentially eligible. All the full texts of 128 publications were assessed. Among them, 107 were excluded from the final analysis; 2 described the use of vaccine as only a booster, 9 included simultaneous injection of PPV and PCV vaccines, 11 included simultaneous injection of PPV and influenza vaccines, and 85 only examined the antibody response. Finally, 21 studies were included in the meta-analysis involving 826109 adult participants, of which 6 (28.6%) were RCTs, 10 (47.6%) were cohort studies and 5 (23.8%) were case-control studies (Supplementary Tables S1, S2, and S3).
Among the six RCTs, five (83%) were described as double-blind, but placebo was used as a control. One (17%) was described as open and no intervention was used as a control. In this group, the follow-up time ranged from 1 to 6.7 years (mean 4.1 years); one trial (17%) did not provide information on the duration of follow-up. Two trials (33.3%) were performed in Uganda-a low income country-, one (16.7%) in Spain, one (16.7%) in Japan, one (16.7%) in Sweden and one (16.7%) in America. The study characteristics are shown in Supplementary Table S1. In the cohort group, the follow-up time ranged from 3 to 10 years (mean 6 years); one study (10%) did not provide information on the duration of the follow-up. All studies in this group were performed in high income countries; six (60%) in Spain, two (20%) in Canada, one (10%) in Sweden, and one (10%) in China, Taiwan (Province of China). Supplementary Table S2 determines the features of cohort studies which were included in the meta-analyses.
In the case-control group, two studies (40%) were performed in the US, two (40%) in Spain, and one (20%) in Brazil. Supplementary Table S3 shows the characteristics of case-control studies that were included in the meta-analyses.

Meta-analyses.
Depending on the outcomes, 3 to 6 RCTs and 3 to 13 cohort/case-control studies were included in the meta-analyses. Each analysis included 2293 to 156812 precipitants for RCTs, and 173532 to 655106 precipitants for cohort/case-control studies. The effect of the PPV-23 vaccine on the relative risks of clinical outcomes according to the random-effects meta-analysis of combined study designs (RCTs and observational studies).
The pooled RR (95% CI) for all-cause pneumonia was 0.870 (0.692-1.092; P = 0.230). The quantity I 2 of 74.335% (P = 0) was obtained, indicating that the degree of variability between studies was inconsistent with what would be expected to occur by chance alone. No evidence of publication bias was found; the P-value for Egger's test was 0.972 ( Table 1).
The pooled RR (95% CI) for death from pneumonia was 0.538 (0.162-1.790; P = 0.312). The Cochran's Q test had a P-value of 0, which corresponded to the quantity I 2 of 99.185%, indicating strong evidence of heterogeneity between studies. No evidence of publication bias was found; the P-value for Egger's test was 0.564 (Table 1). When the two studies of Imaz et al. and Ortqvist et al. 33,34 with the largest variances ( Fig. 2) were excluded, the results did not change substantially (RR = 0.567, 95% CI; 0.151-2.135, P = 0.402). However, there was still strong evidence of heterogeneity (Cochran's Q test P = 0, I 2 = 99.389%) but no publication bias was found (Egger's P = 0.581).
In the analysis for pneumococcal pneumonia, after removing Alfageme et al. 's study 29 which had the largest variance, the RR altered from 0.837 to 0.908 (95% CI; 0.680-1.214; P = 0.515), indicating the high stability of the results. However, there was still strong evidence of heterogeneity between studies (Cochran's Q test P = 0.012, The effect of the PPV-23 vaccine on the relative risks of clinical outcomes according to the random-effects meta-analysis of the cohort/case-control studies. Among the cohort/case-control studies that compared the 23-valent pneumococcal vaccinated with unvaccinated events, the pooled RR from a random-effects meta-analysis indicated a significant effect of PPV-23 on all-cause pneumonia (RR = 0.598, 95% CI; 0.386-0.927; P = 0.022), with no evidence of heterogeneity between studies (Cochran's Q test P > 0.1, I 2 = 54.397%). The P-value for the Begg's test was 0.296, while Egger's test had a P-value of 0.033, suggesting a possible bias (Table 4). After removing Ochoa-Gondar et al. 's study 36 using a leave-one-out sensitivity analysis, the RR altered from 0.598 to 0.445 (95% CI; 0.270-0.733, P = 0.001).
Estimation of overall log odds ratios of the clinical outcomes by combining different study types in a two-stage Bayesian hierarchical meta-analysis. The overall results were obtained as the median and 2.5th and 97.5th percentiles of the posterior distribution of the parameter mu. We did a random-effects meta-analysis where mu was the mean of the distribution of study treatment effects (the tau_i), which was assumed normally distributed. Estimates were expressed as posterior medians and 95% credible intervals for logarithm of odds ratio. No significant protective effect of the PPV-23 vaccine was found on all-cause pneumonia, where the estimated overall log OR (95% credible interval) was −0.0002 (−0.0241, 0.0142). The vaccination had a negative effect in the Ortqvist et al. 's study. In the other studies, the vaccination did not have any effect on the outcome ( Table 5).
The overall log OR (95% credible interval) for pneumococcal pneumonia was −0.0002 (−0.0110, 0.0122), indicating no significant effect of the vaccination on the outcome. The vaccination had a significant effect in Maruyama et al. 's study. In the other studies, the vaccination did not have any effect on the outcome (  Table 1. Relative risks of all-cause pneumonia, pneumococcal pneumonia, death from pneumonia, and invasive pneumococcal diseases according to the combination of different study types (RCTs and observational studies) of the 23-valent pneumococcal polysaccharide vaccine in a random-effects meta-analysis.
No significant protective effect of the PPV-23 vaccine was found for death from pneumonia, where the estimated overall log OR (95% credible interval) was −6.3912 × 10 −5 (−0.0219, 0.0131). The vaccination had a negative effect in the Johnstone et al.'s study. In the other studies, the vaccination did not have any effect on the outcome (Table 5). In the other studies, the vaccination did not have any effect on the outcome (Table 5).

Discussion
There is a serious debate concerning the efficacy of the PPV-23 vaccine on clinical outcomes in adults. In this regard, a detailed meta-analysis was conducted including 826109 adult participants distributed into 21 studies with different types of study designs. Each study was involved according to the predefined eligibility criteria. In order to avoid manipulation of these criteria, we did not straightforwardly exclude outliers based on a statistical test of homogeneity. The analyses of either the combined study designs (RCTs and OSs) or observational studies showed that PPV-23 significantly reduced the incidence of IPDs, but not pneumococcal pneumonia and death from pneumonia. The RR ranged from 0.581 to 0.738. The only meta-analysis of cohort studies showed a significant protective effect of PPV-23 against all-cause pneumonia (RR = 0.598). No significant effect of PPV-23 on any of the clinical outcomes was found in the RCTs. When the analysis was restricted to high income countries, for all-cause pneumonia and pneumococcal pneumonia, although the RRs altered, the difference failed to reach significance (RR = 0.852, 95% CI; 0.644-1.128 for ACP and RR = 0.540, 95% CI; 0.178-1.641 for PP). What was remarkable in most analyses, whether all studies were combined or stratified by study designs, was the high levels of heterogeneity, ranged from 57.295% to 99.490%, that might undermine the credibility of the results. Although I 2 does not depend on the number of studies or the effect measure used 37 , it depends on the size of the individual studies in the meta-analysis 38 . Given the large sample size of the individual studies (i.e., the within-study variation is small) I 2 could then be large (like >75%), even if the between-study variation is small. This was therefore a problem particularly when the data of cohort studies were being analysed (having typically large sample sizes). Though any amount of heterogeneity was acceptable as the predefined eligibility criteria were sound and met 39 Table 3. Relative risks of all-cause pneumonia, pneumococcal pneumonia, death from pneumonia, and invasive pneumococcal diseases according to the random-effects meta-analysis of the RCTs. we used a random-effects model, assuming a particular form to heterogeneity (often, but not necessarily, a normal distribution), to overcome it. In addition, controlling the outliers on the forest plot and performing sensitivity analyses 41 which indicated the stability of the pooled RR estimates and a close correlation between I 2 and τ 2 (among-study variance)-that means excluding studies to reduce I 2 resulted in reducing τ 2 39,40 -reaffirmed the accuracy and power of the results. The next problem was dealing with the positive publication bias, which is often tricky 42,43 . The analysis of the PPV-23 vaccine effectiveness on all-cause pneumonia among cohort studies revealed potential sources of bias (P < 0.1 for Egger's test). It can be misleading if we ignore between-study heterogeneity when assessing publication bias 43 . As the meta-analysis was not large (i.e., the number of studies was small (<10) for all-cause pneumonia), methods to test or adjust for publication bias in the presence of heterogeneity might not be powerful. In contrast, it might be a serious threat to the validity of meta-analysis for IPDs (P = 0.028 for Egger's test) as the analysis was done on 13 eligible cohort/case-control studies. However, after stratification by the study design, the potential bias was no longer statistically significant (Egger's P = 0.624 and 0.488 for case-control and cohort studies, respectively) and was effectively ruled out, further strengthening the validity and robustness of the results. Supplementary Tablesncies between the meta-analyses of RCTs and observational studies, largely influenced by the study design. The meta-analyses of RCTs failed to identify a significant overall protective effect of PPV-23 on clinical outcomes, whereas the others (observational studies) indicated a protective effect of PPV-23 against all-cause pneumonia and IPDs. Both RCT and observational study estimations of the PPV-23 vaccine effectiveness are significantly affected by the quality of the study designs and both classes of the study have strengths and weaknesses 44 . Residual confounding, which is a fundamental criticism of observational studies, is always a potential source of discrepancy between RCT and observational studies. Although randomization removes the chances of confounding, there are other biases inherent to RCTs, and the external validity of RCTs may be also limited 45 . In the present meta-analysis, however, the 5 out of 6 RCTs were described as double-blinding. This might be the most acceptable method to minimize biases caused by the placebo effect. However, it could not always provide us with the unbiased estimations of the effect. High quality scores (3-5 points on the Jadad scale) were assigned for all the included RCTs (Supplementary Table S5). Nine cohort studies had high quality scores (7 or 8 points) on the 9-star NOS, and one had a low score (6 points) (Supplementary Table S6). High quality scores (8 or 9 points on the 9-star NOS) were assigned for all the included case-control studies (Supplementary Table S7).

Random-effects model
Most of the previous studies did not evaluate PPV-23 independently; they considered all valencies and were inconsistent due to several methodological flaws 8,46,47 . We not only evaluated PPV-23 independently, but also excluded studies which included simultaneous injection of PPV and PCV or PPV and IVs, and those in which the other valencies of the PPV vaccine or any other specific vaccine (i.e., PCV or IVs) were used as a control. Otherwise, it could influence the net impact of the 23-valent PPV vaccine on different outcomes and the assumption of the synergistic and additive effects of two vaccines on the outcomes could not be ruled out. However, Diao et al. 's study 48 included different interventions in a final meta-analysis of RCTs, which caused the real effectiveness of the 23-valent PPV vaccine to be controversial.
The potential adverse effects of serotype replacement are still a big challenge when implementing routine childhood PCV programs. In particular, where the lost vaccine serotype carriage is almost consistently replaced by non-vaccine serotype carriage, the net effectiveness of a vaccine is less than expected, in general. Moreover, the degree of replacement in disease, which is resulted from replacement in carriage, depends on the invasive potential of the serotypes involved 49,50 . In the present study, based on the pre-defined exclusion criteria, a limited  Table 4. Relative risks of all-cause pneumonia, pneumococcal pneumonia, death from pneumonia, and invasive pneumococcal diseases according to the random-effects meta-analysis of the cohort/case-control studies.
SCIeNTIFIC  Table 5. Estimation of overall log odds ratios on all cause pneumonia, pneumococcal pneumonia, death from pneumonia, and invasive pneumococcal disease by combining different study types into a two-stage Bayesian hierarchical meta-analysis. a Estimates were expressed as posterior medians and 95% credible intervals for logarithm of odds ratio. effectiveness on CAP and IPDs were conducted following PCV-7-or PCV-13-based childhood programs that have decreased the prevalence of PCV-7-or PCV-13-type pneumococcal serotypes in adults, respectively 51 . By the use of two-stage Bayesian hierarchical meta-analysis, which was directly used for combining studies of different designs, the overall log OR (95% credible interval) for IPDs was −0.1048 (−0.3920, −0.0250), indicating a significant protective effect of the vaccination against IPDs. No significant effect of PPV-23 was found on all-cause pneumonia, pneumococcal pneumonia, and death from pneumonia, which confirmed the results obtained by a traditional method. The estimated overall log OR values (95% credible interval) were −0.0002 (−0.0241, 0.0142), −0.0002 (−0.0110, 0.0122), and −6.3912 × 10 −5 (−0.0219, 0.0131), respectively. As mentioned by Lunn et al. 28 , two limitations exist in the two-stage Bayesian hierarchical modelling employed in this article. One is the assumption of marginal independence between studies in stage 1 and another is dealing with sparse data. It is worthwhile to note that both of these limitations did not occur here since our Metropolis sampler was non-degenerate in stage 1 and the data was not sparse.
In conclusion, the PPV-23 vaccine might be effective in preventing the most severe invasive forms of pneumococcal diseases, but not effective in preventing other clinical outcomes, in the adult population of 18 years and older. It was validated by a fully Bayesian hierarchical modelling in two stages, which provides a high-performance approach to more complex evidence syntheses, such as multi-parameter evidence synthesis or mixed treatment comparisons. Although the meta-analysis of cohort studies, but not RCTs, indicated a significant protective effect against all-cause pneumonia, the number was not large and thus the results should be interpreted with caution. The high-quality large RCTs are required to more confidently validate the efficacy of the PPV-23 vaccine in protecting against all-cause pneumonia.
Determining the most common circulating pneumococcal serotypes in the community, designing more RCT and analytical studies to evaluate the effectiveness of pneumococcal vaccines, and finally evaluating the effectiveness of PCVs administration along with PPV23 can help to effectively measure the protective effect of PPV23. Regarding the cost benefits of vaccination against IPD compared to the antibiotic therapy strategies, and due to the increasing rate of IPD worldwide, the vaccination of the elderly is recommended.

Data sources. We searched MEDLINE, EMBASE, LILACS (Latin American and Caribbean Health Sciences
Literature), AIM (African Index Medicus), and IndMed (Indian Medlars Centre) databases by using the keywords "23-valent pneumococcal vaccine", "PPSV-23", or "PPV-23" in combination with "outcomes", "clinical outcomes", or "clinical effectiveness". These keywords were combined with terms describing the study design; "randomized-controlled trial", "controlled clinical trial", "clinical trial", "cohort study", and "case-control study". We attempted to include all randomized comparisons of PPV-23 with a placebo in any type of population.
Inclusion and exclusion criteria. We included RCTs that compared the PPV-23 vaccine with a placebo or no intervention, and cohort studies and case-control studies that compared outcomes between pneumococcal polysaccharide vaccinated and unvaccinated groups. Only English language articles were applied in this study. We excluded uncontrolled and observational intervention studies; animal and laboratory studies; studies examining antibody responses only; studies in which the PPV-23 vaccine was used as a booster after vaccination with conjugate pneumococcal vaccine; studies in which the other valencies of the PPV vaccine or any other specific vaccine (i.e., PCV or influenza vaccines & IVs) were used as a control; studies that included simultaneous injection of PPV and PCV vaccines, and those including the simultaneous injection of PPV and IVs; studies performed in child populations; the results coming from abstracts only; and review articles. Data from each potentially relevant article were extracted independently by two investigators (H.L.-N. and S.L.-N.) and discussed to solve any disagreement. Moreover, the information on the methodologic quality of the studies was extracted. The observed interrater agreement was measured using the kappa statistic. Finally, 21 original papers were left for analyses, which were divided into three groups according to their study design: RCTs 29,34,35,[52][53][54] , cohort studies 20,30-32,36,55-59 , and case-control studies 14,15,33,60,61 (Supplementary Tables S1, S2 and S3; Fig. 1). More detailed information about how pneumonia is diagnosed (for example by clinical features such as cough, fever, and pleuritic chest pain, or by chest X-ray or another method) and how pneumococcal pneumonia is confirmed (for example by cultivation or PCR-based methods on different clinical samples such as bronchoalveolar lavage, naso-pharyngeal-throat swabs and nasal swabs) is summarized in Supplementary Table S8. Quality assessment of RCTs and observational studies was determined on the basis of Jadad 62 and Newcastle-Ottawa (NOS) 63 quality assessment scales, respectively.
Meta-analysis Outcomes. The following four outcomes were considered: (a) all-cause pneumonia; (b) pneumococcal pneumonia; (c) death from pneumonia; and (d) bacteremia, septicemia or invasive pneumococcal diseases (defined as isolation of S. pneumoniae from sterile body fluids).
Classical meta-analysis method. Combined relative risk (RR) and 95% confidence intervals (CIs) were estimated for each study. The significance of pooled RR was tested by Z-test, and a P-value less than 0.05 was considered as statistically significant. The Cochran's Q statistic was used to test heterogeneity among the included studies. The DerSimonian and Laird random-effects 64 model was used to conduct the meta-analyses. Besides, we examined the I 2 (I-squared) statistic describing the amount of variations due to true differences (heterogeneity) rather than random errors. The I 2 values of 25%, 50%, and 75% corresponded to the low, moderate, and high levels of heterogeneity 40 , respectively. To explore the source of heterogeneity, we performed subgroup analyses according to the types of study design. Furthermore, the studies which had both the largest variance (wide intervals) and the extreme outlier weight in each clinical outcome group were identified. Then, a leave-one-out sensitivity analysis was conducted to assess the impact of individual studies, and thus the average RR was estimated in the absence of each study and heterogeneity was quantified using both the I 2 and τ 2 statistics 40 . The Egger's regression asymmetry test and the Begg-Mazumdar adjusted rank correlation test were used to statistically assess the potential publication bias 65,66 . P-values less than 0.10 indicated the presence of bias. All the analyses were performed using a Comprehensive Meta-Analysis software (version 2, Biostat, Englewood, NJ), followed by PRISMA guidelines 67 , and 2-tailed P-values were calculated in all the tests.

Specifications for a two-stage Bayesian hierarchical model. Let y C i
and y T i denote the number of control and treatment groups of study i, respectively. In addition, let C i π and T i π represent the corresponding underlying probabilities (the probability of observing the disease again) of these groups. The total number of individuals in these groups are n C i and n T i . We have π π = … . y Binomial n y Binomial n i N ( , ), ( , ) 1, , where ξ s i are nuisance parameters and θ i is the treatment effect for study i. Note that θ i is equal to logarithm of the odds ratio for treatment compared with control. So in our cases, the negative values of θ i represent the positive effect of the vaccination while the zero values indicate no effect.
We used the two-stage Bayesian hierarchical modeling proposed in Lunn et al. 28 . In the first stage, it is assumed that i ξ s are independent from Normal(0, 100 ) 2 and i θ s are independently distributed as Normal(0, 100 ) 2 . Also, in the second stage the underlying distributions arẽ Therefore the conditional distribution of 2 σ given ( y , , ) θ µ is a truncated gamma distribution on the interval (0, 100).