A Bayesian network analysis quantifying risks versus benefits of the Pfizer COVID-19 vaccine in Australia

Sinclair, Jane E.; Mayfield, Helen J.; Short, Kirsty R.; Brown, Samuel J.; Puranik, Rajesh; Mengersen, Kerrie; Litt, John C. B.; Lau, Colleen L.

doi:10.1038/s41541-022-00517-6

Download PDF

Article
Open access
Published: 11 August 2022

A Bayesian network analysis quantifying risks versus benefits of the Pfizer COVID-19 vaccine in Australia

npj Vaccines volume 7, Article number: 93 (2022) Cite this article

3591 Accesses
6 Citations
38 Altmetric
Metrics details

Subjects

Abstract

The Pfizer COVID-19 vaccine is associated with increased myocarditis incidence. Constantly evolving evidence regarding incidence and case fatality of COVID-19 and myocarditis related to infection or vaccination, creates challenges for risk-benefit analysis of vaccination. Challenges are complicated further by emerging evidence of waning vaccine effectiveness, and variable effectiveness against variants. Here, we build on previous work on the COVID-19 Risk Calculator (CoRiCal) by integrating Australian and international data to inform a Bayesian network that calculates probabilities of outcomes for the delta variant under different scenarios of Pfizer COVID-19 vaccine coverage, age groups (≥12 years), sex, community transmission intensity and vaccine effectiveness. The model estimates that in a population where 5% were unvaccinated, 5% had one dose, 60% had two doses and 30% had three doses, there was a substantially greater probability of developing (239–5847 times) and dying (1430–384,684 times) from COVID-19-related than vaccine-associated myocarditis (depending on age and sex). For one million people with this vaccine coverage, where transmission intensity was equivalent to 10% chance of infection over 2 months, 68,813 symptomatic COVID-19 cases and 981 deaths would be prevented, with 42 and 16 expected cases of vaccine-associated myocarditis in males and females, respectively. These results justify vaccination in all age groups as vaccine-associated myocarditis is generally mild in the young, and there is unequivocal evidence for reduced mortality from COVID-19 in older individuals. The model may be updated to include emerging best evidence, data pertinent to different countries or vaccines and other outcomes such as long COVID.

Age and sex-specific risks of myocarditis and pericarditis following Covid-19 messenger RNA vaccines

Article Open access 25 June 2022

Risks of myocarditis, pericarditis, and cardiac arrhythmias associated with COVID-19 vaccination or SARS-CoV-2 infection

Article Open access 14 December 2021

Systematic review and meta-analysis of myocarditis and pericarditis in adolescents following COVID-19 BNT162b2 vaccination

Article Open access 09 June 2023

Introduction

In December 2020, the Pfizer vaccine (BNT162b2; Cormirnaty) became the first COVID-19 vaccine to be authorised for public use¹, and has since had >1.5 billion doses delivered to 131 countries^2,3. In June 2021, reports linking the Pfizer vaccine to myocarditis, especially in male adolescents and young adults, started to emerge in Israel⁴ and the USA⁵. Despite low case numbers, this association informed government policies surrounding a slower vaccine rollout in younger age groups around the world⁶. Furthermore, intense media focus on this uncommon adverse event may have contributed to an increase in vaccine hesitancy in younger age groups⁷, especially in Australia where it was the only COVID-19 vaccine recommended for those aged under 60 years at the time⁸.

Having access to transparent information on the risks and benefits based on the current best available evidence is crucial for individuals to make an informed decision on whether to get vaccinated^9,10, and also for informing public health policy regarding the recommendation of different vaccine types for different population subgroups. The Australian Technical Advisory Group on Immunisation (ATAGI) produced a helpful document on ‘Weighing up the potential benefits against risk of harm from COVID-19 vaccine AstraZeneca’¹¹ to address concerns of vaccine-associated thrombosis with thrombocytopenia syndrome. While ATAGI released a clinical ‘Guidance on myocarditis and pericarditis after mRNA COVID-19 vaccines’¹², there have not been any documents focused on risk-benefit analysis.

By October 2021, 23.4% and 55.1% of Australians aged over 16 years had received one and two doses of a COVID-19 vaccine, respectively, and an unspecified but small percentage had received a third dose¹³. Because of concerns related to the risk of thrombosis and thrombocytopenia syndrome with the AstraZeneca COVID-19 vaccine, the Pfizer vaccine was the standard recommendation for those aged < 60 years¹⁴. However, 6-month Pfizer vaccine effectiveness data that became available in October 2021 showed concerning reductions in protection against symptomatic infection each month after administration of the second dose¹⁵. In the context of the reopening of Australian borders in December 2021 and the introduction of the highly transmissible omicron variant, this decrease in vaccine effectiveness may leave even those who have had two doses of a COVID-19 vaccine at substantial risk of developing symptomatic COVID-19. Even for the highly vaccinated population of Australia, it was therefore crucial to communicate the necessity of third doses for maintaining optimal protection against symptomatic infection, serious illness and death.

To effectively facilitate this communication, a risk-benefit analysis tool capable of integrating best evidence from multiple data sources (both Australian and international) and formats (government reports, published literature and expert opinion) was required¹⁶. Furthermore, this tool must be easy to update as the pandemic landscape rapidly evolves and as more data become available. We have previously developed a Bayesian network (BN) model to analyse the risks and benefits of the COVID-19 AstraZeneca vaccine in the Australian population^17,18. This model was used to programme the COVID-19 Risk Calculator (CoRiCal)¹⁹, a freely available user-friendly online tool that enables scenario analysis based on user inputs (age, sex, vaccination status, transmission scenario). The tool provides risk estimates for targeted subgroups and can be used by health managers as well as individuals alone or in conjunction with their general practitioner for shared decision-making on vaccination. This study describes the development of the BN model used to programme the second version of the CoRiCal tool, which focuses on the risks and benefits of the COVID-19 Pfizer vaccine for the Australian context, and details the results of the population-level risk-benefit analysis performed using this model, factoring in emerging evidence of waning vaccine effectiveness over time and the new recommendation for third doses.

Results

Model description

The BN model was designed to predict five outcomes:

i.
Probability of developing and dying from Pfizer vaccine-associated myocarditis (n5, n12)—depending on vaccine dose (n1), age (n2) and sex (n3);
ii.
Background probability of developing and dying from myocarditis (in those who have not had Pfizer vaccine or COVID-19) (n6, n13). Estimates were converted to probability of events over 2 months to enable comparison with the probability of vaccine-associated (n5, n12) and infection-associated outcomes (n10, n14) over 2-month periods;
iii.
Probability of symptomatic COVID-19 (n10)—depending on intensity of community transmission (n4), vaccine effectiveness against symptomatic infection (n7), relative risk of symptomatic infection by age and sex (n9);
iv.
Probability of dying from COVID-19 (n14)—depending on age (n2), sex (n3), vaccine effectiveness against death (n8); and
v.
Probability of developing and dying from COVID-19-related myocarditis (n11, n15)—depending on age (n2), sex (n3).

The BN (Fig. 1) displays the links between variables and outcomes based on the assumptions presented in Table 1^{11,15,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35} and Supplementary Tables S1–9. Supplementary Table 10 summarises each of the 15 nodes and their parent/child associations. While some links are largely association-driven (e.g., the links between age and sex and relative risk of symptomatic infection), others are more causal (e.g., the link between developing COVID-19-related myocarditis and dying from it).

**Fig. 1: Bayesian network for assessing risks versus benefits of the Pfizer COVID-19 vaccine in Australia.**

Table 1 Summary of data sources, assumptions and prior distributions for a Bayesian network to assess risks versus benefits of the Pfizer COVID-19 vaccine.

Full size table

The BN includes four input nodes (orange) for use in scenario analyses: Pfizer vaccine dose and time since second dose (n1), age (n2), sex (n3) and intensity of community transmission (n4). Community transmission scenarios were presented as probability of infection over 2 months to enable comparison of vaccination risks versus benefits, as vaccine effectiveness is expected to decrease over time (modelled using 2-month intervals for time since second dose). Transmission scenarios were based on ATAGI definitions of low/medium/high risk¹¹ (equivalent to 0.016%, 0.149%, 1.920% chance of infection over 2 months), and 1%, 2%, 5% and 10% chance of infection over 2 months. The model contains six intermediate nodes (yellow): Pfizer vaccine-associated myocarditis (n5), background incidence of myocarditis (n6), vaccine effectiveness (n7, n8), relative risk of symptomatic infection based on age and sex (n9), and incidence of COVID-19-related myocarditis (n11).

Two model versions were constructed employing distinct definitions of the ‘Pfizer vaccine dose and time since dose 2’ node (n1):

Version 1: Pfizer vaccine doses defined as no doses, first dose, second dose and third dose. This version allows estimation of the probability of vaccine-associated myocarditis with each dose of vaccine, and was used in the coding of the CoRiCal online tool for providing individualised risk estimates.
Version 2: Pfizer vaccine doses defined as no doses, received only one dose, received two doses and received three doses. This version allows estimation of the probability of deaths in the target population based on vaccine coverage rates, and is the model used henceforth for public health risk-benefit analyses.

Model validation

All authors agreed that the final model accurately represented the variables, their states and associations within the model’s scope, in a manner consistent with the best current evidence. Manual calculations of risk estimates for multiple scenarios performed using the data sources and pre-defined assumptions matched the model outputs, validating the BN’s predictive behaviour (Supplementary Table 11).

Risk-benefit analysis

The first risk-benefit analysis estimated the risks of background myocarditis, Pfizer vaccine-associated myocarditis and myocarditis in patients with symptomatic COVID-19. Based on background rates of myocarditis reported by Li et al.²⁷ and Barda et al.²⁸, our model estimated 2-month incidence of 10.0 (females aged 12–19 years) to 53.9 (males aged ≥70 years) cases per million, and overall case fatality rate (CFR) ranging from 1.2% to 4.3% for different age-sex subgroups (Supplementary Table 6).

Up to 09/12/2021 in Australia, age-sex-specific incidence of Pfizer vaccine-associated myocarditis cases ranged from zero to 24 per million after the first dose, and zero to 103 per million after the second dose (Supplementary Table 7), with no reported deaths. Our model assumed an overall CFR of 0.17% (two deaths out of 1195 cases) based on reports from the Centers for Disease Control and Prevention Vaccine Adverse Event Reporting System in the USA³³ (Table 1).

At the time of writing, Australian data on myocarditis in COVID-19 patients were limited (Table 1). Model assumptions on the incidence and CFR of myocarditis in COVID-19 patients were obtained from an international cohort study by Buckley et al.³⁴, and additional unpublished age-sex specific data from the study via personal communication with the lead author. Data showed incidence ranging from 1.66% to 13.74%, and CFR ranging from < 1% to 15.14%, depending on age and sex (Supplementary Table 8). Based on estimates from model version 2, Fig. 2 shows that, in a population aged ≥12 years, with vaccine coverage of 5% unvaccinated, 5% had one dose, 60% had two doses and 30% had three doses, the probability of developing myocarditis related to symptomatic COVID-19 once already infected with SARS-CoV-2 was 471 to 5847 times higher than developing Pfizer vaccine-associated myocarditis, depending on age group and sex (Fig. 2a, dashed lines). The probability of dying from myocarditis related to symptomatic COVID-19 was 1430 to 384,684 times higher than dying from vaccine-associated myocarditis, again depending on age group and sex (Fig. 2a, solid lines).

**Fig. 2: Comparison of the estimated risks of developing and dying from Pfizer vaccine-associated or COVID-19-related myocarditis.**

The second risk-benefit analysis performed estimated symptomatic COVID-19 cases and deaths prevented. Model version 2 was used to calculate expected symptomatic COVID-19 cases and deaths prevented over 2 months per million population aged ≥12 years, where 5% were unvaccinated, 5% had one dose, 60% had two doses (20% each with the last dose administered 0 to < 2, 2 to < 4 and 4 to < 6 months ago) and 30% had three doses. Figure 3a, b show the expected cases and deaths, respectively, prevented by age group under different community transmission intensities:

1% chance of symptomatic infection over 2 months (green), equivalent to average of 3645 cases per day in Australia (Supplementary Table 4);
5% chance of symptomatic infection over 2 months (yellow), equivalent to average of 7290 cases per day in Australia; and
10% chance of symptomatic infection over 2 months (orange), equivalent to average of 18,225 cases per day in Australia.

**Fig. 3: Comparison of estimated Pfizer vaccine-associated myocarditis cases to symptomatic COVID-19 cases and deaths prevented.**

The model estimates that for a million 12–19 year-olds with this vaccine coverage, 11,029 symptomatic COVID-19 cases and one death would be expected to be prevented under 1% transmission (green) versus 110,288 cases and 11 deaths prevented under 10% transmission (orange) (Fig. 3a, b), with 146 expected cases of Pfizer vaccine-associated myocarditis in males and 35 cases in females (Fig. 3c). In contrast, for a million people aged ≥70 years, 2757 cases and 98 deaths would be expected to be prevented under the 1% transmission scenario, 27,566 cases and 981 deaths prevented under the 10% transmission scenario, with less than five expected vaccine-associated myocarditis cases in males or females. Calculations are detailed in Supplementary Table 12.

The third risk-benefit analysis estimated symptomatic COVID-19 cases and deaths under different vaccination coverage scenarios. Model version 2 was further used to estimate expected symptomatic COVID-19 cases and deaths per million people if transmission intensity was equivalent to a 10% chance of infection over 2 months, if 5% were unvaccinated, 5% had one dose, 60% had two doses and 30% had three doses (scenario one) (Fig. 4, orange), versus if 0% of the population received no doses, 5% received the first dose only, 15% had two doses (5% each with the second dose administered 0 to < 2, 2 to < 4 and 4 to < 6 months ago), and 80% had three doses (scenario two) (Fig. 4, blue).

**Fig. 4: Estimated symptomatic COVID-19 cases and deaths under different vaccination coverage scenarios.**

The model shows that for a million people aged 12–19 years with the vaccine coverage described in scenario one, 27,391 symptomatic COVID-19 cases and less than one death from COVID-19 would be expected under 10% transmission over 2 months, versus 11,042 cases and less than one death in scenario two. For one million people aged 20–29 years, 36,249 cases and two deaths could be expected in scenario one versus 13,168 cases and less than one death under scenario two. In contrast, for a million people aged ≥70 years, 12,694 cases and 404 deaths would be expected in scenario one versus 5487 cases and 68 deaths under scenario two.

Sensitivity analysis

The first sensitivity analysis investigated the incidence of Pfizer vaccine-associated myocarditis. Therapeutic Goods Administration (TGA) reports between 14/10/2021 and 09/12/2021³⁶ presented slight fluctuations in Pfizer vaccine-associated myocarditis incidence in Australia ranging from two to 37 cases per million depending on age-sex subgroup (Table 2). These small changes exerted no substantive impact on population-level estimates of the number of deaths. Model calculations also showed expected Pfizer vaccine-associated myocarditis deaths per million second doses to change only slightly during this time; differences ranged from 0.000 to 0.063 deaths per million by age-sex subgroup when comparing data from 14/10/2021 and 09/12/2021.

Table 2 Evolving evidence on incidence of Pfizer vaccine-associated myocarditis by age and sex in Australia in October–December 2021.

Full size table

The second sensitivity analysis focused on vaccine effectiveness against developing and dying from symptomatic COVID-19. The model calculated that in a population where 5% are unvaccinated, 5% had one dose, 60% had two doses and 30% had three doses, a hypothetical 5% or 10% decrease in vaccine effectiveness against the delta variant would result in a 17.8% or 35.7% increase in estimated symptomatic cases, respectively, and a 23.9% or 54.7% increase in estimated expected deaths, respectively (Table 3). Thus, model estimates of cases and deaths are highly sensitive to reductions in vaccine effectiveness, necessitating frequent monitoring of and updating with emerging vaccine effectiveness data, particularly against new variants.

Table 3 Impact of theoretical reduction in vaccine effectiveness against delta variant on estimated deaths, assuming 5% of population of ages ≥12 years is unvaccinated, 5% had one dose, 60% had two doses and 30% had three doses.

Full size table

Discussion

We developed a BN model to facilitate risk-benefit analysis of the Pfizer COVID-19 vaccine for the Australian population. Results from this model highlight the importance of both individual factors such as age, sex and vaccination status, and location-specific factors that reflect the current pandemic landscape, such as transmission intensity, case incidence and CFR from COVID-19, and COVID-19- and Pfizer vaccine-associated myocarditis. Our model could be used to help inform discussions and decision-making for population health managers, individuals and clinicians. In this way, the model may aid in policy development, public health management, increased public awareness and improved shared decision-making in medical consultations.

For Australians ≥12 years, we compared the risk of developing Pfizer vaccine-associated myocarditis, with the benefit of protection against developing and dying from symptomatic COVID-19 over 2 months under different transmission scenarios, if 5% were unvaccinated, 5% had a first dose 3 weeks ago, 60% had two doses (20% each with the last dose administered 0 to < 2, 2 to < 4 and 4 to < 6 months ago), and 30% had three doses with the last dose administered < 4 months ago. Overall, an Australian is 471 to 5847 times more likely to develop COVID-19-related myocarditis if infected with SARS-CoV-2 than vaccine-associated myocarditis, and 1430 to 384,684 times more likely to die from it, depending on age and sex (Fig. 2a). Under a transmission scenario where the chance of infection was 5%–10% over 2 months, an Australian who is not already infected with SARS-CoV-2 is 5–88 times more likely to develop myocarditis from COVID-19 than the Pfizer vaccine, and 13–2466 times more likely to die from it (Fig. 2c, d). Under a lower transmission scenario where the chance of infection was 1% over 2 months, all age-sex subgroups except for 20–29-year-old males are more likely to develop myocarditis from COVID-19 than the Pfizer vaccine, and all age-sex subgroups are more likely to die from it (Fig. 2b), an important finding for informing individual decision-making on Pfizer COVID-19 vaccination. Crucial also for informing public health decision-making, modelling showed that under any transmission level, younger age groups benefited the most from protection against symptomatic COVID-19 while older age groups benefited the most from protection against fatal COVID-19 (Fig. 3), both critical markers of vaccine effectiveness^37,38,39. Younger age groups were at higher risk of developing vaccine-associated myocarditis than older groups, and males were at greater risk than females. We note that myocarditis was more common after COVID-19 compared to the background rates, especially in younger men. In comparison, vaccine-associated myocarditis also has a predilection for younger males but at a much lower prevalence than cases associated with symptomatic COVID-19. Importantly, in the main, vaccination is justified in all age groups because myocarditis is generally mild in the young^40,41,42, and there is unequivocal evidence for reduced mortality in older individuals across all levels of community transmission.

While the above risk-benefit analyses were conducted assuming the Australian vaccine coverage at the time of writing, outcomes under other coverage rates can be assessed by the model to further inform public health decision-making. We compared the number of COVID-19 cases and deaths expected if the chance of infection was 10% over 2 months under a scenario where 5% are unvaccinated, 5% had a first dose 3 weeks ago, 60% had two doses (20% each with the last dose administered 0 to < 2, 2 to < 4 and 4 to < 6 months ago) and 30% had three doses with the last dose administered < 4 months ago, to those expected under a second scenario where 0% are unvaccinated, 5% had a first dose, 15% had two doses (5% each with the last dose administered 0 to < 2, 2 to < 4 and 4 to < 6 months ago) and 80% had three doses with the last dose administered < 4 months ago (Fig. 4). Younger age groups benefited from the steepest decline in expected case rates, with at least 23,000 fewer cases per million in 20–29 year olds. In contrast, older age groups benefited from the greatest decrease in expected deaths from COVID-19, with 337 fewer deaths per million expected in those aged ≥70 years.

The evidence on the risks versus benefits of COVID-19 vaccination was derived from an range of sources, each associated with some level of uncertainty. The use of probabilistic modelling techniques, such as BNs, can contribute somewhat to communicating this uncertainty by framing the outcomes in terms of probabilities. However, the nature of the evolving pandemic landscape means that the most update-to-date evidence will rarely be free of uncertainty. While we have not attempted here to quantify the uncertainty levels of the data used to parameterise individual components of the BN, all assumptions surrounding their use have been fully explained and justified. Sensitivity analysis was also used to examine the impact of some of the data uncertainties on the model estimates.

Sensitivity analysis showed model estimates to be robust against minor changes in the number of Pfizer vaccine-associated myocarditis cases (Table 2), but highly affected by changes in vaccine effectiveness against symptomatic infection and death (Table 3). At a public health level, this holds important implications for COVID-19 burden if new variants such as omicron, for which vaccine effectiveness is decreased, continue to emerge or if vaccine effectiveness proves to wane over time. While vaccine effectiveness would have to drop to a very low threshold for the associated myocarditis risk to outweigh the benefit of protection against symptomatic infection and death from COVID-19 in any age-sex-subgroup, this result highlights the importance of updating the model as new evidence becomes available, or new variants emerge.

Model estimates must be contextualised within the scope of the BN model, which does not currently consider comorbidities or personal behaviour that may influence an individual’s risks of acquiring COVID-19, their response to the infection, or their individual risk of myocarditis. Furthermore, limitations to the availability of Australian data introduces uncertainty in the model inputs, so results may change as more data become available. For example, the incidence of vaccine-associated myocarditis in Australia reported by TGA defines myocarditis based on varying degrees of certainty (e.g., confirmed diagnosis by clinical evidence, tests and imaging versus ECG versus possible diagnosis based on symptoms and a doctor’s report that myocarditis is the most likely diagnosis in the absence of medical tests and investigations)³¹. The rate also includes cases of myocarditis that occurred after vaccination but may not be vaccine-related (possibly due to background risk instead)³¹. This uncertainty in the diagnosis of myocarditis may also introduce reporting bias, e.g., self-reporting of adverse events may be influenced by exposure to relevant news in the media, resulting in temporary fluctuation in the reported incidence. In another example, at the time of writing, no Australian data were available on the incidence of Pfizer vaccine-associated myocarditis after the third dose and international data were deemed inappropriate as a substitute (see Table 1 assumptions), necessitating the use of rates for the second dose as a worst-case scenario. In a third example, when calculating the delta variant-specific CFR from COVID-19, ideally CFR for the unvaccinated population would be used, and the 2-to-3 week lag between diagnosis and death accounted for. This information was not available in Australia, so the assumptions were made that the time-window of a few months for the delta wave was long enough to minimise the effect of time lag from infection to death, and the great majority of deaths during the delta wave was in unvaccinated people. Other limitations arise from the model development process, where the use of expert elicitation may be perceived to introduce bias in the evidence viewed. This was minimised through broad literature searches and frequent meetings with external experts such as cardiologists about the quality of the data sources used in the model assumptions.

Despite these limitations, the use of an evidence-based BN to model the risks and benefits of COVID-19 vaccination has many advantages. BNs allow for interactive scenario analysis so the model was well-suited for use in programming CoRiCal, a free online tool aimed at better informing the public and helping clinicians to best advise patients on the risks and benefits of COVID-19 vaccination¹⁹. Another benefit of BNs is the ease of updating, allowing for future model updates to incorporate other outcomes such as long COVID, different patient groups such as those < 12 years and those with comorbidities, other vaccines such as Moderna, or different vaccine adverse events such as anaphylaxis. Finally, BNs are advantageous due to their integration of new data and different data sources in informing different aspects of the model. While this model has been designed for the Australian context, conditional probability tables (CPTs) can easily be re-populated wherever possible using data from another country.

In summary, we developed a BN to compare the risks and benefits of Pfizer COVID-19 vaccination in the Australian population, that incorporated emerging evidence of waning vaccine effectiveness over time and the recent recommendation of third doses, in order to assist clinicians with providing guidance about the Pfizer COVID-19 vaccine. In a community rather than individual context, the final model can also be used to calculate population-level estimates to help inform policy development and public health management. Although designed to compare risks of developing and dying from COVID-19, COVID-19- and Pfizer vaccine-associated myocarditis for the delta variant, the model can be updated to consider the omicron or other variants, other inputs such as patient comorbidities, and other outcomes such as long COVID.

Methods

Bayesian networks

BNs are graphical displays of directional associations between variables, as defined by conditional probabilities⁴³. Nodes represent variables and have multiple potential states (e.g., male and female), and associations are represented by arrows in the direction of parent (independent) to child (dependent) variable (Fig. 5). Probabilities are assigned to each potential node state via CPTs depending on parent node states or, in the case of no parents, prior distributions. The use of CPTs allows for integration of multiple data sources and formats including published figures, other literature and expert opinion, as well as easy updating when new data are presented⁴⁴. BNs are also appropriate for analysing estimated or uncertain risks as they allow for sensitivity analysis to test multiple possible inputs⁴⁴.

**Fig. 5: Example Bayesian network (BN) for modelling the risk of developing background myocarditis over 2 months based on age and sex.**

Throughout the COVID-19 pandemic, BNs have been used in decision-making⁴⁵, risk assessment⁴⁶ and analysis^47,48. We have previously developed the first BN model for risk–benefit analysis of a COVID-19 vaccine, and used the model outputs to design an online tool to communicate the risks and benefits of the AstraZeneca COVID-19 vaccine in the Australian context^17,18,19.

Model design

The model was based on best evidence from multiple sources, designed through collaboration between subject matter experts (KRS, RP, JL, JES, CLL) including clinicians and researchers with expertise in virology, infectious disease epidemiology, cardiology, general practice and public health, and modellers (CLL, HJM, KM, JES), who facilitated the design process and generated the model. Subject matter experts identified relevant variables to be considered and their relationships, and agreed on reliable information sources for each variable. A conceptual model was drafted based on this and node states defined. The model was then evaluated and refined, with nodes and relationships being removed in the absence of reliable, quantitative evidence. The model focuses on ages ≥12 years due to insufficient data on younger age groups at the time of development.

Myocarditis

Acute myocarditis can result in myocardial inflammation from either an infectious or immune-mediated aetiology⁴⁹. Thus, our model compared the risk of Pfizer vaccine-associated myocarditis with the risk of myocarditis in COVID-19 patients. While often asymptomatic, myocarditis may present as chest pain, palpitations and/or dysrhythmias^49,50,51 and can cause dilated cardiomyopathy, arrhythmia and/or sudden cardiac death^50,51. In Australia, myocarditis is often diagnosed using electrocardiogram, serum troponin levels, inflammatory markers, chest X-ray, echocardiography and occasionally endomyocardial biopsy¹². However, these methods can underestimate the presence of myocarditis in comparison to more sensitive cardiac magnetic resonance imaging (MRI), which is considered the gold-standard for non-invasive diagnosis worldwide^52,53. The 2018 Lake Louise criteria for MRI-based diagnosis of myocarditis targets tissue-based imaging markers of oedema, hyperaemia, necrosis and fibrosis^54,55. To ensure the diagnosis of myocarditis was made robustly in our model, data reporting myocarditis cases diagnosed via cardiac MRI were used wherever possible.

While both the Pfizer COVID-19 vaccine and COVID-19 itself may also be associated with pericarditis, either separately or simultaneously with myocarditis, this model focuses solely on myocarditis. This is because diagnostic criteria for pericarditis are not well-defined, and because it is less common than myocarditis. In studies that reported ‘myocarditis/pericarditis’, we estimated that ~65% of cases were attributable to myocarditis, based on proportions of cases reported in studies that differentiate between them^28,29.

The definitions for vaccine-associated and infection-induced myocarditis used for the model reflect those used within the studies from which data were drawn. Vaccine-associated myocarditis was defined as confirmed or suspected myocarditis occurring within 2 months of vaccine administration (with most cases occurring within the first 10 days)³¹, and COVID-19-related myocarditis was defined as myocarditis that occurred within 6 months of COVID-19 diagnosis³⁴.

Data sources

CPTs were derived from data compiled by experts from published material, government reports and through dialogue with external clinical experts (e.g., cardiologists regarding the evidence for Pfizer vaccine-associated, COVID-19-related and background rate of myocarditis). Official Australian authority-issued data were employed whenever possible (e.g., national data on Pfizer vaccine-associated myocarditis). When this was unavailable, data were retrieved from other reliable and publicly available sources (e.g., background rates of myocarditis). Where Australian data were not readily available and international data were not suitable to use for the Australian context, expert opinion was sought. For example, there were limited data in Australia about Pfizer vaccine-associated myocarditis incidence and CFR after the third dose. While rates were reported in Israel and Singapore, these were deemed inappropriate to use in the model as reported rates from first and second doses in these countries were much lower than in Australia. However, both reported lower incidence of myocarditis after the third dose than the second dose. Therefore, to avoid underestimating the risk, the decision was made by the subject experts to use a conservative assumption that incidence after the third dose was the same as the second dose. For some variables, data analysis was required to obtain probabilities for the CPTs, e.g., converting COVID-19 case incidence into probability of infection over 2 months for the community transmission intensity node, or averaging data to fit the BN age categories. Table 1 and Supplementary Tables S1–9 summarise data sources, model assumptions and rationale.

The BN incorporates default prior distributions for age group (based on the Australian population’s age distribution), sex (50% male, 50% female) and vaccine coverage (5% of the population unvaccinated, 5% of received one dose, 60% received two doses [20% with the second dose administered 0 to < 2 months ago, 20% 2 to < 4 months ago and 20% 4 to < 6 months ago], and 30% received three doses [administered ~3 weeks ago]). Prior distributions do not influence scenario analyses results, e.g., once male sex is selected, outputs relate only to males regardless of the entered prior distribution of sexes. Prior distributions can also be altered to model-specific scenarios, e.g., different levels of vaccine coverage.