key: cord-0317586-dyx18233 authors: Lau, C. L.; Mayfield, H. J.; Sinclair, J. E.; Brown, S. J.; Waller, M.; Enjeti, A. K.; Baird, A.; Short, K. R.; Mengersen, K.; Litt, J. title: Risk-benefit analysis of the AstraZeneca COVID-19 vaccine in Australia using a Bayesian network modelling framework date: 2021-10-03 journal: nan DOI: 10.1101/2021.09.30.21264337 sha: 3b023aa67e403219d9d235292f18c2fc37976e4b doc_id: 317586 cord_uid: dyx18233 Thrombosis and Thromobocytopenia Syndrome (TTS) has been associated with the AstraZencea (AZ) COVID-19 vaccine. Australia has reported low TTS incidence of <3/100,000 after the first dose, with case fatality rate (CFR) of 5-6%. Risk-benefit analysis of vaccination has been challenging because of rapidly evolving data, changing levels of transmission, and age-specific variation in rates of TTS, COVID-19, and CFR. We aim to optimise risk-benefit analysis by developing a model that enables inputs to be updated rapidly as evidence evolves. A Bayesian network was used to integrate local and international data, government reports, published literature and expert opinion. The model estimates probabilities of outcomes under different scenarios of age, sex, low/medium/high transmission (0.05%/0.45%/5.76% of population infected over 6 months), SARS-CoV-2 variant, vaccine doses, and vaccine effectiveness. We used the model to compare estimated deaths from vaccine-associated TTS with i) COVID-19 deaths prevented under different scenarios, and ii) deaths from COVID-19 related atypical severe blood clots (cerebral venous sinus thrombosis & portal vein thrombosis). For a million people aged >70 years where 70% received first dose and 35% received two doses, our model estimated <1 death from TTS, 25 deaths prevented under low transmission, and >3000 deaths prevented under high transmission. Risks versus benefits varied significantly between age groups and transmission levels. Under high transmission, deaths prevented by AZ vaccine far exceed deaths from TTS (by 8 to >4500 times depending on age). Probability of dying from COVID-related atypical severe blood clots was 58-126 times higher (depending on age and sex) than dying from TTS. To our knowledge, this is the first example of the use of Bayesian networks for risk-benefit analysis for a COVID-19 vaccine. The model can be rapidly updated to incorporate new data, adapted for other countries, extended to other outcomes (e.g., severe disease), or used for other vaccines. The AstraZeneca ChAdOx1 (AZD1222) COVID-19 vaccine (AZ vaccine) has been widely used globally, with one billion doses released in over 170 countries by August 2021 [1] . The vaccine is 86 highly effective against symptomatic infection, serious illness and death from COVID-19 [2] [3] [4] . In knowledge, BNs have not yet been applied for risk-benefit analysis of COVID-19 vaccines. Our BN model structure was based on the best available scientific evidence from multiple sources. Technical Advisory Group on Immunisation (ATAGI) and the Therapeutic Goods Administration 184 (TGA), e.g., using the same age groups and definitions of low/medium/high community transmission 185 intensity (equivalent to 0.05%/0.45%/5.76% of population infected over 6 months) [15] . is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101/2021.09.30.21264337 doi: medRxiv preprint TTS is a rare and atypical form of blood clotting associated paradoxical thrombosis and low platelets. clots such as CVST and splanchnic vein thrombosis (SVT) [27, 28] , which have also been reported in 192 30] . CVST While the assumptions and model structure are defined by experts, the CPTs were based on empirical 202 data. Experts compiled evidence from peer-reviewed literature, government websites and reports, and 203 through discussion with external clinical experts (e.g., haematologists regarding the evidence for AZ 204 vaccine-associated TTS, and background rates of CVST and PVT). Official data from Australian 205 authorities were used wherever possible (e.g., local data on AZ vaccine-associated TTS). Otherwise, 206 data were obtained from other robust and publicly available sources (e.g., background rates of CVST To determine how often model assumptions need to be updated, sensitivity analysis was conducted for 234 three predictor variables that were considered most likely to change over time. We examined actual 235 changes in the reported incidence of AZ vaccine-associated TTS cases and deaths in Australia in August-September 2021. We also examined i) plausible differences in age-specific CFR for COVID- . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 ii) plausible reductions in vaccine effectiveness against symptomatic COVID-19 infection and death. Incidence and CFR of vaccine-associated TTS: From July-September 2021, reported incidence of AZ 240 vaccine-associated TTS and related deaths in Australia fluctuated by week because of low numbers. We examined ATAGI reports from 25/8/2021, 1/9/2021, 8/9/2021, and 15/9/2021 [36-39] to determine 242 how fluctuations in data affected our model predictions of age-specific TTS-related deaths over these 243 weeks. Age-specific CFR for COVID-19 in Australia: By 31/8/2021, COVID-19 CFR in Australia were very 245 low for younger age groups, with less than five deaths during the entire pandemic in each male/female 246 subgroups in those aged 0-9, 10-19, 20-29, 30-39, and 40-49 is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. A minor anomaly was detected in the model through assessment of biological plausibility. For COVID-298 19 cases in younger age groups (females <30 years and males <10 years), the estimated probability of 299 dying from COVID-19 was lower than the probability of dying from COVID-19 related atypical severe 300 blood clots. The reason for this anomaly is that data for COVID-19 CFR in Australia were used for the 301 CPTs, and there have been no deaths to date in these age-sex subgroups (Table A3) , while data on 302 COVID-19 related atypical blood clots were extracted from a study where data were predominantly 303 sourced from the USA and Europe, where high CFRs could have resulted from an overwhelmed health 304 system during the outbreak. Although the discrepancies in probabilities were extremely low (probability 305 of dying from COVID-19 related atypical blood clots <0.0002% higher than dying from COVID-19 306 itself), the anomaly highlights that the model predictions should be used as broad estimates rather than 307 exact risks. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint Up to 25/8/2021 in Australia, age-specific incidence of AZ vaccine-associated TTS cases ranged from 336 16 to 27 per million first doses (Table 1) , with an overall CFR of ~5% (6 deaths out of 115 cases). Based on annual background rates of CVST and PVT reported by Kristoffersen et al. [27] and Ageno 338 et al. [28] , our model estimated 6-week incidence of 0.38 (age <20 years) to 2.69 (age ³70 years) cases 339 per million, and overall CFR ranging from 7.0% to 21.6% from youngest to oldest age groups (Table 340 . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101/2021.09.30.21264337 doi: medRxiv preprint from a retrospective cohort study using linked electronic health records predominantly from the USA 346 and Europe [30] because there were insufficient data from Australia. Model version 1 estimated that 347 overall fatality from atypical severe blood clots (CVST and PVT combined) in COVID-19 patients were 348 51.1 and 37.3 per 100,000 in males and females, respectively. Figure 4 shows that the probability of 349 developing atypical blood clots in COVID-19 patients was 14-28 times more likely than developing 350 TTS after the first dose of the AZ vaccine, depending on age group and sex ( Figure 4 , dashed lines). The probability of dying from COVID-19 infection-related atypical blood clots in COVID-19 patients 352 was 58-126 times more likely than dying from TTS after the first dose of AZ vaccine, again depending vaccine-associated TTS in Australia, and CFR ranging from 5.2% to 6.4% (Table 3) . The model outputs 360 showed that estimated deaths from TTS per million first doses did not change significantly over this 361 time period, and ranged from differences of -0.08 to 0.37 deaths per million (depending on age group) 362 when comparing data from 25/8/2021 with subsequent reports. Therefore, minor fluctuations in CFR 363 from TTS did not have any significant influence on the point estimates of the number deaths at a 364 population level. Because of the small number of TTS cases and deaths in Australia so far, the 95% 365 confidence intervals (CI) for CFR were wide, ranging from 1.9% to 12.2% over the four reports (Table 366 3). Our model currently assumes a CFR of 5%; it is plausible that true CFR is twice as high (10%), and 367 if so, model estimates of TTS related deaths would be doubled. . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101/2021.09.30.21264337 doi: medRxiv preprint age-sex subgroup during this time, CFR would have increased the most in 60-69 year-olds and the least 373 in 20-29 year-olds, but by less than 0.06% in any subgroup ( Figure 5 ). The model was most sensitive 374 to changes in the 60-69 year-old age group because of the small case numbers (small denominator). If 375 there were five extra deaths in each age-sex subgroup, CFR would have increased by less than 0.3% in 376 any subgroup. Therefore, our model was not very sensitive to minor changes in number of reported 377 deaths, and model estimates of deaths per million would have differed by less than 0.06% or 0.3% if 378 there were one or five extra deaths in any age-sex subgroup, respectively. Our model shows that for a population where 70% has received first dose and 35% has received two 382 doses, a theoretical 5% or 10% reduction in vaccine effectiveness against the delta variant results in a 383 7.1% or 15.1% increase in estimated deaths, respectively (Table 4) . Therefore, sensitivity analyses 384 show that model predictions of deaths are much more sensitive to changes in vaccine effectiveness than 385 to changes in incidence and CFR of TTS, or changes in CFR from COVID-19 infection. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. To date, the CFR of TTS in Australia has been lower than those reported in other countries [7, 9] . Although numbers have been small, CFR have remained relatively stable throughout August-September 412 2021 (Table 3) is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10. 1101 /2021 Sensitivity analyses showed that the estimates of COVID-19 related deaths were very sensitive to changes in vaccine effectiveness. Therefore, model assumptions need to be updated to reflect evolving on analysis of millions (e.g., QCOVID [44] ) or hundreds of thousands (e.g., ISARIC4C [47]) of medical 452 records. Our modelling approach provides an alternative option of developing risk assessment tools if 453 large datasets are not available, and model inputs need to be sourced from alternative sources. We have 454 not identified any web-based interactive tools that use Bayesian networks for risk-benefit analysis of 455 COVID-19 vaccines for use in clinical or public health settings. Our results should be interpreted considering the model's limitations. There are uncertainties associated 458 with some of our model inputs, either because of limited data, or the use of data from other countries. As illustrated with the anomaly regarding the risk of dying in younger persons, model outputs should 460 be considered as broad estimates rather than exact risks, but estimates can be improved over time as 461 more data become available. Our model provides population level estimates and does not consider 462 individual risks such as behaviour and comorbidities. We plan to develop future models that include 463 the individual's comorbidities, similar to the QCOVID tool [44] but specific for the Australian context. In the results provided, we have assumed that 100% of infections were from the delta variant. Assumptions of age distribution of delta cases (if unvaccinated) were obtained from data during the 466 early stages of the delta outbreak in NSW from June 2021. While vaccination rates were relatively low 467 then, older ages had higher vaccine coverage so infection rates for delta may have been underestimated 468 in these groups. Data on CVST and PVT were obtained from studies outside Australia and may not 469 reflect the local experience. The current model focuses on fatalities from COVID-19, TTS, and atypical 470 blood clots, but does not consider other risks (e.g., adverse events) or other benefits (e.g., cases of severe 471 COVID prevented, or broader societal benefits). Our model was not parameterised from any specific 472 datasets, so model outputs could not be directly validated by data. Nevertheless, the model provides a 473 powerful mechanism for complex synthesis of multiple sources of data, and the outputs reflect the latest 474 available knowledge. In conclusion, we developed a novel approach to risk-benefit analysis for the AZ vaccine by using an 477 adaptable BN modelling framework. Our model enables more precise risk analysis based on 478 demographics, the outbreak situation, local data on vaccine-associated TTS, and the best available 479 . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101/2021.09.30.21264337 doi: medRxiv preprint international evidence on vaccine effectiveness and atypical blood clots. Although use of the AZ 480 vaccine is expected to gradually decrease in Australia over coming months, the model can be readily 481 adapted for use in other countries or risk-benefit assessment of other vaccines. https://www.ox.ac.uk/news/2021-07-29-oxford-vaccine-reaches-one-billion-doses-released. . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; an adaptable Bayesian network modelling framework Table 1 . Summary of data sources, assumptions, and prior distributions for a Bayesian network to assess risks versus benefits of the AstraZeneca COVID-19 vaccine. Age distribution of infections from delta variant NSW COVID-19 case data from 1/6/2021 to 13/8/2021 were used to provide estimates of age distribution of infections from delta variant. Case data published daily by NSW Health, for the following age categories: 0-19, 5-year age groups from 20-69 years, and 70+. For cases in the 0-19 age group, assumed that 40% were aged 0-9, and 60% aged 10-19 (based on age distribution of cases reported by NNDSS). Date range used was selected to reflect the first 6 weeks of delta outbreak, when vaccination coverage was relatively low. No significant change in age distribution of cases to 29/8/2021. See Appendix A, Table A1 . Age distribution of infections from alpha/wild variants COVID-19 cases reported in Australia from January to December 2020 were used to provide estimates of age distribution from alpha/wild variants. Data sourced from National Notifiable Diseases Surveillance System. See Appendix A, Table A2 . 14,33 Case fatality rates of COVID-19 cases COVID-19 cases reported in Australia from January 2020 to 13/8/2021 were used to provide estimates of age-specific case fatality. Data sourced from National Notifiable Diseases Surveillance System. See Appendix A, Table A3 . 14,33 Chance of infection over 6 months calculated for different levels of community transmission. See Appendix A, Table A4 . Definitions of low, medium, and high transmission as defined by ATAGI document 'Weighing up the potential benefits and risk of harm from COVID-19 Vaccine AstraZeneca'. Low -similar to first wave in Australia (equivalent to 0.05% of population infected over 6 months). Medium -similar to second wave in VIC in 2020 (equivalent to 0.045% of population infected over 6 months). High -similar to Europe in January 2021 (equivalent to 5.76% of population infected over 6 months). Also included transmission scenarios equivalent to: zero transmission; 1% and 2% chance of infection over 6 months; 200 cases/day and 1000 cases/day in NSW; 1000 cases/day in VIC; 1000 cases/day in QLD. Other transmission levels can be added to model. Estimated rate per 100,000 after 2 nd dose of AZ vaccinations: 0.18 per 100,000 (no age specific rates available). Case fatality rate in Australia ~5% (noting that higher rates reported in UK ~18%). For sensitivity analysis, data from ATAGI reports on 1/9/2021, 8/9/2021, and 15/9/2021 were used. Background rates of atypical venous thrombotic disorders Background rates (in population not infected with and not vaccinated for COVID-19) of atypical venous thrombotic disorder (CVST and PVT) over 6 weeks were calculated for each age group to provide a comparison with chance of TTS after AZ vaccine. CVST data from Kristoffersen et al. • -9,10-19, 20-29,30-39, 40-49, 50-59, 60-69 ) . Data in figure are fictitious and for illustrative purposes only. The model assumes 90% vaccine effectiveness against death and even distribution of males or females in the population; these numbers are entered as priors because the nodes do not have parents. The model assumes case fatality rate of 20% in males and 10% in females. The CPT for Die from Covid shows that if the vaccine is effective, probability of dying is 0% for both genders. If the vaccine is not effective, the probability of dying is 20% for males and 10% and females. Fig 1a) shows the BN in its default state; if the vaccine was 90% effective, we expect an overall 1.5% chance of dying from COVID (e.g., in a population of 1000 people, the vaccine was not effective in 100 people. Of these 100 people, we estimate 10 deaths out of 50 males, and 5 deaths out of 50 females, i.e., total of 15 deaths out of 1000 people, or 1.5%). Fig 1b) shows an example scenario analysis of 'what is the chance of a male dying if the vaccine was not effective?' Selected states in each node are underlined, and the model updates the chance of dying from COVID to 20% under this scenario. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101/2021.09.30.21264337 doi: medRxiv preprint Figure 3 . Estimated COVID-19 deaths prevented over 6 months per million population of each age group if 70% had first dose, and 35% had two doses of AZ vaccine under a) low, b) medium, and c) high levels of community transmission; and d) estimated deaths from AZ vaccine-associated TTS if 70% of the population had first dose, and 35% had two doses. (Note the large variations in scale in y-axes between each graph). . CC-BY-ND 4.0 International license It is made available under a perpetuity. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint The copyright holder for this this version posted October 3, 2021. ; https://doi.org/10.1101/2021.09.30.21264337 doi: medRxiv preprint Figure 4 . Number of times more likely to develop and die from atypical blood clots (CVST and PVT) from COVID-19 infection than AZ vaccine-associated TTS, by age group and sex. of Oxford Oxford vaccine reaches one billion doses released Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) (COVID-19) case numbers and statistics Risk Assessment and Decision Analysis with Bayesian 20 Extending the range of COVID-19 risk factors in a 28 Incidence rates and case fatality rates of portal vein thrombosis and Budd-29 Living risk prediction algorithm (QCOVID) for risk of hospital admission 45 COVID-19 Risk Tools. COVID-19 Risk Tooks 2021 Impact of an interactive web tool on patients' intention to receive 647 COVID-19 vaccination: a before-and-after impact study among patients with chronic 648 conditions in France ISARIC4C (Coronavirus Clinical Characterisation Consortium). ISARIC4C Mortality Score Appendix B. All authors evaluated the biological plausibility of estimates, e.g., for COVID-19 patients, A7). The background CFR for CVST and PVT (combined) range from 0.03 (age <20 years) to 0.58