key: cord-0876123-00ygsikk
authors: Feikin, Daniel R.; Abu-Raddad, Laith J.; Andrews, Nick; Davies, Mary-Ann; Higdon, Melissa M.; Orenstein, Walter A.; Patel, Minal K.
title: Assessing vaccine effectiveness against severe COVID-19 disease caused by omicron variant. Report from a meeting of the World Health Organization
date: 2022-05-02
journal: Vaccine
DOI: 10.1016/j.vaccine.2022.04.069
sha: 664165ed3dfa5f5e8b86c512c03f22fbd2a1210b
doc_id: 876123
cord_uid: 00ygsikk

Vaccine effectiveness is lower and wanes faster against infection and symptomatic disease caused by the omicron variant of SARS-CoV-2 than was observed with previous variants. Vaccine effectiveness against severe omicron disease, on average, is higher, but has shown variability, including rapid apparent waning, in some studies. Assessing vaccine effectiveness against omicron severe disease using hospital admission as a measure of severe disease has become more challenging because of omicron’s attenuated intrinsic severity and its high prevalence of infection. Many hospital admissions likely occur among people with incidental omicron infection or among those with infection-induced exacerbation of chronic medical conditions. To address this challenge, the World Health Organization held a virtual meeting on March 15, 2022, to review evidence from several studies that assessed Covid-19 vaccine effectiveness against severe omicron disease using several outcome definitions. Data was shown from studies in South Africa, the United States, the United Kingdom and Qatar. Several approaches were proposed that better characterize vaccine protection against severe Covid-19 disease caused by the omicron variant than using hospitalization of omicron-infected persons to define severe disease. Using more specific definitions for severe respiratory Covid-19 disease, such as indicators of respiratory distress (e.g. oxygen requirement, mechanical ventilation, and ICU admission), showed higher vaccine effectiveness than against hospital admission. Second, vaccine effectiveness against progression from omicron infection to hospitalization, or severe disease, also showed higher vaccine protection. These approaches might better characterize vaccine performance against severe Covid-19 disease caused by omicron, as well as future variants that evade humoral immunity, than using hospitalization with omicron infection as an indicator of severe disease.

become more challenging because of omicron's attenuated intrinsic severity and its high prevalence of infection. Many hospital admissions likely occur among people with incidental omicron infection or among those with infection-induced exacerbation of chronic medical conditions. To address this challenge, the World Health Organization held a virtual meeting on March 15, 2022, to review evidence from several studies that assessed Covid-19 vaccine effectiveness against severe omicron disease using several outcome definitions. Data was shown from studies in South Africa, the United States, the United Kingdom and Qatar. Several approaches were proposed that better characterize vaccine protection against severe Covid-19 disease caused by the omicron variant than using hospitalization of omicron-infected persons to define severe disease. Using more specific definitions for severe respiratory Covid-19 disease, such as indicators of respiratory distress (e.g. oxygen requirement, mechanical ventilation, and ICU admission), showed higher vaccine effectiveness than against hospital admission. Second, vaccine effectiveness against progression from omicron infection to hospitalization, or severe disease, also showed higher vaccine protection. These approaches might better characterize vaccine performance against severe Covid-19 disease caused by omicron, as well as future variants that evade humoral immunity, than using hospitalization with omicron infection as an indicator of severe disease.

Since the emergence of the omicron variant of SARS-CoV-2 in November 2021, mounting evidence has demonstrated significant immune evasion from infection-induced and vaccineinduced immunity. Vaccine effectiveness is lower against infection and symptomatic disease caused by omicron than other variants, including delta. 1 Moreover, vaccine effectiveness against these outcomes appears to wane faster after the primary series of vaccination. Vaccine effectiveness against severe omicron disease, on average, is higher, perhaps because of the role of preserved cellular immunity. 2 Nonetheless, assessing vaccine effectiveness against omicron severe disease has become more challenging because of its attenuated intrinsic severity and its high prevalence of infection. To address this challenge, the World Health Organization held a virtual meeting of the Covid-19 Vaccine Effectiveness Methods Group to review the evidence from several studies that assessed Covid-19 vaccine effectiveness against severe omicron disease using several outcome definitions. Data was shown from studies in South Africa, the United States, the United Kingdom and Qatar. This report summarizes the results of these studies, as well as other relevant studies in the pre-print or published literature and discusses approaches to optimize evaluations of vaccine effectiveness against severe Covid-19 disease caused by omicron or future variants with immune evasion.

Since June 2021, the World Health Organization and International Vaccine Access Center at Johns Hopkins Bloomberg School of Public Health have undertaken a living systematic review of the emerging evidence for COVID-19 vaccine effectiveness. The methods have been described elsewhere. 3,4 Between December 3, 2021 and April 7, 2022, there were 21 vaccine effectiveness studies that met our inclusion criteria that reported results for severe omicron disease for five vaccines (Table 1 and Figure) . 5-26 The majority (n=13, 62%) of studies used hospitalization with some clinical evidence of Covid-19 disease as the outcome, while six (29%) studies used hospitalization with PCR-confirmed infection without clinical criteria, and 4 (17%) used other outcomes besides hospitalization. (Two studies evaluated vaccine effectiveness for more than one severe outcome.) In contrast to vaccine effectiveness against delta severe disease, the majority of vaccine effectiveness estimates for omicron severe disease were below 75%; for example, fourteen (82%) of seventeen vaccine effectiveness estimates within three months of vaccination with the primary series were below 75% (Figure) . Moreover, twelve vaccinespecific vaccine effectiveness estimates fell below 50% at some point in time after vaccination. 5, 9, 14, [17] [18] [19] 21, 24, 26 Vaccine effectiveness after receipt of a booster dose increased to >75% for all vaccines within the first 3 months after a booster dose, with the exception of one study that reported a vaccine effectiveness of 71% at 8-59 days after a homologous CoronaVac booster. 21 Few studies have evaluated vaccine effectiveness against severe omicron disease three months or more after the booster dose. There is a suggestion that the vaccine effectiveness after the primary series is lower when severe disease is defined as hospitalization without requirement for clinical criteria of Covid-19 than hospitalization with clinical criteria, particularly after 3 months since vaccination, although too few studies (n=3) are available to make a definitive comparison (Figure) .

Hospitalization is an accessible and easily defined measure of severe disease, particularly when using electronic databases. However, criteria for hospitalization vary significantly by geographic An example of the changing distribution of types of Covid-19 hospitalization with omicron was presented from Western Cape Province, South Africa. 33 A detailed assessment of deaths among persons admitted with SARS-CoV-2 infection found that the percentage of deaths due to severe COVID-19 decreased from 78% during the delta wave to 50% during the omicron wave.

Conversely, Covid-19 associated deaths (where SARS-CoV-2 infection may have played a role in exacerbation of underlying illnesses) and incidental infection increased from 2% and 0%, respectively, during the delta wave to 24% and 6%, respectively, during the omicron wave.

Others studies have shown similar findings. In a California hospital, 19.8% of admissions with omicron infection were deemed be not likely due to Covid-19; the median age of these admissions was 38 years old compared to 67 years old for those admitted likely due to Covid-19. 34 In one large medical center in the Netherlands, medical records review of all admissions with omicron infection during a two month period revealed that 45% were admitted for primary Covid-19 disease, 21% due to omicron infection contributing to an underlying illness, 31% due to incidental omicron infection, and 3% with an indeterminant role of omicron infection. 35

At the meeting, data were presented from studies that evaluated vaccine effectiveness using other approaches to define vaccine effectiveness against severe omicron disease besides hospitalization. First, outcomes that reflect greater severity than hospital admission, particularly those more specific for hypoxic respiratory disease, such as use of high-flow oxygen, mechanical ventilation and admission to the intensive care unit, likely better assess the protection of vaccines against severe Covid-19 disease. An analysis from the United Kingdom was presented that showed that the more specific the case definition was for respiratory disease (i.e., primary ICD-10 code for respiratory illness) and severe disease (i.e., oxygen use, mechanical ventilation or ICU admission) caused by omicron variant, the higher the vaccine effectiveness. 23 For example, among SARS-CoV-2-positive 18-64 year old persons admitted for at least one day who did not have respiratory disease as their primary diagnosis the vaccine effectiveness at 14-174 days after vaccination with an mRNA vaccine or AstraZeneca-Vaxzevria was 29.5% (15.1 to 41.5), which increased to 79.1% (-36.9 to 96.8) when the admission was two or more days, had acute respiratory illness in the primary diagnosis, and required supplemental oxygen. This difference in vaccine effectiveness among hospitalized cases based on case definition was of greater magnitude with omicron than delta. Moreover, waning of the effectiveness against "severe" omicron disease over time was substantial using all admissions, but was much less when using more specific definitions for severe Covid-19 disease; whereas, with severe delta disease minimal waning of vaccine effectiveness was observed using all case definitions of severity, including hospital admission.

Two studies from the United States showed similar differences in the vaccine effectiveness for severe omicron disease based on the definition used. One study presented from the IVY network of 21 hospitals in the United States showed that vaccine effectiveness against two doses of mRNA vaccines for hospital admission with omicron was 65% (95% CI, 51-75%), while it was 79% (95% CI, 66-87%) for invasive mechanical ventilation or in-hospital death. 15,16 The difference in vaccine effectiveness against these same outcomes was less during the delta period for two mRNA doses -88% (95% CI 86-90%) and 85% (95% CI, 83-87%), respectively. In general, omicron patients were older and more medically complex than delta patients, suggesting a higher likelihood of exacerbation of comorbid conditions. A study that became available as a preprint subsequent to the meeting found that among hospitalized adolescents 12-18 years of age in the U.S. the vaccine effectiveness against hospitalization during the Omicron-predominant-period was 40% (95% CI 9-60%) for the Pfizer-BioNTech-Comirnaty vaccine, but 79% (95% CI 51-91%) for critical Covid-19 disease (i.e., requiring life support or progressing to death); in contrast, there was minimal difference during the Delta-predominantperiod -92% (95% CI 89-95%) and 96% (95% CI 90-98%), respectively. 26 A second approach is assessing the effectiveness against progression to severe disease conditional upon being infected. Halloran et. al. conceptualized vaccine effectiveness against a disease outcome as a product of vaccine effectiveness against susceptibility to infection (i.e., VE s ) and vaccine effectiveness against progression from infection to the disease outcome (i.e., VE p ). 36 As such, if VE s for omicron infection decreases, the effectiveness against severe omicron disease would also apparently decrease, even if VE p from infection to severe disease was maintained, thereby obscuring the component of effectiveness than prevents progression to severe disease (VE p ). For example if VE s reduces from 80% to 50%, but VE p is maintained at 70%, then the overall effectiveness against hospitalization, which is 1-(1-VE s )*(1-VE p ), reduces

Data was presented from an unpublished analysis from Qatar using multivariable logistic regression to assess associations with progression to COVID-19 hospitalization and death among infected cases. (Supplement S1 for methods). In this setting of a young population, two or three doses of either mRNA vaccine reduced hospital admission among infected persons by 25% (95% CI, 19-31%) and 31% (95% CI 21-40%), respectively (Table 2, noting that VE p = 1adjusted odds ratio). In contrast, vaccine protection against progression to ICU admission, mechanical ventilation or death increased to 67% (54-76%) and 84% (71-91%) for two or three doses, respectively. Additionally, the analysis showed that vaccine protection against progression from infection to severe outcomes was significantly higher when using WHO disease classifications based on clinical criteria of severity (i.e., severe Covid-19, critical Covid-19 and fatal Covid-19), than when using hospital admission (Table 2 ).

In the IVY network in the United States, the overall vaccine effectiveness for two or three doses of mRNA vaccines among immunocompetent adults was 44% (95% CI 0-69) against progression among persons admitted with omicron infection to invasive mechanical ventilation or death, similar to what they found for delta variant (50%, 95% CI 37-60%). 16 In the Western Cape Province, South Africa, protection of the primary series of Janssen-Ad26.COV2.S or Pfizer-BioNTech-Comirnaty vaccines against progression from infection to severe admission or death was similar during the omicron wave (adjusted HR 0.45, 95% confidence intervals, 0.36-0.56) as during the delta wave (adjusted HR 0.53, 95% confidence intervals, 0.44-0.64). 37 In a study among members of Kaiser Permanente Southern California (not presented at meeting), the primary series of the Janssen-Ad26.COV2.S and both mRNA vaccines both showed approximately a halving of the probability of progression from omicron infection diagnosed in the outpatient setting to hospital admission (hazards ratio for progression to admission for Janssen-Ad26.COV2.S of 0.51, 95% CI 0.33-0.78, and for mRNA vaccines given <90 days prior to testing of 0.49, 95% CI 0.32-0.76.) 38 Vaccine protection against progression of omicron infection was similar to that found for delta for Ad26.COV2.S (HR 0.46, 95% CI 0.30-0.70), although vaccine protection was less with omicron than delta with the mRNA vaccines (HR 0.28, 95% CI 0.23-0.33). Minimal waning was seen in protection against progression of the mRNA vaccines with time since vaccination.

Most studies showed high vaccine effectiveness against omicron-associated death. In Qatar, the adjusted odds ratio of progression from omicron infection to death was 0.12 (95% CI, 0.05-0.28) for two doses and 0.03 (95% CI, 0.00-0.25) for three doses of mRNA vaccines (Table 2 ). In South Africa, the hazard ratio of progression from omicron infection to death during the omicron wave was 0.24 (95% CI, 0.10-0.58). 37 In a study (not presented at the meeting), among U.S. veterans, two doses of the mRNA vaccines had a vaccine effectiveness of 44% (26-58) against hospitalization with omicron, compared with 75% (52-87%) against death with omicron. 5 Despite these findings, potential concern was raised in using death as an outcome for vaccine effectiveness evaluations. Death among persons who have tested positive for SARS-CoV-2 is clearly a more severe outcome than hospitalization, however, it might also be nonspecific for Covid-19, particularly during the omicron wave with high infection rates. This can occur because most definitions of Covid-19-associated deaths include a positive test up to a month prior to death. Misclassification of the cause of death might be a particular concern among elderly persons with comorbidities who are at higher risk of dying from other causes.

When using death as an outcome, verification of the cause of death as due to Covid-19 should be done, if feasible.

Covid-19 vaccines likely have higher effectiveness against severe omicron Covid-19 disease than indicated by effectiveness estimates that use hospital admission of omicron-infected persons to define severe disease. This is because a greater proportion of admissions are associated with, but not caused by, omicron infection, against which current covid-19 vaccines are less effective.

To evaluate vaccine protection against severe omicron disease, we recommend using more specific definitions for severe Covid-19 respiratory disease among hospitalized persons. As a second approach to measuring vaccine protection against severe disease, we suggest evaluating progression from omicron infection to more severe outcomes, like intensive care using admission and ventilatory support. While fatal outcomes can be used to evaluate vaccine effectiveness against severe omicron disease, caution should be taken to prevent misclassification of the cause of death. It may also be useful to use ecological analyses on end points not dependent on testing, such as all cause deaths or all respiratory deaths/hospitalizations /ICU admissions, as a sense check, because in the context of high infection, it would be surprising to see these indicators remaining at low levels (as has been the case in many countries) if vaccine effectiveness against these end points was not high. Which type of severe outcomes are prevented by Covid-19 vaccines has implications for vaccine policy.

Preserved high vaccine effectiveness against severe Covid-19 disease attributed to omicron suggests that the current vaccine formulations continue to have utility in preventing the most severe forms of disease. However, because omicron evades vaccine-induced immunity against infection, as perhaps will future emergent variants, a greater proportion of hospitalizations and deaths may be caused by infection-associated exacerbations of chronic diseases in vulnerable adults. Preventing these types of severe outcomes related to SARS-CoV-2 infection might require more frequent boosters or new vaccines that more effectively and durably prevent SARS-CoV-2 infection. 

Walter A. Orenstein is a member of the Scientific Advisory Board for Moderna.

The data in figure and Table 1 are available at https://view-hub.org/covid-19/effectivenessstudies.

☐ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

☒ The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:

Walter Orenstein reports a relationship with Emory University Vaccine Center that includes: consulting or advisory. 

Change in profile of COVID-19 deaths in Western Cape Province, South Africa, during the fourth wave

Clinical Characteristics and Outcomes Among Adults Hospitalized with Laboratory-Confirmed SARS-CoV-2 Infection During Periods of B.1.617.2 (Delta) and B

Admissions to a large tertiary care hospital and Omicron BA.1 and BA.2 SARS-CoV-2 PCR positivity: primary, contributing, or incidental COVID-19. medRxiv 2022

Design and Interpretation of Vaccine Field Studies

Outcomes of laboratory-confirmed SARS-CoV-2 infection in the Omicron-driven fourth wave compared with previous waves in the Western Cape Province, South Africa. medRxiv 2022

Clinical outcomes among patients infected with Omicron (B.1.1.529) SARS-CoV-2 variant in southern California

* Not included in plot for one of following reasons: another similar time interval was available, time interval is large and cannot be place into specific time period for plot, or VE estimate not reliable

We want to acknowledge Manish Patel, U.S. Centers for Disease Control and Prevention, for sharing data from the IVY network. We acknowledge Anurima Baidya and Karoline Walter, International Vaccine Access Center, for their work in extracting data for studies for the ongoing systematic review.

Invasive mechanical ventilation or inhospital death