Real-world effectiveness of mRNA COVID-19 vaccines in the elderly during the Delta and Omicron variants: Systematic review

Harvey Palalay,Riddhi Vyas,Barbara Tafuto

Abstract

Key Words: COVID-19; mRNA Vaccine; Effectiveness; Elderly; Delta; Omicron; Systematic review

INTRODUCTION

As of December 31, 2022, there were over 6.6 million coronavirus disease 2019 (COVID-19) deaths and over 651 million cases across 200 countries worldwide[1].During the second half of 2021, COVID-19 cases and deaths were predominantly influenced by the Delta and Omicron variants, wreaking havoc even in countries with tough COVID-19 restrictions[2,3].Epidemiological studies have shown that the contagious and highly transmissible nature of the Delta and Omicron variants has even put the elderly population in a more disadvantaged position, accounting roughly 14% of all COVID-19 cases and 70%of all COVID-19 deaths as of December 31, 2022[4-7].While there is no broad consensus on the age at which a person can be considered elderly, the approved cutoff age as per the United Nations is 60+years[8].

As a response to the extensive impact of COVID-19, which has become a public health concern and an international health crisis, the Centers for Disease Control and Prevention rolled out a global strategy response framework which outlined a combination of non-pharmaceutical and pharmaceutical interventions[9-11].While the primary method of epidemic control has been non-pharmaceutical measures, pharmaceutical intervention, like vaccine, is expected to be the only effective, long-term defense against infection and death[12,13].Vaccination is critical since the epidemic is still challenging to control due to the dormant symptoms and contagious nature of the virus especially during the incubation period which triggers late detection of infection[12,13].

Of the 356 vaccine candidates, over 12 billion vaccine doses have been administered by 34 different vaccines approved under Emergency Use Authorization[1,14].Despite the increase in vaccinations and booster shots, COVID-19 cases and deaths continue to remain high[1].While the effectiveness of these vaccines has already been established by different manufacturers, the fact remains that these vaccines were created quickly for global emergency use, tested under controlled clinical conditions from voluntary subjects and age groups whose general characteristics may differ from the actual general population[15-17].In spite of the many observational studies providing data on the effectiveness of vaccination in various populations, this study aims to compile the disparate data through systematic review[18-29].This study carefully examines the effectiveness of COVID-19 vaccines in real-world settings in the elderly during the predominance of Delta and Omicron variants.

MATERIALS AND METHODS

The systematic review was designed according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards to ensure a comprehensive and methodical approach[30].

Search strategy and selection criteria

The review searched for qualified studies using a combination of Medical Subject Headings (MeSH) and non–Medical Subject Headings from PubMed, Cochrane, CINAHL, Scopus, ProQuest, EMBASE, Web of Science, and Google Scholar databases, as well as qualified research studies from pre–print servers using medRxiv and Research Square, published from January 1, 2021 – December 31, 2022.The search was independently performed by a single researcher using the following keywords and search terms (Supplementary Table 1: Keywords and Search Terms using PICO): (1) Covid-19; covid 19; covid19;SARS CoV 2*; SARS-CoV-2*; SARS Coronavirus 2 Infection; sars virus; 2019 Novel Coronavirus*; nCoV;2019-nCoV*; COVID-19 Pandemic*; COVID-19 Virus*; Coronavirus; Coronavirus Disease*; Severe Acute Respiratory Syndrome Coronavirus 2 Infection; CV-19; CV19; (2) covid 19 vaccine*; covid-19 vaccine*; Pfizer-BioNTech vaccine; Comirnaty; BNT162b2; Bnt-162b2; Bnt162b2; Tozinameran;Tozinameran [INN]; UNII-5085ZFP6SJ; Moderna vaccine; mRNA-1273; MRNA-1273; Spikevax; CX-024414; Elasomeran; Elasomeran [INN]; M-1273; Moderna covid-19 vaccine rna; TAK-919; UNIIEPK39PL4R4; Covid 19 booster; Covid-19 booster; SARS-CoV-2 vaccine; SARS-CoV-2 booster;vaccinated; inoculat*; immuni*; post-vaccination; antibody; protected; (3) unvaccinated; uninoculated;uninoculated; unimmunized; unprotected; susceptible; and (4) reduce incidence*; reduce admission*;reduce infection*; reduce hospitalization*; reduce morbidity*; reduce mortality*; reduce death*; lessen infection*; lessen admission*; lessen hospitalization*; lessen morbidity*; lessen mortality*; lessen death*; prevent incidence*; prevent infection*; prevent admission*; prevent hospitalization*; prevent morbidity*; prevent mortality*; prevent death*; minimize incidence*; minimize admission*; minimize infection*; minimize hospitalization*; minimize morbidity*; minimize mortality*; minimize death*;control incidence*; control admission*; control infection*; control hospitalization*; control morbidity*;control mortality*; control death*; combat incidence*; combat admission*; combat infection*; combat hospitalization*; combat morbidity*; combat mortality*; combat death*; eliminate incidence*; eliminate admission*; eliminate infection*; eliminate hospitalization*; eliminate morbidity*; eliminate mortality*;eliminate death*; diminish incidence*; diminish admission*; diminish infection*; diminish hospitalization*; diminish morbidity*; diminish mortality*; diminish death*; solve incidence*; solve admission*; solve infection*; solve hospitalization*; solve morbidity*; solve mortality*; solve death*.

Eligibility standards: inclusion and exclusion criteria

In accordance to the inclusion criteria, the systematic review identified relevant English-published observational studies, which examined the effectiveness of COVID-19 vaccines among the: (1) Elderly populations who were ≥ 60 years old; (2) recipient of at least 2 doses of mRNA (Pfizer-BioNTech and Moderna) vaccines; (3) during the predominance of Delta (B.1.617.2) or Omicron (B.1.1.529/BA); and (4)studies which examined subjects as COVID-19 positive based on a positive Reverse Transcription Polymerase Chain Reaction (RT-PCR or PCR) tests as well as studies which compared and examined the incidence of COVID-19, infection, hospitalization, admission to intensive care unit (ICU), intubation,and death.This systematic review, however, will not include: (1) Systematic review and meta-analysis studies, case reports, case series, reviews, editorials, conference papers, letters, and correspondence; (2)studies on animals; (3) studies with mathematical modelling analysis; (4) studies with insufficient data to calculate the prevention rate of COVID-19; (5) studies with immunocompromised subjects; (6) studies which did not have an unvaccinated subjects to compare; (7) studies that did not use SARS-CoV-2 vaccination as the exposure; (8) duplicate studies or studies with overlapping participants; and (9)studies that did not explain how COVID-19 subjects were determined (Figure 1).

Data extraction and outcomes

The review process underwent 4 stages: (1) All the papers found within the identified databases were examined and the publication year, study titles, authors, and abstracts were imported into an Excel spreadsheet; (2) the records were managed, screened and duplicates were eliminated manually by assessing the study title, authors, and abstract for inclusion; (3) only those with titles and abstracts that match the inclusion criteria were retrieved and carefully evaluated for full text review; and (4) using a separate Excel spreadsheet, a data extraction sheet was developed to independently extract the general study characteristics (author and publication year, study design, location, purpose, study population,including age of study population, variant of concern, vaccine type, number of doses received, outcome measures, vaccine effectiveness, and results).All qualified studies for systematic reviews were imported, stored, and managed in EndNote20.

Figure 1 Flow diagram of study identification and selection process.

The studies included in the review were assessed based on: (1) Age of study population; (2) variant of concern; (3) type of vaccine used; and (4) effectiveness of vaccines based on outcome measures.The effectiveness of COVID-19 mRNA vaccines in reducing morbidity and mortality were examined by comparing the following outcomes amongst the selected studies:

(1) Effectiveness of COVID-19 mRNA vaccines to reduce morbidity in terms of infections, hospitalization, admission to ICU and intubation;

And (2) effectiveness of COVID-19 mRNA vaccines to reduce mortality or deaths.

This study did not require the approval of an ethical committee or an Institutional Review Board since data collection and synthesis were gathered from already published studies in which proper consent or approvals would have been obtained by the researchers.

Quality assessment and data synthesis

The methodological quality of these observational studies was assessed through the risk of bias using ROBINS-I tool (risk of bias in non-randomized studies of interventions) and were analyzed using a narrative synthesis method which gathered the information from several sources and employed words and text to summarize and explain the findings since meta-analysis is not practical due to significant heterogeneity between the studies[31,32].

RESULTS

Study selection process and study characteristics

After searching 9 different databases, 1,021 studies were identified from PubMed (n= 454), Cochrane (n= 22), CINAHL (n= 54), Scopus (n= 20), EMBASE (n= 58), Web of Science (n= 10), medRxiv (n= 312),Research Square (n= 37), and Google Scholar (n= 54).From the preliminary review, 20 duplicates, 626 unrelated, and 12 unretrieved studies were excluded, leaving 363 studies were moved for title andabstract screening.As per the inclusion and exclusion conditions in the eligibility criteria, 354 studies were excluded for the following reasons: (1) Irrelevant sample population and studies (n= 89); (2)mathematical modelling, systematic reviews, and case studies (n= 47); (3) no matched control group (n= 7); (4) studied different vaccine and variant of concern (n= 168); (4) did not have sub-analysis for elderly population (n= 43); and (5) overlap in study population (n= 1).As a result, only 8 studies were included for systematic review[33-40].PRISMA Flow Diagram summarized the literature selection process (Figure 1).

Table 1 Characteristic of studies included for vaccine effectiveness

Among these studies, 3 were published and 5 were published on the pre-print platforms[33-40].All of the 8 studies used observational study designs such as cohort, case control, and cross-sectional studies[33-40].These studies reported the effectiveness of Pfizer-BioNTech (n= 8) and Moderna (n= 4)vaccines, with 7 studies examining 2 doses, 2 studies examining 2nd and booster doses, and 1 study examining booster dose in reducing COVID-19 morbidity and mortality during the prevalence of Delta(B.1.617.2) and Omicron (B.1.1.529/BA) variants[33-40].Study locations were in New York, Finland,Canada, Costa Rica, Qatar, Greece, and Brazil published between 2021 (n= 3) and 2022 (n= 5)[33-40].The studies compared an estimated total sample size of 8740562 vaccinated elderly people and 9658245 unvaccinated elderly cohorts from an estimated total study population of 26535692 which evaluated the effectiveness of mRNA vaccines of an adult population including elderly cohorts who were 50 years old and older[33-40].Although the goal of the study is to focus on elderly subjects who were 60 years old and older, some of the selected studies in this review, grouped the elderly subjects from 50 years old to include 60 years old and older subjects[35-38].The largest sample size of vaccinated elderly people was 3479102 and 8138482 unvaccinated elderly cohorts while the smallest sample size of vaccinated and unvaccinated elderly people was 45345 and 1272, respectively[34,35,39].The outcome measures used by the selected studies defined morbidity as infection (n= 5), hospitalization (n= 6), admission to ICU and intubation (n= 2) and mortality or death (n= 4) as outcome measurements[33-40].The characteristics of included studies are shown in Tables 1 and 2.

Risk of bias

The risk of bias was evaluated by following ROBINS-I tool (risk of bias in non-randomized studies of interventions)[31].All of the 8 observational studies were rated to have moderate risk of bias mainlydue to lack of control for confounders such as comorbidities or socioeconomic status like age and occupation, outbreak data such as location and time of test, and other risk-taking behavior modification[33-40].Due to the dependence in surveillance data which were subject to incomplete information, 5 studies received a moderate risk of bias score because of missing data, while 3 studies due to misclassification of measurement of outcomes, were rated with moderate bias[35-40].Table 3 shows the results of ROBINS-I risk of bias assessment of observational studies.

Table 2 Characteristic of participants included for vaccine effectiveness

Synthesis of results

Vaccine effectiveness against infection:5 of the 8 studies (36%) reported the effectiveness of vaccines using infection as an outcome measure[34,35,37,38,40].Among these, 2 of the studies used booster dose to evaluate vaccine effectiveness against asymptomatic and symptomatic infections while 3 studies assessed the effectiveness of 2 doses of vaccines[34,35,37,38,40].The findings from these studies revealed that 2 doses of mRNA vaccines offer 83%-89% protection against infection, while other studies revealed vaccine’s protection level against infection at 40%-63%, marginally lower for 65 years old when compared to Moderna, Pfizer-BioNTech vaccine was reported to have slightly lower efficacy against infections with indications of declining protection over a period of time[34,35,37,38,40].

Vaccine effectiveness against hospitalization:3 of the 8 studies (21%) reported the effectiveness of vaccines using hospitalization as an outcome measure which demonstrated 92% efficacy for older people[33,35,36].Similarly, when compared to Moderna, Pfizer-BioNTech vaccine have lower marginal protection against hospitalization with indication of waning effectiveness against hospitalization, 6 months after the 2nddose[33,35].

Vaccine effectiveness against ICU admission and intubation:2 out of the 8 studies (14%) reported the effectiveness of vaccines using admission to ICU and intubation as outcome measures[33,39].The study on ICU admissions revealed that Pfizer-BioNTech vaccine’s protection waned from 98% down to 85%after 6 months among 70 years old and older[33].Similar findings was observed when intubation was used as an outcome measure, which revealed diminished vaccine effectiveness from 96.9% down to 86%, in 6 months among 60 years old and older populations but was restored at 97.6% by booster dose[39].

Table 3 ROBINS-I risk of bias assessment of observational studies

Vaccine effectiveness against death:4 out of the 8 studies (29%) reported the effectiveness of vaccines using death with hospitalization as outcome measures[34,38-40].The findings showed that although 2 doses of mRNA vaccine can prevent death, it offers a marginally limited protection against death among 75 years old and older with indications of diminishing protection which was only restored by a booster dose[38-40].Additionally, the finding showed that Pfizer-BioNTech has marginally higher protection level against death at 87% when compared to Moderna at 77%[34].The outcomes of included studies for vaccine effectiveness are shown in Table 4.

DISCUSSION

While clinical trial data on Pfizer-BioNTech and Moderna vaccines demonstrated 94% effectiveness among the elderly, the results in this study showed that the effectiveness of mRNA vaccines in realworld settings is marginally lower against COVID-19 infection, hospitalization, ICU admission and intubation, and deaths during the predominance of Delta and Omicron variants[33-40].

The results in this systematic review further strengthen and supplement the increasing evidence on the real-world effectiveness of mRNA vaccines.While the inclusion and exclusion criteria of this review limits a variety of similar studies in the data analysis, for discussion purposes these studies echoed similar findings.A study conducted in United Kingdom revealed that vaccine effectiveness for ≥ 60 years old is 42.3%[41].The same observed pattern is reported for ≥ 75 years old in a study conducted in Israel, in a case-control study conducted among US military personnel and in a test-negative design study conducted in Malaysia[42-44].Using random-effects model on 15 observational studies to estimate the pooled vaccine effectiveness (VE) with 95% confidence intervals for each vaccine type against each variant, the systematic review and meta-analysis conducted by Zhanget al[45] revealed a limited vaccine effectiveness among ≥ 65 years old.The result in this study also align with the result in our study and with the findings of other studies focusing on vaccine effectiveness in the elderly during the predominance of Delta and Omicron variants[46-48].Utilizing the same research model, a contrasting result was reported by Liet al[49] when they evaluated the effectiveness of vaccine in over 30000 participants aged 60 years and older.This systematic review and meta-analysis however, largely focused on randomized controlled trials which may have skewed the outcomes.Given that clinical trials on COVID-19 vaccines are conducted under controlled clinical conditions from volunteer subjects of targeted age groups, these studies are not able to take into account the abilities of COVID-19 to mutateand evade the vaccine[46,47,50-53].Therefore, the tangible effect of vaccines can be substantially different from the real-world which may not necessarily illustrate the authentic effectiveness of vaccines.Furthermore, although interventional studies such as clinical trials are more methodologically sound, observational studies are more reliable since they produce practical and realistic results that are grounded from real-world experiences.

Table 4 Outcomes of studies included for vaccine effectiveness

old (2 doses):15539/79.84%; 60-74 yr old (3 doses, within 2 mo): 15053/64%; 60-74 yr old (3 doses, after 2 mo): 3273/66.36%; ≥ 75 yr old (3 doses, within 2 mo): 116955/89.86%;≥ 75 yr old (3 doses,after 2 mo):64495/99.36%2750/52.81%; 60-74 yr old (3 doses,within 2 mo):50/68.50%; 60-74 yr old (3 doses, after 2 mo): 51/96.01%; ≥75 yr old (3 doses,within 2 mo):511/90.41%; ≥ 75 yr old (3 doses, after 2 mo): 2964/72.40%doses, within 2 mo): 88.4%; 60-74 yr old (3 doses, after 2 mo):90.4%; ≥ 75 yr old (3 doses,within 2 mo): 77.3%; ≥ 75 yr old (3 doses, after 2 mo):78.5%; Hospitalization or death: 60-74 yr old (2 doses):63.4%; ≥ 75 yr old (2 doses):40.7%; 60-74 yr old (3 doses,within 2 mo): 88.4%; 60-74 yr old (3 doses, after 2 mo):90.4%; ≥ 75 yr old (3 doses,within 2 mo): 77.3%; ≥ 75 yr old (3 doses, after 2 mo): 78.5%

By focusing on Delta and Omicron variants, we hypothesize that much of the previous research on vaccine effectiveness only included earlier variants which may have skewed the results for newer and more dominant variants like Delta and Omicron.We also aim to provide value in understanding the effectiveness of mRNA vaccines by comparing their effectiveness in real-world settings.While the results in this study reported a marginal difference in effectiveness between Moderna and Pfizer-BioNTech vaccines, the minor difference on an absolute scale can be significant when considering world-wide population for vaccination[54].

Furthermore, the observed waning effectiveness of vaccine in this study supports the findings of other studies which suggested that the diminishing effectiveness of vaccine is due to the extensive abilities of COVID-19 virus to evolve and generate new variants which allow them to avoid the effects of the vaccines[51-53].The ability of Delta and Omicron variants to elude sensitivity to antibody neutralization was observed to decline over time making the vaccine less effective[41,55-58].Consistent to the findings of this study, this imply that 2 doses of mRNA vaccines is inadequate and only provides interim protection against COVID-19 infection, hospitalization, ICU admission and intubation, and deaths[58-61].Because of the vaccine’s natural diminishing effectiveness, the importance of booster dose to restore its efficacy is vital in providing additional protection against emerging variants[33,34,39,40,49,62-64].This position is in line with the study conducted in an elderly long-term care facility where the effectiveness of vaccine was observed to only have improved after the second dose, along with other studies pointing out that booster dose can provide significant protection and is the most effective approach to COVID-19 prevention[59,63,65].

Strengths and limitations

This study provides useful information on the effectiveness of mRNA vaccines in the real-world settings which are not under a regulated condition of clinical trials.Specifically, the strengths of this study made use of an inexpensive design that is reproducible since it rests on an organized search strategy, strong procedure with the inclusion of literature from pre-print servers.It also included a broad range of possible outcome measures to include as many studies as possible that can provide relevant information on the topic.Furthermore, this systematic review included research studies from different parts of the world that have relatively large representatives of elderly population with longer follow up which is useful in minimizing selection bias.

The findings of this study should be cautiously interpreted due to certain limitations.First, the included literature were observational studies which have restrictions in statistical power.Second, since there is limitation to access the data used by the research studies, this study has a risk in information bias.Third, due to the missing data and estimation of some studies, a degree of misclassification further delimits this study.Fourth, in exchange of large sample sizes, this study sits on a potential bias of unmeasured confounder such as comorbidities or socioeconomic status like age and occupation,outbreak data such as location and time of test, and other risk-taking behavior modification.Finally, the use of heterogenous outcome measures creates limitation due to potential classification errors.

CONCLUSION

As a response to the rapidly evolving COVID-19 outbreak, many research studies were organized and carried out resulting in highly heterogeneous outcome measurements.From a research perspective, this heterogeneity inhibits the comparison, contrast, and integration of the results which makes data pooling across different studies problematic.Therefore, this systematic review suggests that, while pharmaceutical intervention like vaccination is important to fight an epidemic, utilizing common outcome measurements or carrying out studies with minimal heterogeneity in outcome measurements, is equally crucial to better understand and respond to an international health crisis.Notwithstanding these limitations, the consistent findings of this review indicated waning of vaccine effectiveness over time,implying that a large proportion of the vaccinated population, particularly the elderly, may lose protection unless booster doses are rolled out to restore the effectiveness of the vaccine (Supplementary Table 1).

ARTICLE HIGHLIGHTS

FOOTNOTES

Author contributions:Palalay H conceived the manuscript from study design, literature search, study selection process, data extraction, analysis, and synthesis; Tafuto B reviewed the draft, added critical comment and intellectual content, and participated in most of the study steps; Vyas R provided additional input and guidance; All authors read and approved the final manuscript for submission and take full responsibility for the content.

Conflict-of-interest statement:No conflict of interest to declare.

PRISMA 2009 Checklist statement:The authors have read the PRISMA 2009 Checklist, and the manuscript was prepared and revised according to the PRISMA 2009 Checklist.

Open-Access:This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers.It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BYNC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is noncommercial.See: https://creativecommons.org/Licenses/by-nc/4.0/

Country/Territory of origin:United States

ORCID number:Harvey Palalay 0000-0002-1311-5224; Riddhi Vyas 0009-0007-4322-3174; Barbara Tafuto 0000-0002-7489-3281.

S-Editor:Liu JH

L-Editor:A

P-Editor:Yu HG