Field evaluation of COVID-19 rapid antigen test: Are rapid antigen tests less reliable among the elderly?

Irena Tabain, Djivo Cucevic,Nikola Skreb, Anna Mrzljak, Ivana Ferencak, Zelika Hruskar, Anita Misic, JosipaKuzle,Ana Marija Skoda, Hrvojka Jankovic,Tatjana Vilibic-Cavlek

Abstract

Key Words: Coronavirus disease 2019; Severe acute respiratory syndrome coronavirus-2; Rapid antigen test; Real-time reverse transcription-polymerase chain reaction; Croatia

lNTRODUCTlON

Coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) originated in Wuhan, Hubei province, China, in December 2019 and was declared a pandemic by the World Health Organization (WHO) on March 11, 2020. One of the strategic objectives in response to the pandemic was the early identification of infected individuals[1]. According to the WHO interim guidance published on March 2, 2020, nucleic acid amplification tests such as quantitative reverse transcription-polymerase chain reaction (RT-PCR) are considered a gold standard for diagnostic detection of SARS-CoV-2[2]. RT-PCR takes several hours to detect the SARS-CoV-2 RNA. Furthermore, it requires trained staff and is a high-priced molecular diagnostic tool. Bearing all this in mind, RT-PCR as a diagnostic tool has been challenging worldwide. In addition, highly sensitive and specific tests are consequential in identifying infected individuals and subsequently implementing control measures to limit the outbreak.

In contrast to RT-PCR, antigen-detection rapid diagnostic tests (RATs) are less expensive, easier to use, and provide results more rapidly. These methods are designed to directly detect SARS-CoV-2 proteins produced during the viral replication in respiratory secretions. RAT testing should be conducted by trained personnel within the first 5-7 d following the onset of symptoms[3]. Since the molecular capacity is difficult to scale-up and there was a need to expand the COVID-19 testing capacity due to the increasing numbers of cases at the end of 2020, Croatia implemented the RAT as a part of the national strategy for handling the SARS-CoV-2 pandemic[4].

This study evaluated the performance of the first RAT used at the Croatian Institute of Public Health (CIPH) during the second pandemic wave.

MATERlALS AND METHODS

The evaluation period included samples from November 11, 2020 to January 23, 2021. According to the national strategy protocol, all symptomatic patients who self-reported at least one characteristic COVID-19 symptom no longer than 5 d prior to testing were subjected to RAT. Self-reported symptoms were concisely revised by a medical doctor, after which a nasopharyngeal swab was taken and placed into the viral transport medium. Then the swabs were transported to the laboratory and tested within 60 min from sampling using the Humasis COVID-19 Ag Test Kit (Humasis Co., Ltd., Gyeonggi-do, Republic of Korea), anin vitrodiagnostic test based on an immunochromatographic assay designed for the qualitative detection of SARS-CoV-2 nucleocapsid and receptor binding domain antigens according to the manufacturer’s instructions. According to the internal protocol, positive RAT (containing both test and control lines) was considered SARS-CoV-2-positive and was not further tested. Testing was repeated if the rapid antigen test was invalid (containing no control line). Negative tests (containing only the control line) were subjected to confirmatory RT-PCR testing. RT-PCR testing was performed from the same nasopharyngeal swab. Viral RNA was extracted using the automated extraction system Nextractor-NX-48 Genolution. Real-time RT-PCR was performed on the Applied Biosystem Instrument QuantStudio5 using the RT-PCR assay (GeneFinderTM COVID-19 Plus RealAmp Kit; Osang Healthcare Co Ltd, Republic of Korea) according to the manufacturer’s instructions. Basic demographic analysis and descriptive statistic tests were performed. C2, Mann-Whitney U, and Kruskal-Wallis tests were performed using an online tool (https://www.socscistatistics.com/tests) on the data concerning RTPCR results and average RT-PCR cycle for the three age groups (＜ 18, 19-65, ＞ 65 years). Weekly positive, negative, and false-negative RAT were correlated with average Ct values for false-negative antigen tests and 7 d average COVID-19 incidence in Croatia. P ＜ 0.05 was considered statistically significant.

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Croatian Institute of Public Health (Protocol No. 030-02/20-05/1, Approved on May 7, 2020).

RESULTS

During the evaluation period, 2490 RATs were performed on nasopharyngeal swabs obtained from patients with a positive history of SARS-CoV-2 infection in the last 5 d prior to sampling. The largest number of patients tested was 545 in the week from November 11, 2020 to November 14, 2020. We observed a steady decline in symptomatic patients during the study period (Figure 1).

Of the 2490 performed antigen detection tests on symptomatic patients, 953 patients tested positive, yielding an overall positive RAT prevalence of 37.90%, while 1537 patients tested negative and were subjected to confirmatory RT-PCR test according to WHO and national CIPH guidelines. The dynamics of the weekly ratio of positive results in the observed period are shown in Figure 2. The highest prevalence of 44.83% was seen in the 1stwk of 2021, while the lowest prevalence of 22.62% was observed the week prior. Among the 1537 samples that underwent both RAT and RT-PCR tests, RT-PCR confirmed 1271 patients as true negatives, while 266 patients were RT-PCR positive. Those patients were classified as false negatives on the antigen detection test, and the calculated negative predictive value of the RAT was 82.69%. The highest ratio of false-negative results (28.57%, from 196 negative RATs) was observed in the 49thwk of 2020, whereas the lowest ratio (2.86%) was observed in the 2ndwk of 2021 (Figure 3). False-negative patients were summarized according to the RT-PCR cycle in which they were identified as positive, as shown in Figure 4. The Shapiro-Wilk test of normality revealed a not-normal, asymmetrical distribution with a long left tail. Sixty-five percent of them had a lower viral load (Ct ≥ 25). The lowest Ct value observed was 18, but for only 5.62% of false-negative patients, the Ct value was ≤ 20.

Spearman’s correlation coefficient was calculated to determine the strength of correlation between the total number of tested patients and false negatives and the possible correlation between false negatives and Ct values. The results showed a significant correlation between the total number of patients tested and false negatives (P ＜0.05). There was also a significant correlation between the number of negatives and false negatives (P ＜0.05), which showed that an increase in the sample size of tested patients and an increase in the number of negative results would increase the number of false-negative results. Meanwhile, Spearman's correlation coefficient did not show any significant correlation between Ct values and the prevalence of false-negative RAT results (P= 0.52) (Figure 5). This showed that during this time, an increase in the number of cycles in the confirmatory RT-PCR tests did not decrease the occurrence or prevalence of positive RT-PCR results.

Interestingly, Spearman’s coefficient of correlation between the weekly average of newly diagnosed in Croatia and the number or percentage of false-negative RATs also did not reach a significance level (P= 0.2) (Table 1). Graphical representation of the correlation between the total number of tested patients and the number of false-negative results, together with the corresponding calculated predictive model, showed that approximately 10% of false-negative results could be expected each time (Figure 6). We also analyzed if age causes any variation in the occurrence of negative and false-negative results and if there is any difference in mean Ct value with different age groups. Patients were subdivided into three age groups: ＜ 18, 19-65, ＞ 65. Statistical analysis with theχ2test showed a significant difference in the occurrence of false negatives between the youngest group (＜ 18 years of age) and the adult (19-65 years of age) or elderly group (＞ 65 years of age) (P ＜0.0001), as well as the difference between the adults and elderly (P= 0.033). The overall difference between the three groups was significant (χ2test;P ＜0.01).

Table 1 Percentage of false-negative rapid antigen-detection tests and reverse transcription-polymerase chain reaction cycle values by age groups

Figure 1 Weekly number of symptomatic patients tested by rapid antigen-detection tests.

Figure 2 Ratio of positive rapid antigen-detection tests results by week. RAT: Rapid antigen.

Figure 3 Ratio of false-negative rapid antigen-detection tests results by week.

Figure 4 Number of rapid antigen-detection tests false-negative patients testing positive per each reverse transcription-polymerase chain reaction cycle. Ct: Reverse transcription-polymerase chain reaction cycle.

Figure 5 Correlation between reverse transcription-polymerase chain reaction cycle values and prevalence of false-negative results.

Kruskal-Wallis test showed a significant statistical difference in mean Ct value among all three age groups (P= 0.012). Subsequently, Mann-Whitney U-tests revealed that the overall differences arose from the differences between the oldest group compared to any of the younger ones, whereas the difference between the 0-18 and 19-65 age groups did not reach a significance level.

不斷強(qiáng)化長(zhǎng)江流域大型生產(chǎn)建設(shè)項(xiàng)目水土保持監(jiān)督檢查。對(duì)2010—2012年部批在建的139個(gè)項(xiàng)目進(jìn)行全面監(jiān)督檢查，加強(qiáng)對(duì)丹江口庫(kù)區(qū)及上游水土保持二期工程等項(xiàng)目的技術(shù)指導(dǎo)。組織開(kāi)展流域農(nóng)村水利數(shù)據(jù)建設(shè)用戶需求分析及數(shù)據(jù)框架建設(shè)，實(shí)施流域大中型灌區(qū)配套改造調(diào)研。加強(qiáng)水利安全監(jiān)督工作，認(rèn)真完成水利部委派的水利稽察任務(wù)，積極開(kāi)展委屬工程建設(shè)項(xiàng)目稽察工作。

Figure 6 Predictive model of correlation between the total number of patients and the number of false negatives.

DlSCUSSlON

Published studies have reported a wide range of data concerning RAT accuracy and consequently developed various opinions on their use in COVID-19 diagnostics. However, these data evaluated numerous different RAT; therefore, the results are not entirely comparable. Two studies evaluating Abbot's Panbio COVID-19 test reported negative predictive values of 99.5%[5] and 94.6%[6]. The evaluation of COVID-19 Ag Respi-Strip (Coris BioConcept) gave the results for negative predictive values (85.25%), similar to our results (82.69%)[7]. There are only a few studies that evaluated Humasis Covid-19 RAT. Klajmonet al[8] tested 189 samples using RT-PCR and RAT simultaneously. The calculated negative predictive value (97.22%) was higher than our study but more similar to Abbot's PanBio RAT (94.6% and 99.5%, respectively). Magyaret al[9] evaluated the performance of ten commercially available SARS-CoV-2 RAT, including the Humasis COVID-19 Ag test, compared to RT-PCR. The mean Ct value (24.1 ± 5.53) was similar to our findings (23.86-26.58) and overall negative predictive value (79.0%).

During the study period, declining numbers of tested and confirmed COVID-19 cases were observed, which can be attributed to the epidemiological situation in Croatia characterized with declining incidence and prevalence of COVID-19[4,10]. The main strength of this in-field evaluation is the sample size, whereas the limitations include various swab-takers and six RAT operators/interpreters; however, they were always competent and well-educated personnel (e.g.,laboratory personnel or medical doctors). Furthermore, the lower negative predictive value could partially result from the declining incidence and prevalence of COVID-19 in Croatia during the studied period. The retrospective nature of this study can be seen as a disadvantage in terms of reproducibility and scientific accuracy. However, on the other hand, it depicts a realistic, day-to-day approach to COVID-19 diagnostics. National and European Centre for Disease Prevention guidelines were, however, strictly followed. According to WHO recommendations, RAT with a minimum of 80% sensitivity and 97% specificity was used in patients with symptoms consistent with COVID-19[11]. Since symptomatic persons may carry a higher risk of transmitting the infection, this study investigated only symptomatic people. Therefore, RATpositive samples were not further tested; thus, the sensitivity, specificity, and positive predictive value of the rapid antigen test included in the study could not be determined. The negative predictive value of 83% was lower than those stated by the manufacturer (2 false-negative tests of 22, giving a negative predictive value of 91%). However, the patients self-reported the symptoms and their durations, and although reviewed by a doctor before sampling, this could cause a decrease in the negative predictive value.

To the best of our knowledge, this study is the first to address the age groups differences, finding out significant differences concerning the false-negative RAT rates and the Ct values of these specimens. The youngest age group was significantly less likely to be false negative, whereas the oldest age group's Ct values were significantly lower than those of the remaining two groups. Although due to the difference in the number of tested patients between the age groups, further research is needed. Based on our results, we recommend that the elderly patients could preferably test by RT-PCR test due to their increased vulnerability towards COVID-19 infection and possibly lower performance of rapid antigen tests in that age group.

The national recommendation that negative RAT results in symptomatic patients suggestive of COVID-19 should be further tested by RT-PCR seems like a good solution because negative RAT results do not rule out the infection, especially in the high prevalence settings during study period[4]. Identifying and confirming COVID-19 in symptomatic persons is important to provide appropriate care and implement public health measures without delay[11]. Therefore, despite the limitations of RAT, including inferior results to RT-PCR[8,12], their previously described use is effective and can be a support for RT-PCR testing of SARS-CoV-2. Nevertheless, the RAT cost is significantly lower than RTPCR[13].

CONCLUSlON

In conclusion, RAT is an acceptable tool in COVID-19 diagnostics that saves time, human and financial resources and has reasonably accurate results if the viral load is high (Ct ＜ 25), but caution is needed. In order to prevent viral spread, in the settings when it is feasible, all RAT negative samples should be subjected to RT-PCR analysis.

ARTlCLE HlGHLlGHTS

FOOTNOTES

lnstitutional review board statement:The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Croatian Institute of Public Health (protocol code 030-02/20-05/1, approved on May 7th, 2020).

lnformed consent statement:Patients were not required to give informed consent to the study because the analysis used anonymous clinical data that were obtained after each patient agreed to treatment by written consent.

Conflict-of-interest statement:The authors declare no conflict of interest.

Data sharing statement:No additional data are available.

Open-Access:This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BYNC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is noncommercial. See: https://creativecommons.org/Licenses/by-nc/4.0/

Country/Territory of origin:Croatia

ORClD number:Irena Tabain 0000-0002-2518-522X; Djivo Cucevic 0000-0002-0969-4657; Nikola Skreb 0000-0003-1730-8768; Anna Mrzljak 0000-0001-6270-2305; Ivana Ferencak 0000-0002-9285-9052; Zeljka Hruskar 0000-0002-9078-1773; Anita Misic 0000-0001-7246-3205; Josipa Kuzle 0000-0003-0434-7514; Ana Marija Skoda 0000-0002-3868-225X; Hrvojka Jankovic 0000-0002-9497-3393; Tatjana Vilibic-Cavlek 0000-0002-1877-5547.

S-Editor:Wang LL

L-Editor:Filipodia

P-Editor:Wang LL