key: cord-0004025-k8wfyj9b
authors: Paweska, Janusz T.; Moolla, Naazneen; Storm, Nadia; Msimang, Veerle; Conteh, Ousman; Weyer, Jacqueline; van Vuren, Petrus Jansen
title: Evaluation of Diagnostic Performance of Three Indirect Enzyme-Linked Immunosorbent Assays for the Detection of IgG Antibodies to Ebola Virus in Human Sera
date: 2019-07-24
journal: Viruses
DOI: 10.3390/v11080678
sha: 50897189565f10bfa3b4884058b5a51cf75c091e
doc_id: 4025
cord_uid: k8wfyj9b

Filovirus serological diagnosis and epidemiological investigations are hampered due to the unavailability of validated immunoassays. Diagnostic performance of three indirect enzyme-linked immunosorbent assays (I-ELISA) was evaluated for the detection of IgG antibody to Ebola virus (EBOV) in human sera. One I-ELISA was based on a whole EBOV antigen (WAg) and two utilized recombinant nucleocapsid (NP) and glycoproteins (GP), respectively. Validation data sets derived from individual sera collected in South Africa (SA), representing an EBOV non-endemic country, and from sera collected during an Ebola disease (EBOD) outbreak in Sierra Leone (SL), were categorized according to the compounded results of the three I-ELISAs and real time reverse-transcription polymerase chain reaction (RT-PCR). At the cut-off values selected at 95% accuracy level by the two-graph receiver operating characteristic analysis, specificity in the SA EBOV negative serum panel (n = 273) ranged from 98.17% (GP ELISA) to 99.27% (WAg ELISA). Diagnostic specificity in the SL EBOV negative panel (n = 676) was 100% by the three ELISAs. The diagnostic sensitivity in 423 RT-PCR confirmed EBOD patients was dependent on the time when the serum was collected after onset of disease. It significantly increased 2 weeks post-onset, reaching 100% sensitivity by WAg and NP and 98.1% by GP I-ELISA.

High-mortality and occurrence of Ebola disease (EBOD) outbreaks almost each year in the last three decades [1, 2] are of great public health concern. The unprecedented and first epidemics of EBOD in West Africa from 2013 to 2016 [3] [4] [5] [6] and the large-scale re-emergence of EBOD in the Democratic Republic of the Congo in 2018 and 2019 [2, 7] exemplify the devastating health, humanitarian, and socio-economic impacts and challenges in containing EBOD outbreaks in resource-poor and politically conflicted settings [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] .

The increasing incidence, severity and size of EBOD outbreaks highlight the importance of developing and standardizing diagnostic tools for EBOD rapid diagnosis and post-epidemic surveillance. The most devastating EBOD outbreak to date, the West African outbreak, prompted determining seroprevalence rates, infection risk population studies, and assessing occurrence of asymptomatic infections [35] .

The purpose of this study was to evaluate and compare the diagnostic performance of EBOV IgG-indirect ELISAs based on antigens produced by classical virological and recombinant protein expression methods using human serum panels from EBOV non-infected and EBOV infected humans.

A total of 1372 individual banked sera collected in South Africa (SA) and Sierra Leone (SL), were used. SA sera (n = 273) were originally submitted for various routine diagnostic testing to the Centre for Emerging Zoonotic and Parasitic Diseases of the National Institute for Communicable Diseases (NICD), Johannesburg. These sera represented specimens collected from 273 individuals in EBOV non-endemic country and are regarded as IgG EBOV negative reference serum panel. Ethics clearance for using SA human banked sera in the development and validation of diagnostic assays was obtained from Human Ethics Committee, University of the Witwatersrand, Johannesburg, SA, Clearance Certification No. M1809219, 23 February 2018.

SL blood specimens were originally submitted for EBOV RT-PCR testing to the SA modular high-biosafety field Ebola diagnostic laboratory (FEDL) established in August 2014 near Freetown, in international response to the rapidly increasing number of EBOD cases in SL [36] . Selected aliquots of processed sera were shipped on dry ice from FEDL to the NICD's biosafety level 4 (BSL- 4) In total, 1099 sera obtained from SL patients suspected of having EBOD between August 2014 and March 2015 [36] were used. Of those sera 423 were from EBOV RT-PCR confirmed cases for whom date of disease onset was recorded on EBOD case submission form. Ribonucleic acid from blood was extracted using the QIAamp viral RNA kit (Qiagen, Hilden, Germany) according to the manufacturer's instructions. RT-PCR was performed using the QIAGEN One-Step RT-PCR kit (Qiagen, Hilden, Germany) as per the manufacturer's instructions using previously described primers and probes targeting the EBOV L gene [37] . Specimens with Ct values below 40 were considered positive for EBOV RNA [36] and thus confirming EBOV infectious status of a patient. Serum specimens from the SL EBOV RT-PCR positive patients were regarded as SL reference positive serum panel.

The remaining specimens were from 676 EBOV RT-PCR negative individuals whose sera tested negative for anti-EBOV IgG by all IgG ELISAs evaluated in this study using cut-offs derived from SA IgG EBOV negative reference serum panel. This serum panel was regarded as SL reference negative serum panel.

The source of positive control serum (C++) was the 1996 imported EBOD case from Gabon to SA [38, 39] . The case was a Gabonese physician who had been brought from Libreville, Gabon on 27 October and admitted to a private hospital in Johannesburg where a nurse died after being infected through exposure to his blood. Negative control serum (reference no. CEZPD SVPL 502/2017) was obtained from South African Blood Service. It tested negative for anti-EBOV IgG and IgM antibodies by in-house ELISA using procedures published by Ksiazek et al. [40] .

Internal quality control (IQC) data were generated as described previously [41] . Upper and lower control IQC limits together with coefficients of variations ≤10% for replicates of positive internal control serum and test sera were applied as an assay acceptance criteria.

Diagnostic performance of three indirect ELISAs (I-ELISA) for the detection of anti-EBOV IgG antibody in human sera was evaluated. These ELISAs were based on antigens prepared from infected cell lysate, or on recombinant antigens produced in a bacterial or mammalian expression system.

The EBOV whole antigen (WAg) was produced as previously described with some modifications [40] . Vero C1008 cells (ATCC, Manassas, VA, USA) were infected with the SPU 220/96 isolate of EBOV (4th passage in Vero cells) isolated from the serum of a nurse who contracted a fatal infection from a Gabonese physician admitted to a private hospital in South Africa in 1996 [39] . After incubation at 37 • C when cytopathic effect was observed, cells and supernatant were collected and snap freeze-thawed at −70 • C and 37 • C. Lysed cells and supernatant were separated by centrifugation (10,000× g, 4 • C, 15 min). The collected supernatant was gamma irradiated with 30,000 Gy to inactivate the virus. Saturated ammonium sulphate solution (100%) (Sigma Aldrich, Merck, Kenilworth, NJ, USA) was slowly added with constant stirring to the irradiated supernatant to a final concentration of 35%, and the mixture was incubated at 4 • C overnight. The antigen-containing pellet precipitate was collected by centrifugation at 20,000× g for 30 min at 4 • C and resuspended in one tenth of the original supernatant volume of phospate buffered saline (PBS) pH 7.2. The antigen was further dialysed against PBS to remove the ammonium suphate (3-4 buffer changes). The WAg preparation was aliquoted and stored at −70 • C until use. Uninfected Vero C1008 cells were prepared in the same way and used as control antigen.

The codon optimized nucleotide sequence for EBOV nucleocapsid (NP) (GenBank accession number AF086833.2) was synthesized with a C-terminal glycine linker to a 6× Histidine tag (GenScript, Piscataway, NJ, USA) and subcloned into the NcoI and XhoI restriction sites of the pET-15b expression vector (Novagen, Merck, USA). The sequence verified plasmid (pET-15b ZEbov NP) was used to transform competent BL21 Star (DE3) E. coli. A starter culture was grown overnight at 37 • C, then diluted 10 fold and allowed to reach exponential-phase growth (OD 600 of between 0.6-0.8) before protein expression was induced using 1 mM IPTG (Sigma Aldrich, Merck, USA) for 20 h at 37 • C with vigorous shaking. Cells were harvested by centrifugation, resuspended in sodium phosphate buffer (50 mM NaH 2 HPO 4 , 300 mM NaCl, pH 8.0) and lysed using a combination of Bugbuster and Lysonase (Novagen, Merck, USA) treatment, freeze-thaw cycles and sonication. The recombinant NP protein-containing soluble phase was collected by high speed centrifugation (15,000× g, 30 min, 4 • C) and loaded onto Profinity IMAC (Biorad, Hercules, CA, USA) cobalt-charged resin. The protein was left to bind overnight with gentle shaking at 4 • C, using a ratio of 1 mL resin to 16 mL soluble protein fraction. Contaminating proteins were removed by washing the resin twice with 20 packed-resin volumes of sodium phosphate buffer containing 12.5 mM imidazole (pH 8.0), using gentle centrifugation (1000× g, 2 min, 4 • C). The recombinant NP was then eluted overnight with gentle shaking at 4 • C in 2 packed-resin volumes of sodium phosphate buffer containing 250 mM imidazole (pH 8.0). Eluted protein was dialysed into 0.05 M carbonate/bicarbonate buffer, pH 9.6 (Sigma Aldrich, Merck, USA). To remove any residual contaminating proteins, the purified protein was passed through a SEC 650 size exclusion column (Biorad, Hercules, CA, USA) and the fractions containing the NP were collected. Collected fractions were pooled and concentrated using Amicon Ultra filters (Millipore, Merck, USA) with a 50 kD cut-off. The purified NP was quantified using the Bradford Concentration Assay kit (Pierce, ThermoFisher Scientific, Waltham, MA, USA) and aliquots were stored at −70 • C for later use. For the control antigen, the same process was followed using an expression vector (pET-15b) without an insert.

Recombinant EBOV glycoprotein (GP) antigen expressed in human embryonic kidney 293 T cells was obtained from Integrated BioTherapeutics (Rockville, MD, USA).

Optimal immunoreagents concentrations/working dilutions for I-ELISA were determined using standard checkerboard titration procedures [42] . Microtiter 96-wells plates (Maxisorb Immunoplates, Nunc, Roskilde, Denmark) for WAg, NP, and GP I-ELISAs were respectively coated with WAg and corresponding control antigen, NP and corresponding control antigen or with GP. For WAg and NP I-ELISAs virus and control antigens were added to rows of the top half of the plate (rows A-D: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] and the bottom half of the plate (rows E-H: 1-12), respectively. For GP I-ELISA all rows (rows A-H: 1-12) were coated with the commercial GP for which control antigen was not available. The NP and corresponding control antigen were diluted 1:2000 (stock concentration 0.4 mg/mL) in carbonate-bicarbonate buffer, pH 9.6. All the other antigens were diluted in phosphate PBS without Mg and Ca, pH 7.4; WAg and uninfected Vero C1008 cell culture antigen was diluted 1:300 and the GP was diluted 1:6000 (stock concentration 3.83 mg/mL).

Each 96-well plate had four replicates of positive serum control (C++), two replicates of negative serum control (C−) and two replicates of conjugate control (Cc). For WAg and NP I-ELISA C++ was added to wells A 1-2 and B 1-2 coated with virus antigen and to corresponding wells E 1-2 and F 1-2 coated with a control antigen. Accordingly, C− was added to C 1-2 and G 1-2, and Cc (diluent buffer) was added to D 1-2 and H 1-2. The remaining wells (A-D 3-12 and E-H 3-12) were used for testing 20 test sera in duplicate with test serum no. 1 added to wells A3 and B3 and corresponding E3 and F3; adding of the remaining test sera followed the same layout principle. For GP I-ELISA, not having control antigen, internal controls were placed as follows: C++ in wells A 1-2 and B 1-2; C− in wells C 1-2; Cc in wells D 1-2. The remaining wells (A-D 3-12 and E-H 1-12) were used for testing 44 test sera in duplicate with test serum no. 1 added to wells A3 and B3 and accordingly test serum no. 44 added to wells G12 and H12.

All reagents were added to the immunoplates at a volume of 100 µL/well unless otherwise stated. Passive adsorption onto ELISA plates was performed at 4 • C overnight and all subsequent incubations (except for substrate addition) were performed at 37 • C in a humidified chamber for 1 h. Following coating, plates were washed 3 times with 300 µL of PBS containing 0.1% Tween 20; the same washing procedure followed each subsequent stage of I-ELISAs. Plates were blocked with 200 µL 10% non-fat milk powder in PBS. After incubation, plates were washed, and control and test sera diluted 1:400 in 2% non-fat milk powder in PBS (diluent buffer) were added. Each test serum, negative control serum and conjugate control was tested in duplicate, and positive control was tested in quadruplicate. Following incubation with the sera, the plates were washed and goat anti-human IgG HRPO conjugate (Invitrogen, ThermoFisher Scientific, USA), diluted 1:10,000 in diluent buffer, was added to the wells. After incubation, plates were washed and 2,2'-azinodiethylbenzothiazoline sulfonic acid (ABTS) peroxidase (Seracare Lifesciences, Milford, MA, USA) substrate added to wells. Plates were incubated in the dark at room temperature (±22 • C) for 30 min.

Reactions were stopped by the addition of 1% sodium dodecyl sulphate (SDS) and the optical densities (OD) readings were measured at 405 nm. OD readings were converted into a percentage of the positive internal control serum (PP) using the equation as previously described [41] . Briefly, a specific activity of each serum (net OD) was calculated by subtracting the non-specific background OD in the wells with control antigen from the OD in wells with virus antigen (for NP and WAg I-ELISA only). The mean net OD readings were converted to PP using the formula: PP = (mean net of test serum/mean net OD of positive control) × 100 [41] .

Cut-off values were determined as mean +3 standard deviations (SD) of PP values recorded in SA and SL IgG EBOV negative serum panels and also selected at 95% accuracy level by the misclassification cost term (MCT) option of the two-graph receiver operating characteristics (TG-ROC) analysis available from Microsoft Excel 2010 (Redmond, WA, USA) [43, 44] using SL reference IgG EBOV negative and positive serum panels. Coefficient of variation (CV) values were calculated to measure relative variability using the formula CV = (SD of replicates/mean of replicates) × 100.

Optimisation of cut-off values by MCT TG-ROC was based on the following equation: To compare the diagnostic performance of the assays and agreements between results of matched test samples, data were analyzed in Stata 13 using McNemar's test and Cohen's kappa statistic (κ) [46] .

To evaluate the effect of serum inactivation on the levels of the detectable anti-EBOV IgG by WAg, NP and GP I-ELISAs, laboratory protocols previously shown to completely inactivate EBOV virus in human serum were used [47, 48] . Briefly, the C++ was first diluted 1:10 in PBS containing either 0.5% Triton X-100 or 0.5% Tween-20 (Sigma-Aldrich, Taufkichen, Germany) and heated at 60 • C for 15 min. Then, two-fold log 10 dilutions (from 1:100 to 1:200,480; log 10 10 2 −log 10 10 5.3 ) of untreated and treated serum was tested by each of the I-ELISAs. Each dilution was tested in duplicate on 3 separate runs. A serum titer was considered the highest sample dilution at which its PP value was ≥ the I-ELISA cut-off for at least 50% of six replicates.

Both within and between the runs, the C++ OD readings for WAg, NP and GP I-ELISAs were within the IQC lower (LCL) and upper (UCL) control limits. Within the runs, the average CV ranged between 3.54 ± 1.99 (NP I-ELISA) and 4.76 ± 2.16 (GP I-ELISA). Between the runs, the average CV ranged between 3.11 ± 1.05 (NP I-ELISA) and 4.8 ± 1.56 (GP I-ELISA) ( Table 1) . Both within and between the runs, the C− and Cc OD readings were within the IQC LCL and UCL. For WAg, NP, and GP I-ELISA, the IQC C− OD control limits ranged from −0.08 to 0.06, −0.08 to 0.09, 0.1 to 0.17, and the IQC Cc− OD control limits ranged from −0.05 to 0.06, −0.05 to 0.04, and 0.03 to 0.11, respectively. 

The distribution of I-ELISA PP values and determination of cut-offs as mean plus three standard deviations of PP values recorded by each test in SA and SL EBOV IgG negative serum panels are shown in Figure 1 . Selection and optimization of cut-offs by the MCT TG-ROC in SL serum panels is shown in Figure 2 . MCT TR-ROC was based on the non-parametric program option due to departure from a normal distribution of data sets analyzed. PP threshold values for each of the assays were similar irrespective of serum panels analyzed and the methods used for the determination of cut-offs: 13.46, 13.12 ( Figure  1A ) and 13.35 (Figure 2A) for WAg I-ELISA; 16.13, 15.15 ( Figure 1B ) and 16.44 ( Figure 2B ) for NP I-ELISA; 26.52, 25.12 ( Figure 1C ) and 26.28 ( Figure 2C ) for GP I-ELISA, respectively. The higher cut-off values for GP I-ELISA were likely due to higher ELISA noise resulting from not including a control antigen in this assay.

Irrespective of data set analyzed and cut-offs used, all the three assays evaluated in this study had high estimates of D-Sp, ranging from 97.4% to 99.6% in SA, and from 99.7% to 100% in SL negative serum panels. However, D-Sp for GP I-ELISA was generally lower in the SA negative serum panel ( Table 2 ). MCT TR-ROC was based on the non-parametric program option due to departure from a normal distribution of data sets analyzed. PP threshold values for each of the assays were similar irrespective of serum panels analyzed and the methods used for the determination of cut-offs: 13.46, 13.12 ( Figure 1A Figure 1C ) and 26.28 ( Figure 2C ) for GP I-ELISA, respectively. The higher cut-off values for GP I-ELISA were likely due to higher ELISA noise resulting from not including a control antigen in this assay.

Irrespective of data set analyzed and cut-offs used, all the three assays evaluated in this study had high estimates of D-Sp, ranging from 97.4% to 99.6% in SA, and from 99.7% to 100% in SL negative serum panels. However, D-Sp for GP I-ELISA was generally lower in the SA negative serum panel ( Table 2) .

Individual results yielded by WAg, NP and GP I-ELISAs in sera from 423 RT-PCR confirmed EBOD cases at different times post disease onset and using different cut-off values are given in Table 3 . Irrespective of the cut-offs used, the results were similar for the same assay, but the NP I-ELISA was more sensitive in detecting IgG antibody during the first two weeks post disease onset compared to WAg and GP I-ELISAs, the latter being the least sensitive. For example, when using the TG-ROC derived threshold, of 335 EBOD patients bled during the two weeks post onset, 92 (27.5%), 114 (34%), and 75 (22.4%) were positive by WAg, NP, and GP I-ELISA, respectively. Mc Nemar test indicate a disagreement of diagnostic capacity (combined DSe and DSp) between NP and WAg (p = 0.0196) or GP I-ELISA (p = 0.0002) respectively using all Sierra Leone data (n = 1099). Agreement was found between WAg and GP I-ELISA (p = 0.0707; 98.18% κ = 0.9300 ± 0.0348). Most of the discrepant (non-matching) results were recorded during the first two weeks post disease onset. After two weeks post onset, detection of IgG antibody and agreement between assays significantly improved. Of 88 sera tested 15-31 days post onset, all tested positive by WAg and NP I-ELISs irrespective of the cut-off used, and depending on the cut-off, 87 or 86 were positive by GP I-ELISA (Table 3) . Mc Nemar test indicate agreement between NP, Wag and GP I-ELISAs ranging from 99.35% (κ = 0.9684 ± 0.0362) to 100% using data from non-EBOD and diseased patients bled on day 15 or later post-onset (n = 764), except for GP and Wag or NP I-ELISAs with the cut-offs of respectively 26.28, 13.53, 16.44 (p < 0.001). Estimates of D-Se in sera collected at different times post disease onset are given in Table 4 . Mean levels of IgG responses measured by WAg, NP, and GP I-ELISAs in 423 EBOD RT-PCR confirmed cases bled at different times after disease onset are shown in Figure 3 . On average, the first seroconversions were detectable by NP on days 2-4, then on days 5-7 by NP and WAg, and from day 8 post onset by all I-ELISAs. The mean levels of IgG measured by all the I-ELISAs were not significantly different.

Estimates of D-Se in sera collected at different times post disease onset are given in Table 4 . Between 8-36 days post onset, the D-Se ranged from 82.6% (GP I-ELISA) to 92.65 (NP I-ELISA). Between 15-36 days post onset, the D-Se was 100% for both WAg and NP I-ELISA irrespective of the cut-off used, and ranged from 97.7% to 98.9% for the GP I-ELISA. Table 4 . Diagnostic sensitivity of a whole antigen (WAg), nucleocapsid (NP), and glycoprotein (GP) I-ELISAs for the detection of anti-IgG EBOV antibody in humans.

Cut-off PP 1

WAg I-ELISA 13.46 6 19/130 87.2 (80.8-92.1) 0/88 100 (95.9-100) WAg I-ELISA 13.12 7 18/131 87.9 (81.6-92.7) 0/88 100 (95.5-100) WAg I-ELISA 13.53 8 19 Mean levels of IgG responses measured by WAg, NP, and GP I-ELISAs in 423 EBOD RT-PCR confirmed cases bled at different times after disease onset are shown in Figure 3 . On average, the first seroconversions were detectable by NP on days 2-4, then on days 5-7 by NP and WAg, and from day 8 post onset by all I-ELISAs. The mean levels of IgG measured by all the I-ELISAs were not significantly different. The different inactivation protocols used did not have an adverse effect on the kinetics and the detectable levels of the anti-EBOV IgG in C++ by either I-ELISA evaluated in this study. The titers ( Table 5 ) as well as the kinetics of dose-response curves ( Figure 4) were similar before and after inactivation in each assay. 

The diagnostic decision limit or cut-off represents a serological assay test value used to dichotomize negative and positive results, and by inference, to define the infection status of an individual against a specific pathogen of disease. The relevance of data used for the determination of cut-off consequently impacts on estimates of D-Se and D-Sp and other measures of test performance [49] . Important consideration in determining a serological assay cut-off is to select sera from unequivocally infected individuals and sera from individuals who have never been infected with the agent in question. Also, in order to account for the distribution of covariate factors (genetic, nutritional, geographical, and stage of infection) that may influence the estimates of D-Se and D-Sp, the target population should preferably be sampled using simple random, systematic or stratified sampling methods [45] . These ideal conditions could not be applied during this study. An assay validation data should preferably be derived not only from testing samples from reference individuals of known history and infection status but also from the country or region in which the test is to be used. Traditionally, gold standards for selection of truly infected and uninfected subjects include isolation of the agent or pathognomonic histopathological criteria. Because a true gold standard is difficult to accomplish, relative standards of comparison are often necessary, and include results from other serological assays [50] . In order to ensure EBOV true infection status of individuals whose sera were used in our study, RT-PCR negative sera that tested negative for anti-EBOV IgG by all IgG ELISAs using cut-offs derived from SA IgG EBOV negative reference serum panel were regarded as SL reference negative serum panel. Serum specimens from the SL EBOV RT-PCR positive patients with known dates of disease onset were regarded as SL reference positive serum panel.

Various statistical analyses used in our study for the selection of the cut-off values provided similar results. A cut-off value determined as two or three SD above the mean in uninfected individuals is frequently used for the interpretation of serodiagnostic tests. However, this assumes a normal distribution of the test values in population targeted by an assay, and provides only an estimate of D-Sp [49] . Deviations from normality are often recorded in serological data and should be addressed in the selection of threshold values [51] . Therefore, we also used the TG-ROC analysis for the selection and optimization of cut-off values to account for parametric versus nonparametric distribution of test values. All I-ELISAs evaluated in our study had high estimates of D-Sp, but the D-Se was dependent on the time when the serum was taken post disease onset. High detection rate 

The diagnostic decision limit or cut-off represents a serological assay test value used to dichotomize negative and positive results, and by inference, to define the infection status of an individual against a specific pathogen of disease. The relevance of data used for the determination of cut-off consequently impacts on estimates of D-Se and D-Sp and other measures of test performance [49] . Important consideration in determining a serological assay cut-off is to select sera from unequivocally infected individuals and sera from individuals who have never been infected with the agent in question. Also, in order to account for the distribution of covariate factors (genetic, nutritional, geographical, and stage of infection) that may influence the estimates of D-Se and D-Sp, the target population should preferably be sampled using simple random, systematic or stratified sampling methods [45] . These ideal conditions could not be applied during this study. An assay validation data should preferably be derived not only from testing samples from reference individuals of known history and infection status but also from the country or region in which the test is to be used. Traditionally, gold standards for selection of truly infected and uninfected subjects include isolation of the agent or pathognomonic histopathological criteria. Because a true gold standard is difficult to accomplish, relative standards of comparison are often necessary, and include results from other serological assays [50] . In order to ensure EBOV true infection status of individuals whose sera were used in our study, RT-PCR negative sera that tested negative for anti-EBOV IgG by all IgG ELISAs using cut-offs derived from SA IgG EBOV negative reference serum panel were regarded as SL reference negative serum panel. Serum specimens from the SL EBOV RT-PCR positive patients with known dates of disease onset were regarded as SL reference positive serum panel.

Various statistical analyses used in our study for the selection of the cut-off values provided similar results. A cut-off value determined as two or three SD above the mean in uninfected individuals is frequently used for the interpretation of serodiagnostic tests. However, this assumes a normal distribution of the test values in population targeted by an assay, and provides only an estimate of D-Sp [49] . Deviations from normality are often recorded in serological data and should be addressed in the selection of threshold values [51] . Therefore, we also used the TG-ROC analysis for the selection and optimization of cut-off values to account for parametric versus nonparametric distribution of test values. All I-ELISAs evaluated in our study had high estimates of D-Sp, but the D-Se was dependent on the time when the serum was taken post disease onset. High detection rate of anti-EBOV IgG in RT-PCR EBOV confirmed cases was only recorded after two weeks post disease onset. Results of previous study in a small number of the 1995 EBOD patients in Kikwit, Democratic Republic of Congo, suggested that many patients do not have antibody early in the course of their illness, and that many may die without developing detectable antibodies to EBOV. Therefore, measurement of IgG antibody is of rather limited use in the diagnosis of acute EBOD cases [52] .

I-ELISA, represents one of the simplest ELISA formats, but can be difficult to validate because of signal amplification of both specific and non-specific components [42] . For these reasons, to determine the specific binding of antibody, sera should be tested with both a specific viral antigen and its corresponding control or comparison antigen to account for possible non-specific background activity. The GP I-ELISA evaluated in our study was based on a commercially available recombinant GP for which a negative control could not be obtained. Compared to WAg and NP I-ELISAs, which included control antigens, the higher GP I-ELISA test values in EBOV negative serum panels and consequently the higher PP cut-off values derived for this assay are likely due to not having a control antigen. Due to inherent differences amongst assay systems, binding-antibody levels should be expressed in relative rather than absolute terms. One of the advantages of using PP values as a measure of antibody activity in the I-ELISA is that this method of OD readings conversion does not assume a uniform background activity, and therefore it is also more suitable for inter-laboratory standardization [53] .

The first ELISAs for the detection of antibodies to EBOV were based on whole antigen prepared from infected cell lysate [40] , and variations of this assay are still widely used in diagnostic and research laboratories [3] . While whole filovirus antigens can be relatively easily produced in large volumes, their preparation pose health risks, restricting their production to biosafety level 4 (BSL-4) facilities. These facilities are not only very expensive to construct and operate, but also are not easily available or accessible for countries where fatal filoviruses are endemic. In addition, the binding of antibodies to cellular contaminants present in EBOV-infected cell lysates may lead to cross-reactivity, resulting in reduced specificity [54] . High quality filovirus recombinant protein antigens can be safely prepared without the need for high BSL-4 biocontainment facilities and outside EBOV endemic areas. Their use in ELISA has potential to reduce the risk of false positive results and allows for better standardization [55, 56] . Testing clinical specimens potentially containing a BSL-4 viral agent presents a serious biohazard. While a number of inactivation methods were shown to completely inactivate EBOV [47, 48] , they also markedly alter the protein components in human blood, e.g., enzymes and coagulation factors [47] . Results of viral inactivation protocols evaluated in our study indicate that they do not alter detectable levels of anti EBOV-IgG, thus together with recombinant antigen based ELISAs, provide a safe and reliable testing platform.

It has previously been reported that the use of single filoviral proteins as antigens is disadvantageous for filovirus serology [28, 29, 55] . Sera from patients infected with EBOV contain antibodies to several viral proteins and therefore might display reduced activity or later seroconversion in ELISAs based on a single recombinant antigen [55] . Results of our study indicate that anti-EBOV NP IgG is detectable earlier then anti-EBOV GP IgG antibody during the first week post disease onset. This is likely due to higher abundance of this highly immunogenic protein in infected cells [24] .

Long-lasting persistence of IgG antibody in humans after infection with EBOV [40, 57, 58] , renders IgG detection ELISA a suitable tool for epidemiological investigations.

Evaluation of the efficacy of filovirus vaccines and therapeutics require monitoring of immune responses that correlate with protection and survival using reliable and reproducible serological methods. Anti-EBOV GP IgG ELISA was recently shown to reproducibly quantify levels of anti-EBOV IgG antibodies in sera from EBOD survivors and immunized individuals [59] , thus offering an important laboratory tool for assessing immunogenicity of candidate EBOV vaccines. The NP IgG I-ELISA evaluated in this study has a potential to be used for testing immunity and evaluating protection against natural EBOV exposure in individuals immunized with EBOV GP-based vaccines or receiving anti-EBOV GP passive immunization.

As highly accurate, robust and safe tests, the ELISAs based on recombinant EBOV antigens have a potential to replace traditional serological diagnostic methods which pose health risks thus necessitating their use only in high biocontainment facilities. Practically simple viral inactivation protocols evaluated in our study do not alter measurable levels of anti EBOV-IgG, thus together with recombinant antigen based ELISAs provide a safe testing application. Long-lasting IgG antibody makes their detection useful for epidemiological investigations. 

New filovirus disease classification and nomenclature

Ebola virus outbreaks in Africa: Past and present

Emergence of Zaire Ebola virus disease in Guinea

The international Ebola emergency

WHO Ebola Response Team. Ebola virus disease in West Africa-the first 9 months of the epidemic and forward projections

World Health Organization-Ebola virus Disease-Democratic Republic of the Congo

DRC Ebola outbreak crisis update

11 313 unnecessary Ebola-related deaths: Building citizen trust in health systems

The economic impact of the 2014 Ebola epidemic: Short-and medium-term estimates for West Africa

Health-care worker mortality and the legacy of the Ebola epidemic

A retrospective and prospective analysis of the West African Ebola virus disease epidemic: Robust national health systems at the foundation and an empowered WHO at the apex

The link between the West African Ebola outbreak and health systems in Guinea, Liberia and Sierra Leone: A systematic review

Institutional trust and misinformation in the response to the 2018-19 Ebola outbreak in North Kivu, DR Congo; a population-based survey

Comparative evaluation of the diagnostic performance of the prototype Cepheid GeneXpert Ebola assay

Establishing Ebola virus disease (EVD) diagnostics using GeneXpert technology at a mobile laboratory in Liberia: Impact on outbreak response, case management and laboratory systems strengthening

Lateral flow immunoassays for Ebola virus disease detection in Liberia

Comparative performance of four rapid Ebola antigen-detection lateral flow immunoassays during the 2014-2016 Ebola epidemic in West Africa

Serologic cross-reactivity of human IgM and IgG antibodies to five species of Ebola virus

Serological investigation of laboratory-confirmed and suspected Ebola virus disease patients during the late phase of the Ebola outbreak in Sierra Leone

Production of antigens for ELISA

Analysis of linear B-cell epitopes of the nucleoprotein of Ebola virus that distinguish Ebola virus subtypes

Sequence analysis of the Ebola virus genome: Organization, genetic elements, and comparison with the genome of Marburg virus

Enzyme-linked immunosorbent assays for detection of antibodies to Ebola and Marburg viruses using recombinant nucleoproteins

Human survivors of disease outbreaks caused by Ebola or Marburg virus exhibit cross-reactive and long-lived antibody responses

Enzyme-linked immunosorbent assay for detection of filovirus species-specific antibodies

Protective efficacy of neutralizing antibodies against Ebola virus infection

Recombinant Ebola virus nucleoprotein and glycoprotein (Gabon 94 strain) provide new tools for the detection of human infections

Serological reactivity of baculovirus-expressed Ebola virus VP35 and nucleoproteins

Nucleoprotein-based indirect enzyme-linked immunosorbent assay (indirect ELISA) for detecting antibodies specific to Ebola virus and Marburg virus

Quantitative serology assays for determination of antibody responses to Ebola virus glycoprotein and matrix protein in nonhuman primates and humans

Recombinant protein expression in Escherichia coli: Advances and challenges

Engineering N-glycosylation pathways in the baculovirus-insect cell system

Asymptomatic infection and unrecognised Ebola virus disease in Ebola-affected households in Sierra Leone: A cross-sectional study using a new non-invasive assay for antibodies to Ebola virus

Standardization of the filovirus plaque assay for use in preclinical studies

South African Ebola diagnostic response in Sierra Leone: A modular high biosafety field laboratory

Diagnostic reverse-transcription polymerase chain reaction kit for filoviruses based on the strain collections of all European biosafety level 4 laboratories

World Health Organization. Ebola hemorrhagic fever-South Africa

Unexpected Ebola virus in a tertiary setting: Clinical and epidemiological aspects

ELISA for the detection of antibodies to Ebola viruses

Validation of IgG-sandwich and IgM-capture ELISA for the detection of antibody to Rift Valley fever virus in humans

Elisa Theory and Practice

Two-graph receiver operating characteristics (TG-ROC)-A Microsoft-Excel template for the selection of cut-off values in diagnostic tests

Two-graph receiver operating characteristics (TG-ROC)-A Microsoft-Excel template for the selection of cut-off values that minimize overall misclassification costs

Application of diagnostic tests in veterinary epidemiological studies

The measurement of observer agreement for categorical data

Physicochemical inactivation of Lassa, Ebola and Marburg viruses and effect on clinical laboratory analyses

Effective chemical inactivation of Ebola virus

Validation of serological assays for diagnosis infectious diseases

Principles of validation of diagnostic assays for infectious diseases

Determination of the optimum cut-off value of a diagnostic test

Clinical virology of Ebola hemorrhagic fever (EHF): Virus, virus antigen, and IgG and IgM antibody findings among EHF patients in Kiwit, Democratic Republic of the Congo

Standarization and validation of enzyme-linked immunosorbent assay techniques for the detection of antibody in infectious disease diagnosis

Development of an antibody capture ELISA using inactivated Ebola Zaire Makona virus

Profiling the native specific human humoral immune response to Sudan ebolavirus strain Gulu by chemiluminescence enzyme-linked immunosorbent assay

Antigen-capture blocking enzyme-linked immunosorbent assay based on a baculovirus recombinant antigen to differentiate transmissable gastroenteritis virus from porcine respiratory coronavirus antibodies

Clinical, virologic and immunologic follow-up of convalescent Ebola hemorrhagic fever patientsand their household contacts

Immunoglobulin G in Ebola outbreak survivors

Development, qualification, and validation of the Filovirus Animal Nonclinical Group anti-Ebola virus glycoprotein immunoglobulin G enzyme-linked immunosorbent assay for human serum samples

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.