1. Background
Since its emergence in 2019, COVID-19 caused by 2019nCoV has rapidly evolved into a global pandemic, profoundly affecting work and daily life worldwide ( 1 ). Laboratory testing has remained a critical pillar of epidemic prevention and control ( 2 ). To mitigate further harm, rapid and reliable detection of COVID-19 is essential.
Even as the acute phase of the outbreak has subsided, nucleic acid testing continues to play a pivotal role. It enables accurate diagnosis and informs clinical management for individuals infected with the virus ( 3 ). For example, a high viral load on a nucleic acid test may warrant intensified therapy and close monitoring.
At present, there are no standardized national methods in China for verifying the performance of COVID-19 detection reagents. In this study, we evaluated the analytical performance of a 2019-nCoV RNA real-time fluorescence PCR assay in accordance with the Guidelines for Performance Verification of Molecular Diagnostic Test Procedure (CNAS-GL039) ( 4 , 5 ). We further employed 2019-nCoV RNA liquid performance verification reference materials to assess the performance of real-time fluorescence RT-PCR reagents and to determine whether the proposed kit meets the requirements for clinical 2019-nCoV nucleic acid testing ( 6 , 7 ).
2. Objectives
In this study, we evaluated the analytical performance of a 2019nCoV RNA realtime fluorescence PCR assay in accordance with the Guidelines for Performance Verification of Molecular Diagnostic Test Procedure (CNASGL039). We further employed 2019nCoV RNA liquid performance verification reference materials to assess the performance of realtime fluorescence RTPCR reagents and to determine whether the proposed kit meets the requirements for clinical 2019nCoV nucleic acid testing.
3. Materials and Methods
3.1. Research Objects
The various references (P1-P10 with 10 different concentrations), precision references (R1 = 2.0E+03 copies/mL; R2 = 2.0E+04 copies/mL), detection limit references (L4 = 5.0E+02 copies/mL), specificity refe-rences (positive samples of six common viruses (deactivated)) and interferent references (containing the hemoglobin, bilirubin, blood fat and some commonly used drugs,as Oseltamivir, azithromycin, ceftriaxone, sodium chloride) were obtained from BDS Biotech, China. Eight samples obtained from COVID-19 positive patients.
3.2. Reagents
2019-nCoV nucleic acid test kit (Wuhan Easydiagno-sis Biomedicine Co, Ltd) (Wuhan, China) and nucleic acid extraction kit (Da’An Gene Co., Ltd., Sun Yat-Sen University) (Guangzhou, China) were used in this study.
3.3. Instruments
Nucleic acid extractor (SP96, Da’An Gene Co., Ltd., Sun Yat-Sen University) and PCR instrument (SLAN-96S, Shanghai Hongshi Medical Technology Co., Ltd., Shanghai, China) were used in this study.
3.4. Test Method
The proposed kit integrates multiplex PCR–fluore-scence probe technology with one-step RT-PCR. We targeted the 2019-nCoV ORF1ab region and a conserved sequence of the N gene encoding the nucleocapsid protein to design a dual-target primer–probe set, which was then used in the RT-PCR assay. Following amplification, results were interpreted by analyzing the threshold cycle (Ct) values in each fluorescence channel.
To ensure process integrity, the human housekeeping gene RNase P was included as an internal control to monitor sampling, nucleic acid extraction, sample loading, and amplification. Source of reagent kit: Wuhan Easydiagnosis Biomedicine Co., Ltd. 2019-nCoV Nucleic Acid Detection Reagent Manual, RDB-PCR-IFU001-CN.
Complete information on the primers used in this work (sequences, gene name, locatiion, length, etc) was listed in the Supplementary materials.
3.5. Nucleic Acid Extraction
Da’An Gene’s automated extraction workstation and matching reagent kits were used for nucleic acid extraction. All reference materials under evaluation, along with six randomly selected negative clinical samples, were processed using the extraction kit. All study procedures were approved by the Ethics Committee of Jiamusi University Clinical Medical College (Approval No.
0612).
Extraction was carried out on the Da’An Gene Stream SP96 automated nucleic acid extractor. For a reagent comparison study, a separate nucleic acid extraction instrument with its corresponding reagents was employed, and eight positive samples were loaded into a deep-well plate for parallel processing.
3.6. Sample Loading
According to the requirements of the nucleic acid detection kit, a PCR reaction solution was prepared, consisting of 20 μL of 2019-nCoV reaction solution and 5 μL of RNA sample, yielding a total reaction volume of 25 μL. Subsequently, the nucleic acid extracted using the Da Safe Automatic Extraction Instrument was added to the Mingde PCR reaction tube for performance verification. Additionally, the nucleic acid extracted using the Zhongyuan Automatic Nucleic Acid Extraction Instrument was added to three contrast reagents’ PCR reaction tubes.
3.7 Nucleic Acid Test
PCR reaction plate with relevant samples was placed into the amplification area for PCR amplification. Table 1 shows the amplification layout.
3.8. Scheme of the Performance Verification
3.8.1. +/- Coincidence Rate
Due to the particularity of the 2019-nCoV positive samples, we did not include any positive samples in the clinical laboratory. The 2019-nCoV positive samples were replaced with a third-party certified reference substance which can act as a positive reference. Moreover, the third-party known reference substance (the product manual indicated that the concentration of the positive quality control was determined by the fluorescence digital quantitative PCR) was used for validating the tests as the reference method.
Positive references and six negative clinical samples were also included in this study. Nucleic acid extractor and matching nucleic acid extraction reagent were used to extract the nucleic acids, followed by amplification. +/- Coincidence rates were thereafter also recorded, if it met the indicators declared by the manufacturer, the proposed kit +/- coincidence rate was considered qualified. We detected the precision references10 times and calculated the CV value of ct value.
Evaluation criteria used was that the intra-batch precision CV value did not exceed 5.0%.
3.8.2. Lower Detection Limit
We evaluated the LoD using reference materials at multiple concentrations, each tested in five replicates. The acceptance criteria were as follows: results had to meet the LoD specified in the manufacturer’s instruc-tions, with a 100% detection rate across the five replicates. If any concentration failed to yield detection in all five tests, that level was re-evaluated with 20 replicates, for which the required detection rate was ≥ 90%.
3.8.3. Specificity
The specificity was validated by comparing the virus samples with similar infection sites or similar symptoms of 2019-nCoV infection. Six different positive samples references, including Influenza A virus, Influenza B virus, respiratory syncytial virus, human parainfluenza virus, EB virus, and mycobacterium tuberculosis were selected for extraction. The specific test was performed with 2019-nCoV nucleic acid test reagents (the test was to be repeated twice), and the results were required to be negative.
3.8.4. Interference Resistance
Seven distinct references, including hemoglobin, bilirubin, triglyceride, Oseltamivir, Azithromycin, ceftriaxone, and NaCl, were selected for interference resistance verification. Experimental group: The above references were interferent with 2.0E+03 copies/mL positive samples. Control group: 2.0E+03copies/mL positive samples had no interference.
A nucleic acid test was performed after the sample extraction. The results of both experimental and control groups were found corresponding to positive. The CV value of ct value was calculated and was not expected to exceed 5.0%.
3.9. Comparison of Amplification Efficiency of Three Reagents
After thorough shaking, 200 μL of throat swab specimens from 8 positive patients were sequentially added to the Zhongyuan deep well plate. The SARS-CoV-2 nucleic acid was extracted using the Zhongyuan automatic extraction instrument. Subsequently, the extracted nucleic acid was individually added to the reaction solution and then subjected to amplification in the Hongshi amplification instrument.
3.10. Comparison of CT Values Between Pre-Shocked and Unshocked Samples Prior to Sample Addition
The throat swab specimens (200 μL) from 8 positive patients were sequentially added to the Zhongyuan deep-hole plate without agitation. Subsequently, another set of throat swab specimens (200 μL) from the same 8 positive patients was added to the Zhongyuan deep-hole plate in a sequential manner, again without agitation. Following this, SARS-CoV-2 nucleic acid extraction was performed using the Zhongyuan automated extraction instrument.
3.11. Comparison of CT Values of Three Different Loading Amounts
After thorough agitation of the throat swab specimens from 8 positive patients, 100 μL, 200 μL, and 300 μL were sequentially dispensed into the Zhongyuan extraction deep well plate according to their respective serial numbers. Subsequently, SARS-CoV-2 nucleic acid was extracted utilizing the Zhongyuan automated extraction instrument. Following extraction, the nucleic acid was individually introduced into the reaction solution and subsequently subjected to amplification in the Hongshi amplification instrument.
3.12. Statistical Analysis
Microsoft Office Excel software was used for the statistical analysis. Imprecision was expressed as standard deviation (s) and the coefficient of variation (CV) whereas the counting data was expressed as a rate (%) or mean ± standard deviation (x ± s). The comparison of the three reagents was analyzed using Statistic Package for Social Science (SPSS) 25.0 statistical software (IBM, Armonk, NY, USA), employing univariate analysis of variance as the analytical method.
A significance level of p<0.05 was considered to indicate statistical significance.
4. Results
4.1. Coincidence Rate
The amplification curve of FAM and HEX channels of positive references in 10 different cases increased exponentially, and all CT values were <38. The amplification curve of ROX channel also increased exponentially, and the CT value was < 35, which was considered as positive. FAM and HEX channels of the six negative samples displayed no exponential growth, while the amplification curve of ROX channel increased exponentially and the CT value was <35, which was considered as negative (Fig.
1A). The various coincidence rates obtained have been listed in Table 2.
4.2. Precision
We used precision references in this study and the test was performed 10 times, respectively. The CV value of intra-batch precision was observed to be not more than 5.0%, which was qualified. The CT values have been indicated in Figure 1B.
Table 3 shows the statistical analysis results.
4.3. Lower Detection Limit
We selected the detection limit references and repeated the detection 5 times independently. The results of 5 repeated tests were found to be positive and qualified. The CT values have been shown in Figure 1C.
4.4. Specificity
Six positive samples references, including Influenza A virus, Influenza B virus, respiratory syncytial virus, human parainfluenza virus, EB virus and myco-bacterium tuberculosis, were selected for extraction. The specificity of the different positive sample references was tested with 2019-nCoV nucleic acid test, and all results were observed to be negative.
4.5. Interference Resistance
Seven potential interferents—hemoglobin, bilirubin, triglycerides, oseltamivir, azithromycin, ceftriaxone, and NaCl—were evaluated to verify interference resistance. Extracted samples spiked with each interferent were subsequently tested by the nucleic acid assay. Both the experimental (interference) and control groups yielded positive results.
The coefficient of variation (CV) of the calculated Ct values did not exceed 5.0% for any interferent, indicating acceptable precision in the presence of these substances. The corresponding Ct values are presented in Figure 1D.
4.6. Reagent Amplification Comparison Among Three Distinct Formulations
The three reagents exhibited significant variations in the amplification efficiency of the N gene, while there were also significant differences in the amplification efficiency of the O gene between reagent 1 and reagent 2, as well as between reagent 2 and reagent 3. Reagent 2 demonstrated a reduced requirement for cycles and shorter reaction time, whereas reagent 1 displayed greater stability compared to reagent 3 (Table 4).
4.7. Comparison of Sample Shock and Non-Shock Prior to Nucleic Acid Extraction Was Conducted
The CT values of ORF and N gene exhibited a significant disparity between samples that underwent thorough shaking prior to nucleic acid extraction and those that were not shaken. Moreover, the efficiency of nucleic acid extraction was enhanced when samples were subjected to complete agitation (Table 5).
4.8. Comparison of Three Distinct Sample Sizes for Nucleic Acid Extraction
With the increase in sample volume, the CT values of ORF and N target genes exhibited an upward trend, implying that sample volume exerted a discernible impact on nucleic acid extraction efficiency (Table 6).
5. Discussion
After the end of the COVID-19 epidemic, the nucleic acid test of COVID-19 remains an important means of epidemic prevention and control. It can play an important role in monitoring the epidemic, tracking virus mutation, evaluating the effectiveness of vaccination, and assisting diagnosis and treatment ( 8 ). In the nucleic acid test of COVID-19, the CT value has important clinical value.
It can help judge the severity of the disease: The severity of the patient’s condition can be judged according to the CT value ( 9 ). Generally speaking, the lower the CT value, the more serious the disease. If the CT value is above 35, it is generally considered that the patient is no longer infectious and the disease is under control.
It can also evaluate the effectiveness of treatment: The effectiveness of treatment can be evaluated by comparing the changes in CT value before and after treatment. If the treatment is effective, the CT value will gradually increase and the viral load will gradually decrease ( 10 , 11 ). Most laboratories across the country perform 2019-nCoV nucleic acid test through real-time fluorescence PCR ( 12 ).
However, there are several categories of 2019-nCoV nucleic acid kits available for testing. Before use, the laboratories should carefully verify whether the performance of 2019-nCoV nucleic acid test kit can accurately meet the performance indicators and clinical needs claimed by the manufacturer. Laboratories should also conduct performance verification to validate the performance of the kit that has used, but without performance verification ( 13 , 14 ).
Therefore, in this study, we have described the method, process and results of performance verification of 2019-nCoV nucleic acid test kit used in our laboratory. The results showed that +/- coincidence rate of the 2019-nCoV nucleic acid test kit was 100%, and the intra-batch precision CV value was no more that 5.0%, thus establishing that the claimed detection limit was 5.0E+02 copies/mL.
To further validate the kit’s performance, we assessed six positive samples containing different pathogens—influenza A virus, influenza B virus, respiratory syncytial virus, human parainfluenza virus, Epstein–Barr virus, and Mycobacterium tuberculosis. All samples tested negative with the 2019nCoV assay, demonstrating no cross-reactivity. To evaluate robustness against potential interferents, we tested positive controls in the presence of seven reference substances: hemoglobin, bilirubin, triglycerides, oseltamivir, azithromycin, ceftriaxone, and sodium chloride.
Detection remained positive under all interference conditions, indicating that the kit’s performance is not adversely affected by these substances.These findings collectively indicate that the 2019 nCoV nucleic acid testing kit used in our laboratory not only meets the performance standards specified by the manufacturer, but also meets clinical requirements. However, it is worth noting that this study is limited to validating the performance of the specific batch of reagent kits currently used in our laboratory. After successfully verifying the current batch, our laboratory is able to evaluate the performance of the new batch by comparing it with the existing batch.
In addition, inter laboratory comparisons can be conducted to evaluate the results of various reagent kits, extraction reagents, and operators, thereby helping our laboratory select the most suitable extraction and detection reagent kits. In addition to confirming the effectiveness of the reagent kit during use, it is crucial to consider the potential impact of sample shock and non-shock conditions on the efficiency and stability of target gene amplification before nucleic acid extraction and sample addition. In practical applications, selecting appropriate reagents and operating methods according to experimental requirements can greatly improve the accuracy and reliability of nucleic acid testing.
In addition, ensuring consistency and quality in all steps of the nucleic acid testing process, from sample collection and processing to final analysis, is crucial. Proper training on proper procedures and protocols must be provided to laboratory personnel in clinical practice to minimize the risk of errors and contamination. Regular quality control checks should be conducted, such as using positive and negative controls, to monitor the performance of the testing kit and ensure accurate and reliable results.
In the rapidly evolving field of molecular diagnostics, our laboratory recognizes the critical importance of continuously integrating technological advancements to maintain diagnostic excellence. This necessitates the timely adoption of improved testing kits, enhanced extraction reagents, and upgraded instrumentation to optimize our analytical performance and deliver more accurate, efficient results to clinicians and patients. Concurrently, we remain acutely aware of the substantial ethical and regulatory responsibilities inherent in nucleic acid testing.
Our commitment extends beyond technical proficiency to strict compliance with all applicable regulations, rigorous protection of patient confidentiality, proper informed consent procedures, and the ethical application of test results - fundamental principles that are systematically reinforced through ongoing staff training and comprehensive quality management protocols.
In conclusion, the successful verification of the 2019-nCoV nucleic acid test kit in our laboratory is an important step towards providing accurate and reliable diagnostic services. However, it is just one aspect of a comprehensive diagnostic process that requires ongoing attention, evaluation, and improvement. By staying vigilant, adapting to new technologies, and adhering to ethical and regulatory guidelines, we can continue to enhance our diagnostic capabilities and better serve the needs of our patients and the healthcare community.