(320 days)
cobas SARS-CoV-2 Qualitative for use on the cobas 6800/8800 Systems is a real-time RT-PCR test intended for the qualitative detection of nucleic acids from SARS-CoV-2 in nasal and nasopharyngeal specimens collected from symptomatic individuals suspected of COVID-19 by their healthcare provider.
Results are for the detection of SARS-CoV-2 RNA. Positive results are indicative of the presence of SARS-CoV-2 RNA; clinical correlation with patient history and other diagnostic information is necessary to determine patient infection status. Positive results do not rule out bacterial infection or co-infection with other pathogens.
Negative results do not preclude SARS-CoV-2 infection and should not be used as the sole basis for patient management decisions. Results are meant to be used in conjunction with clinical observations, patient history, recent exposures and epidemiological information, and laboratory data, in accordance with the guidelines provided by the relevant public health authorities. cobas SARS-CoV-2 is intended for use by qualified clinical laboratory personnel specifically instructed and trained in the techniques of real-time PCR and on the use of the cobas 6800/8800 Systems.
cobas SARS-CoV-2 Qualitative is based on fully automated sample preparation (nucleic acid extraction and purification) followed by PCR amplification and detection. The cobas 6800/8800 Systems consist of the sample supply module, the transfer module, the processing module, and the analytic module. Automated data management is performed by the cobas 6800/8800 Systems software(s), which assigns test results for all tests. Results can be reviewed directly on the system screen and printed as a report.
Nucleic acid from patient samples and added internal control RNA (RNA IC) molecules are simultaneously extracted. Nucleic acid is released by addition of proteinase and lysis reagent to the sample. The released nucleic acid binds to the silica surface of the added magnetic glass particles. Unbound substances and impurities, such as denatured protein, cellular debris and potential PCR inhibitors, are removed with subsequent wash steps and purified nucleic acid is eluted from the magnetic glass particles with elution buffer at elevated temperature. External controls (positive and negative) are processed in the same way.
Selective amplification of target nucleic acid from the sample is achieved by the use of target-specific forward and reverse primers for ORF1 a/b non-structural region that is unique to SARS-CoV-2. Additionally, a conserved region in the structural protein envelope E-gene were chosen for pan-Sarbecovirus detection. The pan-Sarbecovirus detection sets will also detect SARS-CoV-2 virus.
Selective amplification of RNA Internal Control is achieved by the use of non-competitive sequence specific forward and reverse primers which have no homology with the coronavirus genome. A thermostable DNA polymerase enzyme is used for amplification.
The cobas SARS-CoV-2 Qualitative master mix contains detection probes which are specific for the coronavirus type SARS-CoV-2, members of the Sarbecovirus subgenus, and the RNA Internal Control nucleic acid. The coronavirus and RNA Internal Control detection probes are each labeled with unique fluorescent dyes that act as a reporter. Each probe also has a second dye which acts as a quencher. When not bound to the target sequence, the fluorescent signals of the intact probes are suppressed by the quencher dye. During the PCR amplification step, hybridization of the probes to the specific single-stranded DNA template results in cleavage of the probe by the 5' to 3' exonuclease activity of the DNA polymerase resulting in separation of the reporter and quencher dyes and the generation of a fluorescent signal. With each PCR cycle, increasing amounts of cleaved probes are generated and the cumulative signal of the reporter dye increases concomitantly. Each reporter dye is measured at defined wavelengths, which enables simultaneous detection and discrimination of the amplified coronavirus target and the RNA Internal Control. The master mix includes deoxyuridine triphosphate (dUTP), instead of deoxythimidine triphosphate (dTTP), which is incorporated into the newly synthesized DNA (amplicon). Any contaminating amplicons from previous PCR runs are destroyed by the AmpErase enzyme [uracil-N-glycosylase], which is included in the PCR mix, when heated in the first thermal cycling step. However, newly formed amplicons are not destroyed since the AmpErase enzyme is inactivated once exposed to temperatures above 55℃.
Here's a detailed breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
1. Table of Acceptance Criteria and Reported Device Performance
Note: The document does not explicitly state "acceptance criteria" for clinical performance in a pass/fail format with numerical targets. Instead, it presents the achieved performance. Therefore, the "Acceptance Criteria" column below reflects typical expectations for such devices based on the reported performance.
| Criterion | Acceptance Criteria (Inferred) | Reported Device Performance |
|---|---|---|
| Analytical Sensitivity (Limit of Detection - LoD) | ||
| SARS-CoV-2 (Target 1) | Detect at 95% hit rate at specified concentration | LoD confirmed at 250 IU/mL (at least 23/24 replicates detected at this concentration for all three reagent lots). |
| pan-Sarbecovirus (Target 2) | Detect at 95% hit rate at specified concentration | LoD confirmed at 125 IU/mL (at least 23/24 replicates detected at this concentration for all three reagent lots). |
| Inclusivity | 100% positivity for tested strains | Achieved 100% positivity for all nine tested SARS-CoV-2 strains (including six variants) at varying concentrations. In silico analysis indicated >99.99% detection of known sequences. |
| Precision (Within-laboratory) | Low Coefficient of Variation (CV%) | Overall CV percentage ranging from 1.1% to 2.2% for positive panel members. |
| Reproducibility | High percent agreement across sites, lots, days, runs | 99.1% Negative Percent Agreement (95% CI: 96.7% - 99.9%) for negative samples. |
| Good lot-to-lot, instrument (site), day-to-day, and between run variation for positive panel members. | ||
| Analytical Specificity (Cross-Reactivity) | No false positives/negatives in presence of non-target organisms | None of the 47 tested viruses, bacteria, and fungi (including SARS-CoV-1, which showed an expected pan-Sarbecovirus positive result) interfered. In silico analysis confirmed no concerns. |
| Interference (Exogenous Substances) | No false positives/negatives/invalid results | None of the 14 tested exogenous substances (e.g., nasal sprays, medications) interfered. |
| Interference (Endogenous Substances) | No false positives/negatives/invalid results | None of the 5 tested endogenous substances (e.g., human genomic DNA, mucus, whole blood) interfered. |
| Matrix Equivalency | Acceptable performance across matrix types | Simulated clinical matrix, nasopharyngeal swabs (NPS), and nasal swabs (NS) found acceptable. 100% positive replicates at 1x LoD and 3x LoD for all three types. |
| Collection Media Equivalency | Acceptable performance across media types | Universal Transport Media (UTM-RT), cobas PCR Media (CPM), and 0.9% physiological saline (NaCl) found acceptable. All replicates positive at low and moderate positive concentrations. |
| Clinical Performance: | ||
| Positive Percent Agreement (PPA) | High PPA (e.g., >95%) | NPS: 98.7% (77/78) (95% CI: 93.1%, 99.8%) |
| NS: 97.4% (76/78) (95% CI: 91.1%, 99.3%) | ||
| Negative Percent Agreement (NPA) | High NPA (e.g., >95%) | NPS: 99.7% (857/860) (95% CI: 99.0%, 99.9%) |
| NS: 99.9% (855/856) (95% CI: 99.3%, 100.0%) |
2. Sample Size Used for the Test Set and Data Provenance
-
Clinical Performance Test Set (Clinical Specimens):
- Total Participants enrolled: 1,154
- Participants included in evaluation: 968
- Reasons for exclusion: 184 specimens due to shipment issues/testing completion issues, 2 subjects for being previously enrolled.
- Evaluable NPS results: 938 (961 participants contributed NPS, 4 failed/invalid cobas results)
- Evaluable NS results: 934 (8 invalid/missing cobas NS results)
- Confirmed infected status (NPS comparator): 942 participants
- Data Provenance: Multi-center study with three external testing sites, involving prospectively collected clinical specimens (NPS and NS) from individuals with signs and symptoms of respiratory infection. Participants from 12 geographically distributed enrollment centers. The specific countries are not mentioned, but "US Food & Drug Administration" implies a US-based study or one for US market approval.
-
Analytical Performance Test Sets:
- Analytical Sensitivity (LoD): 5 concentration levels, 24 replicates per concentration, 3 independent dilution series with 3 lots of reagents.
- Inclusivity: 9 SARS-CoV-2 strains, 4 replicates each.
- Precision: 3 concentration levels, 90 replicates per concentration (over 15 instrument days).
- Reproducibility: 4-member panel (positive/negative), 216 tests per concentration (at 3 testing sites, 3 reagent lots, 6 days, 2 runs/day, 3 replicates/run).
- Analytical Specificity/Cross-reactivity: 47 microorganisms (at specified concentrations) tested in absence and presence of SARS-CoV-2 target.
- Interference (Exogenous/Endogenous): Multiple substances at specified concentrations, tested in absence and presence of SARS-CoV-2 target.
- Matrix Equivalency: 3 matrices (simulated, NPS, NS), 3 concentration levels (approx. 3x LoD, 1x LoD, 0.3x LoD), 25 replicates per concentration.
- Collection Media Equivalency: 3 media types (UTM-RT, CPM, NaCl), 2 concentration levels (approx. 2x LoD, 4x LoD), at least 20 replicates for low positive, 10 for moderate positive.
- Data Provenance: All these analytical studies appear to be laboratory-based and controlled, usually performed by the manufacturer or authorized labs. No specific country of origin is stated for these analytical studies, but they are part of the submission to the US FDA.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document refers to "healthcare provider" for collecting specimens and for associating patient history and diagnostic information, but does not specify the number or qualifications of experts (e.g., radiologists) used to establish the ground truth for the test set specifically for interpreting images or providing expert consensus.
Instead, for the clinical performance evaluation, the ground truth was established using a composite comparator method (described below), rather than expert consensus on a subjective measure. For laboratory-based analytical studies, ground truth is based on known concentrations of viral material.
4. Adjudication Method for the Test Set
The ground truth for the clinical test set was established using a composite comparator method:
- Method: Laboratory sites used up to three highly sensitive EUA SARS-CoV-2 molecular assays.
- Initial Comparison: The composite comparator result was defined as the concordant results from two initial comparator assays (Test A and Test B) on the NPS specimen from each subject.
- Discordance Resolution: In case of discordance between the initial two comparator assays, the sample was tested by a third assay (Test C), and the result of the third test determined the composite comparator result.
- Indeterminate Results: The composite comparator result was deemed indeterminate if valid results could not be obtained from two assays (e.g., insufficient volume for repeat testing or invalid/failed results).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This document describes the performance of a diagnostic assay (RT-PCR test), not an AI-assisted diagnostic imaging device that involves human readers. Therefore, there is no mention of human readers or improvement with AI assistance.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, the performance study presented is a standalone (algorithm only) performance. The device is an automated RT-PCR test system (cobas SARS-CoV-2 Qualitative for use on the cobas 6800/8800 Systems). Its performance characteristics (analytical sensitivity, specificity, precision, clinical PPA/NPA) are measured directly against established standards or a composite comparator, without human interpretation of the assay results changing its output. The system software assigns test results.
7. The Type of Ground Truth Used
- Clinical Performance: Composite comparator method using up to three highly sensitive EUA SARS-CoV-2 molecular assays on NPS specimens.
- Analytical Performance:
- Known concentrations of inactivated quantified SARS-CoV-2 virus (WHO International Standard) for LoD and reproducibility.
- Known SARS-CoV-2 strains for inclusivity.
- Known concentrations of various microorganisms or substances for analytical specificity and interference studies.
8. The Sample Size for the Training Set
The document describes a cobas SARS-CoV-2 Qualitative test, which is a real-time RT-PCR assay and not an AI/Machine Learning model. Therefore, the concept of a "training set" in the context of machine learning does not apply here. The document details validation studies for the assay, not the training of an algorithmic model.
9. How the Ground Truth for the Training Set Was Established
As stated above, this is an RT-PCR assay, not an AI/ML model. Therefore, there is no training set and consequently, no ground truth established for a training set. The performance is validated through various analytical and clinical studies as outlined in the document.
§ 866.3981 Device to detect and identify nucleic acid targets in respiratory specimens from microbial agents that cause the SARS-CoV-2 respiratory infection and other microbial agents when in a multi-target test.
(a)
Identification. A device to detect and identify nucleic acid targets in respiratory specimens from microbial agents that cause the SARS-CoV-2 respiratory infection and other microbial agents when in a multi-target test is an in vitro diagnostic device intended for the detection and identification of SARS-CoV-2 and other microbial agents when in a multi-target test in human clinical respiratory specimens from patients suspected of respiratory infection who are at risk for exposure or who may have been exposed to these agents. The device is intended to aid in the diagnosis of respiratory infection in conjunction with other clinical, epidemiologic, and laboratory data or other risk factors.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use in the labeling required under § 809.10 of this chapter must include a description of the following: Analytes and targets the device detects and identifies, the specimen types tested, the results provided to the user, the clinical indications for which the test is to be used, the specific intended population(s), the intended use locations including testing location(s) where the device is to be used (if applicable), and other conditions of use as appropriate.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt (standalone or as part of a test system) for the collection of specimen types claimed by this device; alternatively, the sample collection device must be cleared in a premarket submission as a part of this device.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including reagents, instruments, ancillary materials, all control elements, and a detailed explanation of the methodology, including all pre-analytical methods for processing of specimens;
(ii) Detailed descriptions of the performance characteristics of the device for each specimen type claimed in the intended use based on analytical studies including the following, as applicable: Limit of Detection, inclusivity, cross-reactivity, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, precision, reproducibility, and clinical studies;
(iii) Detailed descriptions of the test procedure(s), the interpretation of test results for clinical specimens, and acceptance criteria for any quality control testing;
(iv) A warning statement that viral culture should not be attempted in cases of positive results for SARS-CoV-2 and/or any similar microbial agents unless a facility with an appropriate level of laboratory biosafety (
e.g., BSL 3 and BSL 3+, etc.) is available to receive and culture specimens; and(v) A prominent statement that device performance has not been established for specimens collected from individuals not identified in the intended use population (
e.g., when applicable, that device performance has not been established in individuals without signs or symptoms of respiratory infection).(vi) Limiting statements that indicate that:
(A) A negative test result does not preclude the possibility of infection;
(B) The test results should be interpreted in conjunction with other clinical and laboratory data available to the clinician;
(C) There is a risk of incorrect results due to the presence of nucleic acid sequence variants in the targeted pathogens;
(D) That positive and negative predictive values are highly dependent on prevalence;
(E) Accurate results are dependent on adequate specimen collection, transport, storage, and processing. Failure to observe proper procedures in any one of these steps can lead to incorrect results; and
(F) When applicable (
e.g., recommended by the Centers for Disease Control and Prevention, by current well-accepted clinical guidelines, or by published peer-reviewed literature), that the clinical performance may be affected by testing a specific clinical subpopulation or for a specific claimed specimen type.(4) Design verification and validation must include:
(i) Detailed documentation, including performance results, from a clinical study that includes prospective (sequential) samples for each claimed specimen type and, as appropriate, additional characterized clinical samples. The clinical study must be performed on a study population consistent with the intended use population and compare the device performance to results obtained using a comparator that FDA has determined is appropriate. Detailed documentation must include the clinical study protocol (including a predefined statistical analysis plan), study report, testing results, and results of all statistical analyses.
(ii) Risk analysis and documentation demonstrating how risk control measures are implemented to address device system hazards, such as Failure Modes Effects Analysis and/or Hazard Analysis. This documentation must include a detailed description of a protocol (including all procedures and methods) for the continuous monitoring, identification, and handling of genetic mutations and/or novel respiratory pathogen isolates or strains (
e.g., regular review of published literature and periodic in silico analysis of target sequences to detect possible mismatches). All results of this protocol, including any findings, must be documented and must include any additional data analysis that is requested by FDA in response to any performance concerns identified under this section or identified by FDA during routine evaluation. Additionally, if requested by FDA, these evaluations must be submitted to FDA for FDA review within 48 hours of the request. Results that are reasonably interpreted to support the conclusion that novel respiratory pathogen strains or isolates impact the stated expected performance of the device must be sent to FDA immediately.(iii) A detailed description of the identity, phylogenetic relationship, and other recognized characterization of the respiratory pathogen(s) that the device is designed to detect. In addition, detailed documentation describing how to interpret the device results and other measures that might be needed for a laboratory diagnosis of respiratory infection.
(iv) A detailed device description, including device components, ancillary reagents required but not provided, and a detailed explanation of the methodology, including molecular target(s) for each analyte, design of target detection reagents, rationale for target selection, limiting factors of the device (
e.g., saturation level of hybridization and maximum amplification and detection cycle number, etc.), internal and external controls, and computational path from collected raw data to reported result (e.g., how collected raw signals are converted into a reported signal and result), as applicable.(v) A detailed description of device software, including software applications and hardware-based devices that incorporate software. The detailed description must include documentation of verification, validation, and hazard analysis and risk assessment activities, including an assessment of the impact of threats and vulnerabilities on device functionality and end users/patients as part of cybersecurity review.
(vi) For devices intended for the detection and identification of microbial agents for which an FDA recommended reference panel is available, design verification and validation must include the performance results of an analytical study testing the FDA recommended reference panel of characterized samples. Detailed documentation must be kept of that study and its results, including the study protocol, study report for the proposed intended use, testing results, and results of all statistical analyses.
(vii) For devices with an intended use that includes detection of Influenza A and Influenza B viruses and/or detection and differentiation between the Influenza A virus subtypes in human clinical specimens, the design verification and validation must include a detailed description of the identity, phylogenetic relationship, or other recognized characterization of the Influenza A and B viruses that the device is designed to detect, a description of how the device results might be used in a diagnostic algorithm and other measures that might be needed for a laboratory identification of Influenza A or B virus and of specific Influenza A virus subtypes, and a description of the clinical and epidemiological parameters that are relevant to a patient case diagnosis of Influenza A or B and of specific Influenza A virus subtypes. An evaluation of the device compared to a currently appropriate and FDA accepted comparator method. Detailed documentation must be kept of that study and its results, including the study protocol, study report for the proposed intended use, testing results, and results of all statistical analyses.
(5) When applicable, performance results of the analytical study testing the FDA recommended reference panel described in paragraph (b)(4)(vi) of this section must be included in the device's labeling under § 809.10(b) of this chapter.
(6) For devices with an intended use that includes detection of Influenza A and Influenza B viruses and/or detection and differentiation between the Influenza A virus subtypes in human clinical specimens in addition to detection of SARS-CoV-2 and similar microbial agents, the required labeling under § 809.10(b) of this chapter must include the following:
(i) Where applicable, a limiting statement that performance characteristics for Influenza A were established when Influenza A/H3 and A/H1-2009 (or other pertinent Influenza A subtypes) were the predominant Influenza A viruses in circulation.
(ii) Where applicable, a warning statement that reads if infection with a novel Influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to State or local health departments for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
(iii) Where the device results interpretation involves combining the outputs of several targets to get the final results, such as a device that both detects Influenza A and differentiates all known Influenza A subtypes that are currently circulating, the device's labeling must include a clear interpretation instruction for all valid and invalid output combinations, and recommendations for any required followup actions or retesting in the case of an unusual or unexpected device result.
(iv) A limiting statement that if a specimen yields a positive result for Influenza A, but produces negative test results for all specific influenza A subtypes intended to be differentiated (
i.e., H1-2009 and H3), this result requires notification of appropriate local, State, or Federal public health authorities to determine necessary measures for verification and to further determine whether the specimen represents a novel strain of Influenza A.(7) If one of the actions listed at section 564(b)(1)(A) through (D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those influenza viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized influenza viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's labeling required under § 809.10(b) of this chapter that accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that accompanies the device, prominently providing a hyperlink to the manufacturer's public website where the analytical reactivity testing data can be found. The manufacturer's website, as well as the primary part of the manufacturer's website that discusses the device, must provide a prominently placed hyperlink to the website containing this information and must allow unrestricted viewing access.