(175 days)
The cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test is an automated rapid multiplex real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A virus, influenza B virus and respiratory syncytial virus (RSV) nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens from individuals exhibiting signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory viral infection due to SARS-CoV-2, influenza and RSV can be similar. This test is intended to aid in the differential diagnosis of SARS-CoV-2, influenza A, influenza B, and RSV infections in humans and is not intended to detect influenza C virus infections.
Nucleic acids from the viral organisms identified by this test are generally detectable in nasopharyngeal and nasal swab specimens during the acute phase of infection. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of respiratory tract infection are indicative of the presence of the identified virus, and aid in diagnosis if used in conjunction with other clinical and epidemiological information, and laboratory findings.
The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Positive results do not rule out coinfection with other organisms. The organism(s) detected by the cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test may not be the definite cause of disease. Negative results do not preclude SARS-CoV-2, influenza A virus, influenza B virus, or RSV infections.
The cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test is performed on the cobas liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2, a well-conserved region of the matrix gene of influenza A (Flu A target), the nonstructural protein 1 (NS1) gene of influenza B (Flu B target) and the matrix gene of RSV (RSV target). An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.
The provided text describes the analytical and clinical performance evaluation of the cobas® liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test, which is a real-time RT-PCR assay. The information mainly focuses on the performance characteristics required for FDA clearance (510(k)).
Here's a breakdown of the requested information based on the provided document:
Acceptance Criteria and Device Performance
The document does not explicitly present a table of "acceptance criteria" in a pass/fail format for clinical performance. Instead, it demonstrates the device's performance through various analytical studies and clinical agreement percentages relative to a comparator method. The acceptance for a 510(k) submission is typically that the device is "substantially equivalent" to a legally marketed predicate device, which implies demonstrating comparable performance characteristics.
The key performance metrics are the Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) in clinical studies. While there aren't explicit numeric acceptance criteria stated, the achieved performance values are presented as evidence of substantial equivalence.
Here’s a table summarizing the reported clinical device performance based on the prospective study (Table 13) and the retrospective study (Table 15). These are the metrics by which the device's clinical performance would be "accepted" as substantially equivalent.
Table of Reported Device Performance (Clinical)
| Target | Specimen Type | PPA (%) (95% CI) | NPA (%) (95% CI) |
|---|---|---|---|
| SARS-CoV-2 | NPS | 94.5 (90.7-96.8) | 97.6 (96.7-98.3) |
| SARS-CoV-2 | ANS | 96.7 (93.4-98.4) | 97.2 (96.2-97.9) |
| Influenza A | NPS | 100.0 (93.4-100.0) | 99.3 (98.8-99.6) |
| Influenza A | ANS | 100.0 (93.2-100.0) | 99.3 (98.8-99.6) |
| Influenza B | NPS (Prospective) | 100.0 (85.1-100.0) | 99.3 (98.8-99.6) |
| Influenza B | ANS (Prospective) | 100.0 (86.2-100.0) | 99.5 (99.0-99.7) |
| Influenza B | NPS (Retrospective) | 100.0 (89.8-100.0) | 97.9 (94.7-99.2) |
| Influenza B | ANS (Retrospective) | 100.0 (89.8-100.0) | 98.3 (95.0-99.4) |
| RSV | NPS | 100.0 (94.8-100.0) | 99.0 (98.3-99.3) |
| RSV | ANS | 97.5 (91.4-99.3) | 98.8 (98.2-99.3) |
Note on "Acceptance Criteria" for Analytical Performance: The document describes detailed analytical studies (LoD, inclusivity, cross-reactivity, interference, reproducibility), and the reported hit rates and concentrations demonstrate that the device met the internal analytical performance specifications, which are implicitly the "acceptance criteria" for these aspects. For instance, for LoD, the acceptance criterion is implied to be ≥95% hit rate at the determined concentration. For inclusivity, it's detection at or near 3x LoD. For cross-reactivity and interference, the acceptance criterion is no cross-reactivity/interference observed. The document states that "none of the organisms tested cross reacted or interfered" and that "substances... did not interfere," indicating successful meeting of these criteria. For reproducibility, the agreement percentages for positive and negative samples are above 99% for most categories.
Study Details
-
Sample sizes used for the test set and the data provenance:
- Prospective Clinical Study (Category I):
- NPS specimens: 1729 enrolled subjects leading to 1704 evaluable specimens for SARS-CoV-2, Flu A, Flu B, and 1705 for RSV.
- ANS specimens: 1729 enrolled subjects leading to 1705 evaluable specimens for SARS-CoV-2, Flu B, 1703 for Flu A, and 1706 for RSV.
- Data Provenance: Fresh specimens, prospective, collected between September 2023 and March 2024 at 14 point-of-care testing sites in the United States (US).
- Retrospective Clinical Study (Category III):
- Specimens: Frozen archived clinical NPS (n=223) and ANS (n=206) specimens.
- Data Provenance: Retrospective, collected between 2019 and 2023. Distributed to 6 sites for testing. Country of origin not explicitly stated but implied to be US given the overall context of a US FDA clearance (though not definitively stated for the retrospective part).
- Prospective Clinical Study (Category I):
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
The ground truth for the clinical test set was established by comparing the results of the cobas® liat test to the "results of an FDA-cleared Nucleic Acid Amplification Test (NAAT)." The document does not specify the number of human experts, their qualifications, or their role in establishing this ground truth. The "ground truth" here is the result from the comparator NAAT, not human expert interpretation of images or clinical data. -
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
The document describes a discrepant analysis for cases where the cobas® liat test results differed from the comparator NAAT. For the prospective study, the discrepancy analysis showed how many of the discrepant results (e.g., cobas® liat negative, comparator positive) were ultimately confirmed as positive or negative by further investigation (implied to be by the comparator method or potentially a third method, though not explicitly detailed beyond "discrepant NAAT results"). The details provided are:- SARS-CoV-2 NPS: Of 12 negative cobas® liat/positive comparator, 8 were positive and 4 negative. Of 35 positive cobas® liat/negative comparator, 12 were positive and 23 negative.
- Similar analyses are provided for other targets and specimen types.
This implies an adjudication method where discrepant results were further investigated, likely with repeat testing or a confirmatory reference method, but the specific "2+1" or "3+1" reader/expert adjudication model (common in imaging studies) is not applicable or described for this in vitro diagnostic (IVD) device.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
No, an MRMC comparative effectiveness study was not done. This type of study is primarily relevant for imaging devices that assist human readers (e.g., AI for radiology). The cobas® liat test is an automated molecular diagnostic test directly detecting nucleic acids, not an AI-assisted interpretation device for human "readers." -
If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
Yes, the performance reported (PPA, NPA, and analytical studies) represents the standalone performance of the cobas® liat device. It is an automated system where the "algorithm" (the RT-PCR assay and its interpretation software) directly produces a qualitative result (Detected/Not Detected), without human "in-the-loop" interpretation for the primary result. Human operators load samples and review results, but the analytical detection and differentiation itself is automated. -
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
The ground truth for both prospective and retrospective clinical studies was based on the results of an FDA-cleared Nucleic Acid Amplification Test (NAAT), which served as the comparator method. For discrepant samples, further re-testing against the comparator or a reference method was performed for adjudication. This falls under a "reference standard" or "comparator method" type of ground truth. -
The sample size for the training set:
The document does not specify the sample size for a "training set." This type of molecular diagnostic device typically relies on analytical validation (LoD, inclusivity, specificity) and clinical validation through comparison to a reference method, rather than a machine learning model that requires explicit training data. The development process would involve iterative optimization of primers, probes, and assay conditions, but this is not typically referred to as a "training set" in the context of IVD submissions, especially for traditional PCR assays. -
How the ground truth for the training set was established:
As no explicit "training set" for a machine learning model is described, the question of how its ground truth was established is not applicable based on the provided text. The "ground truth" in the context of this traditional IVD development refers to the reliable identification of the target analytes in samples for analytical and clinical validation, often through established reference methods or characterized materials.
§ 866.3981 Device to detect and identify nucleic acid targets in respiratory specimens from microbial agents that cause the SARS-CoV-2 respiratory infection and other microbial agents when in a multi-target test.
(a)
Identification. A device to detect and identify nucleic acid targets in respiratory specimens from microbial agents that cause the SARS-CoV-2 respiratory infection and other microbial agents when in a multi-target test is an in vitro diagnostic device intended for the detection and identification of SARS-CoV-2 and other microbial agents when in a multi-target test in human clinical respiratory specimens from patients suspected of respiratory infection who are at risk for exposure or who may have been exposed to these agents. The device is intended to aid in the diagnosis of respiratory infection in conjunction with other clinical, epidemiologic, and laboratory data or other risk factors.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use in the labeling required under § 809.10 of this chapter must include a description of the following: Analytes and targets the device detects and identifies, the specimen types tested, the results provided to the user, the clinical indications for which the test is to be used, the specific intended population(s), the intended use locations including testing location(s) where the device is to be used (if applicable), and other conditions of use as appropriate.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt (standalone or as part of a test system) for the collection of specimen types claimed by this device; alternatively, the sample collection device must be cleared in a premarket submission as a part of this device.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including reagents, instruments, ancillary materials, all control elements, and a detailed explanation of the methodology, including all pre-analytical methods for processing of specimens;
(ii) Detailed descriptions of the performance characteristics of the device for each specimen type claimed in the intended use based on analytical studies including the following, as applicable: Limit of Detection, inclusivity, cross-reactivity, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, precision, reproducibility, and clinical studies;
(iii) Detailed descriptions of the test procedure(s), the interpretation of test results for clinical specimens, and acceptance criteria for any quality control testing;
(iv) A warning statement that viral culture should not be attempted in cases of positive results for SARS-CoV-2 and/or any similar microbial agents unless a facility with an appropriate level of laboratory biosafety (
e.g., BSL 3 and BSL 3+, etc.) is available to receive and culture specimens; and(v) A prominent statement that device performance has not been established for specimens collected from individuals not identified in the intended use population (
e.g., when applicable, that device performance has not been established in individuals without signs or symptoms of respiratory infection).(vi) Limiting statements that indicate that:
(A) A negative test result does not preclude the possibility of infection;
(B) The test results should be interpreted in conjunction with other clinical and laboratory data available to the clinician;
(C) There is a risk of incorrect results due to the presence of nucleic acid sequence variants in the targeted pathogens;
(D) That positive and negative predictive values are highly dependent on prevalence;
(E) Accurate results are dependent on adequate specimen collection, transport, storage, and processing. Failure to observe proper procedures in any one of these steps can lead to incorrect results; and
(F) When applicable (
e.g., recommended by the Centers for Disease Control and Prevention, by current well-accepted clinical guidelines, or by published peer-reviewed literature), that the clinical performance may be affected by testing a specific clinical subpopulation or for a specific claimed specimen type.(4) Design verification and validation must include:
(i) Detailed documentation, including performance results, from a clinical study that includes prospective (sequential) samples for each claimed specimen type and, as appropriate, additional characterized clinical samples. The clinical study must be performed on a study population consistent with the intended use population and compare the device performance to results obtained using a comparator that FDA has determined is appropriate. Detailed documentation must include the clinical study protocol (including a predefined statistical analysis plan), study report, testing results, and results of all statistical analyses.
(ii) Risk analysis and documentation demonstrating how risk control measures are implemented to address device system hazards, such as Failure Modes Effects Analysis and/or Hazard Analysis. This documentation must include a detailed description of a protocol (including all procedures and methods) for the continuous monitoring, identification, and handling of genetic mutations and/or novel respiratory pathogen isolates or strains (
e.g., regular review of published literature and periodic in silico analysis of target sequences to detect possible mismatches). All results of this protocol, including any findings, must be documented and must include any additional data analysis that is requested by FDA in response to any performance concerns identified under this section or identified by FDA during routine evaluation. Additionally, if requested by FDA, these evaluations must be submitted to FDA for FDA review within 48 hours of the request. Results that are reasonably interpreted to support the conclusion that novel respiratory pathogen strains or isolates impact the stated expected performance of the device must be sent to FDA immediately.(iii) A detailed description of the identity, phylogenetic relationship, and other recognized characterization of the respiratory pathogen(s) that the device is designed to detect. In addition, detailed documentation describing how to interpret the device results and other measures that might be needed for a laboratory diagnosis of respiratory infection.
(iv) A detailed device description, including device components, ancillary reagents required but not provided, and a detailed explanation of the methodology, including molecular target(s) for each analyte, design of target detection reagents, rationale for target selection, limiting factors of the device (
e.g., saturation level of hybridization and maximum amplification and detection cycle number, etc.), internal and external controls, and computational path from collected raw data to reported result (e.g., how collected raw signals are converted into a reported signal and result), as applicable.(v) A detailed description of device software, including software applications and hardware-based devices that incorporate software. The detailed description must include documentation of verification, validation, and hazard analysis and risk assessment activities, including an assessment of the impact of threats and vulnerabilities on device functionality and end users/patients as part of cybersecurity review.
(vi) For devices intended for the detection and identification of microbial agents for which an FDA recommended reference panel is available, design verification and validation must include the performance results of an analytical study testing the FDA recommended reference panel of characterized samples. Detailed documentation must be kept of that study and its results, including the study protocol, study report for the proposed intended use, testing results, and results of all statistical analyses.
(vii) For devices with an intended use that includes detection of Influenza A and Influenza B viruses and/or detection and differentiation between the Influenza A virus subtypes in human clinical specimens, the design verification and validation must include a detailed description of the identity, phylogenetic relationship, or other recognized characterization of the Influenza A and B viruses that the device is designed to detect, a description of how the device results might be used in a diagnostic algorithm and other measures that might be needed for a laboratory identification of Influenza A or B virus and of specific Influenza A virus subtypes, and a description of the clinical and epidemiological parameters that are relevant to a patient case diagnosis of Influenza A or B and of specific Influenza A virus subtypes. An evaluation of the device compared to a currently appropriate and FDA accepted comparator method. Detailed documentation must be kept of that study and its results, including the study protocol, study report for the proposed intended use, testing results, and results of all statistical analyses.
(5) When applicable, performance results of the analytical study testing the FDA recommended reference panel described in paragraph (b)(4)(vi) of this section must be included in the device's labeling under § 809.10(b) of this chapter.
(6) For devices with an intended use that includes detection of Influenza A and Influenza B viruses and/or detection and differentiation between the Influenza A virus subtypes in human clinical specimens in addition to detection of SARS-CoV-2 and similar microbial agents, the required labeling under § 809.10(b) of this chapter must include the following:
(i) Where applicable, a limiting statement that performance characteristics for Influenza A were established when Influenza A/H3 and A/H1-2009 (or other pertinent Influenza A subtypes) were the predominant Influenza A viruses in circulation.
(ii) Where applicable, a warning statement that reads if infection with a novel Influenza A virus is suspected based on current clinical and epidemiological screening criteria recommended by public health authorities, specimens should be collected with appropriate infection control precautions for novel virulent influenza viruses and sent to State or local health departments for testing. Viral culture should not be attempted in these cases unless a BSL 3+ facility is available to receive and culture specimens.
(iii) Where the device results interpretation involves combining the outputs of several targets to get the final results, such as a device that both detects Influenza A and differentiates all known Influenza A subtypes that are currently circulating, the device's labeling must include a clear interpretation instruction for all valid and invalid output combinations, and recommendations for any required followup actions or retesting in the case of an unusual or unexpected device result.
(iv) A limiting statement that if a specimen yields a positive result for Influenza A, but produces negative test results for all specific influenza A subtypes intended to be differentiated (
i.e., H1-2009 and H3), this result requires notification of appropriate local, State, or Federal public health authorities to determine necessary measures for verification and to further determine whether the specimen represents a novel strain of Influenza A.(7) If one of the actions listed at section 564(b)(1)(A) through (D) of the Federal Food, Drug, and Cosmetic Act occurs with respect to an influenza viral strain, or if the Secretary of Health and Human Services determines, under section 319(a) of the Public Health Service Act, that a disease or disorder presents a public health emergency, or that a public health emergency otherwise exists, with respect to an influenza viral strain:
(i) Within 30 days from the date that FDA notifies manufacturers that characterized viral samples are available for test evaluation, the manufacturer must have testing performed on the device with those influenza viral samples in accordance with a standardized protocol considered and determined by FDA to be acceptable and appropriate.
(ii) Within 60 days from the date that FDA notifies manufacturers that characterized influenza viral samples are available for test evaluation and continuing until 3 years from that date, the results of the influenza emergency analytical reactivity testing, including the detailed information for the virus tested as described in the certificate of authentication, must be included as part of the device's labeling in a tabular format, either by:
(A) Placing the results directly in the device's labeling required under § 809.10(b) of this chapter that accompanies the device in a separate section of the labeling where analytical reactivity testing data can be found, but separate from the annual analytical reactivity testing results; or
(B) In a section of the device's label or in other labeling that accompanies the device, prominently providing a hyperlink to the manufacturer's public website where the analytical reactivity testing data can be found. The manufacturer's website, as well as the primary part of the manufacturer's website that discusses the device, must provide a prominently placed hyperlink to the website containing this information and must allow unrestricted viewing access.