Search Results

The cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test is an automated rapid multiplex real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A virus and influenza B virus nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens from individuals exhibiting signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory tract infection due to SARS-CoV-2 and influenza can be similar. This test is intended to aid in the differential diagnosis of SARS-CoV-2, influenza A and influenza B infections in humans and is not intended to detect influenza C virus infections.

Nucleic acids from the viral organisms identified by this test are generally detectable in nasopharyngeal and nasal swab specimens during the acute phase of infection. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of respiratory tract infection are indicative of the presence of the identified virus, and aid in diagnosis if used in conjunction with other clinical and epidemiological information and laboratory findings.

The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Positive results do not rule out coinfection with other organisms. The organism(s) detected by the cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test may not be the definite cause of disease. Negative results do not preclude SARS-CoV-2, influenza A virus or influenza B virus infections.

The cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test is performed on the cobas liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2, a well-conserved region of the matrix gene of influenza A (Flu A target), and the nonstructural protein 1 (NS1) gene of influenza B (Flu B target). An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.

This document describes the validation study for the cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test.

Here's an analysis of the acceptance criteria and the study proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

Since this is a diagnostic test, the primary acceptance criteria revolve around analytical and clinical performance metrics like Limit of Detection, Inclusivity, Cross-Reactivity, Reproducibility, and Clinical Agreement (Positive Percent Agreement and Negative Percent Agreement). The document doesn't explicitly state "acceptance criteria" values in a separate table, but these are implied by the performance metrics reported and the general standards for diagnostic device clearance. I will extract the reported device performance from the provided text.

2. Sample Sizes Used for the Test Set and Data Provenance

• Prospective Clinical Study:

  • Sample Size: 1729 symptomatic subjects enrolled.
    • 1705 evaluable NPS specimens for analysis (19 non-evaluable due to missing/invalid results, 5 due to handling).
    • 1706 evaluable ANS specimens for SARS-CoV-2 and Influenza B analysis (22 non-evaluable due to missing/invalid results, 1 due to handling).
    • 1704 evaluable ANS specimens for Influenza A analysis (2 additional found inconclusive for comparator).
  • Data Provenance: Prospective, collected between September 2023 and March 2024 at 14 point-of-care testing sites in the United States (US).

• Retrospective Clinical Study (Influenza B Supplement):

  • Sample Size: 223 archived NPS specimens and 206 archived ANS specimens (total 429).
    • One NPS sample pre-characterized as positive for influenza B was non-evaluable.
  • Data Provenance: Retrospective, frozen archived (Category III) specimens collected between 2019 and 2023. Distributed to 6 sites for testing.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Experts

The document does not mention the use of experts to establish ground truth for the clinical test sets. For molecular diagnostic tests like this, the ground truth is typically established by comparing the investigational device's results against a highly accurate, accepted comparator method (another FDA-cleared Nucleic Acid Amplification Test specific for the target analytes). The expertise lies in the development and validation of these comparator methods, not in individual expert review of each sample for ground truth in this context.

4. Adjudication Method for the Test Set

The document describes discrepant result analysis for both prospective and retrospective clinical studies.

• For the prospective study, "discrepant NAAT results" are detailed for SARS-CoV-2 (NPS and ANS), Influenza A (NPS and ANS), and Influenza B (NPS and ANS).
• For the retrospective study, discrepant NAAT results are detailed for Influenza B (NPS and ANS).

The method appears to be:

• The cobas liat test result is compared to the FDA-cleared comparator NAAT result.
• When there's a discrepancy (e.g., cobas liat positive, comparator negative), it explicitly states how many were "positive" and "negative" upon further investigation or re-evaluation (e.g., with "discrepant NAAT results").
  • For example: "Of 12 specimens negative on cobas® liat and positive on the comparator, 8 were positive and 4 were negative." This implies some form of re-testing or deeper analysis (not specified as "adjudication by experts" but rather "discrepant NAAT results"). It's more of a re-confirmation of the comparator or a third method, rather than a human expert consensus process. Such re-evaluation often involves re-testing using the comparator or a reference method.

Therefore, while there's no "2+1" or "3+1" expert adjudication method described as would be seen in imaging studies, there is a discrepant resolution process based on further NAAT results. It's not "none" in the sense that discrepancies are just reported without follow-up; rather, they are further investigated using additional NAAT results to re-confirm the original comparator status if possible.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

No. This is a standalone diagnostic test (RT-PCR), not an AI-assisted imaging device or a test that involves human "readers" interpreting results. Therefore, an MRMC comparative effectiveness study involving human readers and AI assistance is not applicable and was not performed.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, implicitly. This is a fully automated RT-PCR test run on the cobas liat analyzer. The performance metrics (LoD, inclusivity, cross-reactivity, reproducibility, and clinical agreement) are measures of the device's performance on its own against established ground truth (comparator NAAT). While humans load samples and interpret the final digital result (positive/negative), the core detection and differentiation is algorithm-driven within the instrument, making its performance essentially "standalone" in the context of diagnostic accuracy.

7. The Type of Ground Truth Used

• Clinical Performance (Prospective and Retrospective): The ground truth for clinical sample testing was established by comparing the cobas liat results against an FDA-cleared Nucleic Acid Amplification Test (NAAT), which serves as the reference or "ground truth" method for molecular diagnostic assays. The document explicitly states: "PPA and NPA were determined by comparing the results of cobas® liat SARS-CoV-2 & Influenza A/B v2 to the results of an FDA-cleared Nucleic Acid Amplification Test (NAAT)." and "The comparator method was an acceptable FDA-cleared molecular assay."
• Analytical Studies (LoD, Inclusivity, Cross-Reactivity, Interference, Reproducibility): Ground truth was established by preparing precisely known concentrations of viral material (cultured or inactivated viruses) or specific microorganisms in controlled laboratory settings. For these studies, the "ground truth" is meticulously prepared and verified laboratory standards.

8. The Sample Size for the Training Set

The document does not specify a separate "training set" sample size. For an RT-PCR diagnostic platform, the "training" involves the fundamental biochemical and optical engineering, and the optimization of assay (reagent) design to achieve sensitivity and specificity. This is distinct from machine learning models that often require large, labeled datasets for "training." The analytical and clinical validation studies described here are verification and validation (V&V) studies, akin to a "test set" to prove the device's performance against its design specifications and clinical utility.

9. How the Ground Truth for the Training Set Was Established

Since no explicit "training set" for a machine learning algorithm is mentioned (as this is a molecular diagnostic test), this question is not directly applicable. However, the ground truth for assay development and optimization (which can be considered analogous to "training" in a broader sense of device development) would have been established through extensive laboratory work using:

• Highly characterized viral cultures or purified nucleic acids: Used to define target sequences, optimize primer/probe design, and determine initial analytical sensitivity.
• Spiked samples: Adding known quantities of targets or interferents to negative clinical matrices to mimic real-world conditions during early development.
• Early clinical samples: Used to refine assay performance and resolve initial issues prior to formal validation studies.

These processes ensure the assay correctly identifies the target nucleic acids.

Ask a specific question about this device

The cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test is an automated rapid multiplex real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A virus, influenza B virus and respiratory syncytial virus (RSV) nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens from individuals exhibiting signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory viral infection due to SARS-CoV-2, influenza and RSV can be similar. This test is intended to aid in the differential diagnosis of SARS-CoV-2, influenza A, influenza B, and RSV infections in humans and is not intended to detect influenza C virus infections.

Nucleic acids from the viral organisms identified by this test are generally detectable in nasopharyngeal and nasal swab specimens during the acute phase of infection. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of respiratory tract infection are indicative of the presence of the identified virus, and aid in diagnosis if used in conjunction with other clinical and epidemiological information, and laboratory findings.

The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Positive results do not rule out coinfection with other organisms. The organism(s) detected by the cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test may not be the definite cause of disease. Negative results do not preclude SARS-CoV-2, influenza A virus, influenza B virus, or RSV infections.

The cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test is performed on the cobas liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2, a well-conserved region of the matrix gene of influenza A (Flu A target), the nonstructural protein 1 (NS1) gene of influenza B (Flu B target) and the matrix gene of RSV (RSV target). An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.

The provided text describes the analytical and clinical performance evaluation of the cobas® liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test, which is a real-time RT-PCR assay. The information mainly focuses on the performance characteristics required for FDA clearance (510(k)).

Here's a breakdown of the requested information based on the provided document:

Acceptance Criteria and Device Performance

The document does not explicitly present a table of "acceptance criteria" in a pass/fail format for clinical performance. Instead, it demonstrates the device's performance through various analytical studies and clinical agreement percentages relative to a comparator method. The acceptance for a 510(k) submission is typically that the device is "substantially equivalent" to a legally marketed predicate device, which implies demonstrating comparable performance characteristics.

The key performance metrics are the Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) in clinical studies. While there aren't explicit numeric acceptance criteria stated, the achieved performance values are presented as evidence of substantial equivalence.

Here’s a table summarizing the reported clinical device performance based on the prospective study (Table 13) and the retrospective study (Table 15). These are the metrics by which the device's clinical performance would be "accepted" as substantially equivalent.

Table of Reported Device Performance (Clinical)

Note on "Acceptance Criteria" for Analytical Performance: The document describes detailed analytical studies (LoD, inclusivity, cross-reactivity, interference, reproducibility), and the reported hit rates and concentrations demonstrate that the device met the internal analytical performance specifications, which are implicitly the "acceptance criteria" for these aspects. For instance, for LoD, the acceptance criterion is implied to be ≥95% hit rate at the determined concentration. For inclusivity, it's detection at or near 3x LoD. For cross-reactivity and interference, the acceptance criterion is no cross-reactivity/interference observed. The document states that "none of the organisms tested cross reacted or interfered" and that "substances... did not interfere," indicating successful meeting of these criteria. For reproducibility, the agreement percentages for positive and negative samples are above 99% for most categories.

Study Details

1. Sample sizes used for the test set and the data provenance:

   • Prospective Clinical Study (Category I):
     • NPS specimens: 1729 enrolled subjects leading to 1704 evaluable specimens for SARS-CoV-2, Flu A, Flu B, and 1705 for RSV.
     • ANS specimens: 1729 enrolled subjects leading to 1705 evaluable specimens for SARS-CoV-2, Flu B, 1703 for Flu A, and 1706 for RSV.
     • Data Provenance: Fresh specimens, prospective, collected between September 2023 and March 2024 at 14 point-of-care testing sites in the United States (US).
   • Retrospective Clinical Study (Category III):
     • Specimens: Frozen archived clinical NPS (n=223) and ANS (n=206) specimens.
     • Data Provenance: Retrospective, collected between 2019 and 2023. Distributed to 6 sites for testing. Country of origin not explicitly stated but implied to be US given the overall context of a US FDA clearance (though not definitively stated for the retrospective part).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
   The ground truth for the clinical test set was established by comparing the results of the cobas® liat test to the "results of an FDA-cleared Nucleic Acid Amplification Test (NAAT)." The document does not specify the number of human experts, their qualifications, or their role in establishing this ground truth. The "ground truth" here is the result from the comparator NAAT, not human expert interpretation of images or clinical data.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
   The document describes a discrepant analysis for cases where the cobas® liat test results differed from the comparator NAAT. For the prospective study, the discrepancy analysis showed how many of the discrepant results (e.g., cobas® liat negative, comparator positive) were ultimately confirmed as positive or negative by further investigation (implied to be by the comparator method or potentially a third method, though not explicitly detailed beyond "discrepant NAAT results"). The details provided are:

   • SARS-CoV-2 NPS: Of 12 negative cobas® liat/positive comparator, 8 were positive and 4 negative. Of 35 positive cobas® liat/negative comparator, 12 were positive and 23 negative.
   • Similar analyses are provided for other targets and specimen types.
     This implies an adjudication method where discrepant results were further investigated, likely with repeat testing or a confirmatory reference method, but the specific "2+1" or "3+1" reader/expert adjudication model (common in imaging studies) is not applicable or described for this in vitro diagnostic (IVD) device.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
   No, an MRMC comparative effectiveness study was not done. This type of study is primarily relevant for imaging devices that assist human readers (e.g., AI for radiology). The cobas® liat test is an automated molecular diagnostic test directly detecting nucleic acids, not an AI-assisted interpretation device for human "readers."

5. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
   Yes, the performance reported (PPA, NPA, and analytical studies) represents the standalone performance of the cobas® liat device. It is an automated system where the "algorithm" (the RT-PCR assay and its interpretation software) directly produces a qualitative result (Detected/Not Detected), without human "in-the-loop" interpretation for the primary result. Human operators load samples and review results, but the analytical detection and differentiation itself is automated.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
   The ground truth for both prospective and retrospective clinical studies was based on the results of an FDA-cleared Nucleic Acid Amplification Test (NAAT), which served as the comparator method. For discrepant samples, further re-testing against the comparator or a reference method was performed for adjudication. This falls under a "reference standard" or "comparator method" type of ground truth.

7. The sample size for the training set:
   The document does not specify the sample size for a "training set." This type of molecular diagnostic device typically relies on analytical validation (LoD, inclusivity, specificity) and clinical validation through comparison to a reference method, rather than a machine learning model that requires explicit training data. The development process would involve iterative optimization of primers, probes, and assay conditions, but this is not typically referred to as a "training set" in the context of IVD submissions, especially for traditional PCR assays.

8. How the ground truth for the training set was established:
   As no explicit "training set" for a machine learning model is described, the question of how its ground truth was established is not applicable based on the provided text. The "ground truth" in the context of this traditional IVD development refers to the reliable identification of the target analytes in samples for analytical and clinical validation, often through established reference methods or characterized materials.

Ask a specific question about this device

The cobas® liat SARS-CoV-2 v2 nucleic acid test is an automated real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the qualitative detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens collected from individuals exhibiting signs and symptoms of respiratory tract infection (i.e., symptomatic). Additionally, this test is intended to be used with nasal and nasopharyngeal swab specimens collected from individuals without signs and symptoms of COVID-19 (i.e., asymptomatic).

The cobas® liat SARS-CoV-2 v2 nucleic acid test is intended for use as an aid in the diagnosis of COVID-19 if used in conjunction with other clinical and epidemiological information and laboratory findings. SARS-CoV-2 RNA is generally detectable in nasal swab and nasopharyngeal swab specimens during the acute phase of infection.

Positive results are indicative of the presence of SARS-CoV-2 RNA. Positive results do not rule out co-infection with other microorganisms. Negative results do not preclude SARS-CoV-2 infection. Negative results must be combined with clinical observations, patient history, and epidemiological information. The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

A negative result from an asymptomatic individual is presumptive. Additionally, a negative result obtained with a nasal or nasopharyngeal swab collected from an asymptomatic individual should be followed up by testing at least twice over three days with at least 48 hours between tests.

The cobas® liat SARS-CoV-2 v2 nucleic acid test is performed on the cobas® liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2. An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.

The provided text is a 510(k) Clearance Letter for a medical device which does not include information about AI/ML models. Therefore, it's not possible to extract the information you requested about Acceptance Criteria and a study proving an AI/ML device meets those criteria.

The device described, the "cobas liat SARS-CoV-2 v2 nucleic acid test," is an in vitro diagnostic (IVD) device based on real-time RT-PCR technology. It directly detects viral targets and its performance is evaluated through analytical and clinical studies common for IVDs, not against AI/ML performance metrics like sensitivity, specificity, MRMC studies, or multi-reader reviews.

Here's a breakdown of why your requested information isn't present in this document:

• No AI/ML Component: The document describes a traditional RT-PCR assay. There is no mention of algorithms, machine learning, deep learning, or any AI component.
• Performance Metrics Differ: The performance metrics provided (Limit of Detection, Inclusivity, Cross-reactivity, Reproducibility, Positive Percent Agreement, Negative Percent Agreement) are standard for IVD assays. They are not analogous to metrics used for evaluating AI/ML models (e.g., AUC, F1-score, accuracy in image classification, or diagnostic improvement from AI-assistance).
• No Human Reader Interaction: Since it's an automated lab test, there's no "human reader" (like a radiologist) involved in interpreting the device's output in a way that an AI would assist. The output is qualitative (Detected/Not Detected).
• No Ground Truth Experts in the AI Sense: Ground truth for this IVD is established by a "comparator" (another FDA-cleared NAAT) and clinical/epidemiological information, not by multiple human experts reviewing AI outputs or images.
• No Training/Test Set Split for AI: The "test set" and "training set" concepts described in your request are fundamental to AI/ML model development and validation. For this IVD, there's a "clinical performance evaluation" using prospective and retrospective samples, which serves as the validation dataset, but it's not structured as a training/test split for an AI.

Therefore, I cannot provide the requested table and detailed points because the provided document does not pertain to an AI/ML device.

If you have a document describing an AI/ML medical device, please provide that, and I can attempt to extract the relevant information.

Ask a specific question about this device