Search Results

The cobas® Respiratory 4-flex for use on the cobas® 5800/6800/8800 Systems is an automated, multiplex, nucleic acid test that utilizes real-time polymerase chain reaction (PCR) technology intended for simultaneous in vitro qualitative detection and differentiation of severe acute respiratory syndrome coronavirus (SARS-CoV-2), influenza A virus, influenza B virus, and respiratory syncytial virus (RSV) in nasopharyngeal swab specimens obtained from individuals with signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory viral infection due to SARS-CoV-2, influenza A, influenza B and RSV can be similar. This test is intended to aid in the differential diagnosis of SARS-CoV-2, influenza A, influenza B, and RSV infections in humans and is not intended to detect influenza C virus infections.

Nucleic acids from the viral organisms identified by this test are generally detectable in nasopharyngeal swab specimens during the acute phase of infection. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of respiratory tract infection are indicative of the presence of the identified virus, and aid in diagnosis if used in conjunction with other clinical and epidemiological information, and laboratory findings.

The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Negative results do not preclude SARS-CoV-2, influenza A virus, influenza B virus, or RSV infections. Conversely, positive results do not rule out coinfection with other organisms, and the agent(s) detected by the cobas® Respiratory 4-flex for use on the cobas® 5800/6800/8800 Systems may not be the definite cause of disease.

cobas® Respiratory 4-flex for use on the cobas® 5800/6800/8800 Systems (cobas® Respiratory 4-flex) is based on fully automated sample preparation (nucleic acid extraction and purification) followed by PCR amplification and detection. The cobas® 5800 System is designed as one integrated instrument. The cobas® 6800/8800 Systems consist of the sample supply module, the transfer module, the processing module, and the analytic module. Automated data management is performed by the cobas® 5800 System or cobas® 6800/8800 Systems software(s), which assigns results for all tests. Results can be reviewed directly on the system screen and printed as a report.

Nucleic acid from patient samples and added Internal Control RNA (RNA IC) molecules are simultaneously extracted. Nucleic acid is released by addition of proteinase and lysis reagent to the sample. The released nucleic acid binds to the silica surface of the added magnetic glass particles. Unbound substances and impurities, such as denatured protein, cellular debris and potential PCR inhibitors, are removed with subsequent wash steps and purified nucleic acid is eluted from the magnetic glass particles with elution buffer at elevated temperature. External controls (positive and negative) are processed in the same way.

Selective amplification of target nucleic acid from the sample is achieved by the use of target-specific forward and reverse primers detecting conserved viral genome regions as shown in Table 1.

Selective amplification of RNA IC is achieved by the use of non-competitive, sequence specific forward and reverse primers, which have no homology with the viral-target specific genomes. Amplified target is detected by the cleavage of fluorescently labeled oligonucleotide probes. Roche's temperature assisted generation of signal (TAGS) technology, short TAGS technology, is introduced to differentiate up to three targets per fluorescence channel, enabling the detection of up to14 targets, and the Internal Control, per well. A thermostable DNA polymerase enzyme is used for amplification.

Multiplicity of target detection is enabled with temperature-dependent quenching of cleaved fluorescent target-specific probes. This is achieved by separating signals from probes into introduced thermal channels, where fluorescence is acquired at two additional fixed temperatures for each amplification cycle.

During the PCR amplification step, hybridization of the probes to the specific single-stranded DNA template results in cleavage of the probe by the 5' to 3' exonuclease activity of the DNA polymerase, resulting in separation of the reporter and quencher dyes, and the generation of a fluorescent signal. Conventional probes release fluorescence signal immediately upon separation of reporter from quencher. TAGS probes rely on temperature dependent fluorescence activation, requiring both nuclease cleavage during the extension phase, as well as an increase in reaction temperature, to activate the otherwise dormant fluorophore. For this reason, during each PCR cycle the TAGS technology captures fluorescence in five available fluorescence channels in combination with three thermal channels (detection of fluorescence at three defined temperatures T1, T2 and T3).

The cobas® Respiratory 4-flex master mix contains detection probes which are specific for influenza A virus, influenza B virus, RSV, SARS-CoV-2 and the RNA Internal Control (RNA IC) nucleic acid, which enables simultaneous detection and differentiation of influenza A virus, influenza B virus, RSV, and SARS-CoV-2 viral targets and the RNA IC.

The master mix includes deoxyuridine triphosphate (dUTP), instead of deoxythymidine triphosphate (dTTP), which is incorporated into the newly synthesized DNA (amplicon). Any contaminating amplicons from previous PCR runs are destroyed by the AmpErase enzyme [uracil-N-glycosylase], which is included in the PCR mix, when heated in the first thermal cycling step. However, newly formed amplicons are not destroyed since the AmpErase enzyme is inactivated once exposed to temperatures above 55°C.

The RESP-4FLEX ASAP enables the system to differentiate and report the qualitative results of the four targets influenza A virus, influenza B virus, RSV and SARS-CoV-2. For each specimen the customer can test for any combination of the four enabled virus targets. Also, additional target calculation (digital reflex) can be ordered for the four enabled virus targets (influenza A virus, influenza B virus, RSV and SARS-CoV-2) on the cobas® 5800 System.

Here's an analysis of the acceptance criteria and study detailed in the provided FDA clearance letter for the cobas Respiratory 4-flex, structured according to your request:

1. Table of Acceptance Criteria and Reported Device Performance

The FDA clearance letter does not explicitly state pre-defined acceptance criteria for the clinical performance. Instead, it reports the observed performance metrics (PPA and NPA) from the clinical studies. For the purpose of this table, I will present the reported clinical performance as the "met acceptance criteria," assuming these values were deemed acceptable by the FDA for clearance.

Study Proving Device Meets Criteria: The detailed clinical performance evaluation described in "5. CLINICAL PERFORMANCE EVALUATION" (pages 23-26) demonstrates that the cobas Respiratory 4-flex achieves the reported Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) values against a U.S. FDA-cleared molecular comparator assay. These reported percentage agreements, along with their 95% confidence intervals, serve as the evidence that the device meets the performance expectations for clinical use.

2. Sample Size Used for the Test Set and Data Provenance

• Prospective Clinical Study:

  • Total NPS specimens enrolled: 4,475
  • Total NPS specimens tested: 4,378 (1,832 fresh, 2,546 frozen)
  • Total NPS specimens evaluable: 4,341 (1,827 fresh, 2,514 frozen)
  • Data Provenance: Fresh prospective specimens collected at eleven collection sites during the 2023-2024 respiratory viral season. Frozen prospective specimens collected during parts of the 2022-2023 respiratory viral season at seven sites and the 2023-2024 respiratory viral season from 14 collection sites. The specific country of origin is not explicitly stated but implies U.S. based on the "U.S. testing sites" mention for retrospective samples, and the FDA clearance context. The data is prospective.

• Retrospective Clinical Study:

  • Total NPS specimens enrolled: 770
  • Total NPS specimens evaluable:
    • Influenza A: 657
    • Influenza B: 647
    • RSV: 659
  • Data Provenance: Archived NPS specimens collected between 2014 and 2022 from individuals with signs and symptoms of respiratory viral infection. Tested at three (3) U.S. testing sites. The data is retrospective.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not mention the use of human experts to establish ground truth for the test set. The ground truth was established by a "U.S. FDA-cleared molecular assay" as the comparator method.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method involving experts for discrepant results. Instead, it states that an "FDA 510(k) cleared comparator" method was used to establish the "ground truth." For the influenza A target in the prospective study, "three (3) additional specimens (two (2) fresh and one (1) frozen) were excluded from analysis due to inconclusive results obtained from the comparator test," implying a reliance on the comparator's definitive result rather than external adjudication. The same pattern is noted for retrospective samples, with exclusions for failed or invalid comparator test results.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of Human Readers' Improvement with AI vs. Without AI Assistance

This section is Not Applicable to the provided document. The cobas Respiratory 4-flex is an automated, multiplex nucleic acid detection test, not an AI-powered diagnostic tool requiring human reader interpretation or assistance. Therefore, an MRMC study or analysis of human reader improvement with AI is not relevant.

6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done

Yes, this was a standalone performance evaluation. The device is an automated in vitro diagnostic (IVD) test. The clinical performance evaluation directly compares the cobas Respiratory 4-flex's results (algorithm only, as it's an automated system) against a predicate FDA-cleared molecular assay. There is no human interpretation or intervention in the generation of the primary test result from the cobas system itself.

7. The Type of Ground Truth Used (Expert Consensus, Pathology, Outcomes Data, etc.)

The ground truth for the clinical performance evaluation was established by a "U.S. FDA-cleared molecular assay." This means another legally marketed and validated molecular diagnostic test was used as the reference standard.

8. The Sample Size for the Training Set

The document does not specify a separate training set or its sample size. This is typical for in vitro diagnostic (IVD) devices that use established laboratory techniques (like PCR) rather than machine learning algorithms which require explicit training data. The development of such assays involves extensive analytical validation (LoD, precision, specificity, inclusivity, etc.) and then clinical validation with independent samples.

9. How the Ground Truth for the Training Set Was Established

Since no explicit training set for a machine learning algorithm is mentioned, the concept of "ground truth for the training set" is not applicable in the context of this device's validation as described. The analytical studies (LoD, inclusivity, specificity, etc.) characterize the inherent performance of the assay's chemical and hardware components.

Ask a specific question about this device

Opulus™ Lymphoma Precision is a software device that uses a machine learning-based algorithm to automate segmentation and visualization of lesions along with automation of measurement of total metabolic tumor volume within whole-body FDG-PET/CT scans of patients with FDG-avid lymphomas.

Opulus™ Lymphoma Precision is used to assist trained interpreting physicians with visualization of suspected lesions and calculation of total volume of all lesions in a body. This information can be used in addition to the standard of care image interpretation of FDG-PET/CT scans. Opulus™ Lymphoma Precision annotated images can be reviewed by an appropriately trained physician.

The algorithm is assistive, and requires a radiologist review, who will make the final decision on FDG-PET/CT image interpretation.

Opulus™ Lymphoma Precision is an assistive tool which can be used by physicians to automate the labor intensive task of quantifying disease burden in whole-body FDG-PET/CT scans of patients already diagnosed with FDG-avid lymphomas. It does so by using a machine learning methodology to localize and segment FDG-PET activity ('hot-spots' on FDG-PET scans) of lymphoma lesions within a PET/CT image. Opulus™ Lymphoma Precision does not screen for or diagnose lymphoma. It is intended for patients already diagnosed with FDG-avid lymphoma.

The following is a list of key functionalities algorithm performs to accomplish the proposed intended use.

• localization and segmentation,
• visualization of lymphoma-related tumor lesions
• quantification of Total Metabolic Tumor Volume (TMTV)

Opulus™ Lymphoma Precision aids the efficiency of medical professionals by automatically generating tumor boundary Regions of Interest (ROIs) and quantifying TMTV, which is a tedious task when performed manually. The physician has the option to accept/reject the output generated by the device. The user does not have the ability to modify the device output.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) clearance letter for Opulus™ Lymphoma Precision:

1. Table of Acceptance Criteria and Reported Device Performance

The provided document does not explicitly present a table of "acceptance criteria" with pass/fail thresholds. However, it does state the objectives of the performance validation study and the results that demonstrate the device's agreement and accuracy. We'll infer the implicit acceptance criteria from these objectives.

Ask a specific question about this device

The cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test is an automated rapid multiplex real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A virus and influenza B virus nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens from individuals exhibiting signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory tract infection due to SARS-CoV-2 and influenza can be similar. This test is intended to aid in the differential diagnosis of SARS-CoV-2, influenza A and influenza B infections in humans and is not intended to detect influenza C virus infections.

Nucleic acids from the viral organisms identified by this test are generally detectable in nasopharyngeal and nasal swab specimens during the acute phase of infection. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of respiratory tract infection are indicative of the presence of the identified virus, and aid in diagnosis if used in conjunction with other clinical and epidemiological information and laboratory findings.

The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Positive results do not rule out coinfection with other organisms. The organism(s) detected by the cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test may not be the definite cause of disease. Negative results do not preclude SARS-CoV-2, influenza A virus or influenza B virus infections.

The cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test is performed on the cobas liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2, a well-conserved region of the matrix gene of influenza A (Flu A target), and the nonstructural protein 1 (NS1) gene of influenza B (Flu B target). An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.

This document describes the validation study for the cobas liat SARS-CoV-2 & Influenza A/B v2 nucleic acid test.

Here's an analysis of the acceptance criteria and the study proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

Since this is a diagnostic test, the primary acceptance criteria revolve around analytical and clinical performance metrics like Limit of Detection, Inclusivity, Cross-Reactivity, Reproducibility, and Clinical Agreement (Positive Percent Agreement and Negative Percent Agreement). The document doesn't explicitly state "acceptance criteria" values in a separate table, but these are implied by the performance metrics reported and the general standards for diagnostic device clearance. I will extract the reported device performance from the provided text.

2. Sample Sizes Used for the Test Set and Data Provenance

• Prospective Clinical Study:

  • Sample Size: 1729 symptomatic subjects enrolled.
    • 1705 evaluable NPS specimens for analysis (19 non-evaluable due to missing/invalid results, 5 due to handling).
    • 1706 evaluable ANS specimens for SARS-CoV-2 and Influenza B analysis (22 non-evaluable due to missing/invalid results, 1 due to handling).
    • 1704 evaluable ANS specimens for Influenza A analysis (2 additional found inconclusive for comparator).
  • Data Provenance: Prospective, collected between September 2023 and March 2024 at 14 point-of-care testing sites in the United States (US).

• Retrospective Clinical Study (Influenza B Supplement):

  • Sample Size: 223 archived NPS specimens and 206 archived ANS specimens (total 429).
    • One NPS sample pre-characterized as positive for influenza B was non-evaluable.
  • Data Provenance: Retrospective, frozen archived (Category III) specimens collected between 2019 and 2023. Distributed to 6 sites for testing.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Experts

The document does not mention the use of experts to establish ground truth for the clinical test sets. For molecular diagnostic tests like this, the ground truth is typically established by comparing the investigational device's results against a highly accurate, accepted comparator method (another FDA-cleared Nucleic Acid Amplification Test specific for the target analytes). The expertise lies in the development and validation of these comparator methods, not in individual expert review of each sample for ground truth in this context.

4. Adjudication Method for the Test Set

The document describes discrepant result analysis for both prospective and retrospective clinical studies.

• For the prospective study, "discrepant NAAT results" are detailed for SARS-CoV-2 (NPS and ANS), Influenza A (NPS and ANS), and Influenza B (NPS and ANS).
• For the retrospective study, discrepant NAAT results are detailed for Influenza B (NPS and ANS).

The method appears to be:

• The cobas liat test result is compared to the FDA-cleared comparator NAAT result.
• When there's a discrepancy (e.g., cobas liat positive, comparator negative), it explicitly states how many were "positive" and "negative" upon further investigation or re-evaluation (e.g., with "discrepant NAAT results").
  • For example: "Of 12 specimens negative on cobas® liat and positive on the comparator, 8 were positive and 4 were negative." This implies some form of re-testing or deeper analysis (not specified as "adjudication by experts" but rather "discrepant NAAT results"). It's more of a re-confirmation of the comparator or a third method, rather than a human expert consensus process. Such re-evaluation often involves re-testing using the comparator or a reference method.

Therefore, while there's no "2+1" or "3+1" expert adjudication method described as would be seen in imaging studies, there is a discrepant resolution process based on further NAAT results. It's not "none" in the sense that discrepancies are just reported without follow-up; rather, they are further investigated using additional NAAT results to re-confirm the original comparator status if possible.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

No. This is a standalone diagnostic test (RT-PCR), not an AI-assisted imaging device or a test that involves human "readers" interpreting results. Therefore, an MRMC comparative effectiveness study involving human readers and AI assistance is not applicable and was not performed.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, implicitly. This is a fully automated RT-PCR test run on the cobas liat analyzer. The performance metrics (LoD, inclusivity, cross-reactivity, reproducibility, and clinical agreement) are measures of the device's performance on its own against established ground truth (comparator NAAT). While humans load samples and interpret the final digital result (positive/negative), the core detection and differentiation is algorithm-driven within the instrument, making its performance essentially "standalone" in the context of diagnostic accuracy.

7. The Type of Ground Truth Used

• Clinical Performance (Prospective and Retrospective): The ground truth for clinical sample testing was established by comparing the cobas liat results against an FDA-cleared Nucleic Acid Amplification Test (NAAT), which serves as the reference or "ground truth" method for molecular diagnostic assays. The document explicitly states: "PPA and NPA were determined by comparing the results of cobas® liat SARS-CoV-2 & Influenza A/B v2 to the results of an FDA-cleared Nucleic Acid Amplification Test (NAAT)." and "The comparator method was an acceptable FDA-cleared molecular assay."
• Analytical Studies (LoD, Inclusivity, Cross-Reactivity, Interference, Reproducibility): Ground truth was established by preparing precisely known concentrations of viral material (cultured or inactivated viruses) or specific microorganisms in controlled laboratory settings. For these studies, the "ground truth" is meticulously prepared and verified laboratory standards.

8. The Sample Size for the Training Set

The document does not specify a separate "training set" sample size. For an RT-PCR diagnostic platform, the "training" involves the fundamental biochemical and optical engineering, and the optimization of assay (reagent) design to achieve sensitivity and specificity. This is distinct from machine learning models that often require large, labeled datasets for "training." The analytical and clinical validation studies described here are verification and validation (V&V) studies, akin to a "test set" to prove the device's performance against its design specifications and clinical utility.

9. How the Ground Truth for the Training Set Was Established

Since no explicit "training set" for a machine learning algorithm is mentioned (as this is a molecular diagnostic test), this question is not directly applicable. However, the ground truth for assay development and optimization (which can be considered analogous to "training" in a broader sense of device development) would have been established through extensive laboratory work using:

• Highly characterized viral cultures or purified nucleic acids: Used to define target sequences, optimize primer/probe design, and determine initial analytical sensitivity.
• Spiked samples: Adding known quantities of targets or interferents to negative clinical matrices to mimic real-world conditions during early development.
• Early clinical samples: Used to refine assay performance and resolve initial issues prior to formal validation studies.

These processes ensure the assay correctly identifies the target nucleic acids.

Ask a specific question about this device

The cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test is an automated rapid multiplex real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A virus, influenza B virus and respiratory syncytial virus (RSV) nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens from individuals exhibiting signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory viral infection due to SARS-CoV-2, influenza and RSV can be similar. This test is intended to aid in the differential diagnosis of SARS-CoV-2, influenza A, influenza B, and RSV infections in humans and is not intended to detect influenza C virus infections.

Nucleic acids from the viral organisms identified by this test are generally detectable in nasopharyngeal and nasal swab specimens during the acute phase of infection. The detection and identification of specific viral nucleic acids from individuals exhibiting signs and symptoms of respiratory tract infection are indicative of the presence of the identified virus, and aid in diagnosis if used in conjunction with other clinical and epidemiological information, and laboratory findings.

The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Positive results do not rule out coinfection with other organisms. The organism(s) detected by the cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test may not be the definite cause of disease. Negative results do not preclude SARS-CoV-2, influenza A virus, influenza B virus, or RSV infections.

The cobas liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test is performed on the cobas liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2, a well-conserved region of the matrix gene of influenza A (Flu A target), the nonstructural protein 1 (NS1) gene of influenza B (Flu B target) and the matrix gene of RSV (RSV target). An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.

The provided text describes the analytical and clinical performance evaluation of the cobas® liat SARS-CoV-2, Influenza A/B & RSV nucleic acid test, which is a real-time RT-PCR assay. The information mainly focuses on the performance characteristics required for FDA clearance (510(k)).

Here's a breakdown of the requested information based on the provided document:

Acceptance Criteria and Device Performance

The document does not explicitly present a table of "acceptance criteria" in a pass/fail format for clinical performance. Instead, it demonstrates the device's performance through various analytical studies and clinical agreement percentages relative to a comparator method. The acceptance for a 510(k) submission is typically that the device is "substantially equivalent" to a legally marketed predicate device, which implies demonstrating comparable performance characteristics.

The key performance metrics are the Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) in clinical studies. While there aren't explicit numeric acceptance criteria stated, the achieved performance values are presented as evidence of substantial equivalence.

Here’s a table summarizing the reported clinical device performance based on the prospective study (Table 13) and the retrospective study (Table 15). These are the metrics by which the device's clinical performance would be "accepted" as substantially equivalent.

Table of Reported Device Performance (Clinical)

Note on "Acceptance Criteria" for Analytical Performance: The document describes detailed analytical studies (LoD, inclusivity, cross-reactivity, interference, reproducibility), and the reported hit rates and concentrations demonstrate that the device met the internal analytical performance specifications, which are implicitly the "acceptance criteria" for these aspects. For instance, for LoD, the acceptance criterion is implied to be ≥95% hit rate at the determined concentration. For inclusivity, it's detection at or near 3x LoD. For cross-reactivity and interference, the acceptance criterion is no cross-reactivity/interference observed. The document states that "none of the organisms tested cross reacted or interfered" and that "substances... did not interfere," indicating successful meeting of these criteria. For reproducibility, the agreement percentages for positive and negative samples are above 99% for most categories.

Study Details

1. Sample sizes used for the test set and the data provenance:

   • Prospective Clinical Study (Category I):
     • NPS specimens: 1729 enrolled subjects leading to 1704 evaluable specimens for SARS-CoV-2, Flu A, Flu B, and 1705 for RSV.
     • ANS specimens: 1729 enrolled subjects leading to 1705 evaluable specimens for SARS-CoV-2, Flu B, 1703 for Flu A, and 1706 for RSV.
     • Data Provenance: Fresh specimens, prospective, collected between September 2023 and March 2024 at 14 point-of-care testing sites in the United States (US).
   • Retrospective Clinical Study (Category III):
     • Specimens: Frozen archived clinical NPS (n=223) and ANS (n=206) specimens.
     • Data Provenance: Retrospective, collected between 2019 and 2023. Distributed to 6 sites for testing. Country of origin not explicitly stated but implied to be US given the overall context of a US FDA clearance (though not definitively stated for the retrospective part).

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
   The ground truth for the clinical test set was established by comparing the results of the cobas® liat test to the "results of an FDA-cleared Nucleic Acid Amplification Test (NAAT)." The document does not specify the number of human experts, their qualifications, or their role in establishing this ground truth. The "ground truth" here is the result from the comparator NAAT, not human expert interpretation of images or clinical data.

3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
   The document describes a discrepant analysis for cases where the cobas® liat test results differed from the comparator NAAT. For the prospective study, the discrepancy analysis showed how many of the discrepant results (e.g., cobas® liat negative, comparator positive) were ultimately confirmed as positive or negative by further investigation (implied to be by the comparator method or potentially a third method, though not explicitly detailed beyond "discrepant NAAT results"). The details provided are:

   • SARS-CoV-2 NPS: Of 12 negative cobas® liat/positive comparator, 8 were positive and 4 negative. Of 35 positive cobas® liat/negative comparator, 12 were positive and 23 negative.
   • Similar analyses are provided for other targets and specimen types.
     This implies an adjudication method where discrepant results were further investigated, likely with repeat testing or a confirmatory reference method, but the specific "2+1" or "3+1" reader/expert adjudication model (common in imaging studies) is not applicable or described for this in vitro diagnostic (IVD) device.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
   No, an MRMC comparative effectiveness study was not done. This type of study is primarily relevant for imaging devices that assist human readers (e.g., AI for radiology). The cobas® liat test is an automated molecular diagnostic test directly detecting nucleic acids, not an AI-assisted interpretation device for human "readers."

5. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
   Yes, the performance reported (PPA, NPA, and analytical studies) represents the standalone performance of the cobas® liat device. It is an automated system where the "algorithm" (the RT-PCR assay and its interpretation software) directly produces a qualitative result (Detected/Not Detected), without human "in-the-loop" interpretation for the primary result. Human operators load samples and review results, but the analytical detection and differentiation itself is automated.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
   The ground truth for both prospective and retrospective clinical studies was based on the results of an FDA-cleared Nucleic Acid Amplification Test (NAAT), which served as the comparator method. For discrepant samples, further re-testing against the comparator or a reference method was performed for adjudication. This falls under a "reference standard" or "comparator method" type of ground truth.

7. The sample size for the training set:
   The document does not specify the sample size for a "training set." This type of molecular diagnostic device typically relies on analytical validation (LoD, inclusivity, specificity) and clinical validation through comparison to a reference method, rather than a machine learning model that requires explicit training data. The development process would involve iterative optimization of primers, probes, and assay conditions, but this is not typically referred to as a "training set" in the context of IVD submissions, especially for traditional PCR assays.

8. How the ground truth for the training set was established:
   As no explicit "training set" for a machine learning model is described, the question of how its ground truth was established is not applicable based on the provided text. The "ground truth" in the context of this traditional IVD development refers to the reliable identification of the target analytes in samples for analytical and clinical validation, often through established reference methods or characterized materials.

Ask a specific question about this device

The cobas® liat SARS-CoV-2 v2 nucleic acid test is an automated real-time reverse transcription polymerase chain reaction (RT-PCR) test intended for the qualitative detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleic acids in anterior nasal (nasal) and nasopharyngeal swab specimens collected from individuals exhibiting signs and symptoms of respiratory tract infection (i.e., symptomatic). Additionally, this test is intended to be used with nasal and nasopharyngeal swab specimens collected from individuals without signs and symptoms of COVID-19 (i.e., asymptomatic).

The cobas® liat SARS-CoV-2 v2 nucleic acid test is intended for use as an aid in the diagnosis of COVID-19 if used in conjunction with other clinical and epidemiological information and laboratory findings. SARS-CoV-2 RNA is generally detectable in nasal swab and nasopharyngeal swab specimens during the acute phase of infection.

Positive results are indicative of the presence of SARS-CoV-2 RNA. Positive results do not rule out co-infection with other microorganisms. Negative results do not preclude SARS-CoV-2 infection. Negative results must be combined with clinical observations, patient history, and epidemiological information. The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

A negative result from an asymptomatic individual is presumptive. Additionally, a negative result obtained with a nasal or nasopharyngeal swab collected from an asymptomatic individual should be followed up by testing at least twice over three days with at least 48 hours between tests.

The cobas® liat SARS-CoV-2 v2 nucleic acid test is performed on the cobas® liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the ORF1 a/b non-structural region and membrane protein gene that are unique to SARS-CoV-2. An Internal Control (IC) is included to control for adequate processing of the target virus through all steps of the assay process and to monitor the presence of inhibitors in the RT-PCR processes.

The provided text is a 510(k) Clearance Letter for a medical device which does not include information about AI/ML models. Therefore, it's not possible to extract the information you requested about Acceptance Criteria and a study proving an AI/ML device meets those criteria.

The device described, the "cobas liat SARS-CoV-2 v2 nucleic acid test," is an in vitro diagnostic (IVD) device based on real-time RT-PCR technology. It directly detects viral targets and its performance is evaluated through analytical and clinical studies common for IVDs, not against AI/ML performance metrics like sensitivity, specificity, MRMC studies, or multi-reader reviews.

Here's a breakdown of why your requested information isn't present in this document:

• No AI/ML Component: The document describes a traditional RT-PCR assay. There is no mention of algorithms, machine learning, deep learning, or any AI component.
• Performance Metrics Differ: The performance metrics provided (Limit of Detection, Inclusivity, Cross-reactivity, Reproducibility, Positive Percent Agreement, Negative Percent Agreement) are standard for IVD assays. They are not analogous to metrics used for evaluating AI/ML models (e.g., AUC, F1-score, accuracy in image classification, or diagnostic improvement from AI-assistance).
• No Human Reader Interaction: Since it's an automated lab test, there's no "human reader" (like a radiologist) involved in interpreting the device's output in a way that an AI would assist. The output is qualitative (Detected/Not Detected).
• No Ground Truth Experts in the AI Sense: Ground truth for this IVD is established by a "comparator" (another FDA-cleared NAAT) and clinical/epidemiological information, not by multiple human experts reviewing AI outputs or images.
• No Training/Test Set Split for AI: The "test set" and "training set" concepts described in your request are fundamental to AI/ML model development and validation. For this IVD, there's a "clinical performance evaluation" using prospective and retrospective samples, which serves as the validation dataset, but it's not structured as a training/test split for an AI.

Therefore, I cannot provide the requested table and detailed points because the provided document does not pertain to an AI/ML device.

If you have a document describing an AI/ML medical device, please provide that, and I can attempt to extract the relevant information.

Ask a specific question about this device

cobas® SARS-CoV-2 Qualitative for use on the cobas®5800/6800/8800 Systems is a real-time RT-PCR test intended for the qualitative detection of nucleic acids from SARS-CoV-2 in nasopharyngeal swab specimens collected from individuals with signs and symptoms of COVID-19 and in anterior nasal swab specimens collected from any individuals with or without signs and symptoms of COVID-19.

Positive results are indicative of the presence of SARS-CoV-2 RNA. Positive results do not rule out bacterial infection or co-infection with other pathogens.

Negative results do not preclude SARS-CoV-2 infection and should not be used as the sole basis for patient management decisions. Results are meant to be used in conjunction with clinical observations, patient history, recent exposures, epidemiological information, and laboratory data, in accordance with the guided by the relevant public health authorities.

cobas® SARS-CoV-2 Qualitative is based on fully automated sample preparation (nucleic acid extraction and purification) followed by PCR amplification and detection. The cobas® 5800 System is designed as one integrated instrument. The cobas® 6800/8800 Systems consist of the sample supply module, the transfer module, the processing module, and the analytic module. Automated data management is performed by the cobas® 5800 or cobas® 6800/8800 Systems software(s), which assigns test results for all tests. Results can be reviewed directly on the system screen, and printed as a report.

Nucleic acid from patient samples and added internal control RNA (RNA IC) molecules are simultaneously extracted. Nucleic acid is released by addition of proteinase and lysis reagent to the sample. The released nucleic acid binds to the silica surface of the added magnetic glass particles. Unbound substances and impurities, such as denatured protein, cellular debris and potential PCR inhibitors, are removed with subsequent wash steps and purified nucleic acid is eluted from the magnetic glass particles with elution buffer at elevated temperature. External controls (positive and negative) are processed in the same way.

Selective amplification of target nucleic acid from the sample is achieved by the use of targetspecific forward and reverse primers for ORF1 a/b non-structural region that is unique to SARS-CoV-2. Additionally, a conserved region in the structural protein envelope E-gene were chosen for pan-Sarbecovirus detection. The pan-Sarbecovirus detection sets will also detect SARS-CoV-2 virus.

Selective amplification of RNA Internal Control is achieved by the use of non-competitive sequence specific forward and reverse primers which have no homology with the coronavirus genome. A thermostable DNA polymerase enzyme is used for amplification.

The cobas® SARS-CoV-2 Qualitative master mix contains detection probes which are specific for the coronavirus type SARS-CoV-2, members of the Sarbecovirus subgenus, and the RNA Internal Control nucleic acid. The coronavirus and RNA Internal Control detection probes are each labeled with unique fluorescent dyes that act as a reporter. Each probe also has a second dye which acts as a quencher. When not bound to the target sequence, the fluorescent signals of the intact probes are suppressed by the quencher dye. During the PCR amplification step, hybridization of the probes to the specific single-stranded DNA template results in cleavage of the probe by the 5' to 3' exonuclease activity of the DNA polymerase resulting in separation of the reporter and quencher dyes and the generation of a fluorescent signal. With each PCR cycle, increasing amounts of cleaved probes are generated and the cumulative signal of the reporter dye increases concomitantly. Each reporter dye is measured at defined wavelengths, which enables simultaneous detection and discrimination of the amplified coronavirus target and the RNA Internal Control. The master mix includes deoxyuridine triphosphate (dUTP), instead of deoxythimidine triphosphate (dTTP), which is incorporated into the newly synthesized DNA (amplicon). Any contaminating amplicons from previous PCR runs are destroyed by the AmpErase enzyme [uracil-N-glycosylase], which is included in the PCR mix, when heated in the first thermal cycling step. However, newly formed amplicons are not destroyed since the AmpErase enzyme is inactivated once exposed to temperatures above 55°C.

cobas® SARS-CoV-2 Qualitative is a qualitative nucleic acid test for use on the cobas® 5800/6800/8800 System for the detection of the 2019 novel coronavirus (SARS-CoV-2) RNA in individual nasal and nasopharyngeal swab samples collected in Copan Universal Transport Medium System (UTM-RT), BD™ Universal Viral Transport System (UVT), cobas® PCR Media, or 0.9% physiological saline. The RNA Internal Control, used to monitor the entire sample preparation and PCR amplification process, is introduced into each specimen.

Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Device Performance

The acceptance criteria are implicitly defined by the reported performance metrics of the device, which are Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) against a comparator algorithm. The device needs to demonstrate high agreement for both positive and negative results in asymptomatic individuals.

Table of Acceptance Criteria and Reported Device Performance:

Note: The document does not explicitly state numerical acceptance criteria in a dedicated section. The "acceptance criteria" are inferred from the robust performance demonstrated and the claim of "equivalent performance" to the predicate device.

Study Details for Demonstrating Acceptance Criteria:

The device's performance was evaluated through two clinical studies focusing on asymptomatic populations: the 2020 National Football League (NFL) COVID-19 Surveillance Program and the 2021 Test Us at Home (TUAH) study.

1. NFL COVID-19 Surveillance Program (Real-world evidence)

• Sample Size Used for the Test Set: A total of 1776 samples were selected for analysis.
• Data Provenance: The data was collected from the United States (NFL COVID-19 Surveillance Program participants). It was retrospective as it used data collected between August 2020-January 2021. Samples were prospectively collected for the surveillance program itself.
• Number of Experts Used to Establish Ground Truth & Qualifications: The document does not specify the number or qualifications of experts used for clinical adjudication within the NFL testing program. It mentions a "comparator algorithm that was based on molecular comparator test results and/or clinical adjudication."
• Adjudication Method: Not explicitly stated beyond "clinical adjudication" as part of the comparator algorithm.
• Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study: No, this type of study was not done. This device is a diagnostic test, not an AI-assisted human reading system.
• Standalone Performance: Yes, this study evaluates the standalone performance of the cobas® SARS-CoV-2 Qualitative device against a composite comparator algorithm.
• Type of Ground Truth Used: "molecular comparator test results and/or clinical adjudication performed within the NFL testing program." This indicates a composite ground truth method.
• Sample Size for the Training Set: Not applicable for a clinical performance evaluation study; this is a validation study. The training set for the development of the device itself is not provided in this document.
• How the Ground Truth for the Training Set was Established: Not applicable.

2. Test Us at Home (TUAH) Study (Clinical Study)

• Sample Size Used for the Test Set: 38,192 samples from the TUAH study were included.
• Data Provenance: The data was collected from the United States (TUAH study participants). It was prospective as samples were collected between October 2021 and April 2022 specifically for the longitudinal study.
• Number of Experts Used to Establish Ground Truth & Qualifications: The document does not specify the number or qualifications of experts. The comparator algorithm for this study relies on "two consecutive test results (molecular comparator)."
• Adjudication Method: The comparator algorithm used "two consecutive test results (molecular comparator) over 48 hours" to determine the ground truth. This is a form of algorithmic adjudication based on molecular tests, not human expert adjudication in the traditional sense.
• Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study: No, this type of study was not done.
• Standalone Performance: Yes, this study evaluates the standalone performance of the cobas® SARS-CoV-2 Qualitative device against a composite comparator algorithm based on molecular results.
• Type of Ground Truth Used: Comparator algorithm based on "two consecutive test results (molecular comparator)." This indicates a molecular test-based composite ground truth.
• Sample Size for the Training Set: Not applicable for a clinical performance evaluation study.
• How the Ground Truth for the Training Set was Established: Not applicable.

Summary of In Silico Analysis (Non-Clinical Performance):

• An in-silico analysis was conducted in January 2025 using SARS-CoV-2 sequences from the GISAID database (as of January 15, 2025).
• Sample Size for Test Set: 16,156,883 sequences from the GISAID database.
• Data Provenance: Global (various countries submitting to GISAID). Retrospective, as it used existing sequence data.
• Ground Truth: Bioinformatic analysis of shared viral sequences.
• Results: >99.9% of sequences for SARS-CoV-2 had no changes in primer/probe binding sites at both target regions simultaneously. All sequences were predicted to be detected by at least one of the two targets. This addresses potential variations in the virus that might affect detection.

This document clearly outlines the analytical and clinical performance of the device, particularly focusing on its effectiveness in detecting SARS-CoV-2 in asymptomatic individuals against established comparator methods.

Ask a specific question about this device

The cobas liat CT/NG nucleic acid test is an automated, qualitative in vitro nucleic acid diagnostic test that utilizes realtime polymerase chain reaction (PCR) for the direction of Chlamydia (CT) and Neisseria gonorthoeae (NG) nucleic acid in male urine and vaginal swabs, all in cobas PCR Media (Roche Molecular Systems, Inc.).

This test is intended as an aid in the diagnosis of urogenital infections in both symptomatic individuals.

The test is performed on the cobas® liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the Cryptic plasmid and 23S rRNA of Chlamydia trachomatis and the pivNG and NGR9 of Neisseria gonorrhoeae. An Internal Control (IC) is also included. The IC is present to control for adequate processing of the target bacteria through steps of sample purification, nucleic acid amplification, and to monitor the presence of inhibitors in the PCR processes.

Here's a summary of the acceptance criteria and study details for the cobas® liat CT/NG nucleic acid test, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The document primarily provides performance metrics rather than explicitly stated acceptance criteria with numerical targets. However, based on the demonstrated performance and the context of a 510(k) submission, the implicit acceptance criteria would be high sensitivity and specificity, indicating reliable detection of CT and NG infections.

2. Sample Size and Data Provenance

• Clinical Study Test Set (Prospectively collected):
  • Total Evaluated Subjects: 4780 (2304 males, 2476 females)
  • Male Urine Specimens: 2302 (from 2302 male subjects)
  • Vaginal Swabs: 2476 (1240 clinician-collected, 1236 self-collected from 2476 female subjects)
  • Data Provenance: Multi-site, prospective study collected at 13 geographically diverse clinical sites across the US.
• Clinical Study Test Set (Archived Specimens - Supplementation):
  • Archived Male Urine Specimens: 163
  • Archived Vaginal Swabs: 90
  • Data Provenance: Prospectively collected samples from a prior clinical trial (K173887).
• Reproducibility Study Test Set: Total 1618 tests (811 vaginal, 807 urine) across 3 external sites. Each panel member tested in triplicate. Low positive (1-2x LoD), moderate positive (3-5x LoD), and negative panel members used.
• Supplemental Precision Study (for CT in urine): 810 evaluable tests on urine panel members (negative, 1x-2x LoD, 3x-5x LoD).

3. Number of Experts and Qualifications for Ground Truth

The ground truth for the clinical study was established using a Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA), which relied on a combination of three FDA-cleared NAATs (NAAT1, NAAT2, and NAAT3). The document does not specify the number of human experts used to establish the ground truth or their qualifications for the clinical study. The "ground truth" was algorithmically derived from the results of the comparator NAATs.

4. Adjudication Method for the Test Set

The adjudication method for the clinical study ground truth (PIS/CCA) followed a rule-based algorithm:

• If NAAT1 and NAAT2 were concordant, that result was the final PIS/CCA.
• If NAAT1 and NAAT2 were discordant, NAAT3 was performed as the tiebreaker.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, an MRMC comparative effectiveness study was not done. This study assesses the performance of a diagnostic test (the cobas® liat CT/NG nucleic acid test), which is an automated, qualitative in vitro nucleic acid diagnostic test. It replaced human assessment with an automated process, and the comparison was against a PIS/CCA derived from other reference NAATs, not human readers with and without AI assistance. Therefore, there is no effect size for human readers improving with AI.

6. Standalone (Algorithm Only) Performance

• Yes, a standalone (algorithm only) performance study was done. The entire clinical performance evaluation, reproducibility studies, and analytical studies assess the performance of the cobas® liat CT/NG nucleic acid test itself, which is an automated device performing real-time PCR. It is designed to operate without human intervention beyond sample loading and results interpretation from the automated output.

7. Type of Ground Truth Used

• Clinical Study: Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA) derived from the concordant results of FDA-cleared Nucleic Acid Amplification Tests (NAATs).
• Analytical Studies (LoD, Inclusivity, Specificity, Interference): Known concentrations of specific strains or culture subtypes of bacteria/viruses, spiked into negative clinical specimens.

8. Sample Size for the Training Set

The document does not explicitly describe a separate "training set" for an AI/ML model for the cobas® liat CT/NG nucleic acid test. As a nucleic acid diagnostic test (real-time PCR), it operates based on established biochemical principles and does not typically involve machine learning training in the same way an imaging AI algorithm would. All the data presented is for validation and performance evaluation.

9. How Ground Truth for the Training Set Was Established

Since there is no explicitly mentioned "training set" for an AI/ML model in this context, the method for establishing ground truth for such a set is not applicable or described. The clinical performance is evaluated against a PIS/CCA derived from other NAATs, and analytical performance is against known concentrations.

Ask a specific question about this device

The cobas® liat CT/NG/MG nucleic acid test is an automated, qualitative in vitro nucleic acid diagnostic test that utilizes real-time polymerase chain reaction (PCR) for the direct detection of Chlamydia trachomatis (CT), Neisseria gonorrhoeae (NG), and Mycoplasma genitalium (MG) nucleic acid in male urine and vaginal swabs, all in cobas® PCR Media (Roche Molecular Systems, Inc.).

This test is intended as an aid in the diagnosis of urogenital infections in both symptomatic and asymptomatic individuals.

The test is performed on the cobas® liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the Cryptic plasmid and 23S rRNA of Chlamydia trachomatis, the pivNG and NGR9 of Neisseria gonorrhoeae, and the 23S rRNA and mgpC of Mycoplasma genitalium. An Internal Control (IC) is also included. The IC is present to control for adequate processing of the target bacteria through steps of sample purification, nucleic acid amplification, and to monitor the presence of inhibitors in the PCR processes.

The provided document describes the cobas® liat CT/NG/MG nucleic acid test, an automated in vitro diagnostic test for the direct detection of Chlamydia trachomatis (CT), Neisseria gonorrhoeae (NG), and Mycoplasma genitalium (MG) nucleic acid.

Here's the breakdown of the acceptance criteria and the study proving the device meets them:

1. A table of acceptance criteria and the reported device performance:

The document doesn't explicitly state numerical "acceptance criteria" but rather presents the sensitivity/PPA and specificity/NPA as "performance results." Assuming the performance values achieved in the clinical study are the de facto acceptance criteria for market clearance, the table is compiled from the "Clinical Performance Evaluation" section (Tables 20, 21, and 22).

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Sample Size for Clinical Study (Test Set):
  • Total Subjects: 4852 subjects (2512 females, 2340 males) were enrolled.
  • Evaluable Subjects: 4780 evaluable subjects (2304 males, 2476 females).
  • Specimens:
    • 2302 male urine specimens.
    • 1240 clinician-collected vaginal swabs (females).
    • 1236 self-collected vaginal swabs (females).
  • Archived Specimens: Supplementation included archived specimens from a prior clinical trial (K173887) due to low NG prevalence in prospectively collected male urine and vaginal swabs. The exact breakdown of archived vs. prospective in the final evaluable numbers is not explicitly separated for all analytes, but separate tables are provided for "Archived Male Urine" and "Archived Vaginal Swabs" for NG (which states 163 archived male urine and 90 archived vaginal swabs were used for NG).
• Data Provenance:
  • Country of Origin: United States (13 geographically diverse intended use clinical sites across the US).
  • Study Design: Multi-site, prospective study, with supplementation from prospectively collected archived specimens for certain analytes.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

The ground truth was established using a Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA) derived from a combination of three FDA-cleared NAATs (NAAT1, NAAT2, and NAAT3).

• Number of Experts: Not applicable, as the ground truth was established by algorithmic comparison of results from FDA-cleared NAATs, not by human expert opinion or adjudication.
• Qualifications of Experts: Not applicable.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The adjudication method used was a "2+1" algorithm based on FDA-cleared NAATs:

• If NAAT1 and NAAT2 were concordant, that result was taken as the PIS/CCA.
• If NAAT1 and NAAT2 were discordant, NAAT3 was performed as a tiebreaker.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• MRMC Study: No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done.
• Effect Size of Human Readers with/without AI: Not applicable, as this is an automated diagnostic test that detects nucleic acids, not an AI-assisted interpretation device for human readers.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Standalone Performance: Yes, the clinical performance evaluation (Section 6) assesses the standalone performance of the cobas® liat CT/NG/MG nucleic acid test. The device is described as an "automated, qualitative in vitro nucleic acid diagnostic test," indicating it operates without human "interpretation" of the final result. The study compared the device's output directly against the PIS/CCA ground truth.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

The ground truth used was a Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA) result. This PIS/CCA was derived from the results of three FDA-cleared Nucleic Acid Amplification Tests (NAATs). This is a form of reference standard derived from multiple laboratory tests.

8. The sample size for the training set

The document does not provide details about a "training set" for the algorithm. This is typical for PCR-based diagnostic devices, which rely on established molecular biology principles and analytical validation rather than machine learning on large training datasets for their core functionality. The performance data presented are for clinical validation against a reference standard.

9. How the ground truth for the training set was established

Not applicable, as no explicit training set for an algorithm is described. The device's underlying technology (real-time PCR) is not typically "trained" in the machine learning sense. Analytical studies (Limit of Detection, Inclusivity, Specificity, Interference) form the basis of validating the reagent and assay design.

Ask a specific question about this device

The cobas® SARS-CoV-2 Nucleic acid test for use on the cobas® SARS-CoV-2) is an automated, realtime reverse transcriptase polymerase chain reaction (RT-PCR) test intended for the rapid in vitro qualitative detection of nucleic acid from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in anterior nasal (nasal) and nasopharyngeal swab specimens collected from individuals with signs and symptoms of respiratory tract infection (i.e., symptomatic). Additionally, this test is intended to be used with nasal and necimens collected from individuals without signs and symptoms suspected of COVID-19 (i.e., asymptomatic).

The cobas® SARS-CoV-2 performed on the cobas® Liat® System is intended for use as an aid in the diagnosis of COVID-19 if used in conjunction with other clinical, epidemiologic, and laboratory findings. SARS-CoV-2 RNA is generally detectable in nasal and nasopharyngeal swab specimens during the acute phase of infection.

Positive results are indicative of the presence of SARS-CoV-2 RNA. Positive results do not rule out co-infection with other microorganisms.

A negative result from an asymptomatic individual is presumptive. Additionally, a negative result obtained with a nasal swab collected from an asymptomatic patient should be followed up by testing at least twice over three days with at least 48 hours between tests.

Negative results do not preclude SARS-CoV-2 infection.

The results of this test should not be used as the sole basis for diagnosis, treatment management decisions.

This test is intended for prescription use only and can be used in Point-of-Care settings.

The cobas® SARS-CoV-2 Nucleic acid test for use on the cobas® Liat® System (cobas® SARS-CoV-2) uses real-time reverse transcriptase polymerase chain reaction (RT-PCR) technology to rapidly (approximately 20 minutes) detect SARS-CoV-2 virus from nasopharyngeal and nasal swabs. The automation, small footprint, and easy-to-use interface of the cobas® Liat® System enable performance of this test to occur at the POC or in a clinical laboratory setting.

The cobas® SARS-CoV-2 assay is performed on the cobas® Liat® Analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time RT-PCR assays. The assay targets both the ORF1 a/b nonstructural region and nucleocapsid protein gene that are unique to SARS-CoV-2. An Internal Process Control (IPC) is also included. The IPC is present to control for adequate processing of the target virus through steps of sample purification, nucleic acid amplification, and to monitor the presence of inhibitors in the RT-PCR processes.

Here's an analysis of the acceptance criteria and study findings for the "cobas SARS-CoV-2 Nucleic acid test for use on the cobas Liat System" based on the provided text:

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implied by the clinical performance evaluation results, specifically the Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) with a composite comparator method. While explicit acceptance criteria values are not stated (e.g., "PPA must be >X%"), the device's performance is presented to demonstrate its effectiveness.

Study Information

1. Sample sizes used for the test set and data provenance:

   • Clinical Performance Evaluation (Test Set):
     • Evaluable NPS samples: 1874 (fresh and frozen)
       • Symptomatic: 673 NPS
       • Asymptomatic: 1201 NPS (413 suspected, 788 no suspicion)
     • Evaluable NS samples: 1872 (fresh and frozen, including 1 indeterminate result)
       • Symptomatic: 674 NS
       • Asymptomatic: 1198 NS (411 suspected, 787 no suspicion)
     • Frozen Samples (included in above totals): 23 positive and 23 negative NPS specimens, and 23 positive and 23 negative NS specimens.
     • Data Provenance: Prospectively collected clinical specimens from February-June 2022 (fresh samples) and earlier during the COVID-19 pandemic (March-June 2021) for frozen samples. The document does not specify the country of origin, but it implies collection within "10 point-of-care healthcare facilities" without further geographic detail.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The ground truth for the test set was established using a composite comparator method consisting of "three highly sensitive FDA-EUA-authorized laboratory-based RT-PCR assays."
   • The document does not specify the number or qualifications of human experts involved in establishing this ground truth. The ground truth relies on the performance of these reference RT-PCR assays.

3. Adjudication method for the test set:

   • The text describes a "composite comparator method" using "three highly sensitive FDA-EUA-authorized laboratory-based RT-PCR assays." This implies a form of consensus or a predefined algorithm for combining the results of these three assays to establish the definitive positive or negative status. However, the specific adjudication rules (e.g., "2 out of 3 positive for overall positive") are not explicitly described.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC comparative effectiveness study involving human readers or AI assistance is described. This study focuses on the standalone performance of the cobas SARS-CoV-2 test.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, a standalone performance study was done. The entire clinical performance evaluation section (Section 7) describes the performance of the "cobas SARS-CoV-2 test" (the device/algorithm) against a composite comparator method (ground truth) without human intervention in the result interpretation of the device. The device itself is an automated RT-PCR test.

6. The type of ground truth used:

   • The ground truth used was a composite comparator method based on the results from "three highly sensitive FDA-EUA-authorized laboratory-based RT-PCR assays." This is a form of laboratory reference standard.

7. The sample size for the training set:

   • The document describes a clinical performance evaluation (test set) and analytical studies. It does not provide information about the sample size of a specific "training set" for the device, as it is a molecular diagnostic test rather than a machine learning algorithm that typically undergoes explicit training on large datasets in the way an AI image analysis product would. The development (implied "training" in a broader sense for assay optimization) would have used various samples, but these are not explicitly detailed as a distinct training set in the provided text.

8. How the ground truth for the training set was established:

   • As no explicit "training set" with established ground truth is described in the context of typical AI/ML development, this question cannot be answered from the provided text. The assay development would involve extensive analytical validation using characterized viral samples and clinical specimens validated by reference methods, which broadly contribute to optimizing the assay's performance.

Ask a specific question about this device

The cobas® SARS-CoV-2 & Influenza A/B Nucleic acid test for use on the cobas® Liat® System (cobas® SARS-CoV-2 & Influenza A/B) is an automated rapid multiplex real-time, reverse transcriptase polymerase chain reaction (RT-PCR) test intended for the simultaneous qualitative detection and differentiation of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), influenza A, and influenza B virus nucleic acid in nasopharyngeal swab (NPS) and anterior nasal swab (ANS) specimens from individuals with signs and symptoms of respiratory tract infection. Clinical signs and symptoms of respiratory tract infection due to SARS-CoV-2 and influenza can be similar.

cobas® SARS-CoV-2 & Influenza A/B is intended for use as an aid in the differential diagnosis of SARS-CoV-2, influenza A, and/or influenza B infection if used in conjunction with other clinical and epidemiological information, and laboratory findings. SARS-CoV-2, influenza A and influenza B viral nucleic acid are generally detectable in NPS and ANS specimens during the acute phase of infection.

Positive results do not rule out co-infection with other organisms. The agent(s) detected by the cobas SARS-CoV-2 & Influenza A/B may not be the definite cause of disease.

Negative results do not preclude SARS-COV-2, influenza A, and/or influenza B infection. The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

cobas® SARS-CoV-2 & Influenza A/B assay uses real-time reverse transcriptase polymerase chain reaction (RT-PCR) technology to rapidly (approximately 20 minutes) detect and differentiate between SARS-CoV-2, influenza A, and influenza B viruses from nasopharyngeal and nasal swabs. The automation, small footprint, and easy-to-use interface of the cobas® Liat® System enable performance of this test to occur at the POC or in a clinical laboratory setting.

The provided text describes the acceptance criteria and the study that proves the device meets those criteria for the "cobas SARS-CoV-2 & Influenza A/B Nucleic acid test for use on the cobas Liat System". This is a diagnostic test, not an AI/ML device, so many of the requested elements (like "number of experts used to establish ground truth" or "multi reader multi case comparative effectiveness study") are not applicable or described in the same way they would be for an AI/ML product. However, I will extract and present the information as per the prompt's structure, noting where the information is N/A or conceptually different due to the nature of the device.

Device Name: cobas® SARS-CoV-2 & Influenza A/B Nucleic acid test for use on the cobas® Liat® System

Device Type: Automated rapid multiplex real-time, reverse transcriptase polymerase chain reaction (RT-PCR) test for qualitative detection and differentiation of SARS-CoV-2, influenza A, and influenza B virus nucleic acid.

1. Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state "acceptance criteria" as a separate, pre-defined table. Instead, the performance metrics, such as Limit of Detection (LoD), inclusivity, cross-reactivity, and clinical performance (PPA, NPA), serve as the de facto acceptance criteria. The reported device performance is presented against these metrics.

Implicit Acceptance Criteria and Reported Performance for Key Metrics:

Ask a specific question about this device