(357 days)
The cobas liat CT/NG nucleic acid test is an automated, qualitative in vitro nucleic acid diagnostic test that utilizes realtime polymerase chain reaction (PCR) for the direction of Chlamydia (CT) and Neisseria gonorthoeae (NG) nucleic acid in male urine and vaginal swabs, all in cobas PCR Media (Roche Molecular Systems, Inc.).
This test is intended as an aid in the diagnosis of urogenital infections in both symptomatic individuals.
The test is performed on the cobas® liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the Cryptic plasmid and 23S rRNA of Chlamydia trachomatis and the pivNG and NGR9 of Neisseria gonorrhoeae. An Internal Control (IC) is also included. The IC is present to control for adequate processing of the target bacteria through steps of sample purification, nucleic acid amplification, and to monitor the presence of inhibitors in the PCR processes.
Here's a summary of the acceptance criteria and study details for the cobas® liat CT/NG nucleic acid test, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document primarily provides performance metrics rather than explicitly stated acceptance criteria with numerical targets. However, based on the demonstrated performance and the context of a 510(k) submission, the implicit acceptance criteria would be high sensitivity and specificity, indicating reliable detection of CT and NG infections.
| Metric (Implicit Acceptance Criteria) | Device Performance - CT (Male Urine) | Device Performance - CT (Vaginal Swabs) | Device Performance - NG (Overall Male Urine) | Device Performance - NG (Overall Vaginal Swabs) |
|---|---|---|---|---|
| Sensitivity / Positive Percent Agreement (PPA) | 97.3% (92.4%, 99.1%) | 98.2% (93.6%, 99.5%) | 100.0% (97.7%, 100.0%) | 97.7% (92.0%, 99.4%) |
| Specificity / Negative Percent Agreement (NPA) | 99.9% (99.7%, 100.0%) | 99.8% (99.5%, 99.9%) | 99.9% (99.6%, 100.0%) | 99.8% (99.6%, 99.9%) |
| Reproducibility (Low Positive - 1-2x LoD) | CT: 90.7% | CT: 100% | NG: 99.6% | NG: 100% |
| Reproducibility (Moderate Positive - 3-5x LoD) | CT: 96.3% | CT: 100% | NG: 100% | NG: 100% |
| Reproducibility (Negative) | CT: 100% | CT: 100% | NG: 100% | NG: 100% |
| Analytical Sensitivity (LoD) - CT Serovar D | Urine: 0.085 EB/mL | Vaginal Swab: 0.170 EB/mL | N/A | N/A |
| Analytical Sensitivity (LoD) - CT Serovar I | Urine: 0.784 EB/mL | Vaginal Swab: 0.784 EB/mL | N/A | N/A |
| Analytical Sensitivity (LoD) - NG Strain 2948 | Urine: 0.250 CFU/mL | Vaginal Swab: 0.500 CFU/mL | N/A | N/A |
| Analytical Sensitivity (LoD) - NG Strain 891 | Urine: 0.200 CFU/mL | Vaginal Swab: 0.200 CFU/mL | N/A | N/A |
| Invalid Rate (Initial Test) | 0.6% | 0.6% | 0.6% | 0.6% |
| Invalid Rate (After Retesting) | 0.2% | 0.2% | 0.2% | 0.2% |
2. Sample Size and Data Provenance
- Clinical Study Test Set (Prospectively collected):
- Total Evaluated Subjects: 4780 (2304 males, 2476 females)
- Male Urine Specimens: 2302 (from 2302 male subjects)
- Vaginal Swabs: 2476 (1240 clinician-collected, 1236 self-collected from 2476 female subjects)
- Data Provenance: Multi-site, prospective study collected at 13 geographically diverse clinical sites across the US.
- Clinical Study Test Set (Archived Specimens - Supplementation):
- Archived Male Urine Specimens: 163
- Archived Vaginal Swabs: 90
- Data Provenance: Prospectively collected samples from a prior clinical trial (K173887).
- Reproducibility Study Test Set: Total 1618 tests (811 vaginal, 807 urine) across 3 external sites. Each panel member tested in triplicate. Low positive (1-2x LoD), moderate positive (3-5x LoD), and negative panel members used.
- Supplemental Precision Study (for CT in urine): 810 evaluable tests on urine panel members (negative, 1x-2x LoD, 3x-5x LoD).
3. Number of Experts and Qualifications for Ground Truth
The ground truth for the clinical study was established using a Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA), which relied on a combination of three FDA-cleared NAATs (NAAT1, NAAT2, and NAAT3). The document does not specify the number of human experts used to establish the ground truth or their qualifications for the clinical study. The "ground truth" was algorithmically derived from the results of the comparator NAATs.
4. Adjudication Method for the Test Set
The adjudication method for the clinical study ground truth (PIS/CCA) followed a rule-based algorithm:
- If NAAT1 and NAAT2 were concordant, that result was the final PIS/CCA.
- If NAAT1 and NAAT2 were discordant, NAAT3 was performed as the tiebreaker.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, an MRMC comparative effectiveness study was not done. This study assesses the performance of a diagnostic test (the cobas® liat CT/NG nucleic acid test), which is an automated, qualitative in vitro nucleic acid diagnostic test. It replaced human assessment with an automated process, and the comparison was against a PIS/CCA derived from other reference NAATs, not human readers with and without AI assistance. Therefore, there is no effect size for human readers improving with AI.
6. Standalone (Algorithm Only) Performance
- Yes, a standalone (algorithm only) performance study was done. The entire clinical performance evaluation, reproducibility studies, and analytical studies assess the performance of the cobas® liat CT/NG nucleic acid test itself, which is an automated device performing real-time PCR. It is designed to operate without human intervention beyond sample loading and results interpretation from the automated output.
7. Type of Ground Truth Used
- Clinical Study: Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA) derived from the concordant results of FDA-cleared Nucleic Acid Amplification Tests (NAATs).
- Analytical Studies (LoD, Inclusivity, Specificity, Interference): Known concentrations of specific strains or culture subtypes of bacteria/viruses, spiked into negative clinical specimens.
8. Sample Size for the Training Set
The document does not explicitly describe a separate "training set" for an AI/ML model for the cobas® liat CT/NG nucleic acid test. As a nucleic acid diagnostic test (real-time PCR), it operates based on established biochemical principles and does not typically involve machine learning training in the same way an imaging AI algorithm would. All the data presented is for validation and performance evaluation.
9. How Ground Truth for the Training Set Was Established
Since there is no explicitly mentioned "training set" for an AI/ML model in this context, the method for establishing ground truth for such a set is not applicable or described. The clinical performance is evaluated against a PIS/CCA derived from other NAATs, and analytical performance is against known concentrations.
§ 866.3393 Device to detect nucleic acids from non-viral microorganism(s) causing sexually transmitted infections and associated resistance marker(s).
(a)
Identification. A device to detect nucleic acids from non-viral microorganism(s) causing sexually transmitted infections and associated resistance marker(s) is an in vitro diagnostic device intended for the detection and identification of nucleic acids from non-viral microorganism(s) and their associated resistance markers in clinical specimens collected from patients suspected of sexually transmitted infections. The device is intended to aid in the diagnosis of non-viral sexually transmitted infections in conjunction with other clinical and laboratory data. These devices do not provide confirmation of antibiotic susceptibility since mechanisms of resistance may exist that are not detected by the device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use for the labeling required under § 809.10 of this chapter must include a detailed description of targets the device detects, the results provided to the user, the clinical indications appropriate for test use, and the specific population(s) for which the device is intended.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt (standalone or as part of a test system) for the collection of specimen types claimed by this device; alternatively, the sample collection device must be cleared in a premarket submission as a part of this device.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including reagents, instruments, ancillary materials, all control elements, and a detailed explanation of the methodology, including all pre-analytical methods for processing of specimens;
(ii) Detailed discussion of the performance characteristics of the device for all claimed specimen types based on analytical studies, including Limit of Detection, inclusivity, cross-reactivity, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, within lab precision, and reproducibility, as appropriate;
(iii) Detailed descriptions of the test procedure, the interpretation of test results for clinical specimens, and acceptance criteria for any quality control testing;
(iv) Limiting statements indicating that:
(A) A negative test result does not preclude the possibility of infection;
(B) The test results should be interpreted in conjunction with other clinical and laboratory data available to the clinician;
(C) Reliable results are dependent on adequate specimen collection, transport, storage, and processing. Failure to observe proper procedures in any one of these steps can lead to incorrect results; and
(D) If appropriate (
e.g., recommended by the Centers for Disease Control and Prevention, by current well-accepted clinical guidelines, or by published peer reviewed research), that the clinical performance is inferior in a specific clinical subpopulation or for a specific claimed specimen type; and(v) If the device is intended to detect antimicrobial resistance markers, limiting statements, as appropriate, indicating that:
(A) Negative results for claimed resistance markers do not indicate susceptibility of detected microorganisms, as resistance markers not measured by the assay or other potential mechanisms of antibiotic resistance may be present;
(B) Detection of resistance markers cannot be definitively linked to specific microorganisms and the source of a detected resistance marker may be an organism not detected by the assay, including colonizing flora;
(C) Detection of antibiotic resistance markers may not correlate with phenotypic gene expression; and
(D) Therapeutic failure or success cannot be determined based on the assay results, since nucleic acid may persist following appropriate antimicrobial therapy.
(4) Design verification and validation must include:
(i) Detailed device description documentation, including methodology from obtaining sample to result, design of primer/probe sequences, rationale for target sequence selection, and computational path from collected raw data to reported result (
e.g., how collected raw signals are converted into a reported result).(ii) Detailed documentation of analytical studies, including, Limit of Detection, inclusivity, cross-reactivity, microbial interference, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, within lab precision, and reproducibility, as appropriate.
(iii) Detailed documentation and performance results from a clinical study that includes prospective (sequential) samples for each claimed specimen type and, when determined to be appropriate by FDA, additional characterized clinical samples. The study must be performed on a study population consistent with the intended use population and compare the device performance to results obtained from FDA accepted comparator methods. Documentation from the clinical studies must include the clinical study protocol (including a predefined statistical analysis plan) study report, testing results, and results of all statistical analyses.
(iv) A detailed description of the impact of any software, including software applications and hardware-based devices that incorporate software, on the device's functions.