(357 days)
The cobas® liat CT/NG/MG nucleic acid test is an automated, qualitative in vitro nucleic acid diagnostic test that utilizes real-time polymerase chain reaction (PCR) for the direct detection of Chlamydia trachomatis (CT), Neisseria gonorrhoeae (NG), and Mycoplasma genitalium (MG) nucleic acid in male urine and vaginal swabs, all in cobas® PCR Media (Roche Molecular Systems, Inc.).
This test is intended as an aid in the diagnosis of urogenital infections in both symptomatic and asymptomatic individuals.
The test is performed on the cobas® liat analyzer which automates and integrates sample purification, nucleic acid amplification, and detection of the target sequence in biological samples using real-time PCR assays. The assay targets both the Cryptic plasmid and 23S rRNA of Chlamydia trachomatis, the pivNG and NGR9 of Neisseria gonorrhoeae, and the 23S rRNA and mgpC of Mycoplasma genitalium. An Internal Control (IC) is also included. The IC is present to control for adequate processing of the target bacteria through steps of sample purification, nucleic acid amplification, and to monitor the presence of inhibitors in the PCR processes.
The provided document describes the cobas® liat CT/NG/MG nucleic acid test, an automated in vitro diagnostic test for the direct detection of Chlamydia trachomatis (CT), Neisseria gonorrhoeae (NG), and Mycoplasma genitalium (MG) nucleic acid.
Here's the breakdown of the acceptance criteria and the study proving the device meets them:
1. A table of acceptance criteria and the reported device performance:
The document doesn't explicitly state numerical "acceptance criteria" but rather presents the sensitivity/PPA and specificity/NPA as "performance results." Assuming the performance values achieved in the clinical study are the de facto acceptance criteria for market clearance, the table is compiled from the "Clinical Performance Evaluation" section (Tables 20, 21, and 22).
| Specimen Type | Target | Performance Metric | Reported Device Performance (95% CI) |
|---|---|---|---|
| Male Urine (Total) | CT | Sensitivity/PPA | 97.3% (92.4%, 99.1%) |
| CT | Specificity/NPA | 99.9% (99.7%, 100.0%) | |
| Male Urine (Total) | NG | Sensitivity/PPA | 100.0% (95.4%, 100.0%) |
| NG | Specificity/NPA | 99.9% (99.6%, 100.0%) | |
| Male Urine (Total, including archived) | NG | Sensitivity/PPA | 100.0% (97.7%, 100.0%) |
| NG | Specificity/NPA | 99.9% (99.6%, 100.0%) | |
| Male Urine (Total) | MG | Sensitivity/PPA | 97.1% (93.9%, 98.7%) |
| MG | Specificity/NPA | 99.2% (98.8%, 99.5%) | |
| Vaginal Swabs (Total) | CT | Sensitivity/PPA | 98.2% (93.6%, 99.5%) |
| CT | Specificity/NPA | 99.8% (99.5%, 99.9%) | |
| Vaginal Swabs (Total) | NG | Sensitivity/PPA | 95.2% (84.2%, 98.7%) |
| NG | Specificity/NPA | 99.8% (99.6%, 99.9%) | |
| Vaginal Swabs (Total, including archived) | NG | Sensitivity/PPA | 97.7% (92.0%, 99.4%) |
| NG | Specificity/NPA | 99.8% (99.6%, 99.9%) | |
| Vaginal Swabs (Total) | MG | Sensitivity/PPA | 95.2% (91.9%, 97.3%) |
| MG | Specificity/NPA | 97.8% (97.1%, 98.3%) |
2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Sample Size for Clinical Study (Test Set):
- Total Subjects: 4852 subjects (2512 females, 2340 males) were enrolled.
- Evaluable Subjects: 4780 evaluable subjects (2304 males, 2476 females).
- Specimens:
- 2302 male urine specimens.
- 1240 clinician-collected vaginal swabs (females).
- 1236 self-collected vaginal swabs (females).
- Archived Specimens: Supplementation included archived specimens from a prior clinical trial (K173887) due to low NG prevalence in prospectively collected male urine and vaginal swabs. The exact breakdown of archived vs. prospective in the final evaluable numbers is not explicitly separated for all analytes, but separate tables are provided for "Archived Male Urine" and "Archived Vaginal Swabs" for NG (which states 163 archived male urine and 90 archived vaginal swabs were used for NG).
- Data Provenance:
- Country of Origin: United States (13 geographically diverse intended use clinical sites across the US).
- Study Design: Multi-site, prospective study, with supplementation from prospectively collected archived specimens for certain analytes.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
The ground truth was established using a Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA) derived from a combination of three FDA-cleared NAATs (NAAT1, NAAT2, and NAAT3).
- Number of Experts: Not applicable, as the ground truth was established by algorithmic comparison of results from FDA-cleared NAATs, not by human expert opinion or adjudication.
- Qualifications of Experts: Not applicable.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The adjudication method used was a "2+1" algorithm based on FDA-cleared NAATs:
- If NAAT1 and NAAT2 were concordant, that result was taken as the PIS/CCA.
- If NAAT1 and NAAT2 were discordant, NAAT3 was performed as a tiebreaker.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- MRMC Study: No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done.
- Effect Size of Human Readers with/without AI: Not applicable, as this is an automated diagnostic test that detects nucleic acids, not an AI-assisted interpretation device for human readers.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Standalone Performance: Yes, the clinical performance evaluation (Section 6) assesses the standalone performance of the cobas® liat CT/NG/MG nucleic acid test. The device is described as an "automated, qualitative in vitro nucleic acid diagnostic test," indicating it operates without human "interpretation" of the final result. The study compared the device's output directly against the PIS/CCA ground truth.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
The ground truth used was a Patient Infected Status (PIS) or Composite Comparator Algorithm (CCA) result. This PIS/CCA was derived from the results of three FDA-cleared Nucleic Acid Amplification Tests (NAATs). This is a form of reference standard derived from multiple laboratory tests.
8. The sample size for the training set
The document does not provide details about a "training set" for the algorithm. This is typical for PCR-based diagnostic devices, which rely on established molecular biology principles and analytical validation rather than machine learning on large training datasets for their core functionality. The performance data presented are for clinical validation against a reference standard.
9. How the ground truth for the training set was established
Not applicable, as no explicit training set for an algorithm is described. The device's underlying technology (real-time PCR) is not typically "trained" in the machine learning sense. Analytical studies (Limit of Detection, Inclusivity, Specificity, Interference) form the basis of validating the reagent and assay design.
§ 866.3393 Device to detect nucleic acids from non-viral microorganism(s) causing sexually transmitted infections and associated resistance marker(s).
(a)
Identification. A device to detect nucleic acids from non-viral microorganism(s) causing sexually transmitted infections and associated resistance marker(s) is an in vitro diagnostic device intended for the detection and identification of nucleic acids from non-viral microorganism(s) and their associated resistance markers in clinical specimens collected from patients suspected of sexually transmitted infections. The device is intended to aid in the diagnosis of non-viral sexually transmitted infections in conjunction with other clinical and laboratory data. These devices do not provide confirmation of antibiotic susceptibility since mechanisms of resistance may exist that are not detected by the device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use for the labeling required under § 809.10 of this chapter must include a detailed description of targets the device detects, the results provided to the user, the clinical indications appropriate for test use, and the specific population(s) for which the device is intended.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt (standalone or as part of a test system) for the collection of specimen types claimed by this device; alternatively, the sample collection device must be cleared in a premarket submission as a part of this device.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including reagents, instruments, ancillary materials, all control elements, and a detailed explanation of the methodology, including all pre-analytical methods for processing of specimens;
(ii) Detailed discussion of the performance characteristics of the device for all claimed specimen types based on analytical studies, including Limit of Detection, inclusivity, cross-reactivity, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, within lab precision, and reproducibility, as appropriate;
(iii) Detailed descriptions of the test procedure, the interpretation of test results for clinical specimens, and acceptance criteria for any quality control testing;
(iv) Limiting statements indicating that:
(A) A negative test result does not preclude the possibility of infection;
(B) The test results should be interpreted in conjunction with other clinical and laboratory data available to the clinician;
(C) Reliable results are dependent on adequate specimen collection, transport, storage, and processing. Failure to observe proper procedures in any one of these steps can lead to incorrect results; and
(D) If appropriate (
e.g., recommended by the Centers for Disease Control and Prevention, by current well-accepted clinical guidelines, or by published peer reviewed research), that the clinical performance is inferior in a specific clinical subpopulation or for a specific claimed specimen type; and(v) If the device is intended to detect antimicrobial resistance markers, limiting statements, as appropriate, indicating that:
(A) Negative results for claimed resistance markers do not indicate susceptibility of detected microorganisms, as resistance markers not measured by the assay or other potential mechanisms of antibiotic resistance may be present;
(B) Detection of resistance markers cannot be definitively linked to specific microorganisms and the source of a detected resistance marker may be an organism not detected by the assay, including colonizing flora;
(C) Detection of antibiotic resistance markers may not correlate with phenotypic gene expression; and
(D) Therapeutic failure or success cannot be determined based on the assay results, since nucleic acid may persist following appropriate antimicrobial therapy.
(4) Design verification and validation must include:
(i) Detailed device description documentation, including methodology from obtaining sample to result, design of primer/probe sequences, rationale for target sequence selection, and computational path from collected raw data to reported result (
e.g., how collected raw signals are converted into a reported result).(ii) Detailed documentation of analytical studies, including, Limit of Detection, inclusivity, cross-reactivity, microbial interference, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, within lab precision, and reproducibility, as appropriate.
(iii) Detailed documentation and performance results from a clinical study that includes prospective (sequential) samples for each claimed specimen type and, when determined to be appropriate by FDA, additional characterized clinical samples. The study must be performed on a study population consistent with the intended use population and compare the device performance to results obtained from FDA accepted comparator methods. Documentation from the clinical studies must include the clinical study protocol (including a predefined statistical analysis plan) study report, testing results, and results of all statistical analyses.
(iv) A detailed description of the impact of any software, including software applications and hardware-based devices that incorporate software, on the device's functions.