(325 days)
The NeuMoDx CT/NG Assay 2.0, as implemented on the NeuMoDx 96 Molecular System and NeuMoDx 288 Molecular System, is an automated, qualitative test for the direct detection of Chlamydia trachomatis (CT) and or Neisseria gonorrhoeae (NG) DNA as an aid in the diagnosis of chlamydial and gonococcal urogenital disease in symptomatic and asymptomatic individuals. The Assay utilizes real-time Polymerase Chain Reaction (PCR) and may be used to test male and female urine, and self-collected vaginal swab specimens (collected in a clinical setting).
The NeuMoDx CT/NG Assay 2.0 is an automated in vitro diagnostic test for the direct detection of Chlamydia trachomatis and Neisseria gonorrhoeae (CT/NG) DNA from asymptomatic and symptomatic patient specimens. The assay utilizes real-time polymerase chain reaction (PCR) for the amplification of CT and/or NG DNA and fluorogenic targetspecific TaqMan probes for the detection of the amplified DNA. At the end of the test, a determination of the presence/absence of CT and/or NG DNA in the specimen is automatically made based on the amplification status of the CT and/or NG DNA and/or Sample Process Control sequences using pre-established decision criteria. The NeuMoDx CT/NG Assay 2.0 is intended as an aid to diagnose CT and NG infections in symptomatic or asymptomatic individuals, but not to guide or monitor treatment for CT and NG infections. Concomitant cultures may be necessary to recover organisms for epidemiological typing or for further susceptibility testing.
The NeuMoDx CT/NG Assay 2.0 is an automated, qualitative test for the direct detection of Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (NG) DNA, designed to aid in the diagnosis of chlamydial and gonococcal urogenital disease in symptomatic and asymptomatic individuals.
Here's an analysis of its acceptance criteria and the study proving its performance:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for sensitivity and specificity are not explicitly stated as distinct acceptance criteria in the provided text. However, the FDA review process implies that the observed performance must be deemed acceptable. We will present the observed clinical performance as the reported device performance.
Chlamydia trachomatis (CT) Performance
| Specimen Type | Symptom Status | Prevalence | Sensitivity (95% CI) | Specificity (95% CI) |
|---|---|---|---|---|
| Male Urine (MU) | Asymptomatic | 8.5% | 98.1% (93.3%, 99.5%) | 100.0% (99.7%, 100.0%) |
| Male Urine (MU) | Symptomatic | 18.5% | 95.3% (88.6%, 98.2%) | 99.7% (98.5%, 100.0%) |
| Male Urine (MU) | All | 11.2% | 96.8% (93.3%, 98.5%) | 99.9% (99.6%, 100.0%) |
| Female Urine (FU) | Asymptomatic | 4.1% | 93.0% (81.4%, 97.6%) | 99.9% (99.4%, 100.0%) |
| Female Urine (FU) | Symptomatic | 6.4% | 91.8% (82.2%, 96.4%) | 99.7% (99.0%, 99.9%) |
| Female Urine (FU) | All | 5.2% | 92.3% (85.6%, 96.1%) | 99.8% (99.5%, 99.9%) |
| Self-Collected Vaginal Swab (SCVS) | Asymptomatic | 4.1% | 100.0% (91.8%, 100.0%) | 99.8% (99.3%, 99.9%) |
| Self-Collected Vaginal Swab (SCVS) | Symptomatic | 6.3% | 95.1% (86.5%, 98.3%) | 99.2% (98.4%, 99.6%) |
| Self-Collected Vaginal Swab (SCVS) | All | 5.2% | 97.1% (91.9%, 99.0%) | 99.5% (99.1%, 99.8%) |
Neisseria gonorrhoeae (NG) Performance
| Specimen Type | Symptom Status | Prevalence | Sensitivity (95% CI) | Specificity (95% CI) |
|---|---|---|---|---|
| Male Urine (MU) | Asymptomatic | 0.9% | 100.0% (74.1%, 100.0%) | 99.9% (99.5%, 100.0%) |
| Male Urine (MU) | Symptomatic | 17.6% | 98.8% (93.4%, 99.8%) | 99.7% (98.5%, 100.0%) |
| Male Urine (MU) | All | 5.5% | 98.9% (94.2%, 99.8%) | 99.9% (99.5%, 100.0%) |
| Female Urine (FU) | Asymptomatic | 2.3% | 91.7% (74.2%, 97.7%) | 100.0% (99.6%, 100.0%) |
| Female Urine (FU) | Symptomatic | 2.2% | 95.2% (77.3%, 99.2%) | 99.9% (99.4%, 100.0%) |
| Female Urine (FU) | All | 2.2% | 93.3% (82.1%, 97.7%) | 99.9% (99.7%, 100.0%) |
| Self-Collected Vaginal Swab (SCVS) | Asymptomatic | 2.3% | 100.0% (86.2%, 100.0%) | 100.0% (99.6%, 100.0%) |
| Self-Collected Vaginal Swab (SCVS) | Symptomatic | 2.2% | 95.2% (77.3%, 99.2%) | 99.8% (99.2%, 99.9%) |
| Self-Collected Vaginal Swab (SCVS) | All | 2.2% | 97.8% (88.4%, 99.6%) | 99.9% (99.6%, 100.0%) |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size:
- Male Urine (CT): 1691
- Male Urine (NG): 1698
- Female Urine (CT): 2007
- Female Urine (NG): 2006
- Self-Collected Vaginal Swabs (SCVS) (CT): 2016
- Self-Collected Vaginal Swabs (SCVS) (NG): 2016
- Data Provenance: The study was a "multicenter, pivotal, prospective urogenital specimen collection study" conducted at "14 geographically and demographically diverse U.S. sites."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The ground truth was established by a "patient infected status (PIS) algorithm" based on results from two FDA-cleared, legally marketed Nucleic Acid Amplification Tests (NAATs). The text does not mention the involvement of human experts (e.g., radiologists, clinicians) for establishing the ground truth directly. It relies on the performance of existing, cleared diagnostic devices.
4. Adjudication Method for the Test Set
The adjudication method was a pre-specified patient infected status (PIS) algorithm rather than human expert adjudication:
- Female PIS: Established from the results of female urine (FU) and clinician-collected vaginal swab (CCVS) specimens tested by two FDA-cleared NAAT comparator assays. Females were classified as infected if at least one positive result was obtained by each assay. Any other combination was considered non-infected.
- Male PIS: Established using urine results from two FDA-cleared comparator NAATs. If the male urine results were conflicting (one positive, one negative), a third FDA-cleared NAAT method was performed as a tie-breaker to adjudicate male infection status.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance
No MRMC comparative effectiveness study was done. The study evaluated the standalone performance of the NeuMoDx CT/NG Assay 2.0 against a PIS algorithm and did not involve human readers interpreting results with or without AI assistance.
6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, a standalone performance study was done. The clinical performance characteristics (sensitivity and specificity) were established by comparing the results of the NeuMoDx CT/NG Assay 2.0 (an automated molecular test, essentially an algorithm-only device in its interpretation of PCR data) directly to the patient infected status algorithm. No human interpretation of the device's output and subsequent diagnosis was explicitly included in the reported performance metrics.
7. The Type of Ground Truth Used
The type of ground truth used was an expert consensus (of NAATs) / composite reference standard, referred to as a "patient infected status (PIS) algorithm," which was derived from the results of multiple (two, sometimes three) FDA-cleared, legally marketed Nucleic Acid Amplification Tests (NAATs).
8. The Sample Size for the Training Set
The document does not explicitly state a training set sample size. This device is a diagnostic assay (molecular test), not typically an AI/machine learning algorithm that undergoes a distinct training phase in the same way an image recognition AI would. The "training" or development of the assay (e.g., primer design, cutoff optimization) would have utilized various analytical studies, but a 'training set' in the context of machine learning is not reported here.
9. How the Ground Truth for the Training Set Was Established
As noted above, a distinct "training set" with ground truth established for it in the context of machine learning is not described. The assay's analytical performance (e.g., Limit of Detection, linearity, exclusivity) was characterized using contrived samples (spiked with known concentrations of CT/NG) and known negative samples. For example, the LoD was determined by testing separate dilutions of CT elementary bodies and NG cells in negative matrices, confirming 95% positivity. Inclusivity and cross-reactivity studies used known strains and organisms.
§ 866.3393 Device to detect nucleic acids from non-viral microorganism(s) causing sexually transmitted infections and associated resistance marker(s).
(a)
Identification. A device to detect nucleic acids from non-viral microorganism(s) causing sexually transmitted infections and associated resistance marker(s) is an in vitro diagnostic device intended for the detection and identification of nucleic acids from non-viral microorganism(s) and their associated resistance markers in clinical specimens collected from patients suspected of sexually transmitted infections. The device is intended to aid in the diagnosis of non-viral sexually transmitted infections in conjunction with other clinical and laboratory data. These devices do not provide confirmation of antibiotic susceptibility since mechanisms of resistance may exist that are not detected by the device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The intended use for the labeling required under § 809.10 of this chapter must include a detailed description of targets the device detects, the results provided to the user, the clinical indications appropriate for test use, and the specific population(s) for which the device is intended.
(2) Any sample collection device used must be FDA-cleared, -approved, or -classified as 510(k) exempt (standalone or as part of a test system) for the collection of specimen types claimed by this device; alternatively, the sample collection device must be cleared in a premarket submission as a part of this device.
(3) The labeling required under § 809.10(b) of this chapter must include:
(i) A detailed device description, including reagents, instruments, ancillary materials, all control elements, and a detailed explanation of the methodology, including all pre-analytical methods for processing of specimens;
(ii) Detailed discussion of the performance characteristics of the device for all claimed specimen types based on analytical studies, including Limit of Detection, inclusivity, cross-reactivity, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, within lab precision, and reproducibility, as appropriate;
(iii) Detailed descriptions of the test procedure, the interpretation of test results for clinical specimens, and acceptance criteria for any quality control testing;
(iv) Limiting statements indicating that:
(A) A negative test result does not preclude the possibility of infection;
(B) The test results should be interpreted in conjunction with other clinical and laboratory data available to the clinician;
(C) Reliable results are dependent on adequate specimen collection, transport, storage, and processing. Failure to observe proper procedures in any one of these steps can lead to incorrect results; and
(D) If appropriate (
e.g., recommended by the Centers for Disease Control and Prevention, by current well-accepted clinical guidelines, or by published peer reviewed research), that the clinical performance is inferior in a specific clinical subpopulation or for a specific claimed specimen type; and(v) If the device is intended to detect antimicrobial resistance markers, limiting statements, as appropriate, indicating that:
(A) Negative results for claimed resistance markers do not indicate susceptibility of detected microorganisms, as resistance markers not measured by the assay or other potential mechanisms of antibiotic resistance may be present;
(B) Detection of resistance markers cannot be definitively linked to specific microorganisms and the source of a detected resistance marker may be an organism not detected by the assay, including colonizing flora;
(C) Detection of antibiotic resistance markers may not correlate with phenotypic gene expression; and
(D) Therapeutic failure or success cannot be determined based on the assay results, since nucleic acid may persist following appropriate antimicrobial therapy.
(4) Design verification and validation must include:
(i) Detailed device description documentation, including methodology from obtaining sample to result, design of primer/probe sequences, rationale for target sequence selection, and computational path from collected raw data to reported result (
e.g., how collected raw signals are converted into a reported result).(ii) Detailed documentation of analytical studies, including, Limit of Detection, inclusivity, cross-reactivity, microbial interference, interfering substances, competitive inhibition, carryover/cross contamination, specimen stability, within lab precision, and reproducibility, as appropriate.
(iii) Detailed documentation and performance results from a clinical study that includes prospective (sequential) samples for each claimed specimen type and, when determined to be appropriate by FDA, additional characterized clinical samples. The study must be performed on a study population consistent with the intended use population and compare the device performance to results obtained from FDA accepted comparator methods. Documentation from the clinical studies must include the clinical study protocol (including a predefined statistical analysis plan) study report, testing results, and results of all statistical analyses.
(iv) A detailed description of the impact of any software, including software applications and hardware-based devices that incorporate software, on the device's functions.