(55 days)
The Verigene® Clostridium difficile Nucleic Acid Test (CDF) is a qualitative multiplexed in vitro diagnostic test for the rapid detection of toxin A (tcdA), toxin B (tcdB), and tcdC gene sequences of toxigenic Clostridium difficile and for presumptive identification of PCR ribotype 027 strains from unformed (liquid or soft) stool specimens collected from patients suspected of having C. difficile infection (CDI). Presumptive identification of the PCR ribotype 027 strain of C. difficile is by detection of the binary toxin (cdt) gene sequence and the single base pair deletion at nucleotide 117 in the tcdC gene encodes for a negative regulator in C. difficile toxin production. The test is performed on the Verigene System and utilizes automated specimen preparation and polymerase chain reaction (PCR) amplification, combined with a nanoparticle-based array hybridization assay to detect the toxin gene sequences associated with toxin-producing C. difficile.
The CDF Test is indicated for use as an aid in the diagnosis of CDI. Detection of PCR ribotype 027 strains of C. difficile by the CDF Test is solely for epidemiological purposes and is not intended to guide or monitor treatment for C. difficile infections. Concomitant culture is necessary only if further typing or organism recovery is required.
The Verigene C. difficile Nucleic Acid Test (CDF) is a molecular assay which relies on detection of specific nucleic acid targets in a microarray format. For each of the bacterial nucleic acid sequences detected by CDF, unique Capture and Mediator oligonucleotides are utilized, with gold nanoparticle probe-based endpoint detection. The Capture oligonucleotides are covalently bound to the microarray substrate and hybridize to a specific portion of the nucleic acid targets. The Mediator oligonucleotides have a region which bind to a different portion of the same nucleic acid targets and also have a sequence which allows binding of a gold nanoparticle probe. Specific silver enhancement of the bound gold nanoparticle probes at the capture sites results in gold-silver aggregates that scatter light with high efficiency and provide accurate detection of target capture.
The CDF Test is performed on the Verigene System, a "sample-to-result", fully automated, bench-top molecular diagnostics workstation. The System consists of two components: the Verigene Reader and the Verigene Processor SP. The Reader is the user interface, serving as the central control unit for test processing, automated imaging, and result generation. The Processor SP automates sample preparation, target amplification, and hybridization-detection.
The medical device being described is the Verigene® Clostridium difficile Nucleic Acid Test (CDF), which is a qualitative multiplexed in vitro diagnostic test for the rapid detection of toxin B (tcdB), and tcdC gene sequences of toxigenic Clostridium difficile and for presumptive identification of PCR ribotype 027 strains from unformed (liquid or soft) stool specimens collected from patients suspected of having C. difficile infection (CDI).
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as distinct predefined numerical targets in the provided text. Instead, the study's findings demonstrate the performance, which then implies the met "acceptance criteria." The main performance metrics provided are for sensitivity, specificity, and accuracy against reference methods.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance | Study Section |
|---|---|---|---|
| Toxigenic C. difficile Detection | |||
| Sensitivity (Direct Culture & PCR Ribotyping) | High sensitivity (e.g., >90%) | 98.7% (154/156) (95.5-99.8% CI) | Method Comparison |
| Specificity (Direct Culture & PCR Ribotyping) | High specificity (e.g., >85%) | 87.6% (1500/1713) (85.9-89.1% CI) | Method Comparison |
| Sensitivity (Enriched Culture & PCR Ribotyping) | High sensitivity (e.g., >90%) | 91.8% (247/269) (87.9-94.8% CI) | Method Comparison |
| Specificity (Enriched Culture & PCR Ribotyping) | High specificity (e.g., >90%) | 92.5% (1480/1600) (91.1-93.7% CI) | Method Comparison |
| Sensitivity (Direct Culture & BDS) | High sensitivity (e.g., >90%) | 98.7% (156/158) (95.5-99.9% CI) | Method Comparison |
| Specificity (Direct Culture & BDS) | High specificity (e.g., >85%) | 87.5% (1500/1715) (85.8-89.0% CI) | Method Comparison |
| Sensitivity (Enriched Culture & BDS) | High sensitivity (e.g., >90%) | 91.9% (251/273) (88.1-94.9% CI) | Method Comparison |
| Specificity (Enriched Culture & BDS) | High specificity (e.g., >90%) | 92.5% (1480/1600) (91.1-93.7% CI) | Method Comparison |
| Toxigenic C. difficile / 027 Strain Detection | |||
| Positive Agreement (Direct Culture & PCR Ribotyping) | High agreement (e.g., >90%) | 97.5% (39/40) (86.8-99.9% CI) | Method Comparison |
| Negative Agreement (Direct Culture & PCR Ribotyping) | High agreement (e.g., >90%) | 97.8% (1787/1828) (97.0-98.4% CI) | Method Comparison |
| Positive Agreement (Enriched Culture & PCR Ribotyping) | High agreement (e.g., >90%) | 91.4% (53/58) (81.0-97.1% CI) | Method Comparison |
| Negative Agreement (Enriched Culture & PCR Ribotyping) | High agreement (e.g., >90%) | 98.5% (1783/1811) (97.8-99.0% CI) | Method Comparison |
| Positive Agreement (Direct Culture & BDS) | High agreement (e.g., >90%) | 97.7% (42/43) (87.7-99.9% CI) | Method Comparison |
| Negative Agreement (Direct Culture & BDS) | High agreement (e.g., >90%) | 97.8% (1790/1830) (97.0-98.4% CI) | Method Comparison |
| Positive Agreement (Enriched Culture & BDS) | High agreement (e.g., >90%) | 93.7% (59/63) (84.5-98.2% CI) | Method Comparison |
| Negative Agreement (Enriched Culture & BDS) | High agreement (e.g., >90%) | 98.7% (1787/1810) (98.1-99.2% CI) | Method Comparison |
| Analytical Sensitivity (LoD) | LoD ≤ 1250 CFU/mL of stool | 63 to 1250 CFU/ml of stool | Analytical Testing |
| Analytical Specificity (Cross-reactivity) | 100% specificity | 100% | Analytical Testing |
| Microbial/Interference | No interference observed | No interference observed | Analytical Testing |
| Carry-over / Cross-contamination | No carry-over or cross-contamination | No carry-over or cross-contamination | Analytical Testing |
| Precision | Expected agreement rates (e.g., ~95% for LP, ~20-80% for HN) | Met expected agreement rates | Clinical Testing |
| Reproducibility | High inter-site agreement | Met expected agreement rates | Clinical Testing |
Note: The acceptance criteria were not explicitly stated with specific numerical thresholds but are inferred from the reported performance which were considered sufficient for substantial equivalence.
2. Sample size used for the test set and the data provenance
- Sample Size for Clinical (Method Comparison) Test Set: 1,875 specimens were evaluated in the clinical study (n=1875). For specific comparisons, some samples were excluded due to inconclusive or missing reference results:
- 6 specimens were not PCR-ribotyped.
- 2 specimens were not sequenced for tcdC Bi-Directional Sequencing (BDS).
- Data Provenance: The data was collected from a multi-site prospective investigation study at five U.S. institutions.
- This indicates the data is prospective and collected within the United States.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The text does not explicitly state the "number of experts" or their "qualifications" used to establish the ground truth. The ground truth for the clinical test set was established through a combination of reference culture followed by cell cytotoxicity testing on isolates, and subsequent strain typing by PCR Ribotyping and Bi-Directional Sequencing (BDS) at a central laboratory and an external third-party site. This suggests laboratory professionals and/or researchers were involved, but their specific expert qualifications (e.g., years of experience) are not detailed.
4. Adjudication method for the test set
The text describes using a "central laboratory" for initial culture and cytotoxin B testing, and an "external third-party site" for PCR Ribotyping. Bi-Directional Sequencing was done either on culture-confirmed isolates or DNA extracted during PCR Ribotyping. There isn't a described "adjudication method" in the sense of multiple experts reviewing and reaching a consensus on cases where initial results varied. Instead, the ground truth was established hierarchically and sequentially using definitive laboratory methods. For instance, if C. difficile was isolated from direct culture and was cytotoxin positive, it was deemed "toxigenic C. difficile positive." If not, enriched culture was used.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. This device is an automated molecular diagnostic test, not an AI-assisted diagnostic device that human readers interpret. Therefore, the concept of "human readers improving with AI vs. without AI assistance" is not applicable.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, a standalone study was done. The Verigene CDF Test operates on the Verigene System, which is described as a "sample-to-result, fully automated, bench-top molecular diagnostics workstation." The results are generated by a "software-based decision algorithm resident in the Reader." The performance data presented (analytical and clinical) represent the standalone performance of this automated device and its algorithm.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth used for the clinical (method comparison) study was a combination of:
- Reference Culture: Inoculation onto CCFA-D and CCMB-Tal media.
- Cell Cytotoxicity Testing: Performed on isolates from the cultures.
- Molecular Strain Typing:
- PCR Ribotyping (at an external third-party site, with Agilent 2100 Bioanalyzer for added discrimination).
- Bi-Directional Sequencing (BDS) of the
tcdCgene.
This constitutes a robust, multi-faceted laboratory-based "reference standard" or "gold standard."
8. The sample size for the training set
The document does not provide information on the sample size for a training set. As this device is a molecular diagnostic test based on PCR and hybridization, it is likely designed and validated using analytical samples and clinical samples, but the term "training set" in the context of machine learning (where algorithms learn from data) is not used. The "cutoff verification" mentioned in analytical testing involved 59 C. difficile strains to verify cut-off values for the two-tiered filter algorithm, which might be considered an internal validation step, but not a "training set" in the common AI sense.
9. How the ground truth for the training set was established
As no "training set" is explicitly mentioned in the context of the device's development or a machine learning approach, the method for establishing its ground truth is not described. The analytical studies (LoD, reactivity, specificity, etc.) utilized independently-confirmed C. difficile strains and other microorganisms with known characteristics.
§ 866.3130 Clostridium difficile toxin gene amplification assay.
(a)
Identification. AClostridium difficile toxin gene amplification assay is a device that consists of reagents for the amplification and detection of target sequences inClostridium difficile toxin genes in fecal specimens from patients suspected of havingClostridium difficile infection (CDI). The detection of clostridial toxin genes, in conjunction with other laboratory tests, aids in the clinical laboratory diagnosis of CDI caused byClostridium difficile. (b)
Classification. Class II (special controls). The special controls are set forth in FDA's guideline document entitled: “Class II Special Controls Guideline: Toxin Gene Amplification Assays for the Detection ofClostridium difficile; Guideline for Industry and Food and Drug Administration Staff.” See § 866.1(e) for information on obtaining this document.