(158 days)
The Verigene® Enteric Pathogens Nucleic Acid Test (EP) is a multiplexed. qualitative test for simultaneous detection and identification of common pathogenic enteric bacteria and genetic virulence markers from liquid or soft stool preserved in Cary-Blair media, collected from individuals with signs and symptoms of gastrointestinal infection. The test is performed on the automated Nanosphere Verigene System utilizing reverse transcription (RT), polymerase chain reaction (PCR), and array hybridization to detect specific gastrointestinal microbial nucleic acid gene sequences associated with the following pathogenic bacteria:
- Campylobacter Group (comprised of C. coli. C. jejuni. and C. lari) .
- . Salmonella species
- Shigella species (including S. dysenteriae, S. boydii, S. sonnei, and S. flexneri) .
- Vibrio Group (comprised of V. cholerae and V. parahaemolyticus) .
- . Yersinia enterocolitica
In addition, EP detects the Shiga toxin 1 gene and Shiga toxin 2 gene virulence markers. Shiga toxin producing E. coli (STEC) typically harbor one or both genes that encode for Shiga Toxins l and 2.
EP is indicated as an aid in the diagnosis of specific agents of gastrointestinal illness, in conjunction with other clinical, laboratory, and epidemiological information; however, is not to be used to monitor these infections. EP also aids in the detection and identification of acute gastroenteritis in the context of outbreaks.
Due to the limited number of positive specimens collected for certain organisms during the prospective clinical study, performance characteristics for Yersinia enterocolitica, Vibrio Group and Shigella species were primarily established with contrived specimens.
Concomitant culture is necessary for organism recovery and further typing of bacterial agents.
EP results should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Confirmed positive results do not rule out co-infection with other organisms that are not detected by this test, and may not be the sole or definitive cause of patient illness. Negative EP results in the setting of clinical illness compatible with gastroenteritis may be due to infection by pathogens that are not detected by this test or non-infectious causes such as ulcerative colitis, irritable bowel syndrome, or Crohn's disease.
The Verigene Enteric Pathogens Nucleic Acid Test (EP) is a molecular assay which relies on detection of specific nucleic acid targets in a microarray format. For each of the bacterial nucleic acid sequences detected by EP, unique Capture and Mediator oligonucleotides are utilized, with gold nanoparticle probe-based endpoint detection. The Capture oligonucleotides are covalently bound to the microarray substrate and hybridize to a specific portion of the nucleic acid targets. The Mediator oligonucleotides have a region which bind to a different portion of the same nucleic acid targets and also have a sequence which allows binding of a gold nanoparticle probe. Specific silver enhancement of the bound gold nanoparticle probes at the capture sites results in gold-silver aggregates that scatter light with high efficiency and provide accurate detection of target capture.
The EP test is performed on the Verigene System, a "sample-to-result", fully automated, bench-top molecular diagnostics workstation. The System enables automated nucleic acid extraction from unformed stool specimens (liquid or soft) preserved in Cary-Blair media and detection of bacterial-specific target DNA. The Verigene System consists of two components: the Verigene Reader and the Verigene Processor SP.
The Reader is the Verigene System's user interface, which serves as the central control unit for all aspects of test processing, automated imaging, and result generation using a touchscreen control panel and a barcode scanner. The Verigene Processor SP executes the test procedure, automating the steps of (1) Sample Preparation and Target Amplification – cell lysis and magnetic bead-based bacterial DNA isolation and amplification, and (2) Hybridizationdetection and identification of bacterial-specific DNA in a microarray format by using gold nanoparticle probe-based technology. Once the specimen is loaded by the operator, all other fluid transfer steps are performed by an automated pipette that transfers reagents between wells of the trays and finally loads the specimen into the Test Cartridge for hybridization. Single-use disposable test consumables and a self-contained Verigene Test Cartridge are utilized for each sample tested with the EP assay.
To obtain the test results after test processing is complete, the user removes the Test Cartridge from the Processor SP, and inserts the substrate holder into the Reader for analysis. Light scatter from the capture spots is imaged by the Reader and intensities from the microarray spots are used to make a determination regarding the presence (Detected) or absence (Not Detected) of a bacterial nucleic acid sequence/analyte. This determination is made by means of software-based decision algorithm resident in the Reader.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Device: Verigene® Enteric Pathogens Nucleic Acid Test (EP)
Purpose: Multiplexed, qualitative test for simultaneous detection and identification of common pathogenic enteric bacteria and genetic virulence markers from liquid or soft stool preserved in Cary-Blair media.
1. Acceptance Criteria and Reported Device Performance
The document describes analytical and clinical performance studies, which serve to establish the device's meeting of performance criteria. The tables provided in the original document directly illustrate the reported performance against implied acceptance metrics (e.g., target agreement percentages).
Table of Acceptance Criteria and Reported Device Performance:
Since specific acceptance criteria values (e.g., "must achieve X% sensitivity") are not explicitly stated as distinct criteria, I will list the performance metrics presented in the document as reported performance, implying they met the internal acceptance thresholds for regulatory submission. The precision and reproducibility results are strong indicators of meeting defined criteria for consistency.
| Performance Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Analytical Sensitivity / Limit of Detection (LoD) | LoD defined as ≥95% positive result rate at a given concentration | LoD for 16 strains ranged from 4.10x10^3 to 3.33x10^5 CFU/mL of stool, confirmed with 20 replicates (further 20 if 100% initial detection). |
| Analytical Reactivity (Inclusivity) | Expected result for all 111 clinically relevant bacterial strains tested at 3x LoD. | All 111 strains generated the expected result when tested in triplicate at 3x LoD. |
| Analytical Specificity (Cross-reactivity) | No cross-reactivity with 161 non-target organisms (135 bacterial, 21 viruses, 4 parasites, 1 human cell line), besides defined exceptions. | All organisms yielded "Not Detected", except Campylobacter insulaenigrae (1/9 positive for "Campylobacter" - noted as potential low-level cross-reactivity). |
| Microbial Interference | No interference in presence of 14 common fecal microorganisms at high concentrations. | No interference observed for 14 microorganisms (including bacteria and parasites) tested at 10^7 CFU/mL (or 9x10^6/7x10^6 cells/mL for parasites). |
| Exogenous Substances Interference | No inhibitory effect from 22 potentially interfering substances at medically-relevant concentrations. | None of the 22 substances tested showed inhibitory effect. |
| Carryover / Cross-contamination | No carryover or cross-contamination from high positive to negative samples. | No carryover or cross-contamination observed when alternating high-titer (5x10^5 CFU/mL) positive with negative samples. |
| Competitive Inhibition | Correct detection of both organisms in co-infection scenarios. | Correct detection of both bacterial target organisms in 30 unique sample combinations, with one exception (low-titer Campylobacter coli sometimes missed in presence of high-titer E. coli/Stx2 in 1/3 replicates, but repeat testing ruled out competitive inhibition). |
| Precision (Within-lab) | High agreement with expected results for low and moderate positive samples across operators and days. | Agreement with Expected Result for low and moderate positive samples: all 100% (16/16) except Salmonella enterica (Low) at 93.8% (15/16). |
| Reproducibility (Inter-laboratory) | High agreement with expected results for low and moderate positive samples across 3 sites, operators, and days. | Agreement across 3 sites: varied slightly but generally high. Most moderate samples 100%. Low samples ranged from 83.3% (Y. enterocolitica at Site 3) to 100%. Negative samples were 100% across all sites. |
| PPA (Positive Percent Agreement) | High agreement between EP test and reference methods for positive specimens. | Varied by pathogen and specimen type (fresh, frozen, selected, simulated). Overall PPA for target organisms ranged from 91.5% to 100%. |
| NPA (Negative Percent Agreement) | High agreement between EP test and reference methods for negative specimens. | Varied by pathogen and specimen type. Overall NPA for target organisms ranged from 99.0% to 99.9%. |
2. Sample Size Used for the Test Set and Data Provenance
-
Clinical Study (Method Comparison):
- Total Specimens Tested with EP Test: 1975
- Valid/Evaluable Specimens: 1852 (after excluding 98 specimens and 25 indeterminate "No Call" specimens).
- Data Provenance:
- Country of Origin: United States (7 U.S. institutions involved in prospective investigation study).
- Retrospective/Prospective: Primarily prospective collection of fresh and frozen Cary-Blair specimens. Additionally, simulated frozen seeded Cary-Blair specimens were used (408 specimens, prepared from deidentified prospectively-collected glycerol stocks from 12 clinical specimen acquisition sites).
-
Precision Study (Internal): 14-member simulated sample panel tested daily in duplicate by 2 operators for 4 non-consecutive days, yielding 224 total results.
-
Reproducibility Study (External): 14 unique samples tested daily in triplicate by 2 operators for 5 non-consecutive days at 3 external sites, yielding 1260 total results.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document does not specify the number or qualifications of experts used to establish the ground truth. It states that the EP test results were compared to "reference methods, including bacterial culture and automated phenotype identification for the bacterial targets and broth enrichment followed by EIA and PCR amplification/BDS for Stx1/Stx2 typing." This implies laboratory professionals following established protocols, but no specifics on "experts" or their qualifications for ground truth establishment beyond standard lab procedures.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for discrepancies in the clinical test set results (e.g., by multiple experts). It simply states the comparison to reference methods. For the analytical studies (LoD, reactivity), "expected results" were confirmed by replicate testing and quantitative definitions.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
There is no indication of a multi-reader multi-case (MRMC) comparative effectiveness study being done, nor any mention of human readers assisting or being assisted by the AI/device. This device is a diagnostic test intended to be performed by laboratory personnel, not an AI for image interpretation or clinical decision support with physician assistance. Therefore, an effect size of human readers improving with AI vs. without AI assistance is not applicable to this type of device.
6. Standalone Performance
The entire clinical performance section (pages 10-11) describes the standalone performance of the Verigene EP test against reference methods. This means the algorithm/device's performance (results from the Verigene System) without human intervention in result interpretation beyond what is normally done in a lab setting (e.g., verifying automated calls as per standard operating procedures).
7. Type of Ground Truth Used
The ground truth for the clinical study was established using reference laboratory methods, specifically:
- Bacterial targets: Bacterial culture and automated phenotype identification.
- Stx1/Stx2 typing: Broth enrichment followed by EIA and PCR amplification/BDS.
For artificial/simulated specimens, the ground truth was the known composition of the seeded samples.
8. Sample Size for the Training Set
The document describes the analytical and clinical validation of the device. There is no information provided regarding a "training set" or the process of machine learning model development. This is a molecular diagnostic test, likely based on established probes and detection algorithms, not a deep learning AI model that requires a distinct training phase. The "Cutoff Verification" section mentions assessing 3800 data points (1120 expected positive) from LoD testing to verify the assay cutoff, which is more akin to internal algorithm parameter tuning rather than machine learning training.
9. How the Ground Truth for the Training Set Was Established
As noted above, there is no explicit mention of a training set in the context of machine learning model development. If "training" refers to internal development and optimization of the assay's detection parameters and algorithms, the ground truth would have been established through controlled laboratory experiments (e.g., precisely known concentrations of target organisms, pure cultures, etc.) similar to the analytical studies described.
§ 866.3990 Gastrointestinal microorganism multiplex nucleic acid-based assay.
(a)
Identification. A gastrointestinal microorganism multiplex nucleic acid-based assay is a qualitativein vitro diagnostic device intended to simultaneously detect and identify multiple gastrointestinal microbial nucleic acids extracted from human stool specimens. The device detects specific nucleic acid sequences for organism identification as well as for determining the presence of toxin genes. The detection and identification of a specific gastrointestinal microbial nucleic acid from individuals exhibiting signs and symptoms of gastrointestinal infection aids in the diagnosis of gastrointestinal infection when used in conjunction with clinical evaluation and other laboratory findings. A gastrointestinal microorganism multiplex nucleic acid-based assay also aids in the detection and identification of acute gastroenteritis in the context of outbreaks.(b)
Classification. Class II (special controls). The special controls are set forth in FDA's guideline document entitled: “Class II Special Controls Guideline: Gastrointestinal Microorganism Multiplex Nucleic Acid-Based Assays for Detection and Identification of Microorganisms and Toxin Genes from Human Stool Specimens.” For availability of the guideline document, see § 866.1(e).