(205 days)
The Great Basin Stool Bacterial Pathogens Panel is a multiplexed, qualitative test for the detection and identification of DNA targets of enteric bacterial pathogens. The Stool Bacterial Pathogens Panel detects nucleic acids from:
- · Campylobacter (C. coli/C. jejuni)
- · Salmonella
- · Shiga toxin 1 (stx1)
- Shiga toxin 2 (stx2)
- · Escherichia coli serotype 0157
- Shigella
Shiga toxin genes are found in Shiga toxin-producing strains of E. coli (STEC/EHEC/VTEC) and Shigella dysenteriae. The E. coli O157 test result is only reported if a Shiga toxin gene (stx1 and/or stx2) is also detected.
The Stool Bacterial Pathogens Panel is performed directly from Cary Blair or C&S Medium preserved stool specimens from symptomatic patients with suspected acute gastroenteritis, or colitis and is performed on the Portrait™ Analyzer.
The test is intended for use as an aid in the diagnosis of gastrointestinal illness in conjunction with clinical and epidemiological information; however, it is not to be used to monitor these infection . Positive results do not rule out co-infection with other organisms and may not be the definitive cause of patient illness. Negative test results in the setting of clinical illness compatible with gastroenteritis may be due to infection by pathogens that are not detected by this test, or non-infectious causes such as ulcerative colitis, irritable bowel syndrome, or Crohn's disease. Concomitant culture is necessary if organism recovery or further typing of bacterial agents is desired.
The Great Basin Stool Bacterial Pathogens Panel on the PA500 Portrait™ System utilizes automated, hot-start PCR amplification technology to amplify specific nucleic acid sequences that are then detected using hybridization probes immobilized on a modified silicon chip surface, in a single-use, self-contained test cartridge.
An aliquot of the specimen (stool preserved in stool transport media) is first processed using the Sample Preparation Device (SPD). An aliquot of the eluate obtained from the SPD is loaded into the sample port of the SBPP Test Cartridge.
Genomic DNA is extracted from microbial cells and diluted to reduce potential inhibitors of the PCR. During the PCR process, biotin-labeled primers direct the amplification of specific nucleic acid sequences within a conserved region for identification of: a bacterial sample processing control (SPC), Campylobacter coli/Campylobacter jejuni, Salmonella spp., Shiga toxin 1, Shiqa toxin 2, and E. coli serotype 0157.
Following PCR, biotin-labeled, amplified target DNA sequences are hybridized to sequence specific probes immobilized on the silicon chip surface, and incubated with antibody conjugated to the horseradish peroxidase enzyme (HRP). The unbound conjugate is washed away, and tetramethylbenzidine (TMB) is added to produce a colored precipitate at the location of the probe/target sequence complex. The resulting signal is detected by the automated Portrait™ Optical Reader within the PA500 Portrait™ Analyzer System. The SPC undergoes the same extraction, amplification, and detection steps as the sample in order to inhibitory substances, as well as process inefficiency due to instrument or reagent failure. No operator intervention is required once the sample is loaded into the sample port, and the Stool Bacterial Pathogens Panel cartridge is loaded into the Portrait™ Analyzer.
The PA500 Portrait™ Analyzer System is a fully automated system that includes: the Portrait™ Analyzer, single-use Stool Bacterial Pathogen Panel Cartridges, and the Portrait™ Data Analysis Software Program. The Portrait™ System is designed to perform automated sample preparation, PCR, and optical chip-based detection with integrated data analysis in less than two hours.
This detailed document outlines the performance characteristics of the Great Basin Stool Bacterial Pathogens Panel (SBPP). While it does not include AI-specific performance criteria, it provides a comprehensive overview of the device's analytical and clinical validation, which are analogous to acceptance criteria and study data for traditional medical devices. I will extract the relevant information and present it in the requested format, interpreting "acceptance criteria" as the performance benchmarks demonstrated in the studies and "device performance" as the results achieved.
For AI-specific questions (Adjudication method, MRMC, Standalone performance, Training set details), the document does not contain this information as it pertains to a nucleic acid-based assay and not an AI/ML-driven device. I will explicitly state "Not applicable" for these points.
Acceptance Criteria and Device Performance for Great Basin Stool Bacterial Pathogens Panel
1. Table of Acceptance Criteria and the Reported Device Performance
Since this is a diagnostic assay and not an AI-driven image analysis tool, the acceptance criteria are based on analytical and clinical performance metrics. The document describes several studies (Analytical Sensitivity, Analytical Reactivity, Analytical Specificity, Competitive Inhibition, Interfering Substances, Microbial Interference, Carry-over/Cross Contamination, Reproducibility, Specimen Stability, and Clinical Studies).
For simplicity and relevance to a typical "acceptance criteria" table for a diagnostic device, I will focus on the key performance indicators from the reproducibility and clinical studies. The implicit "acceptance criteria" are the demonstrated performance percentages, often with 95% confidence intervals where available.
| Performance Metric | Acceptance Criteria (Implicit from Study Results) | Reported Device Performance |
|---|---|---|
| Reproducibility (1.5X LoD) | ≥ 95% Agreement (Target) | Campylobacter: 100% (90/90) |
| Salmonella: 96.7% (87/90) | ||
| Shiga toxin 1: 97.8% (90/92) | ||
| Shiga toxin 2: 95.7% (88/92) | ||
| E. coli O157: 97.8% (90/92) | ||
| Shigella: 100% (90/90) | ||
| Reproducibility (3X LoD) | 100% Agreement (Target) | Campylobacter: 100% (90/90) |
| Salmonella: 100% (90/90) | ||
| Shiga toxin 1: 100% (91/91) | ||
| Shiga toxin 2: 100% (91/91) | ||
| E. coli O157: 100% (91/91) | ||
| Shigella: 100% (90/90) | ||
| Reproducibility (Negative) | 100% Agreement (Target) | 100% (450/450) |
| Clinical Performance (Positive Percent Agreement - PPA) - Prospective Study (Fresh Samples) | Point estimate ≥ 80%, lower bound of 95% CI ≥ 70% (General expectation for such assays, not explicitly stated as "acceptance criteria" but implied by regulatory review) | Campylobacter: 96.4% (82.3-99.4) |
| Salmonella: 83.3% (55.2-95.3) | ||
| Shiga toxin 1: 100.0% (20.7-100.0) | ||
| Shiga toxin 2: 100.0% (20.7-100.0) | ||
| E. coli O157: 100% (51.0-100.0) | ||
| Shigella: 100% (56.6-100.0) | ||
| Clinical Performance (Negative Percent Agreement - NPA) - Prospective Study (Fresh Samples) | Point estimate ≥ 95%, lower bound of 95% CI ≥ 90% (General expectation) | Campylobacter: 99.2% (98.6-99.5) |
| Salmonella: 99.6% (99.1-99.8) | ||
| Shiga toxin 1: 99.5% (99.0-99.8) | ||
| Shiga toxin 2: 99.4% (98.8-99.7) | ||
| E. coli O157: 75.0% (46.8-91.1) | ||
| Shigella: 99.1% (98.4-99.4) | ||
| Clinical Performance (PPA) - Frozen Retrospective Study | Point estimate ≥ 90%, lower bound of 95% CI ≥ 80% (General expectation) | Salmonella: 94.4% (81.9-98.5) |
| Shiga toxin 1: 100.0% (88.3-100.0) | ||
| Shiga toxin 2: 100.0% (89.0-100.0) | ||
| E. coli O157: 100.0% (81.6-100.0) | ||
| Shigella: 94.7% (75.4-99.1) | ||
| Clinical Performance (NPA) - Frozen Retrospective Study | Point estimate ≥ 95%, lower bound of 95% CI ≥ 90% (General expectation) | Salmonella: 100.0% (97.8-100.0) |
| Shiga toxin 1: 100.0% (97.9-100.0) | ||
| Shiga toxin 2: 100.0% (97.9-100.0) | ||
| E. coli O157: 100.0% (89.0-100.0) | ||
| Shigella: 100.0% (98.0-100.0) | ||
| Clinical Performance (PPA) - Selected Fresh Salmonella Study | Point estimate ≥ 90%, lower bound of 95% CI ≥ 80% (General expectation) | Salmonella: 92.9% (77.4-98.0) |
2. Sample sizes used for the test set and the data provenance:
-
Clinical Test Sets:
- Prospective Study (Fresh Samples): 1479 samples included in the analysis (1506 collected, 24 excluded). Collected from four external, geographically-diverse U.S. clinical study sites (Midwest, Northeast, Southwest and West) from July 2016 to November 2016. These were excess remnants of preserved stool samples. Prospective data.
- Frozen Retrospective Sample Study: 150 frozen archived de-identified specimens initially (for the general panel), with specific numbers for each analyte:
- Salmonella: 206 samples
- Shiga Toxin 1: 206 samples
- Shiga Toxin 2: 206 samples
- E. coli O157: 48 samples
- Shigella: 206 samples
These were de-identified specimens previously characterized (historical result). Retrospective data.
- Selected Fresh Positive Salmonella Samples Study: 28 additional fresh samples. Collected from Intermountain Healthcare (IMC) in Salt Lake City, UT. Prospective data.
-
Analytical Test Sets:
- Analytical Sensitivity (LoD): 10 bacterial strains, serially diluted.
- Analytical Reactivity (Inclusivity): 91 well-characterized bacterial strains, multiple replicates per strain (at least 3).
- Analytical Specificity (Exclusivity): 100 non-target organisms (84 bacterial, 3 yeast, 3 parasites, 9 viruses) and human genomic DNA, multiple replicates per organism (minimum of 3).
- Competitive Inhibition: 48 unique combinations of pathogens, each tested in triplicate.
- Interfering Substances: 19 different substances, tested with 8 target organisms, minimum of 3 replicates per substance/organism combination.
- Microbial Interference: 29 non-target organisms tested in presence of 8 target analytes, minimum of 3 replicates.
- Carry-over/Cross Contamination: 40 high positive samples and 40 negative samples (total 80 tests).
- Reproducibility: 7 different samples tested in triplicate over 5 non-consecutive days by 6 operators, across 3 sites (90-92 replicates per positive analyte, 450 replicates for negative).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
The document describes the ground truth for clinical samples as "standard stool culture-based methods" and "standard of care method used at the institution (historical result)," further confirmed by "FDA cleared Nucleic Acid Amplification Test (NAAT)" for retrospective samples. The experts are implicitly the microbiologists and clinical laboratory personnel at the clinical sites performing these standard methods. No specific number or explicit qualifications (e.g., years of experience) for these "experts" are provided in the document, which is typical for diagnostic assay submissions relying on established clinical laboratory practices.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:
None described. The ground truth for the clinical studies relied on standard microbiological culture methods and FDA-cleared NAATs as reference methods. Discrepant results were investigated by further testing with other FDA-cleared NAATs (BioFire Film Array GI Panel or Nanosphere Verigene® EP test). This is a discrepant analysis approach, not an adjudication process by human experts re-interpreting initial data.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
Not applicable. This device is a molecular diagnostic assay, not an AI-driven imaging or diagnostic tool intended for human-in-the-loop assistance.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
Yes, the device operates as a standalone automated system. The entire analytical and clinical performance evaluation described in the document represents the standalone performance of the SBPP, as it integrates sample preparation, PCR, and detection with automated software interpreting results. It is an "algorithm only" in the sense of a laboratory assay's automated result interpretation, although not a machine learning algorithm.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):
The ground truth for the clinical studies was established primarily by:
- Standard stool culture-based methods (for prospective samples).
- Historical results from clinical sites using their standard of care methods, then confirmed by FDA cleared Nucleic Acid Amplification Tests (NAATs) (for frozen retrospective samples).
- Discrepant analysis was performed using other FDA-cleared NAATs (BioFire Film Array GI Panel or Nanosphere Verigene® EP test) for unconcordant results.
Thus, the ground truth is a combination of established microbiological standards and validated molecular diagnostic tests.
8. The sample size for the training set:
Not applicable. This is a nucleic acid-based diagnostic device, not an AI/ML device that requires a "training set" in the conventional sense. The development of the assay (primer/probe design, assay conditions) would be based on scientific knowledge and wet-lab experiments, not a machine learning training data set.
9. How the ground truth for the training set was established:
Not applicable. See point 8. The "ground truth" for developing the assay would be pure cultures of target and non-target organisms, along with their known biochemical and genetic characteristics, established through standard microbiology laboratory practices.
§ 866.3990 Gastrointestinal microorganism multiplex nucleic acid-based assay.
(a)
Identification. A gastrointestinal microorganism multiplex nucleic acid-based assay is a qualitativein vitro diagnostic device intended to simultaneously detect and identify multiple gastrointestinal microbial nucleic acids extracted from human stool specimens. The device detects specific nucleic acid sequences for organism identification as well as for determining the presence of toxin genes. The detection and identification of a specific gastrointestinal microbial nucleic acid from individuals exhibiting signs and symptoms of gastrointestinal infection aids in the diagnosis of gastrointestinal infection when used in conjunction with clinical evaluation and other laboratory findings. A gastrointestinal microorganism multiplex nucleic acid-based assay also aids in the detection and identification of acute gastroenteritis in the context of outbreaks.(b)
Classification. Class II (special controls). The special controls are set forth in FDA's guideline document entitled: “Class II Special Controls Guideline: Gastrointestinal Microorganism Multiplex Nucleic Acid-Based Assays for Detection and Identification of Microorganisms and Toxin Genes from Human Stool Specimens.” For availability of the guideline document, see § 866.1(e).