(221 days)
The Micronics PanNAT STEC Test is a qualitative, in vitro nucleic acid amplification-based tor the simultaneous detection and identification of the stx1 and stx2 Shiga toxin genes and the O-antigen gene cluster of E.coli O157 (fc/). Testing is performed in a unitized cartridge on the PanNAT System on soft to diarrheal unpreserved or Cary-Blair preserved stool specimens from individuals with signs and symptoms of gastrointestinal infection. The PanNAT STEC Test is intended for use, in conjunction with clinical presentation, laboratory findings and epidemiological risk factors, as an aid in detection of specific Shiga-toxin expressing strains of E. coli ("STEC") from patients with diarrhea.
The results of this test should not be used as the sole basis for diagnosis, treatment or other patient management decisions. Positive PanNAT STEC Test results do not rule out the potential for coinfection with other pathogens that are not detected by this device and may not be indicative of the sole or definitive cause of patient illness. Negative PanNAT STEC Test results in the context of clinical illness compatible with gastroenteritis may be due to infection by pathogens that are not detected by this test or non-infectious causes such as ulcerative colitis, irritable bowel syndrome, or Crohn's disease.
The PanNAT System consists of the instrument with onboard software and 9-inch touchscreen used to process PanNAT STEC Test cartridges. The instrument automates all steps of the assay after sample addition to the test cartridge and insertion into the instrument; including DNA purification, nucleic acid amplification and detection of the target nucleic acid sequences using qualitative real-time PCR.
The system is a portable device that is powered by an external mains supply with a voltage range of 100-240VAC and a frequency range of 50-60 Hz. An onboard battery is included to provide power to the instrument for up to one hour in the event that mains power supply is interrupted. The instrument includes a pneumatic subsystem, a thermal control unit, fluorescent optical detectors, and software needed to process a test cartridge,
The PanNAT STEC Test cartridge is a unique, single use, disposable device in which all test reagents and controls are incorporated and all steps of the assay are performed. The PanNAT STEC Test cartridge uses a PanNAT Sample Transfer Pack accessory that contains a flocked swab, a prefilled Sample Buffer Tube and an Adaptor Cap.
There are three integrated controls for the PanNAT STEC Test: an endogenous human DNA internal process control that is coextracted and coamplified with the target nucleic acids, a negative control, and a positive control. These controls are performed automatically and do not require any action from the operator. External quality controls for the PanNAT STEC Test are also commercially available. Alternatively, laboratories may prepare their own controls.
Here's an analysis of the provided text, focusing on the acceptance criteria and the study that proves the device meets them:
The document describes the PanNAT STEC Test, a qualitative, in vitro nucleic acid amplification-based test for detecting stx1, stx2 Shiga toxin genes, and the E.coli O157 O-antigen gene cluster.
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state "acceptance criteria" in a numerical or percentage format next to performance metrics. However, the results presented in the analytical and clinical performance studies implicitly serve as the demonstration of the device's ability to meet expected performance standards for diagnostic assays. I will summarize the key performance metrics from the studies provided.
| Performance Metric | Acceptance Criteria (Implied / Demonstrated Performance) | Reported Device Performance |
|---|---|---|
| Reproducibility (Overall Agreement %) | High agreement expected across sites, lots, and operators for both positive and negative samples. | Overall: 97.8%. Site A: 98.8%. Site B: 99.2%. Site C: 95.3%. (Lower for "All Low" panel member at Site C: 76.7%) |
| Limit of Detection (LoD) | Consistent and reliable detection at specified concentrations. | Unpreserved stool: 1.00E+06 CFU/mL for EDL933 and 3007-85 (stx1, stx2, O157). Cary-Blair preserved stool: 1.75E+06 CFU/mL for EDL933 and 3007-85. (Confirmed with ≥95% positive replicates.) |
| Analytical Reactivity (Sensitivity) | Detection of targeted Shiga toxin genes (stx1, stx2) and E. coli O157 in various strains, including subtypes. A high percentage of expected results. | stx1: 92% (24/26 strains, with 2 false negatives for stx1d variants due to weak detection). stx2: 96% (25/26 strains, with 1 false negative for an stx2b-like variant due to extensive mismatches). O157 O-antigen (fcl): 100% (26/26 strains). |
| Analytical Specificity (Cross-Reactivity) | No cross-reactivity with common microbial flora or other potential pathogens. | Bacteria: 100% agreement with expected negative results for 37 strains (except Shigella dysenteriae which is expected to be detected due to sequence homology with stxA gene). Viruses: 100% agreement for 8 viruses. Yeast/Parasites: 100% agreement for 4 organisms. |
| Analytical Specificity (Microbial Interference) | No inhibition of target detection in the presence of other organisms. | 100% agreement with expected positive results for stx1, stx2, and O157 targets in the presence of 49 non-target organisms (bacteria, viruses, yeast, parasites). No competitive inhibition observed. |
| Analytical Specificity (Interfering Substances) | No significant interference from common stool contaminants or substances. | Minimal interference: Gaviscon® and Imodium AD® caused one false negative for all three targets in one of three replicates, and stearic acid caused one false negative for all three targets in one of three replicates, when spiked into a positive sample. Otherwise, no interference with 25 substances. |
| Carry Over / Cross Contamination | No false positives due to run-to-run contamination. | No carryover or cross-contamination observed across three instruments over five rounds of alternating high positive and negative samples. (1 invalid result in 15 negative samples, but no false positive). |
| Clinical Performance (PPA - Prospective) | High positive percent agreement for detected analytes. | stx1 (preserved): 100.0% (4/4). stx2 (preserved): 100.0% (5/5). stx2 (unpreserved): 100.0% (2/2). O157 (preserved): 100.0% (3/3). O157 (unpreserved): 100.0% (2/2). |
| Clinical Performance (NPA - Prospective) | High negative percent agreement for detected analytes. | stx1 (preserved): 99.3% (1172/1180). stx1 (unpreserved): 100.0% (96/96). stx2 (preserved): 99.4% (1172/1179). stx2 (unpreserved): 98.9% (93/94). O157 (preserved): 99.8% (1179/1181). O157 (unpreserved): 100.0% (94/94). |
| Clinical Performance (PPA - Retrospective) | High positive percent agreement, especially for samples with low prevalence in prospective studies. | stx1 (unpreserved): 88.5% (23/26). stx2 (unpreserved): 90.0% (27/30). O157 (unpreserved): 89.7% (26/29). |
| Clinical Performance (NPA - Retrospective) | High negative percent agreement. | stx1 (unpreserved): 97.4% (75/77). stx2 (unpreserved): 98.6% (72/73). O157 (unpreserved): 98.6% (72/73). |
2. Sample Size Used for the Test Set and Data Provenance
Test Set for Reproducibility:
- Sample Size: 773 observations overall from 90 panel members (10 for each of 9 panel members evaluated in triplicate over 6+ days at each of 3 sites). Specific counts per panel member vary from 82 to 88.
- Data Provenance: The study was conducted at three sites (2 external and 1 internal). This suggests a mix of internal testing and external validation, likely within the US given the FDA submission. The data appears to be prospective in the sense of being generated specifically for the study under controlled conditions (testing coded panel members).
Test Set for Clinical Performance:
- Prospective Specimens:
- Sample Size: 1331 prospective specimens enrolled, 1301 evaluated after initial exclusions, and 1280 (1184 Cary-Blair preserved and 96 unpreserved) qualified for agreement analysis.
- Data Provenance: Collected at six clinical sites. Specimens were from pediatric and adult patients as part of routine patient care. This is prospective regional data.
- Retrospective Specimens:
- Sample Size: 114 specimens initially enrolled, leading to 103 evaluable results.
- Data Provenance: Frozen, retrospective samples. Likely collected from a broader range of past cases. This is retrospective data.
- Contrived Specimens:
- Sample Size: 16 contrived specimens.
- Data Provenance: Unpreserved, spiked with enumerated stocks of specific E. coli strains and tested at clinical sites. These are contrived samples.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not explicitly state the "number of experts" or their specific qualifications (e.g., "radiologist with 10 years of experience") for establishing the ground truth.
- For Reproducibility: The ground truth was established by the composition of the coded 9-member panel, which consisted of E. coli strains with known analyte presence (stx1, stx2, O157) at low or moderate concentrations, or negative for all. This is a defined ground truth based on strain characteristics.
- For Analytical Reactivity, Specificity, Interference: The ground truth for these studies was based on the known characteristics of the microbial strains, viruses, yeast, parasites, and interfering substances used in the experiments.
- For Clinical Performance:
- Prospective study: The ground truth was established using "standard of care testing (comparator method)" which included "an immunoassay for detection of Shiga toxins 1 and 2 following enrichment culture, and a SMAC plate for presumptive detection of O157, confirmed by latex agglutination and biochemical analysis."
- Discordant analysis: Performed by an "independent molecular reference laboratory" using "alternative PCR and bidirectional (BDS) sequencing methods." This implies expert molecular biologists or laboratory staff established the final ground truth for discrepant cases.
- Retrospective study: The ground truth was established by "alternative PCR (nested)/bidirectional sequencing." This also implies expert molecular biologists or laboratory staff.
- The document doesn't specify the exact number of experts or their years of experience for these reference methods.
4. Adjudication Method for the Test Set
- For Reproducibility and Analytical Studies: No explicit "adjudication method" beyond the inherent known characteristics of the panel members and strains. For the quantitative results in reproducibility, only valid results were used for mean/SD/CV calculation.
- For Clinical Performance:
- Prospective study: A discordant analysis was performed on discrepant samples by an independent molecular reference laboratory. This serves as an adjudication method, where the reference method's result is used to resolve discrepancies between the device and the initial standard of care comparator.
- Retrospective study: The comparator method itself was an "alternative PCR (nested)/bidirectional sequencing," which acted as the reference standard, therefore, no separate adjudication process is described for this stage beyond the initial testing.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study was not done.
This device is a diagnostic test (nucleic acid amplification-based assay) for pathogen detection, not an imaging device or one that relies on human interpretation for its primary output. Therefore, measuring "how much human readers improve with AI vs without AI assistance" is not applicable to this type of device. The study evaluates the device's performance against comparator methods and predefined ground truths, not assistance to human interpretation.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done
Yes, a standalone performance study was done.
The entire performance evaluation described focuses on the PanNAT STEC Test system (instrument and cartridges) operating autonomously after sample loading. The "PanNAT System automates all steps of the assay after sample addition to the test cartridge and insertion into the instrument." The results are displayed on the screen and stored in the instrument database. There is no human interpretation or human-in-the-loop component in the detection and reporting of results for this assay. The human interaction is limited to preparing and loading the sample.
7. The Type of Ground Truth Used
- Analytical Studies (Reproducibility, LoD, Reactivity, Specificity, Interference, Carryover):
- Defined Ground Truth: Known concentrations of specific bacterial strains, or known presence/absence of target analytes in well-characterized cultures, viruses, yeast, parasites, and chemical substances.
- Clinical Performance Studies (Prospective and Retrospective):
- Prospective: Initial ground truth established by "standard of care testing" (immunoassay + SMAC/latex agglutination/biochemical). Discrepant results were then adjudicated by an "independent molecular reference laboratory" using "alternative PCR and bidirectional (BDS) sequencing methods."
- Retrospective: Ground truth established by "alternative PCR (nested)/bidirectional sequencing."
- Contrived: Defined ground truth based on spiking known concentrations of specific strains into negative matrix.
8. The Sample Size for the Training Set
The document describes no separate "training set" in the context of an AI/ML algorithm that would undergo a distinct training phase. This device is a real-time PCR assay, not a machine learning model that is "trained" on data in the traditional sense. The data presented are for performance validation of the established PCR assay.
9. How the Ground Truth for the Training Set Was Established
As noted above, there is no explicit "training set" for an AI/ML algorithm. The ground truth for the analytical studies and clinical studies (which act as validation data) was established as described in section 7.
§ 866.3990 Gastrointestinal microorganism multiplex nucleic acid-based assay.
(a)
Identification. A gastrointestinal microorganism multiplex nucleic acid-based assay is a qualitativein vitro diagnostic device intended to simultaneously detect and identify multiple gastrointestinal microbial nucleic acids extracted from human stool specimens. The device detects specific nucleic acid sequences for organism identification as well as for determining the presence of toxin genes. The detection and identification of a specific gastrointestinal microbial nucleic acid from individuals exhibiting signs and symptoms of gastrointestinal infection aids in the diagnosis of gastrointestinal infection when used in conjunction with clinical evaluation and other laboratory findings. A gastrointestinal microorganism multiplex nucleic acid-based assay also aids in the detection and identification of acute gastroenteritis in the context of outbreaks.(b)
Classification. Class II (special controls). The special controls are set forth in FDA's guideline document entitled: “Class II Special Controls Guideline: Gastrointestinal Microorganism Multiplex Nucleic Acid-Based Assays for Detection and Identification of Microorganisms and Toxin Genes from Human Stool Specimens.” For availability of the guideline document, see § 866.1(e).