(162 days)
The BioStar® OIA® SHIGATOX assay is an Optical Immunoassay (OIA) test for the qualitative, rapid detection of the presence of Shiga Toxins in human diarrheal fecal specimens, broth cultures, and swab sampling of colonies from a culture plate. This test is intended for in vitro diagnostic use as an aid in the diagnosis of infection by Shiga Toxinproducing Escherichia coli (STEC), both 0157 and all non -- 0157 Shiga toxin producing strains.
The BioStar OIA SHIGATOX device is based on a novel thin film optical detection technology that relies on the interaction of white light with thin films to create a destructive interference phenomenon. Characteristic of this phenomenon is the generation of a reflective surface that changes color as a function of the change in optical thickness (refractive index x thickness) of the films on the surface of the device. To take advantage of this phenomenon for monitoring biological binding events, the optical surface with a special background color is coated with a capture reagent specific to the analyte of interest. In the OIA SHIGATOX device, the biological capture film is a combination of affinity-purified polyclonal antibodies to Shiga toxins 1 and 2 (Stx 1 and Stx 2). Samples suspected of containing either or both of the toxins are mixed with cocktail containing polyclonal antibodies to Stx 1 and Stx 2 that have been covalently conjugated to horseradish peroxidase (HRP). Once a sample containing toxins or either toxin is applied to the surface, the immune complex of toxin(s) and the anti-toxin-HRP conjugate(s) are bound to the surface antibodies. Following a wash step, a precipitating substrate for HRP is added, and a thin film generated by the immobilized immune complex is enhanced by the precipitation of the HRP product. Once washed and dried, a simple color change relative to the gold background color is observed as an indication of the presence of Stx 1 or Stx 2 in the original specimen.
The OIA SHIGATOX device produces a qualitative result for the presence or absence of Shiga toxin as the device output. Input to the device is the simple addition of an aliquot of fecal material or broth culture to the reagents contained in the kit. Fecal samples are routinely collected, and no special collection requirements exist beyond the elimination of the use of fecal transport media. Test devices within the kit are single use devices, and disposal instructions are provided in the Package Insert. The kit contains all components necessary for analysis of the direct stool sample with the exception of a timer.
Here's a breakdown of the acceptance criteria and the study details for the BioStar® OIA® SHIGATOX device based on the provided 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
The provided document doesn't explicitly state numerical "acceptance criteria" against which the device performance was measured in the same way a clinical trial might define primary endpoints. Instead, it presents the performance of the device and claims substantial equivalence to predicate devices. For this table, I've extracted the performance metrics reported for the OIA SHIGATOX device from the clinical testing summaries and listed them alongside the performance of the predicate device for comparison, as this is how the document frames its "acceptance." The "acceptance criteria" here are implied by the performance of the predicate devices the OIA SHIGATOX device aims to be equivalent to.
| Acceptance Criteria Category (Implied by Predicate Performance) | BioStar® OIA® SHIGATOX Performance |
|---|---|
| Analytical Sensitivity (Limit of Detection) | Stx 1: 1 ng/mL, Stx 2: 1 ng/mL (in both antigen diluent and liquid stool matrix) |
| Analytical Specificity (Cross-Reactivity) | No cross-reactivity with 49 bacteria (≥ 10^7 organisms/ml) and other organisms/cells (Cryptosporidium, Giardia, Candida albicans) when toxins were absent. Expected positive results when toxins were spiked. |
| Interfering Substances | No false positive or false negative results in the presence of 29% Barium sulfate, 12.5 mg/g mucin, 50% Kaopectate, 50% Pepto Bismol, 50% Liquid Imodium, or 50% Whole blood (tested in antigen diluent, liquid stool, and semi-solid stool). |
| Reproducibility | 100% reproducibility tested across three clinical trial sites, 3 operators, and on three successive days using a blinded panel of 27 specimens (negative, low, and moderate levels of Stx 1 and/or Stx 2). |
| Clinical Sensitivity & Specificity (Direct Stool vs. EIA) | Prospective: Positive Agreement: 100% (95% CI: 73.5-100%), Negative Agreement: 98.1% (95% CI: 95.6 – 99.4%), Overall Percent Agreement: 98.2% (95% CI: 95.8 - 99.4%) (compared to commercial EIA). |
| Frozen: Positive Agreement: 87.5% (95% CI: 67.6-97.3%), Negative Agreement: 97.4% (95% CI: 86.2 – 99.9%), Overall Percent Agreement: 93.6% (95% CI: 84.3-98.2%) (compared to commercial EIA). | |
| Clinical Sensitivity & Specificity (Broth Culture vs. EIA) | Fresh Stools: Positive Agreement: 100% (95% CI: 73.5 -100%), Negative Agreement: 99.6% (95% CI: 97.9 - 100%), Overall Percent Agreement: 99.6% (95% CI: 98.0 - 100%) (compared to commercial EIA). |
| Frozen Stools: Positive Agreement: 100% (95% CI: 84.6 - 100%), Negative Agreement: 95.6% (95% CI: 81.7 - 99.9%), Overall Percent Agreement: 98% (95% CI: 89.4 - 100%) (compared to commercial EIA). | |
| Clinical Sensitivity & Specificity (SMAC Culture) | Direct Fresh Stool vs. SMAC: Positive Agreement: 90% (95% CI: 55.5 - 99.8%), Negative Agreement: 96.9% (95% CI: 94.0 - 98.7%), Overall Percent Agreement: 96.7% (95% CI: 93.7 - 98.5%). |
| Frozen Stool vs. SMAC: Positive Agreement: 100% (95%CI: 66.4 - 100%), Negative Agreement: 75.5% (95%CI: 61.7 - 86.2%), Overall Percent Agreement: 79% (95%CI: 66.8 - 88). | |
| Concordance with Cytotoxicity Test Assay (CTA) | Direct Stool: 12/13 positive samples detected by OIA SHIGATOX where CTA was positive. |
| Broth Culture: 12/12 positive samples detected by OIA SHIGATOX where CTA was positive. | |
| Toxigenic Strain Recognition | All 70 clinical isolates (from Department of Public Health and University Laboratory, covering various serotypes and Stx types) produced expected positive assay results. Shigella dysenteriae (ATCC 13313) also positive. |
2. Sample Sizes Used for the Test Set and Data Provenance
- Clinical Sensitivity and Specificity (Direct Stool): 272 prospective specimens from diarrheal patients.
- Clinical Sensitivity and Specificity (Frozen Direct Stools): 62 additional frozen specimens (prospectively tested).
- Broth Culture (Fresh Stools): 269 prospective specimens from diarrheal patients (3 failed to produce growth, so 266 used for comparison).
- Broth Culture (Frozen Stools): Insufficient data, described as "ten of the frozen specimens were not tested" and "Two of the remaining specimens failed to exhibit growth," implying a small subset of the 62 frozen specimens. The exact number used in the comparison table is not explicitly stated.
- SMAC Culture (Direct Fresh Stool): 269 direct stool samples.
- SMAC Culture (Frozen Stool): 62 frozen specimens.
- Reproducibility: A panel of 27 fecal specimens.
- Toxigenic Strain Testing: 49 isolate strains from a Department of Public Health, 21 isolate strains from a university laboratory, or other commercial source (total 70 isolates).
- Analytical Specificity (Cross Reactivity): 49 bacteria species, Cryptosporidium, Giardia, Candida albicans.
- Interfering Substances: Multiple tests on samples containing Barium Sulfate, Bovine Mucin, Kaopectate, Pepto Bismol, Imodium®, and Whole Blood (tested in duplicate), with and without toxin spikes.
Data Provenance:
- Clinical Studies: Conducted at three clinical trial sites in the Eastern, Southern, and Western regions of the United States.
- Toxigenic Strain Testing: Clinical isolates obtained from a Department of Public Health and a university laboratory, or other commercial source.
- The data appears to be a mix of prospective (for direct stool and broth culture comparisons) and retrospective/archival (for frozen direct stools and toxigenic strain testing).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not specify the number or qualifications of experts used to establish ground truth for the clinical test sets.
- Ground truth for clinical performance was primarily established by:
- Commercial EIA tests (predicate devices).
- Cytotoxicity Testing Assay (CTA) as the confirmatory method for positive immunoassay results.
- SMAC culture (Sorbitol MacConkey plates) for E. coli O157.
- For the toxigenic strain testing, the isolates were "previously analyzed for the presence of Shiga toxin genes and serotyped," implying some form of expert confirmation, but specifics are missing.
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (like 2+1 or 3+1 expert review) for the test set.
- The primary method for confirming positive immunoassay results was Cytotoxicity Testing Assay (CTA). "All positive results from either immunoassay method were confirmed by cytotoxicity testing."
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done
No, a MRMC comparative effectiveness study was not explicitly stated or performed. This device is an in vitro diagnostic test that produces a qualitative result (positive/negative), not an imaging device or AI algorithm designed to assist human readers. The clinical studies compare its performance to other in vitro diagnostic methods (EIA, SMAC, CTA), not to human reader performance with or without AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
The BioStar® OIA® SHIGATOX device is a standalone diagnostic test. It functions as an "algorithm only" in the sense that the device itself, through its chemical and optical reactions, generates a result without direct human interpretation influencing the test outcome beyond visually observing a color change (a "purple spot on the gold background" for positive, "original gold color" for negative). The "human-in-the-loop" component is simply reading the device's output, not interpreting complex data or making a diagnosis with AI assistance.
7. The Type of Ground Truth Used
The types of ground truth used are:
- Predicate Device Performance: Performance of legally marketed EIA tests (Premier EHEC and ProSpecT® Shiga Toxin Microplate Assay).
- Cytotoxicity Testing Assay (CTA): Considered the confirmatory reference method for Shiga toxins.
- SMAC Culture (Sorbitol MacConkey agar): Used specifically for the determination of E. coli O157.
- Molecular/Serological Characterization: For toxigenic strains, they were "previously analyzed for the presence of Shiga toxin genes and serotyped."
8. The Sample Size for the Training Set
The document does not provide information on a specific "training set" as this is an in vitro diagnostic device, not a machine learning model. The various analytical studies (analytical sensitivity, specificity, interfering substances) and the development process itself would have involved testing and refinement, but these are not referred to as a "training set" in the context of AI/ML.
9. How the Ground Truth for the Training Set Was Established
Since there is no explicit "training set" for an AI/ML model, this question is not applicable in the context of this device and report. The analytical performance (sensitivity, specificity, strain recognition) was established through controlled laboratory experiments using purified toxins, characterized bacterial strains, and spiked stool samples.
§ 866.3255
Escherichia coli serological reagents.(a)
Identification. Escherichia coli serological reagents are devices that consist of antigens and antisera used in serological tests to identifyEscherichia coli from cultured isolates derived from clinical specimens. Additionally, some of these reagents consist ofEscherichia coli antisera conjugated with a fluorescent dye used to identifyEscherichia coli directly from clinical specimens or cultured isolates derived from clinical specimens. The identification aids in the diagnosis of diseases caused by this bacterium belonging to the genusEscherichia, and provides epidemiological information on diseases caused by this microorganism. AlthoughEscherichia coli constitutes the greater part of the microorganisms found in the intestinal tract in humans and is usually nonpathogenic, those strains which are pathogenic may cause urinary tract infections or epidemic diarrheal disease, especially in children.(b)
Classification. Class I (general controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 866.9.