Search Results

The Great Basin Stool Bacterial Pathogens Panel is a multiplexed, qualitative test for the detection and identification of DNA targets of enteric bacterial pathogens. The Stool Bacterial Pathogens Panel detects nucleic acids from:

• · Campylobacter (C. coli/C. jejuni)
• · Salmonella
• · Shiga toxin 1 (stx1)
• Shiga toxin 2 (stx2)
• · Escherichia coli serotype 0157
• Shigella

Shiga toxin genes are found in Shiga toxin-producing strains of E. coli (STEC/EHEC/VTEC) and Shigella dysenteriae. The E. coli O157 test result is only reported if a Shiga toxin gene (stx1 and/or stx2) is also detected.

The Stool Bacterial Pathogens Panel is performed directly from Cary Blair or C&S Medium preserved stool specimens from symptomatic patients with suspected acute gastroenteritis, or colitis and is performed on the Portrait™ Analyzer.

The test is intended for use as an aid in the diagnosis of gastrointestinal illness in conjunction with clinical and epidemiological information; however, it is not to be used to monitor these infection . Positive results do not rule out co-infection with other organisms and may not be the definitive cause of patient illness. Negative test results in the setting of clinical illness compatible with gastroenteritis may be due to infection by pathogens that are not detected by this test, or non-infectious causes such as ulcerative colitis, irritable bowel syndrome, or Crohn's disease. Concomitant culture is necessary if organism recovery or further typing of bacterial agents is desired.

The Great Basin Stool Bacterial Pathogens Panel on the PA500 Portrait™ System utilizes automated, hot-start PCR amplification technology to amplify specific nucleic acid sequences that are then detected using hybridization probes immobilized on a modified silicon chip surface, in a single-use, self-contained test cartridge.

An aliquot of the specimen (stool preserved in stool transport media) is first processed using the Sample Preparation Device (SPD). An aliquot of the eluate obtained from the SPD is loaded into the sample port of the SBPP Test Cartridge.

Genomic DNA is extracted from microbial cells and diluted to reduce potential inhibitors of the PCR. During the PCR process, biotin-labeled primers direct the amplification of specific nucleic acid sequences within a conserved region for identification of: a bacterial sample processing control (SPC), Campylobacter coli/Campylobacter jejuni, Salmonella spp., Shiga toxin 1, Shiqa toxin 2, and E. coli serotype 0157.

Following PCR, biotin-labeled, amplified target DNA sequences are hybridized to sequence specific probes immobilized on the silicon chip surface, and incubated with antibody conjugated to the horseradish peroxidase enzyme (HRP). The unbound conjugate is washed away, and tetramethylbenzidine (TMB) is added to produce a colored precipitate at the location of the probe/target sequence complex. The resulting signal is detected by the automated Portrait™ Optical Reader within the PA500 Portrait™ Analyzer System. The SPC undergoes the same extraction, amplification, and detection steps as the sample in order to inhibitory substances, as well as process inefficiency due to instrument or reagent failure. No operator intervention is required once the sample is loaded into the sample port, and the Stool Bacterial Pathogens Panel cartridge is loaded into the Portrait™ Analyzer.

The PA500 Portrait™ Analyzer System is a fully automated system that includes: the Portrait™ Analyzer, single-use Stool Bacterial Pathogen Panel Cartridges, and the Portrait™ Data Analysis Software Program. The Portrait™ System is designed to perform automated sample preparation, PCR, and optical chip-based detection with integrated data analysis in less than two hours.

This detailed document outlines the performance characteristics of the Great Basin Stool Bacterial Pathogens Panel (SBPP). While it does not include AI-specific performance criteria, it provides a comprehensive overview of the device's analytical and clinical validation, which are analogous to acceptance criteria and study data for traditional medical devices. I will extract the relevant information and present it in the requested format, interpreting "acceptance criteria" as the performance benchmarks demonstrated in the studies and "device performance" as the results achieved.

For AI-specific questions (Adjudication method, MRMC, Standalone performance, Training set details), the document does not contain this information as it pertains to a nucleic acid-based assay and not an AI/ML-driven device. I will explicitly state "Not applicable" for these points.

Acceptance Criteria and Device Performance for Great Basin Stool Bacterial Pathogens Panel

1. Table of Acceptance Criteria and the Reported Device Performance

Since this is a diagnostic assay and not an AI-driven image analysis tool, the acceptance criteria are based on analytical and clinical performance metrics. The document describes several studies (Analytical Sensitivity, Analytical Reactivity, Analytical Specificity, Competitive Inhibition, Interfering Substances, Microbial Interference, Carry-over/Cross Contamination, Reproducibility, Specimen Stability, and Clinical Studies).

For simplicity and relevance to a typical "acceptance criteria" table for a diagnostic device, I will focus on the key performance indicators from the reproducibility and clinical studies. The implicit "acceptance criteria" are the demonstrated performance percentages, often with 95% confidence intervals where available.

2. Sample sizes used for the test set and the data provenance:

• Clinical Test Sets:

  • Prospective Study (Fresh Samples): 1479 samples included in the analysis (1506 collected, 24 excluded). Collected from four external, geographically-diverse U.S. clinical study sites (Midwest, Northeast, Southwest and West) from July 2016 to November 2016. These were excess remnants of preserved stool samples. Prospective data.
  • Frozen Retrospective Sample Study: 150 frozen archived de-identified specimens initially (for the general panel), with specific numbers for each analyte:
    • Salmonella: 206 samples
    • Shiga Toxin 1: 206 samples
    • Shiga Toxin 2: 206 samples
    • E. coli O157: 48 samples
    • Shigella: 206 samples
      These were de-identified specimens previously characterized (historical result). Retrospective data.
  • Selected Fresh Positive Salmonella Samples Study: 28 additional fresh samples. Collected from Intermountain Healthcare (IMC) in Salt Lake City, UT. Prospective data.

• Analytical Test Sets:

  • Analytical Sensitivity (LoD): 10 bacterial strains, serially diluted.
  • Analytical Reactivity (Inclusivity): 91 well-characterized bacterial strains, multiple replicates per strain (at least 3).
  • Analytical Specificity (Exclusivity): 100 non-target organisms (84 bacterial, 3 yeast, 3 parasites, 9 viruses) and human genomic DNA, multiple replicates per organism (minimum of 3).
  • Competitive Inhibition: 48 unique combinations of pathogens, each tested in triplicate.
  • Interfering Substances: 19 different substances, tested with 8 target organisms, minimum of 3 replicates per substance/organism combination.
  • Microbial Interference: 29 non-target organisms tested in presence of 8 target analytes, minimum of 3 replicates.
  • Carry-over/Cross Contamination: 40 high positive samples and 40 negative samples (total 80 tests).
  • Reproducibility: 7 different samples tested in triplicate over 5 non-consecutive days by 6 operators, across 3 sites (90-92 replicates per positive analyte, 450 replicates for negative).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

The document describes the ground truth for clinical samples as "standard stool culture-based methods" and "standard of care method used at the institution (historical result)," further confirmed by "FDA cleared Nucleic Acid Amplification Test (NAAT)" for retrospective samples. The experts are implicitly the microbiologists and clinical laboratory personnel at the clinical sites performing these standard methods. No specific number or explicit qualifications (e.g., years of experience) for these "experts" are provided in the document, which is typical for diagnostic assay submissions relying on established clinical laboratory practices.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

None described. The ground truth for the clinical studies relied on standard microbiological culture methods and FDA-cleared NAATs as reference methods. Discrepant results were investigated by further testing with other FDA-cleared NAATs (BioFire Film Array GI Panel or Nanosphere Verigene® EP test). This is a discrepant analysis approach, not an adjudication process by human experts re-interpreting initial data.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

Not applicable. This device is a molecular diagnostic assay, not an AI-driven imaging or diagnostic tool intended for human-in-the-loop assistance.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

Yes, the device operates as a standalone automated system. The entire analytical and clinical performance evaluation described in the document represents the standalone performance of the SBPP, as it integrates sample preparation, PCR, and detection with automated software interpreting results. It is an "algorithm only" in the sense of a laboratory assay's automated result interpretation, although not a machine learning algorithm.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

The ground truth for the clinical studies was established primarily by:

• Standard stool culture-based methods (for prospective samples).
• Historical results from clinical sites using their standard of care methods, then confirmed by FDA cleared Nucleic Acid Amplification Tests (NAATs) (for frozen retrospective samples).
• Discrepant analysis was performed using other FDA-cleared NAATs (BioFire Film Array GI Panel or Nanosphere Verigene® EP test) for unconcordant results.

Thus, the ground truth is a combination of established microbiological standards and validated molecular diagnostic tests.

8. The sample size for the training set:

Not applicable. This is a nucleic acid-based diagnostic device, not an AI/ML device that requires a "training set" in the conventional sense. The development of the assay (primer/probe design, assay conditions) would be based on scientific knowledge and wet-lab experiments, not a machine learning training data set.

9. How the ground truth for the training set was established:

Not applicable. See point 8. The "ground truth" for developing the assay would be pure cultures of target and non-target organisms, along with their known biochemical and genetic characteristics, established through standard microbiology laboratory practices.

Ask a specific question about this device

The Great Basin Bordetella Direct Test is a qualitative in vitro diagnostic test for the detection of Bordetella pertussis DNA from nasopharyngeal swab specimens obtained from patients suspected of having a respiratory tract infection attributable to B. pertussis.

The Bordetella Direct Test is performed on the PA500 Portrait Analyzer and utilizes PCR amplification of the insertion sequence IS481. The IS481 sequence is also found in other organisms including Bordetella holmesii or Bordetella bronchiseptica. Respiratory infection with B. pertussis, B. holmesii ot B. bronchiseptica may yield positive test results with IS481 assays. B. holmesii infection may cause clinical illness similar to B. pertussis, and mixed outbreaks involving both B. pertussis and B. holmesii infection have been reported. Additional testing should be performed if necessary to differentiate B. holmesii and B. pertussis. B. bronchiseptica is a rare cause of infection in humans. When clinical factors suggest that B. pertussis may not be the cause of respiratory infection, other clinically appropriate investigation(s) should be carried out in accordance with published guidelines.

Negative results for the Great Basin Bordetella Direct Test do not preclude B. pertussis infection and positive results do not rule out co-infection with other respiratory pathogens. Results from the Great Basin Bordetella Direct Test should be used in conjunction with information obtained during the patient's clinical evaluation as an aid in diagnosis of Bordetella pertussis infection and should not be used as the sole basis for treatment or other patient management decisions.

The Great Basin Bordetella Direct Test on the PA500 Portrait™ Analyzer System utilizes automated hot-start PCR technology to target and amplify the IS481 insertion sequence of B. pertussis. Genomic DNA is extracted from microbial cells and diluted to reduce potential inhibitors of PCR. During PCR, double-stranded DNA is separated and the target nucleic acid sequence is amplified by thermal cycling using biotin-labeled primers that target the IS481 sequence for identification of B. pertussis. Following PCR, biotin-labeled amplicon is hybridized to sequence-specific capture probes immobilized on the silicon chip surface, then incubated with anti-biotin antibody conjugated to the horseradish peroxidase enzyme (HRP). The unbound conjugate is washed away and tetramethylbenzidine (TMB) is added to produce a visible precipitate at the location of the probe/target sequence complex. The resulting signal is detected by the automated Portrait™ Optical Reader within the PA500 Portrait™ Analyzer System. The Specimen Processing Control (SPC), undergoes the extraction, and detection steps to monitor for inhibitory substances as well as process inefficiency due to instrument or reagent failure. No operator intervention is necessary once the clinical sample is loaded into the sample port and the Bordetella Direct Test cartridge is loaded into the Portrait Analyzer.

The PA500 Portrait™ Analyzer System is a fully automated system that includes: The Portrait™ Analyzer, single-use Bordetella Direct Test Cartridges, and the Portrait™ Data Analysis Software Program. The Portrait™ System is designed to perform automated sample preparation, PCR, and optical chip-based detection with integrated data analysis in less than two hours.

This documentation describes acceptance criteria and the study that proves the Great Basin Bordetella Direct Test meets these criteria. The device is a qualitative in vitro diagnostic test for the detection of Bordetella pertussis DNA from nasopharyngeal swab specimens.

Here's the detailed breakdown of the acceptance criteria and study information:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for the Great Basin Bordetella Direct Test are implicitly defined by the performance observed in the analytical and clinical studies, which are designed to demonstrate the device's acceptable functioning for its intended use. While explicit numerical acceptance criteria (e.g., "PPA must be >X%") are not directly stated as pass/fail thresholds in the provided text, the reported performance metrics are presented as the results that demonstrate substantial equivalence.

• ATCC 8467: 3.3 x 10^3 CFU/mL (20/20 detected)
• ATCC 9797: 1.6 x 10^3 CFU/mL (20/20 detected)
• ATCC BAA-589: 2.3 x 10^3 CFU/mL (19/20 detected) |
  | Analytical Reactivity (Inclusivity) | Correct detection of additional B. pertussis strains. | All 8 additional B. pertussis strains tested were correctly detected (3/3 positive for each strain). |
  | Analytical Specificity (Exclusivity) | No cross-reactivity with non-target organisms commonly found in the human respiratory system, except for known IS481-positive Bordetella species, which should be noted as limitations. | Most non-target organisms (48 bacteria, 20 viruses, 2 yeast, 1 human genomic DNA) showed no cross-reactivity (100% expected negative results for all tested).
  Observed cross-reactivity:
• Bordetella bronchiseptica: 1 of 3 strains showed cross-reactivity (ATCC 4617).
• Bordetella holmesii: All 3 strains showed cross-reactivity.
• Bordetella hinzii: 1 of 2 strains showed cross-reactivity (ATCC 51784).
  These noted cross-reactivities for Bordetella species are consistent with the presence of the IS481 insertion sequence also detected by the test. |
  | Microbial Interference | No interference (i.e., correct detection of B. pertussis) when B. pertussis is present with high concentrations of other organisms. | 83 of 84 tested organisms (48 bacteria, 19 viruses, 2 yeast, 14 Bordetella non-B. pertussis strains, 1 human genomic DNA) showed no interference (100% expected positive results). One exception: 1 replicate for M. tuberculosis initially reported 'NOT DETECTED', but subsequent retesting confirmed no interference. |
  | Interfering Substances | No interference with common chemical substances found in patients with upper respiratory conditions. | All 19 tested chemical substances showed no interference (100% expected positive and negative results). |
  | Carry-over/Cross-Contamination | No false positives in negative samples following high positive samples. | No carry-over or cross-contamination observed (100% correct results for alternating high positive and negative samples). |
  | Swab, Transport Media, Elution Buffer Equivalency | Device compatibility with various swab types, transport media, and elution buffers. | All tested swab types (Flocked Nylon, Polyester, Rayon), transport media (Remel M4/M4RT/M5/M6 VTM, BD Universal VTM, ESwab), and elution buffers (Molecular grade water, PBS, Saline, TE Buffer) showed no interference (3/3 expected results for each).
  Equivalence study confirmed 100% agreement when using different combinations of swab types and transport/elution media. |
  | Sample Stability and Storage | Specimen stability for specified storage conditions (room temperature and refrigerated). | 100% agreement with expected results for samples stored up to 72 hours at room temperature and up to 168 hours (7 days) at 2-8°C, supporting specimen storage claims of 48 hours at room temperature or ≤120 hours at 2-8°C. |
  | Reproducibility | Consistent results across different sites, operators, and reagent lots. | 100% agreement observed across replicates, runs, operators (6), and sites (3 clinical sites) for low positive, moderate positive, and negative samples. |
  | Clinical Studies | | |
  | Positive Percent Agreement (PPA) | High agreement with the Reference NAAT for positive samples, demonstrating clinical sensitivity. (Implicitly, the confidence interval should be acceptable, likely meeting a lower bound, although not explicitly stated). | Prospective Study (Fresh): 85.7% (95% CI: 65.4% - 95.0%) (18/21 true positives)
  Frozen Retrospective Study: 94.6% (95% CI: 86.1% - 98.3%) (56/59 true positives) |
  | Negative Percent Agreement (NPA) | High agreement with the Reference NAAT for negative samples, demonstrating clinical specificity. (Implicitly, the confidence interval should be acceptable, likely meeting a lower bound, although not explicitly stated). | Prospective Study (Fresh): 99.6% (95% CI: 98.9% - 99.8%) (890/894 true negatives)
  Frozen Retrospective Study: 100.0% (95% CI: 94.3% - 100.0%) (63/63 true negatives) |

2. Sample Size Used for the Test Set and Data Provenance

• Prospective Study (Fresh Samples):
  • Sample Size: 915 samples (936 collected, 21 excluded due to improper storage or failed QC).
  • Data Provenance: U.S. clinical study sites (four external, geographically-diverse sites: Northwest, Southwest, Midwest, West). Samples collected prospectively from July 2016 to January 2017. These were excess, de-identified nasopharyngeal (NP) swab specimens submitted for standard of care B. pertussis testing.
• Frozen Retrospective Study:
  • Sample Size: 122 samples (124 initially, 2 excluded due to failed QC).
  • Data Provenance: Archived frozen NP swab specimens from institutions across the U.S. (implied from the context of clinical studies and specimen collection methods), retrospective in nature. These were de-identified specimens previously characterized by a nucleic acid amplification test at their originating institution.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

This document describes a diagnostic test for Bordetella pertussis using molecular methods (NAAT). The "ground truth" for the clinical test sets (both prospective and retrospective) was established by comparison to an FDA-cleared Nucleic Acid Amplification Test (Reference NAAT).

• No human experts (e.g., radiologists) were used to establish the ground truth for classification of positive/negative B. pertussis status in the clinical studies.
• The ground truth relies on the performance of the established Reference NAAT.

4. Adjudication Method for the Test Set

• For the clinical performance studies, discrepant results between the Great Basin Bordetella Direct Test and the initial Reference NAAT were investigated by testing in a second FDA-cleared NAAT (NAAT 2) which also detects Bordetella pertussis. This acts as an adjudication step.
• The document states: "In total, there were six (6) false negative and four (4) false positive results. Two (2) of the six (6) false negatives were also negative by a second FDA cleared NAAT and two (2) of the four (4) false positives were also positive by the second FDA cleared NAAT." This indicates the NAAT 2 was used to help understand the nature of the discrepancies, but it does not specify if the NAAT 2 result was definitively taken as the final "truth" for the agreement calculations in all cases. Typically, in diagnostic studies, a third, independent method or a consensus of multiple reference methods is used for resolving discrepancies, but the exact arbitration rule (e.g., 2-out-of-3, or specific tie-breaking rules) is not detailed for the final truth determination for PPA/NPA beyond the investigation by NAAT 2.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

• No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) test, specifically a nucleic acid amplification test (NAAT). The performance is measured by its ability to detect specific DNA directly from a patient sample, not by a human interpreting images or data to make a diagnosis. Therefore, comparative effectiveness with human readers (e.g., radiologists, pathologists) is not relevant for this type of device. The studies focused on its analytical performance and its agreement with a reference molecular diagnostic test.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Yes, the primary performance evaluation of the Great Basin Bordetella Direct Test is effectively "standalone" performance. This means the device itself, the automated PA500 Portrait Analyzer and its associated assay, is evaluated for its ability to produce positive or negative results. There is no human interpretation or intervention in the diagnostic output generation described beyond loading the sample into the automated system. The results are generated by the instrument's automated software.

7. The Type of Ground Truth Used

• For the clinical studies, the ground truth for classifying samples as positive or negative for B. pertussis was established by a Reference FDA-cleared Nucleic Acid Amplification Test (NAAT). Discrepancies were investigated by a second FDA-cleared NAAT.
• For the analytical studies (LoD, inclusivity, exclusivity, interference), the ground truth was established by known concentrations of cultured bacterial strains (CFU/mL) or genomic copies/TCID50 for viruses and other microbes, which were spiked into negative clinical matrix.

8. The Sample Size for the Training Set

• The provided document is a 510(k) premarket notification summary for a test that is likely based on traditional molecular biology techniques (PCR) rather than a machine learning/AI algorithm that requires a separate training set.
• Therefore, the concept of a "training set" in the context of machine learning, where an algorithm learns patterns from data, does not directly apply to the development and validation of this specific in vitro diagnostic device. Its "training" is more akin to traditional assay development and optimization to ensure primers, probes, and reaction conditions are specific and sensitive.
• The document does not mention a machine learning component or a distinct "training set" data size.

9. How the Ground Truth for the Training Set was Established

• As explained in point 8, the concept of a "training set" for a machine learning algorithm is not applicable here. The device's underlying technology is PCR amplification, which is based on established biological and chemical principles rather than machine learning from a specific training dataset. Development and optimization of such assays involve
  • Designing and testing primers/probes: Based on known genetic sequences of B. pertussis.
  • Optimizing reaction conditions: To achieve desired sensitivity and specificity using known positive and negative controls/samples.
  • Analytical validation: As described in the document (LoD, inclusivity, exclusivity) using characterized bacterial strains and other microbes.
• The "ground truth" during the development and optimization phases would have been established by precisely characterized biological materials, such as pure cultures of B. pertussis at known concentrations, or panels of well-characterized positive and negative patient samples.

Ask a specific question about this device

The Great Basin Staph ID/R Blood Culture Panel is a qualitative, multiplex, nucleic acid-based in vitro diagnostic assay intended for the simultaneous identification of nucleic acid from Staphylococcus lugdumensis and various Staphylococcus species to the genus level and the detection of the mecA gene for methicillin resistance directly from patient positive blood culture specimens. The test utilizes automated hot-start enabled polymerase chain reaction (PCR) for the amplification of specific DNA targets detected by hybridization probes immobilized on a silicon chip surface. The assay is performed directly on positive blood culture specimens identified as positive by continuous monitoring blood culture system that demonstrates the presence of organisms as determined by Gram stain to contain gram-positive cocci in clusters (GPCC) or gram-positive cocci in singles (GPC). The test may be performed using blood culture bottles. The Staph ID/R Blood Culture Panel identifies Staphylococcus aureus (SA), and Staphylococcus lugdunensis, and detects other Staphylococcus species without identification to species level.

The Portrait Staph ID/R Blood Culture Panel is indicated for use in conjunction with other clinical or laboratory findings to aid in the diagnosis of bacterial bloodstream infections; however, it is not used to monitor these infections. Subculturing positive blood cultures is necessary to recover viable organisms for further identification, susceptibility testing, or epidemiological typing to identify organisms in the blood culture that are not detected by the Great Basin Staph ID/R Blood Culture Panel. If detected, mecA may or may not be associated with Staphylococcus spp. detected or the agent responsible for the disease. Negative results for mecA antimicrobial resistance gene assays do not always indicate susceptibility, as other mechanisms of resistance to methicillin exist.

The Great Basin PA500 Portrait™ Analyzer System is a fully automated system that includes the Portrait Analyzer, single-use Staph ID/R Blood Culture Panel Test Cartridges, and the Portrait data analysis software. The PA500 Portrait Analyzer System is designed to perform automated sample preparation, PCR, and optical chip-based detection with integrated data analysis in approximately 110 minutes.

The Great Basin Staph ID/R Blood Culture Panel is a qualitative, multiplex, nucleic acid-based in vitro diagnostic assay intended for the simultaneous identification of nucleic acid from Staphylococcus aureus, Staphylococcus lugdunensis, and various Staphylococcus species to the genus level, and the detection of the mecA gene for methicillin resistance directly from patient positive blood culture specimens. The device uses automated hot-start enabled PCR for amplification and hybridization probes on a silicon chip for detection.

Here's an analysis of its acceptance criteria and the study that proves the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

• Staphylococcus positivity: 100% (540/540) for positive, 96.7% (87/90) for negative.
• Specific Staphylococcus detection: 99.6% (538/540) for positive, 99.3% (1341/1350) for negative.
• mecA detection (no organism): 99.7% (359/360) for present, 100% (270/270) for absent. |
  | Evaluation of Blood Culture Bottle Types | Compatibility with various bottle types. | All 13 tested blood culture bottle types were compatible with the Staph ID/R Blood Culture Panel, with no false negative results. One false positive S. aureus and one false positive Staphylococcus species OTHER than S. aureus or S. lugdunensis were observed in a single test run each, attributed to contamination. Nine invalid calls for E. faecalis. |
  | Prospective Clinical Performance - S. aureus | PPA ≥ 95%, NPA ≥ 95% (implied). | - PPA: 98.6% (211/214) (95% CI: 96.0 - 99.5%)
• NPA: 99.5% (548/551) (95% CI: 98.4 - 99.8%) |
  | Prospective Clinical Performance - S. lugdunensis | PPA ≥ 95%, NPA ≥ 95% (implied). | - PPA: 100.0% (3/3) (95% CI: 43.9 - 100%)
• NPA: 99.9% (761/762) (95% CI: 99.3 - 99.9%) |
  | Prospective Clinical Performance - Other Staphylococcus spp. | PPA ≥ 95%, NPA ≥ 95% (implied). | - PPA: 98.9% (444/449) (95% CI: 97.4 - 99.5%)
• NPA: 97.2% (307/316) (95% CI: 94.7 - 98.5%) |
  | Prospective Clinical Performance - mecA with S. aureus | PPA ≥ 90%, NPA ≥ 98% (implied). | - PPA: 94.4% (68/72) (95% CI: 86.6 - 97.8%)
• NPA: 98.8% (682/690) (95% CI: 97.7 - 99.4%) |
  | Prospective Clinical Performance - mecA with Other Staphylococcus spp. | PPA ≥ 90%, NPA ≥ 95% (implied). | - PPA: 92.7% (243/262) (95% CI: 88.1 - 97.1%)
• NPA: 96.2% (481/500) (95% CI: 92.4 - 98.0%) |
  | Invalid Rate - Clinical | Acceptable low rate, resolvable on retest. | Initial Invalid Rate: 1.39% (11/789). Final Invalid Rate: 0.00% after retest. |
  | Abort Rate - Clinical | Acceptable low rate, resolvable on retest. | Initial Abort Rate: 3.30% (26/789). Final Abort Rate: 0.00% after retest. |

2. Sample Size Used for the Test Set and Data Provenance

• Prospective Clinical Study:

  • Sample Size: A total of 762 prospective specimens were tested. Additionally, 69 archived frozen specimens were tested after the prospective study, making the overall clinical sample set significantly larger. A 'Low Prevalence' panel of 102 contrived or 'simulated' blood culture specimens was also tested.
  • Data Provenance: The prospective specimens were collected at three geographically diverse U.S. sites.

• Analytical Studies (LoD, Inclusivity, Exclusivity, Interference, Bottle Types, Reproducibility):

  • These studies used various numbers of cultured bacterial strains (ATCC, NCTC, CCUG, Clinical isolates where specified) and simulated blood culture specimens, often with multiple replicates per strain (ranging from 2 to 20+ replicates per strain for LoD, 2 replicates per strain for exclusivity, and 90 replicates per analyte for reproducibility). The specific number of unique strains and replicates varies per study as detailed in the tables (e.g., 22 strains for LoD, 116 off-panel strains for exclusivity, 7 simulated blood culture specimens for reproducibility).
  • Data Provenance: Not explicitly stated for each analytical study, but context suggests laboratory-controlled experiments. Strains were from recognized culture collections (ATCC, NCTC, CCUG) or clinical sources.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts

The document does not specify the number of experts or their qualifications used to establish the ground truth for the clinical or analytical test sets.

For the clinical study, the ground truth (Staphylococcus species identification and mecA gene presence) was established using "Reference Method(s)". While not explicitly defined, these reference methods typically involve conventional microbiology techniques such as:

• Sub-culturing of positive blood cultures.
• Phenotypic identification (e.g., Gram stain, catalase, coagulase tests, biochemical panels).
• Molecular methods (e.g., gene sequencing, PCR assays with a different target).
• Antimicrobial susceptibility testing (e.g., oxacillin MIC) to confirm methicillin resistance (though the device detects the mecA gene, not phenotypic resistance itself).

For analytical studies, bacterial strains from recognized collections (ATCC, NCTC, CCUG) and presumably well-characterized clinical isolates were used, implying their identity and mecA status were already established by standard microbiological and genetic methods.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method (like 2+1 or 3+1) for the clinical test set. The comparison is made directly against "Reference Method(s)". For discrepant results in the exclusivity and microbial interference studies, repeat testing was performed (e.g., a minimum of six (6) repeat tests for 'Staphylococcus POSITIVE' calls in exclusivity, sometimes at higher CFU/mL input for interference studies to resolve miscalls).

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not conducted. This device is an automated in vitro diagnostic assay (algorithm only), not an AI-assisted human reading system. Therefore, there is no human-in-the-loop performance to measure improvement with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done

Yes, the studies conducted demonstrate the standalone performance of the Great Basin Staph ID/R Blood Culture Panel. This device is an automated, qualitative, molecular diagnostic system, meaning its output is generated by the instrument itself (the Portrait Analyzer and its software) without direct human interpretation of complex images or data that would typically benefit from AI assistance. The performance metrics (PPA, NPA) directly reflect the device's accuracy against reference methods.

7. The Type of Ground Truth Used

The primary ground truth for both analytical and clinical performance studies was established through reference microbiology methods. This includes:

• Culture-based identification: Sub-culturing, Gram staining, and phenotypic tests for species identification.
• Molecular identification: Implied use of validated molecular techniques or genetic characterization (e.g., for mecA gene presence and confirmation of species for various strains).
• Antimicrobial Susceptibility Testing (AST): Specifically, oxacillin MIC was used in the Analytical Reactivity (Inclusivity) Challenge Panel to determine phenotypic resistance, which indirectly supports the genotypic mecA detection.
• For the analytical studies, well-characterized strains from culture collections (ATCC, NCTC, CCUG) served as a strong foundation for ground truth.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of device development or any specific AI/machine learning models. This is typical for traditional molecular diagnostic assays, where assay design (primer/probe sequences, reaction conditions) is based on known genetic targets and verified through extensive analytical validation rather than data-driven machine learning training. The analytical studies (LoD, inclusivity, exclusivity) serve to validate the analytical performance of the developed assay against a wide range of relevant organisms and conditions.

9. How the Ground Truth for the Training Set Was Established

As no explicit "training set" is described for an AI/ML model, this question is not directly applicable. If one considers the development of the assay (design of specific DNA targets and hybridization probes) as analogous to "training," then the ground truth would have been established through extensive molecular biology research, genetic sequencing, and characterization of Staphylococcus species and the mecA gene. This would involve identifying conserved and variable regions of target genes across a broad collection of known Staphylococcus strains to ensure specificity and sensitivity.

Ask a specific question about this device

The Great Basin Shiga Toxin Direct Test performed on the Portrait™ Analyzer is an automated, in vitro diagnostic assay for the qualitative detection of Shiga toxin 1 (stxl) / Shiga toxin 2 (stx2) genes and specific identification of a conserved genetic region of the E. coli O157 serogroup. Shiga toxin genes are found in Shiga toxin-producing strains of E. coli (STEC) and Shigella dysenteriae. The E. coli 0157 test result is reported only if a Shiga toxin gene is also detected.

The test is performed directly from Cary-Blair or C&S Medium preserved stool specimens from symptomatic patients with suspected acute gastroenteritis, or colitis in hospital laboratories. The assay is intended for use in conjunction with clinical presentation as an aid in the diagnosis of STEC infections. Positive results do not rule out oninfection with other organisms, and may not be the definitive cause of patient illness.

The results of this test should not be used as the sole basis for diagnosis, treatment, or other patient management decisions. Shiga Toxin Direct Test negative results in the setting of clinical illness compatible with gastroenteritis may be due to infection by pathogens that are not detected by this test or non-infectious causes such as ulcerative colitis, irritable bowel syndrome, or Crohn's disease.

The Portrait System is a fully automated system that includes the Portrait Analyzer, single-use Great Basin Shiga Toxin Direct Test cartridges, and the Portrait System data analysis software. The Portrait System is designed to perform automated sample preparation, PCR, and optical chip-based detection with integrated data analysis in approximately 2 hours.

The single-use Test Cartridge contains blister packs, fluidic channels, processing chambers, a waste chamber, and an assay chip coated with an array of sequence-specific detection probes. All reagents are contained within the integrated blister packs with the exception of the amplification reagents and SPC, which are dried into the Amplification Chamber and SPC Chambers of the Cartridge, respectively.

The appropriate specimen for use in the Test Cartridge is an aliquot of stool from symptomatic patients preserved in Cary-Blair or C&S transport media. A preserved stool specimen is placed into the sample port of the Test Cartridge for processing. Multiple fluidic channels move reagents from integrated blister packs to chambers where reagent mixing and sample processing occur. A waste chamber, self-contained and segregated within the Test Cartridge, collects and stores reagent waste.

Here's a breakdown of the acceptance criteria and study details for the Great Basin Shiga Toxin Direct Test, as extracted from the provided text:

1. Table of Acceptance Criteria and Reported Device Performance:

The document doesn't explicitly state "acceptance criteria" as a separate, pre-defined set of values that the device must meet to be approved. Instead, it presents the clinical performance (sensitivity, specificity, PPA, NPA) as results from the studies, which are then compared to a predicate device (though specific performance metrics for the predicate are noted as "Not Reported").

However, based on the performance data presented, here's a summary of the device's reported performance which implicitly serve as the achieved acceptance level, especially when compared against itself in different study types. The comparison chart with the predicate device also indicates a qualitative "same" for intended use and target sequences which are fundamental acceptance points.

2. Sample Size Used for the Test Set and Data Provenance:

• Prospective Clinical Study:

  • Sample Size: 1,082 clinical specimens.
  • Data Provenance:
    • Origin: Collected prospectively (fresh) at five sites.
    • Retrospective/Prospective: Prospective.
    • Timeframe: June to September 2015 (three-month period).

• Frozen Retrospective Clinical Study:

  • Sample Size: 88 unique frozen clinical specimens (from an initial panel of 92).
  • Data Provenance:
    • Origin: Previously characterized clinical specimens.
    • Retrospective/Prospective: Retrospective.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:

The document does not specify the number of experts used or their qualifications to establish the ground truth for the clinical test sets. It mentions "reference clinical microbiology protocols" for the prospective study and "Clinical Characterization - Molecular and/or Shiga Toxin EIA" for the retrospective study as the basis for ground truth.

4. Adjudication Method for the Test Set:

The document does not detail any specific adjudication method (e.g., 2+1, 3+1). It states that the device's performance was compared to "reference clinical microbiology protocols" and "Molecular and/or Shiga Toxin EIA" results. For discrepant results in the prospective study, it notes:

• "Shiga toxin was detected in 8/8 false positive specimens by both bi-directional sequencing and alternate, FDA-cleared comparator NAAT."
• "O157 serogroup was detected in 2/2 false positive specimens by alternate, FDA-cleared comparator NAAT."
  This describes a characterization process for disagreements rather than a multi-expert adjudication method for the initial ground truth.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance:

This device is an in-vitro diagnostic assay for direct detection of nucleic acids, not an imaging device or an AI-assisted diagnostic tool that involves human readers. Therefore, an MRMC comparative effectiveness study involving human readers and AI assistance is not applicable and was not performed.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done:

Yes, the studies evaluate the Great Basin Shiga Toxin Direct Test (a molecular diagnostic assay run on an automated system) as a standalone device. The device's output (qualitative detection of Shiga toxin genes and E. coli O157) is the primary subject of the performance evaluation. It is an automated, in-vitro diagnostic assay.

7. The Type of Ground Truth Used:

• Prospective Clinical Study: "Reference clinical microbiology protocols for the detection of both Shiga Toxin and the E. coli O157 Serotype." This typically involves culture, immunoassay (EIA), and potentially molecular methods. For discrepant results, bi-directional sequencing and FDA-cleared comparator NAAT were used for confirmation.
• Frozen Retrospective Clinical Study: "Clinical Characterization - Molecular and/or Shiga Toxin EIA."

8. The Sample Size for the Training Set:

The document does not explicitly mention a "training set" in the context of machine learning or AI algorithm development because this is a molecular diagnostic test. However, the document details various analytical studies which are foundational to the device's design and calibration. These studies use panels of known strains and concentrations:

• Analytical Sensitivity (LoD): Involved testing specific E. coli strains (ATCC BAA-2191, ATCC 51434, ATCC BAA-2192, ATCC 43895) at varying concentrations (e.g., 25/25, 21/22, 26/26, 20/20 correct results for LoD determination, indicating numerous replicates).
• Analytical Reactivity (Inclusivity): Tested 33 known positive strains (30 STEC, 3 Shigella dysenteriae) with 3 replicates each, totaling 99 tests.
• Analytical Specificity (Exclusivity): Tested 118 microorganisms (bacteria, fungi, yeasts, parasites, viruses) and human genomic DNA, each typically with 3 replicates (e.g., 3/3 results mentioned frequently).
• Microbial Interference: Tested 42 microorganisms, each typically with 3 or more replicates, against two STEC strains (ATCC 43895 and ATCC 43894) at 2X LoD.
• Interfering Substances: Tested 26 different substances, each with 3 replicates, in both positive and negative stool matrices.

9. How the Ground Truth for the Training Set Was Established:

For these analytical studies, the ground truth was established by using well-characterized reference strains (e.g., ATCC strains) with known genetic profiles (presence of stx1, stx2, O157 genes) and established concentrations. This allows for precise control and verification of the device's ability to detect specific targets and its limits of detection, and to ensure it does not cross-react with non-targets. For other studies like specimen stability, known positive samples were contrived by spiking these characterized strains into negative stool matrix.

Ask a specific question about this device