Search Results

The LIAISON PLEX® Gram-Positive Blood Culture Assay (BCP), performed using the automated, sample-to-result LIAISON PLEX® System, is a qualitative multiplexed in vitro diagnostic test for the simultaneous detection and identification of selected gram-positive pathogens and/or selected genetic determinants associated with antimicrobial resistance in positive blood culture bottles. BCP is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system and which contain gram-positive bacteria as determined by Gram stain.

The BCP Assay detects and identifies the following:

Gram Positive Resistance Markers:

• mecA/mecC
• vanA
• vanB

Genera and Species:

• Bacillus spp.
• Enterococcus faecalis
• Enterococcus faecium
• Listeria spp.
• Staphylococcus spp.
• Staphylococcus aureus
• Staphylococcus epidermidis
• Staphylococcus lugdunensis
• Streptococcus spp.
• Streptococcus agalactiae
• Streptococcus anginosus group
• Streptococcus pneumoniae
• Streptococcus pyogenes

Negative results for antimicrobial resistance genes do not indicate bacterial susceptibility as there are multiple mechanisms that can contribute to resistance.

The LIAISON PLEX® BCP Assay contains targets for the detection of genetic determinants associated with resistance to methicillin (mecA/C) and vancomycin (vanA and vanB) to aid in the identification of potentially antimicrobial-resistant organisms in positive blood culture samples. In mixed growth, the LIAISON PLEX BCP Assay does not specifically attribute vanA/vanB-mediated vancomycin resistance to either E. faecalis or E. faecium, or mecA/mecC-mediated methicillin resistance to either Staphylococcus spp., S. aureus, S. epidermidis or S. lugdunensis.

The antimicrobial resistance gene or marker detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of methicillin and vancomycin resistance exist.

The LIAISON PLEX® BCP Assay is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections (BSI). The LIAISON PLEX® BCP Assay is not intended to monitor these infections. Sub-culturing of positive blood cultures is necessary to recover organisms for antimicrobial susceptibility testing (AST), for identification of organisms not detected by the LIAISON PLEX BCP Assay, to detect mixed infections that may not be detected by the LIAISON PLEX BCP Assay, for association of antimicrobial resistance genes to a specific organism, or for epidemiological typing.

The LIAISON PLEX® Gram-Positive Blood Culture Assay (BCP Assay) is an automated test for the detection and identification of nucleic acid from gram-positive bacteria in a positive blood culture media sample. The BCP Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain gram-positive bacteria, as determined by a Gram stain.

The LIAISON PLEX® System is a fully automated, bench-top "sample-to-answer" device that performs sample preparation and microarray-based hybridization for the detection of target-specific nucleic acids. The test reagents are supplied as a single, disposable test cartridge. Target amplification is not performed as part of the BCP Assay workflow, as it is a non-amplified, direct detection test performed on the LIAISON PLEX® System.

The provided text describes the acceptance criteria and the study that proves the LIAISON PLEX® Gram-Positive Blood Culture Assay meets those criteria. However, it's important to note that this document is a 510(k) Clearance Letter and 510(k) Summary, which focuses on demonstrating substantial equivalence to a predicate device for regulatory clearance. It is not an academic paper detailing a clinical trial with specific acceptance criteria that would typically be associated with AI/ML diagnostic performance metrics like those in a multi-reader multi-case (MRMC) study or the establishment of ground truth by multiple experts.

The study described here is for an in vitro diagnostic test for the detection of nucleic acid sequences, not for an AI/ML-based diagnostic imaging device. Therefore, many of the requested points (e.g., number of experts for ground truth, adjudication method, MRMC study, human reader improvement with AI assistance) are not applicable to this type of device and study.

I will interpret the provided information in the context of the device described (a multiplex nucleic acid assay) and its verification and validation.

Acceptance Criteria and Device Performance

The acceptance criteria for this device are implicitly derived from the performance goals demonstrated in the analytical and clinical studies, aiming for high sensitivity and specificity in detecting specified microorganisms and resistance markers. The reported performance is presented in several tables throughout the document.

Table 1: Acceptance Criteria (Implicit) and Reported Device Performance

2. Sample Sizes and Data Provenance

• Test Set (Clinical Study):
  • Prospective Arm (Arm 1): 562 unique specimens enrolled, 509 included in analysis.
  • Pre-selected Arm (Arm 2): 163 pre-selected left-over specimens, 162 included in analysis.
  • Contrived Arm (Arm 3): 225 contrived specimens.
  • Total specimens analyzed for clinical performance: 509 (prospective) + 162 (pre-selected) + 225 (contrived) = 896 unique samples.
• Data Provenance:
  • Prospective Arm: Collected between April 2024 and August 2024 from four geographically diverse clinical sites within the United States. Data is prospective.
  • Pre-selected Arm: Sourced from ten vendors in the United States and one site in Italy. Data are retrospective (left-over, de-identified specimens).
  • Contrived Arm: Specimens were prepared, blinded, randomized, and tested at three external testing sites and one internal testing site between June 2024 to September 2024. These are contrived (synthetic) samples.

3. Number of Experts and Qualifications for Ground Truth

• Not Applicable in the traditional sense for this IVD device. The ground truth for this nucleic acid assay is established through reference methods such as culture with automated microbiological/biochemical identification (VITEK 2), PCR followed by bi-directional sequencing (BDS), or a combination thereof. This is a laboratory-based diagnostic test, not an imaging device requiring human expert interpretation for ground truth.

4. Adjudication Method for the Test Set

• Not Applicable in the traditional sense. The "adjudication" for the gold standard (reference method) involves a hierarchical algorithm (Table 19). If initial VITEK 2 results were insufficient or for specific targets like Bacillus spp. and resistance markers (mecA/mecC, vanA, vanB), PCR followed by bi-directional sequencing was used. This is a technical validation process against established laboratory methods rather than a consensus among clinical experts interpreting an output.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Not done. This type of study (MRMC) is typically used for diagnostic imaging devices where human readers interpret images with and without AI assistance. This device is a molecular diagnostic assay that provides direct results, and therefore, an MRMC study is not relevant.

6. Standalone (i.e., algorithm only without human-in-the-loop performance)

• Yes, in essence. The LIAISON PLEX® BCP Assay is an automated, sample-to-result system. Its performance (sensitivity, specificity) is evaluated independently against a reference method. While trained operators are involved in loading samples and running the system, the diagnostic result itself is generated by the "system" (instrument + assay + software algorithm) without human diagnostic interpretation of the raw signal data.

7. The Type of Ground Truth Used

• Laboratory Reference Methods:
  • Culture followed by Automated microbiological/biochemical identification using VITEK 2 for most bacterial species.
  • PCR followed by bi-directional sequencing (BDS) for Bacillus spp. and resistance markers (mecA/mecC, vanA, vanB). This method was also used to confirm discordant results or for targets with low prevalence in initial testing.
  • In some cases, MALDI-ToF assay (Matrix-Assisted Laser Desorption/Ionization-Time Of Flight) was also mentioned as a Standard of Care method in the footnotes for discordant result explanations.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set" for an AI/ML model. This device is a direct detection assay based on hybridization, not a machine learning model that undergoes a training phase with a distinct dataset. Therefore, the concept of a "training set" in the context of data used to train an AI model is not applicable to this traditional IVD. The development process would involve extensive analytical characterization, probe design, and optimization using various microbial strains and clinical samples, but this is distinct from "training data" for an AI algorithm.

9. How the Ground Truth for the Training Set Was Established

• Not applicable for this type of device. As explained above, this is not an AI/ML device that requires a "training set" with ground truth established in the AI/ML context. The assay's design and optimization would rely on established microbiological methods and genetic sequencing to define target sequences and ensure specificity and inclusivity.

Ask a specific question about this device

The GenMark ePlex Blood Culture Identification Gram-Positive (BCID-GP) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous qualitative detection and identification of multiple potentially pathogenic gram-positive bacterial organisms and select determinants associated with antimicrobial resistance in positive blood culture. In addition, the ePlex BCID-GP Panel is capable of detecting a wide variety of gram-negative bacteria (Pan Gram-Negative assay) and several Candida species (Pan Candida assay). The ePlex BCID-GP Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain gram-positive organism.

The following bacterial organisms and genes associated with antibiotic resistance are identified using the ePlex BCID-GP Panel: Bacillus cereus group, Bacillus subtilis group, Corynebacterium, Cutibacterium acnes (Propionibacterium acnes), Enterococcus, Enterococcus faecalis, Enterococcus faecium, Lactobacillus, Listeria monocytogenes, Micrococcus, Staphylococcus, Staphylococcus aureus, Staphylococcus epidermidis, Staphylococcus lugdunensis, Streptococcus agalactiae (GBS), Streptococcus anginosus group, Streptococcus pneumoniae, Streptococcus pyogenes (GAS), mecA, mecC, vanA and vanB.

The ePlex BCID-GP Panel contains assays for the detection of genetic determinants associated with resistance to methicillin (mecA and mecC) to aid in the identification of potentially antimicrobial resistant organisms in positive blood culture samples. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease.

The ePlex BCID-GP Panel also contains targets designed to detect a broad range of organisms with a potentially misleading Gram stain result or organisms that may be missed by Gram staining altogether, for example in the case of co-infections. These include a broad Pan Gram-Negative assay as well as a Pan Candida assay, which is designed to detect four of the most prevalent Candida species: Candida albicans, Candida glabrata, Candida krusei and Candida parapsilosis.

The detection and identification of specific bacterial and fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-GP Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

Negative results in the setting of a suspected bloodstream infection may be due to infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-GP Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-GP Panel and for susceptibility testing. differentiation of mixed growth and association of antimicrobial resistance marker genes to a specific organism) and clinical presentation must be taken into consideration in the final diagnosis of blood stream infection.

The ePlex Blood Culture Identification Gram-Positive (BCID-GP) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.

Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.

During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.

Here's an analysis of the provided text to extract information about the device's acceptance criteria and the study proving it meets them.

The document is a 510(k) Premarket Notification from the FDA regarding the GenMark ePlex Blood Culture Identification Panel - Gram Positive (BCID-GP) Panel (K181663). The core of the performance data revolves around clinical performance (sensitivity/PPA and specificity/NPA) against comparator methods, and analytical performance (Limit of Detection, Inclusivity, Exclusivity, Reproducibility, Interfering Substances, Competitive Inhibition).

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the clinical and analytical performance targets the device aimed to meet, typically demonstrating non-inferiority or substantial equivalence to a predicate device and adequate analytical performance. The tables provided (Table 6 through Table 30 for clinical performance, and Table 61 onwards for reproducibility) present the device's reported performance. A formal table of acceptance criteria with corresponding results isn't explicitly laid out with a "criteria" column alongside "performance" in the document, but we can infer the criteria from the achieved results and the FDA clearance itself, which implies acceptable performance.

For this device, the primary acceptance criteria would be high Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) with established comparator methods for all targeted organisms and resistance genes, along with robust analytical performance.

Here's a summary of the reported performance, reflecting the device meeting implicit acceptance criteria for clinical accuracy:

Table 1: Derived Acceptance Criteria and Reported Device Performance (Selected Examples)

Detailed Information about the Study:

1. A table of acceptance criteria and the reported device performance

   • See the table above for inferred criteria and reported performance based on clinical PPA/NPA. The acceptance criteria are not explicitly stated as quantitative thresholds in the provided text, but the successful clearance of the device implies that the observed performance was deemed acceptable by the FDA.

2. Sample sizes used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

   • Test Set (Clinical Study):
     • Prospective Samples: Total 711 evaluable samples. Collected from 7 clinical sites. 312 were tested fresh (Jan-Feb 2018), and 399 were frozen (June 2014-July 2016).
     • Retrospective Samples: Total 586 evaluable samples. Collected to supplement low prevalence targets.
     • Combined Clinical Samples: 1297 (711 prospective + 586 retrospective).
     • Contrived Samples: 565 samples prepared by spiking isolates into blood culture bottles.
   • Data Provenance: The document states "A prospective, multicenter clinical study was conducted at 7 clinical sites." While specific countries are not mentioned, FDA clearances typically involve studies predominantly conducted in the USA or with data acceptable to US regulatory standards. The terms "multicenter" and "clinical sites" imply different geographical locations although not explicitly named.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

   • The ground truth for the clinical test set was established using "Standard laboratory procedures" including traditional and automated culture, MALDI-TOF IVD, and microbiological and biochemical techniques for organism identification. Specific PCR assays followed by bi-directional sequencing (or 16S sequencing for Corynebacterium, Staphylococcus epidermidis, Candida parapsilosis) were used for confirmation where necessary. For antibiotic resistance genes (mecA, mecC, vanA, vanB), ground truth was established using analytically validated qPCR amplification assays followed by bi-directional sequencing.
   • The document does not specify the number of experts or their qualifications (e.g., board-certified microbiologists, medical technologists with X years of experience) who performed these standard laboratory procedures or interpreted the sequencing results to establish ground truth.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

   • The document describes a comparison between the ePlex BCID-GP Panel results and "comparator method(s) results." For discrepant results (e.g., false positives, false negatives), additional analytically validated PCR assays and sequencing were used for confirmation (e.g., for Corynebacterium, S. epidermidis, Pan Candida, Staphylococcus for mecA, and Enterococcus for vanA/vanB).
   • This is a form of adjudication, where discrepant results are further investigated. However, it's not a multi-reader adjudication method (like 2+1, 3+1) involving multiple human readers interpreting results, as this device performs automated detection. Instead, it's a discrepancy resolution method using a higher-level, analytically validated diagnostic test (PCR/sequencing) as the tie-breaker/confirmatory method.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

   • No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic test for identifying microorganisms and resistance markers, not an AI-assisted diagnostic imaging or interpretation tool for human readers. Its performance is evaluated against laboratory standard reference methods, not against human reader performance or improvement with AI assistance.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

   • Yes, the primary clinical and analytical performance studies are standalone (algorithm only) performance. The ePlex BCID-GP Panel is an automated test on the ePlex Instrument. Its results are generated by the device's assay and software, and then compared directly to the ground truth established by comparator laboratory methods. There is no explicit human-in-the-loop component described for its diagnostic performance evaluation, although clinical interpretation of the results by medical professionals is clearly indicated as necessary for patient management decisions (as stated in the Indications for Use).

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

   • The ground truth for the clinical study was based on "Standard laboratory procedures" for organism identification (culture, MALDI-TOF IVD, microbiological/biochemical techniques) and analytically validated PCR assays with bi-directional sequencing for confirmation of specific organisms and all resistance genes. This would fall under laboratory reference standard/molecular confirmation. Clinical outcomes data was not primarily used for ground truth.

8. The sample size for the training set

   • The document does not specify a separate "training set" size for the development of the ePlex BCID-GP Panel's algorithm. For in vitro diagnostic devices like this one, the development process might involve internal validation and optimization, but the regulatory submission focuses on the performance of the final, locked version of the device using a clinically representative "test set." The text primarily describes the clinical evaluation (prospective, retrospective, contrived samples) which serves as the independent test set for regulatory submission.

9. How the ground truth for the training set was established

   • As a distinct "training set" is not explicitly mentioned as per typical machine learning contexts, the document doesn't detail how ground truth for such a set was established. It's plausible that internal development and optimization would have used similar laboratory reference methods as those used for the clinical validation.

Ask a specific question about this device

The iCubate iC-GPC Assay for use on the iC-System is a qualitative, multiplexed, in vitro diagnostic test for the detection and identification of potentially pathogenic gram positive bacteria, which may cause bloodstream infection (BSI). The iC-GPC Assay is performed directly on positive blood cultures, confirmed by Gram stain to contain gram positive cocci. Cultures demonstrating mixed Gram stain results should not be tested with the assay. The iC-GPC Assay is validated for use with select BACTEC, BacTIALERT and VersaTREK blood culture bottles. The iC-GPC Assay is indicated for use in conjunction with other clinical and laboratory findings, such as blood culture isolate identification and antimicrobial susceptibility testing, to aid in the diagnosis of bacterial bloodstream infections; however, it is not used to monitor bloodstream infections.

The iC-GPC Assay detects organism DNA and identifies the following bacterial species and resistance markers:

Bacterial Species Staphylococcus aureus Staphylococcus epidermidis Streptococcus pneumoniae Enterococcus faecalis Enterococcus faecium

Resistance Markers

mecA- associated with methicillin resistance vanA- associated with vancomycin resistance vanB- associated with vancomycin resistance

The iC-GPC Assay detects the mecA resistance marker, inferring mecA-mediated methicillin resistance, and the vanA and vanB resistance markers, inferring vanAlvanB-mediated vancomycin resistance. In mixed growth, the iC-GPC Assay does not specifically attribute van-mediated vancomycin resistance to either E. faecalis or E. faecium, or mecA-mediated methicillin resistance to either S. aureus or S. epidermidis.

Sub-culturing of positive blood cultures is necessary to recover organisms for susceptibility testing. identification of organisms not detected by the iC-GPC Assay, differentiation of mixed growth, association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.

The iC-GPC Assay utilizes polymerase chain reaction (PCR) for the amplification of specific targets and detects the amplified target DNA with fluorescence-based microarray hybridization. The iC-GPC Assay uses proprietary ARM-PCR (Amplicon Rescued Multiplex PCR) technology allowing for multiple targets to be amplified in one reaction. Targets are detected directly from patient positive blood cultures, confirmed by Gram stain to contain gram positive cocci. Testing is performed within a closed, disposable cassette that contains all the reagents required to complete the iC-GPC Assay, including a universal microarray.

To operate, the user opens the iC-GPC Cassette cap and pipettes an aliquot of the positive blood culture sample into the sample well of the cassette is then inserted into the iC-Processor, which performs the processes of DNA extraction, multiplex amplification, and microarray After processing is complete, the cassette is inserted into the iC-Reader for hybridization. fluorescence-based detection and data analysis. Final results are generated by iC-Report, computer software for data acquisition, analysis, and display.

Extraction, amplification, and hybridization are defined by an assay script controlled by the iC-Processor. The processing script is defined within a barcode label positioned on the top of each iC-GPC Cassette which communicates with the iC-Processor. To access and pierce the foil-sealed reagent wells located in the bottom well plate of the cassette, the processor manipulates the cassette to move the internal pipette horizontally and vertically. The script directs the transfer of reagents between the wells in the bottom well plate and finally to the array within the cassette. The iC-Processor is capable of processing 4 iC-Cassettes with random access.

Once processing is complete, the cassette is manually transferred from the iC-Processor to the iC-Reader where the microarray within the cassette is read. The iC-Reader is capable of reading up to 4 iC-Cassettes at one time. The results are interpreted via the iC-Report software and displayed for the user on an iMac computer. Raw data and result interpretations are stored within the iMac; raw data is accessible to authorized personnel only and is not available to the end user.

The provided document describes the iC-GPC Assay for use on the iC-System, a qualitative, multiplexed, in vitro diagnostic test for the detection and identification of potentially pathogenic gram-positive bacteria and resistance markers from positive blood cultures. The document details the performance studies conducted to support its substantial equivalence to a legally marketed predicate device (VERIGENE® Gram Positive Blood Culture Nucleic Acid Test (BC-GP)) and its overall analytical and clinical performance.

Here's an breakdown of the acceptance criteria and the studies proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly state "acceptance criteria" in a single, consolidated table with pre-defined thresholds. Instead, it presents performance metrics in various tests (reproducibility, LoD, inclusivity, exclusivity, method comparison). Acceptance for these tests is implied by the successful outcomes. The core performance is summarized in the method comparison study against an FDA-cleared multiplex assay and against traditional culture and susceptibility methods.

Below, I've compiled a table summarizing the key performance metrics from the Method Comparison section, which represents the device's main performance claim for its intended use. The "Acceptance Criteria" here are implicitly High agreement (e.g., >95%) for both positive and negative agreement, as is common for in vitro diagnostics devices aiming for substantial equivalence.

2. Sample Size and Data Provenance

• Test Set Sample Size:
  • Clinical Method Comparison Study: 966 positive blood culture specimens initially enrolled, reduced to 913 (879 fresh prospective, 34 frozen prospective) after withdrawals.
  • Contrived Samples: An additional 168 contrived samples.
• Data Provenance: The general text of the letter indicates "four geographically dispersed clinical sites." Given the context of a US regulatory submission to the FDA, it is highly likely these sites were in the United States. The "Expected Values" table further specifies "U.S. State" data from NY, WI, NM, and FL. The study collected prospective (fresh and frozen) and contrived samples.

3. Number of Experts and Qualifications for Ground Truth

The document does not explicitly state the "number of experts" or their "qualifications" for establishing the ground truth for the clinical method comparison test set.

However, for the comparison methods that served as ground truth:

• FDA-cleared multiplex assay: This is a pre-existing, validated device, implying its results are considered reliable.
• Conventional reference methods: This includes "subculture, identification of isolates using biochemical methods or MALDI-TOF, and phenotypic antimicrobial susceptibility testing (AST)." These methods are standard clinical laboratory practices performed by trained microbiologists and clinical laboratory scientists. The qualification of these individuals would adhere to standard laboratory accreditation and personnel requirements.

4. Adjudication Method for the Test Set

The document details discordant analysis for the method comparison studies:

• Against FDA-Cleared Multiplex Assay: PCR and bidirectional sequencing were performed for all specimens with positive results for resistance markers by either the iC-GPC Assay or the FDA-cleared multiplex assay. These results were used to analyze and investigate discordant results. This implies a form of 2+1 adjudication or investigation where if the iC-GPC and comparator disagreed, an independent gold standard (PCR/sequencing) was used to resolve.
• Against Conventional Reference Methods: PCR and bi-directional sequencing were not used for discordant analysis between iC-GPC and conventional reference method results. This suggests that the conventional methods served as the direct reference, without further independent adjudication for discrepancies with the iC-GPC assay.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

This is not a multi-reader multi-case (MRMC) study. The iC-GPC Assay is an in vitro diagnostic device for laboratory use that provides automated results, not a diagnostic imaging AI algorithm that assists human readers. Therefore, there is no human-in-the-loop performance measurement or effect size reported here regarding human reader improvement with AI assistance.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

Yes, this entire submission revolves around the standalone performance of the iC-GPC Assay, which is an automated system (iC-System) that performs DNA extraction, multiplex amplification, hybridization, and detection to generate results. The reported performance metrics (positive/negative agreement, LoD, inclusivity, exclusivity) are a direct measure of the algorithm's (device's) performance.

7. Type of Ground Truth Used

• For Analytical Studies (LoD, Inclusivity, Exclusivity, Reproducibility, etc.): Ground truth was established by known concentrations/strains of organisms, obtained from characterized sources (e.g., ATCC strains, well-characterized clinical isolates) and confirmed by methods such as colony counts.
• For Clinical Method Comparison Study:
  • Primary Ground Truth 1: An FDA-cleared multiplex assay.
  • Secondary Ground Truth 2: Conventional laboratory reference methods (subculture, biochemical identification, MALDI-TOF, and phenotypic antimicrobial susceptibility testing (AST)).
  • Discordant Resolution (for resistance markers in comparison to FDA-Cleared Multiplex Assay): PCR and bi-directional sequencing as an independent, highly accurate molecular method.

8. Sample Size for the Training Set

The document does not provide information on the training set size for the iC-GPC Assay's development. This FDA 510(k) submission focuses on the validation of the device through performance studies (test sets) rather than detailing the specifics of its development or "training" phase. For in vitro diagnostics that use molecular techniques, the "training" analogous to machine learning models usually involves extensive research and development to optimize primer and probe design, reaction conditions, and analytical algorithms based on large collections of known positive and negative samples, but these are typically internal R&D activities not explicitly reported as "training set size" in the final validation summary.

9. How the Ground Truth for the Training Set Was Established

As the document does not detail a "training set," it also does not describe how its ground truth was established. However, for a molecular diagnostic assay like this, the underlying "ground truth" for developing and optimizing the assay's targets and reaction conditions would typically be established through:

• Known, characterized bacterial strains and clinical isolates: Identified and confirmed using a combination of standard microbiological methods (culture, Gram stain, biochemical tests, phenotypic susceptibility testing) and advanced molecular methods (16S rRNA gene sequencing, whole-genome sequencing) to determine precise species and resistance marker presence.
• Synthetic DNA/RNA targets: Used for initial assay design and analytical performance testing.
• Bioinformatics analysis: To select highly specific gene targets (like gseA, nuc, lytA, ddl, fcm, mecA, vanA, vanB) that are unique to the target organisms and resistance mechanisms.

This developmental phase would involve rigorous, iterative testing and refinement against a comprehensive panel of both target and non-target organisms to ensure high sensitivity and specificity before formal validation studies like those presented in the 510(k) summary are conducted.

Ask a specific question about this device

The Great Basin Staph ID/R Blood Culture Panel is a qualitative, multiplex, nucleic acid-based in vitro diagnostic assay intended for the simultaneous identification of nucleic acid from Staphylococcus lugdumensis and various Staphylococcus species to the genus level and the detection of the mecA gene for methicillin resistance directly from patient positive blood culture specimens. The test utilizes automated hot-start enabled polymerase chain reaction (PCR) for the amplification of specific DNA targets detected by hybridization probes immobilized on a silicon chip surface. The assay is performed directly on positive blood culture specimens identified as positive by continuous monitoring blood culture system that demonstrates the presence of organisms as determined by Gram stain to contain gram-positive cocci in clusters (GPCC) or gram-positive cocci in singles (GPC). The test may be performed using blood culture bottles. The Staph ID/R Blood Culture Panel identifies Staphylococcus aureus (SA), and Staphylococcus lugdunensis, and detects other Staphylococcus species without identification to species level.

The Portrait Staph ID/R Blood Culture Panel is indicated for use in conjunction with other clinical or laboratory findings to aid in the diagnosis of bacterial bloodstream infections; however, it is not used to monitor these infections. Subculturing positive blood cultures is necessary to recover viable organisms for further identification, susceptibility testing, or epidemiological typing to identify organisms in the blood culture that are not detected by the Great Basin Staph ID/R Blood Culture Panel. If detected, mecA may or may not be associated with Staphylococcus spp. detected or the agent responsible for the disease. Negative results for mecA antimicrobial resistance gene assays do not always indicate susceptibility, as other mechanisms of resistance to methicillin exist.

The Great Basin PA500 Portrait™ Analyzer System is a fully automated system that includes the Portrait Analyzer, single-use Staph ID/R Blood Culture Panel Test Cartridges, and the Portrait data analysis software. The PA500 Portrait Analyzer System is designed to perform automated sample preparation, PCR, and optical chip-based detection with integrated data analysis in approximately 110 minutes.

The Great Basin Staph ID/R Blood Culture Panel is a qualitative, multiplex, nucleic acid-based in vitro diagnostic assay intended for the simultaneous identification of nucleic acid from Staphylococcus aureus, Staphylococcus lugdunensis, and various Staphylococcus species to the genus level, and the detection of the mecA gene for methicillin resistance directly from patient positive blood culture specimens. The device uses automated hot-start enabled PCR for amplification and hybridization probes on a silicon chip for detection.

Here's an analysis of its acceptance criteria and the study that proves the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

• Staphylococcus positivity: 100% (540/540) for positive, 96.7% (87/90) for negative.
• Specific Staphylococcus detection: 99.6% (538/540) for positive, 99.3% (1341/1350) for negative.
• mecA detection (no organism): 99.7% (359/360) for present, 100% (270/270) for absent. |
  | Evaluation of Blood Culture Bottle Types | Compatibility with various bottle types. | All 13 tested blood culture bottle types were compatible with the Staph ID/R Blood Culture Panel, with no false negative results. One false positive S. aureus and one false positive Staphylococcus species OTHER than S. aureus or S. lugdunensis were observed in a single test run each, attributed to contamination. Nine invalid calls for E. faecalis. |
  | Prospective Clinical Performance - S. aureus | PPA ≥ 95%, NPA ≥ 95% (implied). | - PPA: 98.6% (211/214) (95% CI: 96.0 - 99.5%)
• NPA: 99.5% (548/551) (95% CI: 98.4 - 99.8%) |
  | Prospective Clinical Performance - S. lugdunensis | PPA ≥ 95%, NPA ≥ 95% (implied). | - PPA: 100.0% (3/3) (95% CI: 43.9 - 100%)
• NPA: 99.9% (761/762) (95% CI: 99.3 - 99.9%) |
  | Prospective Clinical Performance - Other Staphylococcus spp. | PPA ≥ 95%, NPA ≥ 95% (implied). | - PPA: 98.9% (444/449) (95% CI: 97.4 - 99.5%)
• NPA: 97.2% (307/316) (95% CI: 94.7 - 98.5%) |
  | Prospective Clinical Performance - mecA with S. aureus | PPA ≥ 90%, NPA ≥ 98% (implied). | - PPA: 94.4% (68/72) (95% CI: 86.6 - 97.8%)
• NPA: 98.8% (682/690) (95% CI: 97.7 - 99.4%) |
  | Prospective Clinical Performance - mecA with Other Staphylococcus spp. | PPA ≥ 90%, NPA ≥ 95% (implied). | - PPA: 92.7% (243/262) (95% CI: 88.1 - 97.1%)
• NPA: 96.2% (481/500) (95% CI: 92.4 - 98.0%) |
  | Invalid Rate - Clinical | Acceptable low rate, resolvable on retest. | Initial Invalid Rate: 1.39% (11/789). Final Invalid Rate: 0.00% after retest. |
  | Abort Rate - Clinical | Acceptable low rate, resolvable on retest. | Initial Abort Rate: 3.30% (26/789). Final Abort Rate: 0.00% after retest. |

2. Sample Size Used for the Test Set and Data Provenance

• Prospective Clinical Study:

  • Sample Size: A total of 762 prospective specimens were tested. Additionally, 69 archived frozen specimens were tested after the prospective study, making the overall clinical sample set significantly larger. A 'Low Prevalence' panel of 102 contrived or 'simulated' blood culture specimens was also tested.
  • Data Provenance: The prospective specimens were collected at three geographically diverse U.S. sites.

• Analytical Studies (LoD, Inclusivity, Exclusivity, Interference, Bottle Types, Reproducibility):

  • These studies used various numbers of cultured bacterial strains (ATCC, NCTC, CCUG, Clinical isolates where specified) and simulated blood culture specimens, often with multiple replicates per strain (ranging from 2 to 20+ replicates per strain for LoD, 2 replicates per strain for exclusivity, and 90 replicates per analyte for reproducibility). The specific number of unique strains and replicates varies per study as detailed in the tables (e.g., 22 strains for LoD, 116 off-panel strains for exclusivity, 7 simulated blood culture specimens for reproducibility).
  • Data Provenance: Not explicitly stated for each analytical study, but context suggests laboratory-controlled experiments. Strains were from recognized culture collections (ATCC, NCTC, CCUG) or clinical sources.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts

The document does not specify the number of experts or their qualifications used to establish the ground truth for the clinical or analytical test sets.

For the clinical study, the ground truth (Staphylococcus species identification and mecA gene presence) was established using "Reference Method(s)". While not explicitly defined, these reference methods typically involve conventional microbiology techniques such as:

• Sub-culturing of positive blood cultures.
• Phenotypic identification (e.g., Gram stain, catalase, coagulase tests, biochemical panels).
• Molecular methods (e.g., gene sequencing, PCR assays with a different target).
• Antimicrobial susceptibility testing (e.g., oxacillin MIC) to confirm methicillin resistance (though the device detects the mecA gene, not phenotypic resistance itself).

For analytical studies, bacterial strains from recognized collections (ATCC, NCTC, CCUG) and presumably well-characterized clinical isolates were used, implying their identity and mecA status were already established by standard microbiological and genetic methods.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method (like 2+1 or 3+1) for the clinical test set. The comparison is made directly against "Reference Method(s)". For discrepant results in the exclusivity and microbial interference studies, repeat testing was performed (e.g., a minimum of six (6) repeat tests for 'Staphylococcus POSITIVE' calls in exclusivity, sometimes at higher CFU/mL input for interference studies to resolve miscalls).

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not conducted. This device is an automated in vitro diagnostic assay (algorithm only), not an AI-assisted human reading system. Therefore, there is no human-in-the-loop performance to measure improvement with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done

Yes, the studies conducted demonstrate the standalone performance of the Great Basin Staph ID/R Blood Culture Panel. This device is an automated, qualitative, molecular diagnostic system, meaning its output is generated by the instrument itself (the Portrait Analyzer and its software) without direct human interpretation of complex images or data that would typically benefit from AI assistance. The performance metrics (PPA, NPA) directly reflect the device's accuracy against reference methods.

7. The Type of Ground Truth Used

The primary ground truth for both analytical and clinical performance studies was established through reference microbiology methods. This includes:

• Culture-based identification: Sub-culturing, Gram staining, and phenotypic tests for species identification.
• Molecular identification: Implied use of validated molecular techniques or genetic characterization (e.g., for mecA gene presence and confirmation of species for various strains).
• Antimicrobial Susceptibility Testing (AST): Specifically, oxacillin MIC was used in the Analytical Reactivity (Inclusivity) Challenge Panel to determine phenotypic resistance, which indirectly supports the genotypic mecA detection.
• For the analytical studies, well-characterized strains from culture collections (ATCC, NCTC, CCUG) served as a strong foundation for ground truth.

8. The Sample Size for the Training Set

The document does not explicitly mention a "training set" in the context of device development or any specific AI/machine learning models. This is typical for traditional molecular diagnostic assays, where assay design (primer/probe sequences, reaction conditions) is based on known genetic targets and verified through extensive analytical validation rather than data-driven machine learning training. The analytical studies (LoD, inclusivity, exclusivity) serve to validate the analytical performance of the developed assay against a wide range of relevant organisms and conditions.

9. How the Ground Truth for the Training Set Was Established

As no explicit "training set" is described for an AI/ML model, this question is not directly applicable. If one considers the development of the assay (design of specific DNA targets and hybridization probes) as analogous to "training," then the ground truth would have been established through extensive molecular biology research, genetic sequencing, and characterization of Staphylococcus species and the mecA gene. This would involve identifying conserved and variable regions of target genes across a broad collection of known Staphylococcus strains to ensure specificity and sensitivity.

Ask a specific question about this device

The Verigene® Gram-Positive Blood Culture Nucleic Acid Test (BC-GP) performed using the sample-to-result Verigene System is a qualitative, multiplexed in vitro diagnostic test for the simultaneous detection and identification of potentially pathogenic gram-positive bacteria which may cause bloodstream infection (BSI). BC-GP is performed directly on blood culture bottles identified as positive by a continuousmonitoring blood culture system and which contain gram-positive bacteria. BC-GP detects and identifies the following bacterial genera and species: Staphylococcus spp., Staphylococcus aureus, Staphylococcus epidermidis, Staphylococcus lugdunensis, Enterococcus faecalis, Enterococcus faecium, Streptococcus spp., Streptococcus pneumoniae, Streptococcus pyogenes, Streptococcus agalactiae, Streptococcus anginosus group, and Listeria spp.

In addition. BC-GP detects the mecA resistance marker, inferring mecA-mediated methicillin resistance, and the vanA and vanB resistance markers, inferring vanA/vanB-mediated vancomycin resistance. In mixed growth, BC-GP does not specifically attribute van-mediated vancomycin resistance to either E. faecalis or E. faecium, or mecAmediated methicillin resistance to either S. aureus or S. epidermidis.

BC-GP is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections; however, is not to be used to monitor these infections. Sub-culturing of positive blood cultures is necessary to recover organisms for susceptibility testing, identification of organisms not detected by BC-GP, differentiation of mixed growth, association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.

The Verigene® Gram-Positive Blood Culture Nucleic Acid Test (BC-GP) is a molecular assay which relies on detection of specific nucleic acid targets in a microarray format. For each of the bacterial nucleic acid sequences detected by the BC-GP test. Capture and Mediator olizonucleotides are utilized for gold nanoparticle probe-based endpoint detection. The Canture oligonucleotides bind to a specific portion of the nucleic acid target and are themselves bound onto a substrate in the microarray. The Mediator oligonucleotides bind to a different portion of the same nucleic acid target and allow binding of a gold nanoparticle probe to a portion complementary to a gold nanoparticle probe. Specific silver enhancement of the bound gold nanoparticle probes at the capture sites results in gold-silver aggregates that scatter light with high efficiency.

The BC-GP test is performed on the Verigene System, a "sample-to-result", fully automated, bench-top molecular diagnostics workstation. The Verigene System consists of two components: the Verigene Reader and the Verigene Processor SP. The BC-GP test utilizes single-use disposable test consumables and a self-contained Verigene Test Cartridge for each sample tested. For the BC-GP test, the Verigene System allows automated nucleic acid extraction from grampositive bacteria-containing blood culture specimens and target detection of bacteria-specific DNA.

The Reader is the Verigene System's user interface, which serves as the central control unit for all aspects of test processing and results generation. The Reader's graphical user interface guides the user through test processing and test results using a barcode scanner. The user inserts the Test Cartridge into the Verigene Processor SP, which executes the test procedure, automating the steps of ( ) Sample Preparation - Cell lysis and magnetic bead-bacterial DNA isolation from blood culture samples and (2) Hybridization - Detection and identification of bacterialspecific DNA in a microarray format by using gold nanoparticle probe-based technology.

After test processing is complete, to obtain the test results the user removes the Test Cartridge from the Processor SP, removes the reagent pack from the substrate holder, and inserts the substrate holder into the Reader for analysis. Light scatter from the capture spots is imaged by the Reader and intensities from the microarray spots are used to make decisions regarding the presence (Detected) or absence (Not Detected) of a bacterial nucleic acid sequence/analyte.

The user is asking for information about the acceptance criteria and study data for the "Verigene® Gram-Positive Blood Culture Nucleic Acid Test (BC-GP)".

Here's a breakdown of the requested information based on the provided text:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state "acceptance criteria" in a tabular format with corresponding "reported device performance" for this specific 510(k) submission (K122514). This submission is an expansion of the intended use for a previously cleared device (K113450) and largely refers back to the original submission for detailed performance characteristics.

However, the document describes a study to support the expanded use (analytical testing of additional culture bottles) and reports the performance of the device in that study. I can create a table based on the findings of this specific study as it relates to the suitability of the additional bottle types.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Sample Size for Test Set: For the analytical testing of additional culture bottles, the study inoculated each of the six additional bottle types with representative strains into seven replicates each.
• Data Provenance: The document does not specify the country of origin of the data, nor does it explicitly state whether the study was retrospective or prospective. Given it involved controlled inoculations into specific bottle types to test performance, it would be considered a prospective analytical study within a lab setting rather than clinical data provenance.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

The document does not mention the use of experts or their qualifications for establishing ground truth in this analytical study. Ground truth in this context (testing specific bacterial strains in culture bottles) would typically be established by controlled inoculation of known bacterial strains, confirmed by standard microbiological identification methods, rather than expert consensus on interpretation.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The document does not mention any adjudication method. For an analytical study involving known bacterial inoculations, adjudication by experts for ground truth is generally not applicable in the same way it would be for interpreting complex clinical images or diagnoses.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• Was an MRMC study done? No. This device is an in vitro diagnostic test for detecting nucleic acid sequences of bacteria directly from blood cultures, not an imaging device requiring human readers or AI assistance in interpretation in the context of an MRMC study.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• Standalone performance: Yes, the performance described in the document is for the standalone device (Verigene® BC-GP test on the Verigene System). The system is a "sample-to-result, fully automated" system with "automated nucleic acid extraction... and target detection," and interpretation by "Diagnostic Software/Decision Algorithm." This indicates it operates without human-in-the-loop performance for result generation. The results are "Detected" or "Not Detected" for specific analytes based on signal intensity read by the system.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

For the analytical testing of additional culture bottles, the ground truth was established by controlled inoculation of known bacterial strains. The presence or absence of specific bacterial targets was predetermined by the strains inoculated.

8. The sample size for the training set

The document does not provide information about a "training set" for the BC-GP test. This type of molecular diagnostic device is typically developed based on known genetic sequences and optimized through analytical studies, rather than machine learning models that require a distinct training set in the conventional sense. The "Performance Characteristics" section refers to K113450 for the full analytical and clinical performance data, indicating that the core technology and algorithm were established in the previous submission.

9. How the ground truth for the training set was established

As there's no mention of a "training set" in the context of machine learning, ground truth establishment for such a set is not applicable here. The establishment of the assay's detection capabilities would have been through rigorous analytical validation using characterized bacterial cultures and nucleic acid controls.

Ask a specific question about this device

The Verigene® Gram Positive Blood Culture Nucleic Acid Test (BC-GP) performed using the sample-to-result Verigene System is a qualitative, multiplexed in vitro diagnostic test for the simultaneous detection and identification of potentially pathogenic gram-positive bacteria which may cause bloodstream infection (BSI). BC-GP is performed directly on positive blood culture using BACTEC™ Plus Aerobic/F and BacT/ALERT FA FAN® Aerobic blood culture bottles, which contain gram positive bacteria. BC-GP is indicated for use in conjunction with other clinical and laboratory findings, such as culture, to aid in the diagnosis of bacterial bloodstream infections; however, it is not used to monitor bloodstream infections.

BC-GP detects and identifies the following bacterial genera and species:
Staphylococcus spp.
Staphylococcus aureus
Staphylococcus epidermidis
Staphylococcus lugdunensis
Streptococcus spp.
Streptococcus pneumoniae
Streptococcus pyogenes
Streptococcus agalactiae
Streptococcus anginosus group
Enterococcus
faecalis
Enterococcus
faecium
Listeria spp.

In addition, BC-GP detects the mecA resistance marker, inferring mecA-mediated methicillin resistance, and the vanA and vanB resistance markers, inferring vanA/vanB-mediated vancomycin resistance. In mixed growth, BC-GP does not specifically attribute van-mediated vancomycin resistance to either E. faecium, or mecA-mediated methicillin resistance to either S. aureus or S. epidermidis.

BC-GP is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections; however, is not to be used to monitor these infections. Sub-culturing of positive blood cultures is necessary to recover organisms for susceptibility testing. identification of organisms not detected by BC-GP, differentiation of mixed growth, association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.

The Verigene® Gram Positive Blood Culture Nucleic Acid Test (BC-GP) is a molecular assav which relies on detection of specific nucleic acid targets in a microarray format. For each of the bacterial nucleic acid sequences detected by the BC-GP test, Capture and Mediator oligonucleotides are utilized for gold nanoparticle probe-based endpoint detection. The Capture oligonucleotides bind to a specific portion of the nucleic acid target and are themselves bound onto a substrate in the microarray. The Mediator oligonucleotides bind to a different portion of the same nucleic acid target and allow binding of a gold nanoparticle probe to a portion complementary to a gold nanoparticle probe. Specific silver enhancement of the bound gold nanoparticle probes at the capture sites results in gold-silver aggregates that scatter light with high efficiency.

The BC-GP test is performed on the Verigene® System, a 'sample-to-result', fully automated, bench-top molecular diagnostics workstation. The Verigene System consists of two components: the Verigene Reader and the Verigene Processor SP. The BC-GP test utilizes single-use disposable test consumables and a self-contained Verigene Test Cartridge for each sample tested. For the BC-GP test, the Verigene System allows automated nucleic acid extraction from Gram-positive bacteria-containing blood culture specimens and target detection of bacteria-specific DNA.

The Reader is the Verigene System's user interface, which serves as the central control unit for all aspects of test processing and results generation. The Reader's graphical user interface guides the user through test processing and test results using a barcode scanner. The user inserts the Test Cartridge into the Verigene Processor SP, which executes the test procedure. automating the steps of (1) Sample Preparation - Cell lysis and magnetic bead-based bacterial DNA isolation from blood culture samples and (2) Verigene Hybridization Test - Detection and identification of bacterial-specific DNA in a microarray format by using gold nanoparticle probe-based technology.

After test processing is complete, to obtain the test results the user removes the Test Cartridge from the Processor SP, removes the reagent pack from the substrate holder, and inserts the substrate holder into the Reader for analysis. Light scatter from the capture spots is imaged by the Reader and intensities from the microarray spots are used to make decisions regarding the presence (Detected) or absence (Not Detected) of a bacterial nucleic acid sequence/analyte.

The provided text describes the evaluation of the Verigene® Gram Positive Blood Culture Nucleic Acid Test (BC-GP). While specific "acceptance criteria" defined as a numerical threshold for performance are not explicitly stated in a single section, the document outlines the performance requirements for various aspects of the device (limit of detection, inclusivity, specificity, interference, precision, and clinical agreement). The "reported device performance" is then presented in the clinical study results, allowing for an implicit comparison against acceptable ranges for such diagnostic assays.

Here's an organized breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance

The document doesn't present explicit acceptance criteria with specific numerical thresholds for each analyte. Instead, it describes general expectations for "acceptable specificity," "expected results," and "high concordance." For the purpose of this table, I will infer general performance expectations from the overall context and the nature of diagnostic assay validation (e.g., generally aiming for very high agreement for positive and negative cases, especially for critical diagnoses like bloodstream infections). The reported device performance is directly extracted from the clinical study results.

Inferred Acceptance Criteria vs. Reported Device Performance

2. Sample size used for the test set and the data provenance (e.g., country of origin of the data, retrospective or prospective)

• Test Set Sample Size:
  • Clinical Study: 1767 specimens in total were initially enrolled.
    • 1251 fresh specimens
    • 175 frozen specimens
    • 216 contrived blood culture specimens (for rare organisms).
  • 125 specimens were excluded for various reasons (e.g., training sample, culture/reference method results unavailable, sample not tested on BC-GP, QC/protocol error).
  • The total evaluable specimens for analysis are not explicitly stated as a single number after exclusions, but performance tables typically reflect data from the 1426 (1251 fresh + 175 frozen) prospectively/retrospectively collected samples + 216 contrived samples, minus the excluded non-evaluable samples where applicable to each specific analyte calculation.
• Data Provenance:
  • Country of Origin: Not explicitly stated, but the study was conducted at "five external, geographically-diverse clinical study sites," implying multiple locations within a single country (likely the USA, given the FDA context).
  • Retrospective or Prospective: The clinical study incorporated both:
    • Most specimens were collected and tested as "fresh" (implying prospective at time of collection relative to the BC-GP testing).
    • Some were "stored frozen prior to testing" (implying retrospective collection for later testing).
    • A portion was "contrived" (simulated specimens for rare organisms).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience)

• The document does not specify the number of experts or their specific qualifications (e.g., "radiologist with 10 years of experience").
• However, it details the reference methods used to establish ground truth:
  • "conventional biochemical, culture, and bidirectional sequencing reference methods."
  • "culture followed by testing blood culture isolates with conventional biochemicals, Vitek2, and cefoxitin disc testing."
  • "Cefoxitin discs were used as the reference method for confirming mecA mediated resistance in S. aureus and S. epidermidis."
  • "Vancomycin resistance and the presence of vanA or vanB in E. faecium and E. faecalis was performed using vancomycin E-tests followed by bidirectional sequencing on resistant organisms."
  • This implies that the ground truth was established by laboratory professionals using established clinical microbiology techniques, but their individual expertise is not quantified.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

• The document does not explicitly describe a formal adjudication method (like "2+1" or "3+1") for resolving discrepancies between the BC-GP test and the reference methods.
• It states that "BC-GP Test results were compared with results from traditional laboratory reference methods." Discrepancies are reported (e.g., in mixed cultures, 6 "false positives" and 25 "false negatives" compared to reference culture). However, the process for investigating or resolving these discrepancies to arrive at a final "ground truth" for the discrepant cases is not detailed.
• It does mention that the "initial No-call rate in the clinical study was 4.7% (77/1642), and the final No-call rate was 1.1% (18/1642)," implying that some initial no-calls were resolved or re-tested successfully, but this isn't an "adjudication" of BC-GP vs. reference method.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• No, a multi-reader multi-case (MRMC) comparative effectiveness study was NOT done.
• This device is an automated molecular diagnostic assay, not an imaging device that requires human "readers" or interpretation in the same way an AI for radiology would. Its performance is compared directly against established laboratory reference methods, not against human reader performance, nor is it designed to "assist" human readers in a direct interpretation task. The "Reader" component of the Verigene System is an automated instrument for analyzing the microarray, not a human.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

• Yes, the provided study describes standalone performance of the algorithm.
• The Verigene BC-GP test is a "sample-to-result, fully automated, bench-top molecular diagnostics workstation." The entire process, from sample preparation to target detection and result generation, is automated by the Verigene System. The clinical study evaluates the performance of this automated system (algorithm only) against traditional culture-based reference methods. There is no human "in-the-loop" for interpreting the BC-GP test results; the system generates "Detected" or "Not Detected" calls.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• The ground truth for the clinical study was established using a combination of traditional microbiological reference methods and gold-standard molecular techniques:
  • Culture: Isolation and identification of microorganisms through conventional culture methods.
  • Biochemicals: Conventional biochemical tests for species identification.
  • Vitek2: An automated system for microbial identification and antimicrobial susceptibility testing.
  • Cefoxitin Disk Diffusion: For mecA-mediated methicillin resistance.
  • Vancomycin E-tests: For vancomycin resistance.
  • Bidirectional Sequencing: For vanA and vanB resistance markers.
  • This represents a robust and well-accepted set of methods for establishing ground truth in clinical microbiology.

8. The sample size for the training set

• The document mentions "Training Sample" as a reason for exclusion from the evaluable clinical dataset (e.g., 5 at OSU, 4 at LIJ), but it does not specify the total sample size used for the training set for the BC-GP device itself.
• The "Assay Cutoff" section mentions that "a set of bacterial strains were tested to represent all the analytes detected by the BC-GP Test" to initially determine cutoff values, and "A set of retrospective blood culture specimens were then tested with the BC-GP Test to verify the previously determined cut off values." This internal process would constitute a form of training or optimization, but the specific number of samples for this phase is not provided.
• The clinical study mentions the exclusion of 125 specimens, some labeled as "Training Sample" (indicating they were likely used for internal development or training purposes prior to the formal clinical evaluation).

9. How the ground truth for the training set was established

• For the "Assay Cutoff" determination (an internal optimization process analogous to training), the ground truth was established by:
  • "Blood culture bottles were spiked with each individual organism and grown in automated blood culture instruments to 'bottle ring.'"
  • "The test decisions from each test were compiled to generate a data set that was used to initially determine BC-GP Assay Test cutoff values."
  • The verification of these cutoffs used "retrospective blood culture specimens" where "Results were analyzed using logistic fit and ROC statistics. The results obtained from this the cutoff determination matched the results from culture-based biochemical results, thus verifying the final cutoff values."
• This indicates that for the internal optimization/training phases, the ground truth was established by known spiked organisms and confirmed by culture-based biochemical results.

Ask a specific question about this device