The LIAISON PLEX® Gram-Positive Blood Culture Assay (BCP), performed using the automated, sample-to-result LIAISON PLEX® System, is a qualitative multiplexed in vitro diagnostic test for the simultaneous detection and identification of selected gram-positive pathogens and/or selected genetic determinants associated with antimicrobial resistance in positive blood culture bottles. BCP is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system and which contain gram-positive bacteria as determined by Gram stain.

The BCP Assay detects and identifies the following:

Gram Positive Resistance Markers:

• mecA/mecC
• vanA
• vanB

Genera and Species:

• Bacillus spp.
• Enterococcus faecalis
• Enterococcus faecium
• Listeria spp.
• Staphylococcus spp.
• Staphylococcus aureus
• Staphylococcus epidermidis
• Staphylococcus lugdunensis
• Streptococcus spp.
• Streptococcus agalactiae
• Streptococcus anginosus group
• Streptococcus pneumoniae
• Streptococcus pyogenes

Negative results for antimicrobial resistance genes do not indicate bacterial susceptibility as there are multiple mechanisms that can contribute to resistance.

The LIAISON PLEX® BCP Assay contains targets for the detection of genetic determinants associated with resistance to methicillin (mecA/C) and vancomycin (vanA and vanB) to aid in the identification of potentially antimicrobial-resistant organisms in positive blood culture samples. In mixed growth, the LIAISON PLEX BCP Assay does not specifically attribute vanA/vanB-mediated vancomycin resistance to either E. faecalis or E. faecium, or mecA/mecC-mediated methicillin resistance to either Staphylococcus spp., S. aureus, S. epidermidis or S. lugdunensis.

The antimicrobial resistance gene or marker detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of methicillin and vancomycin resistance exist.

The LIAISON PLEX® BCP Assay is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections (BSI). The LIAISON PLEX® BCP Assay is not intended to monitor these infections. Sub-culturing of positive blood cultures is necessary to recover organisms for antimicrobial susceptibility testing (AST), for identification of organisms not detected by the LIAISON PLEX BCP Assay, to detect mixed infections that may not be detected by the LIAISON PLEX BCP Assay, for association of antimicrobial resistance genes to a specific organism, or for epidemiological typing.

The LIAISON PLEX® Gram-Positive Blood Culture Assay (BCP Assay) is an automated test for the detection and identification of nucleic acid from gram-positive bacteria in a positive blood culture media sample. The BCP Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain gram-positive bacteria, as determined by a Gram stain.

The LIAISON PLEX® System is a fully automated, bench-top "sample-to-answer" device that performs sample preparation and microarray-based hybridization for the detection of target-specific nucleic acids. The test reagents are supplied as a single, disposable test cartridge. Target amplification is not performed as part of the BCP Assay workflow, as it is a non-amplified, direct detection test performed on the LIAISON PLEX® System.

The provided text describes the acceptance criteria and the study that proves the LIAISON PLEX® Gram-Positive Blood Culture Assay meets those criteria. However, it's important to note that this document is a 510(k) Clearance Letter and 510(k) Summary, which focuses on demonstrating substantial equivalence to a predicate device for regulatory clearance. It is not an academic paper detailing a clinical trial with specific acceptance criteria that would typically be associated with AI/ML diagnostic performance metrics like those in a multi-reader multi-case (MRMC) study or the establishment of ground truth by multiple experts.

The study described here is for an in vitro diagnostic test for the detection of nucleic acid sequences, not for an AI/ML-based diagnostic imaging device. Therefore, many of the requested points (e.g., number of experts for ground truth, adjudication method, MRMC study, human reader improvement with AI assistance) are not applicable to this type of device and study.

I will interpret the provided information in the context of the device described (a multiplex nucleic acid assay) and its verification and validation.

Acceptance Criteria and Device Performance

The acceptance criteria for this device are implicitly derived from the performance goals demonstrated in the analytical and clinical studies, aiming for high sensitivity and specificity in detecting specified microorganisms and resistance markers. The reported performance is presented in several tables throughout the document.

Table 1: Acceptance Criteria (Implicit) and Reported Device Performance

2. Sample Sizes and Data Provenance

• Test Set (Clinical Study):
  • Prospective Arm (Arm 1): 562 unique specimens enrolled, 509 included in analysis.
  • Pre-selected Arm (Arm 2): 163 pre-selected left-over specimens, 162 included in analysis.
  • Contrived Arm (Arm 3): 225 contrived specimens.
  • Total specimens analyzed for clinical performance: 509 (prospective) + 162 (pre-selected) + 225 (contrived) = 896 unique samples.
• Data Provenance:
  • Prospective Arm: Collected between April 2024 and August 2024 from four geographically diverse clinical sites within the United States. Data is prospective.
  • Pre-selected Arm: Sourced from ten vendors in the United States and one site in Italy. Data are retrospective (left-over, de-identified specimens).
  • Contrived Arm: Specimens were prepared, blinded, randomized, and tested at three external testing sites and one internal testing site between June 2024 to September 2024. These are contrived (synthetic) samples.

3. Number of Experts and Qualifications for Ground Truth

• Not Applicable in the traditional sense for this IVD device. The ground truth for this nucleic acid assay is established through reference methods such as culture with automated microbiological/biochemical identification (VITEK 2), PCR followed by bi-directional sequencing (BDS), or a combination thereof. This is a laboratory-based diagnostic test, not an imaging device requiring human expert interpretation for ground truth.

4. Adjudication Method for the Test Set

• Not Applicable in the traditional sense. The "adjudication" for the gold standard (reference method) involves a hierarchical algorithm (Table 19). If initial VITEK 2 results were insufficient or for specific targets like Bacillus spp. and resistance markers (mecA/mecC, vanA, vanB), PCR followed by bi-directional sequencing was used. This is a technical validation process against established laboratory methods rather than a consensus among clinical experts interpreting an output.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Not done. This type of study (MRMC) is typically used for diagnostic imaging devices where human readers interpret images with and without AI assistance. This device is a molecular diagnostic assay that provides direct results, and therefore, an MRMC study is not relevant.

6. Standalone (i.e., algorithm only without human-in-the-loop performance)

• Yes, in essence. The LIAISON PLEX® BCP Assay is an automated, sample-to-result system. Its performance (sensitivity, specificity) is evaluated independently against a reference method. While trained operators are involved in loading samples and running the system, the diagnostic result itself is generated by the "system" (instrument + assay + software algorithm) without human diagnostic interpretation of the raw signal data.

7. The Type of Ground Truth Used

• Laboratory Reference Methods:
  • Culture followed by Automated microbiological/biochemical identification using VITEK 2 for most bacterial species.
  • PCR followed by bi-directional sequencing (BDS) for Bacillus spp. and resistance markers (mecA/mecC, vanA, vanB). This method was also used to confirm discordant results or for targets with low prevalence in initial testing.
  • In some cases, MALDI-ToF assay (Matrix-Assisted Laser Desorption/Ionization-Time Of Flight) was also mentioned as a Standard of Care method in the footnotes for discordant result explanations.

8. The Sample Size for the Training Set

• Not explicitly stated in terms of a "training set" for an AI/ML model. This device is a direct detection assay based on hybridization, not a machine learning model that undergoes a training phase with a distinct dataset. Therefore, the concept of a "training set" in the context of data used to train an AI model is not applicable to this traditional IVD. The development process would involve extensive analytical characterization, probe design, and optimization using various microbial strains and clinical samples, but this is distinct from "training data" for an AI algorithm.

9. How the Ground Truth for the Training Set Was Established

• Not applicable for this type of device. As explained above, this is not an AI/ML device that requires a "training set" with ground truth established in the AI/ML context. The assay's design and optimization would rely on established microbiological methods and genetic sequencing to define target sequences and ensure specificity and inclusivity.

Ask a specific question about this device

The LIAISON PLEX® Gram‐Negative Blood Culture (BCN) Assay, performed using the automated, sample‐to‐result LIAISON PLEX® System, is a qualitative multiplexed in vitro diagnostic test for the simultaneous detection and identification of selected gram‐negative pathogens and/or selected genetic determinants associated with antimicrobial resistance in positive blood culture bottles. LIAISON PLEX® BCN Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system which contain gram‐negative bacteria as determined by Gram stain.

The LIAISON PLEX® BCN Assay detects and identifies the following:

Resistance Markers:

• CTX‐M (blaCTX‐M)
• IMP (blaIMP)
• KPC (blaKPC)
• NDM (blaNDM)
• OXA (blaOXA)
• VIM (blaVIM)
• MCR
• SME (blaSME)

Gram Negative Genera and Species:

• Enterobacteriaceae / Morganellaceae
• Acinetobacter baumannii
• Acinetobacter spp.
• Citrobacter spp.
• Enterobacter spp. (1)
• Escherichia coli (2)
• Haemophilus influenzae
• Klebsiella oxytoca
• Klebsiella pneumoniae
• Klebsiella variicola
• Morganella morganii
• Neisseria meningitidis
• Proteus spp.
• Pseudomonas aeruginosa
• Pseudomonas spp.
• Salmonella spp.
• Serratia marcescens
• Stenotrophomonas maltophilia

(1) Due to reclassification, Klebsiella aerogenes will be reported Enterobacter spp.
(2) LIAISON PLEX® BCN Assay will not distinguish between Escherichia coli and Shigella spp. (S. dysenteriae, S. boydii, S. flexneri and S. sonnei)

LIAISON PLEX® BCN Assay contains targets for the detection of genetic determinants associated with resistance to carbapenems (blaCTX‐M, blaIMP, blaKPC, blaNDM, blaOXA48‐like, blaVIM, blaSME) to aid in the identification of potentially antimicrobial‐resistant organisms in positive blood culture samples. In addition, the panel includes an assay for the detection of the mobilized genetic determinant MCR, an emerging marker of public health importance. The antimicrobial resistance gene or marker detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of resistance to ß‐lactams and colistin exist.

LIAISON PLEX® BCN Assay is indicated for use in conjunction with other clinical and laboratory findings to aid in the diagnosis of bacterial bloodstream infections (BSI). LIAISON PLEX® BCN Assay is not intended to monitor treatment of these infections. Sub‐culturing of positive blood cultures is necessary to recover organisms for antimicrobial susceptibility testing (AST), for identification of organisms not detected by LIAISON PLEX® BCN Assay, to detect mixed infections that may not be detected by LIAISON PLEX® BCN Assay, for association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.

The LIAISON PLEX® Gram‐Negative Blood Culture (BCN) Assay is an automated test for the detection and identification of nucleic acid from gram‐negative bacteria in a positive blood culture media sample. The BCN Assay is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain gram‐negative bacteria, as determined by a Gram stain.

The LIAISON PLEX® System is a fully automated, bench‐top "sample‐to‐answer" device that performs sample preparation, polymerase chain reaction (PCR) and microarray‐based hybridization for the detection of target‐specific nucleic acids. The test reagents are supplied as a single, disposable test cartridge. PCR is not performed on the LIAISON PLEX® BCN Assay, as it is a non‐amplified, direct detection test performed on the LIAISON PLEX® System.

Here's a breakdown of the acceptance criteria and study details for the LIAISON PLEX Gram-Negative Blood Culture Assay, based on the provided FDA 510(k) clearance letter:

1. Table of Acceptance Criteria and Reported Device Performance

Note: The reported performance for individual targets (Sensitivity/PPA and Specificity/NPA) is explicitly given for the "Combined" data set (Prospective + Pre-selected). For Resistance Markers, some N/A entries indicate insufficient positive reference cases in the clinical study.

2. Sample Size and Data Provenance

• Test Set (Clinical Study):
  • Prospective Samples: 381 unique specimens enrolled, 351 included in analysis (30 excluded due to duplicates or incomplete reference testing).
  • Pre-selected Samples: 231 left-over, de-identified specimens.
  • Contrived Samples: 746 specimens.
  • Total Clinical Samples (Prospective + Pre-selected): 582 specimens included in analysis.
  • Total Contrived Samples: 746 specimens included in analysis.
  • Data Provenance:
    • Prospective: Collected from four geographically diverse clinical sites within the United States between March 2024 and July 2024.
    • Pre-selected: Sourced from seven vendors in the United States and one site in Italy.
    • Contrived: Tested at four testing sites during April 2024 – August 2024.
  • Retrospective/Prospective: Prospective data was collected prospectively. Pre-selected and Contrived data were essentially retrospective/prepared, then tested blindly and randomized.

3. Number of Experts and Qualifications for Ground Truth

The document does not mention the use of "experts" in the sense of clinical specialists (e.g., radiologists) establishing ground truth for the test set. Instead, the ground truth was established by laboratory methods.

4. Adjudication Method for the Test Set

The document describes the "Reference Method Algorithm" (Table 19) where the LIAISON PLEX BCN Assay results were compared to:

• Culture followed by Automated microbiological/biochemical identification using VITEK 2 for most organism targets.
• Culture followed by Automated microbiological/biochemical identification using VITEK 2 with positive confirmation of the clinical isolate by PCR/BDS for Acinetobacter baumannii, Klebsiella oxytoca, Klebsiella pneumoniae, and Klebsiella variicola.
• PCR followed by bi-directional sequencing (BDS) for all resistance markers (CTX-M, IMP, KPC, NDM, OXA, VIM, MCR, SME).

This is a hierarchical or sequential method rather than an "adjudication" by multiple human readers for discrepancies. For pre-selected specimens, initial identification was by Standard of Care (SoC) and/or PCR followed by BDS, then confirmed.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, this was not an MRMC comparative effectiveness study where human readers' improvement with AI vs. without AI assistance was measured. This device is an in vitro diagnostic test for direct detection of nucleic acids, not an AI software intended to assist human readers.

6. Standalone Performance (Algorithm Only)

• Yes, this was a standalone performance study. The LIAISON PLEX BCN Assay is an automated, qualitative multiplexed in vitro diagnostic test performed on the LIAISON PLEX System. The performance metrics (Sensitivity/PPA and Specificity/NPA) described in the clinical study (Table 20 and 21) represent the standalone performance of the device without human interpretation of its results.

7. Type of Ground Truth Used

The ground truth for the clinical study (test set) was established using a combination of:

• Culture followed by Automated microbiological/biochemical identification (VITEK 2).
• PCR followed by bi-directional sequencing (BDS).
• In some cases, a combination of VITEK 2 and PCR/BDS for confirmation.
• For resistance markers, ground truth was solely by PCR followed by bi-directional sequencing (BDS).

8. Sample Size for the Training Set

The document does not explicitly state the sample size used for the training set of the LIAISON PLEX BCN Assay. The provided information focuses on the analytical and clinical validation of the final device. For IVD products like this, internal development and optimization (which would involve data that could be considered a "training set" in a broad sense for assay design) are typically not detailed in the same way as machine learning model training sets in FDA submissions. The inclusivity and exclusivity studies (Analytical Reactivity and Specificity) use laboratory-tested strains and in silico analysis respectively, which are key parts of ensuring the assay's design covers the intended targets and avoids false positives.

9. How the Ground Truth for the Training Set Was Established

Since an explicit "training set" sample size isn't provided, the method for establishing its ground truth isn't detailed. However, the comprehensive Analytical Reactivity (Inclusivity) and Analytical Specificity (Exclusivity) studies describe how the assay's design was validated:

• Analytical Reactivity (Inclusivity): Laboratory testing of 246 on-panel organisms (in triplicate) and in silico analysis using sequences from GenBank and WGS databases (February to April 2024). The ground truth for these would be the known identity of the characterized strains and the genetic sequences from public databases.
• Analytical Specificity (Exclusivity): Laboratory testing of 113 off-panel species (in triplicate) and in silico exclusivity assessment against on-panel and off-panel organisms from GenBank (June 7, 2024). The ground truth here is the known identity of these off-panel organisms and their genetic sequences.

These analytical studies effectively serve as the validation of the assay's "knowledge" or "training" on what to detect and what not to detect.

Ask a specific question about this device

The BIOFIRE Blood Culture Identification 2 (BCID2) Panel is a multiplexed nucleic acid test intended for use with BIOFIRE FILMARRAY 2.0 or BIOFIRE FILMARRAY TORCH Systems for the simultaneous qualitative detection and identification of multiple bacterial and yeast nucleic acids and select genetic determinants associated with antimicrobial resistance. The BIOFIRE BCID2 Panel test is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system. Results are intended to be interpreted in conjunction with Gram stain results. The following organism types and subtypes are identified using the BIOFIRE BCID2 Panel: Gram Positive Bacteria Enterococcus faecalis Staphylococcus spp. Streptococcus spp. Enterococcus faecium Staphylococcus aureus Streptococcus agalactiae (Group B) Listeria monocytogenes Staphylococcus epidermidis Streptococcus pneumoniae Staphylococcus lugdunensis Streptococcus pyogenes (Group A) Gram Negative Bacteria Acinetobacter calcoaceticus-baumannii complex Enterobacterales Bacteroides fragilis Enterobacter cloacae complex Haemophilus influenzae Escherichia coli Neisseria meningitidis (encapsulated) Klebsiella aerogenes Pseudomonas aeruginosa Klebsiella oxytoca Stenotrophomonas maltophilia Klebsiella pneumoniae group Proteus spp. Salmonella spp. Serratia marcescens Yeast Candida albicans Candida krusei Cryptococcus neoformans/gattii Candida auris Candida parapsilosis Candida glabrata Candida tropicalis The BIOFIRE BCID2 Panel contains assays for the detection of genetic determinants associated with resistance to methicillin (mecA/C and mecA/C in conjunction with MREJ, vancomycin (vanA and vanB), ß-lactams including penicillins, cephalosporins, monobactams, and carbapenems (blaCTX-M, blaIMP, blaKPC, blaNDM, blaOXA48-like, blaVIM) to aid in the identification of potentially antimicrobial-resistant organisms in positive blood culture samples. In addition, the panel includes an assay for the mobilized genetic determinant mor-1, an emerging marker of public health importance. The antimicrobial resistance gene or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of resistance to methicillin, vancomycin, b-lactams, and colistin exist. Antimicrobial Resistance Genes CTX-M KPC mecA/C NDM vanA/B IMP mcr-1 mecA/C and MREJ (MRSA) OXA-48-like VIM The BIOFIRE BCID2 Panel is indicated as an aid in the diagnosis of bloodstream infection and results should be used in conjunction with other clinical and laboratory findings. Positive results do not rule out co-infection with organisms not included in the BIOFIRE BCID2 Panel. The BIOFIRE BCID2 Panel is not intended to monitor treatment for bloodstream infection. Subculturing of positive blood cultures is necessary to recover organisms for susceptibility testing and epidemiological typing, to identify organisms in the blood culture that are not detected by the BIOFIRE BCID2 Panel, and for determination of species detected but not identified within complexes, groups, or genera by the BIOFIRE BCID2 Panel assays.

The BIOFIRE Blood Culture Identification 2 (BCID2) Panel is designed to simultaneously identify 43 bacteria and yeast responsible for bloodstream infections, as well as select genetic determinants of antimicrobial resistance (see Table 1), in a timeframe (about an hour) that allows the test results to be used in determining appropriate patient treatment and management. The BIOFIRE BCID2 Panel is performed directly on positive blood culture samples. The BIOFIRE BCID2 Panel is compatible with BioFire's PCR-based in vitro diagnostic BIOFIRE FILMARRAY 2.0 and FILMARRAY TORCH systems for infectious disease testing. A specific software module (i.e., BIOFIRE BCID2 Panel pouch module) is used to perform BIOFIRE BCID2 Panel testing on these systems. A test is initiated by loading Hydration Solution into one port of the BIOFIRE pouch and positive blood culture specimen mixed with the provided Sample Buffer into the other port of the BIOFIRE BCID2 Panel pouch and placing it in a BIOFIRE System. The pouch contains all the reagents required for specimen testing and analysis in a freeze-dried format; the addition of Hydration Solution and Sample/Buffer Mix rehydrates the reagents. After the pouch is prepared, the BIOFIRE Software guides the user though the steps of placing the the instrument, scanning the pouch barcode, entering the sample identification, and initiating the run. The BIOFIRE System contains a coordinated system of inflatable bladders and seal points, which act on the pouch to control the movement of liquid between the pouch blisters. When a bladder is inflated over a reagent blister, it forces liquid from the blister into connecting channels. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically-controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the pouch to drive the PCR reactions and the melt curve analysis. Nucleic acid extraction occurs within the BIOFIRE pouch using mechanical and chemical lysis followed by purification using standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, the BIOFIRE system performs a nested multiplex PCR that is executed in two stages. During the first stage, the BIOFIRE System performs a single, large volume, highly multiplexed reverse transcription PCR (rt-PCR) reaction. The products from first stage PCR are then diluted with a fresh, primer-free master mix and a fluorescent double stranded DNA binding dye (LC Green® Plus, BioFire Diagnostics). The solution is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The 2nd stage PCR, or nested PCR, is performed in single plex fashion in each well of the array. At the end of the 2nd stage PCR, the array is interrogated by melt curve analysis for the detection of signature amplicons denoting the presence of specific targets. A digital camera placed in front of the 2nd stage PCR captures fluorescent images of the PCR reactions and software interprets the data. The BIOFIRE Software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel.

Here's a breakdown of the acceptance criteria and the study information based on the provided FDA 510(k) summary for the BioFire Blood Culture Identification 2 (BCID2) Panel:

1. Table of Acceptance Criteria and Reported Device Performance:

The document primarily focuses on a software update to mitigate a risk of false negative results for C. tropicalis specifically. It states that the re-analysis with the modified software led to a "very minor adjustment to clinical sensitivity for S. epidermidis from 96.5% [93.0-98.2%] to 96.9%".

The summary does not provide a comprehensive table of acceptance criteria and the overall device performance for all targets. Instead, it highlights the impact of the software update on C. tropicalis detection and the resulting minor change in S. epidermidis sensitivity.

Acceptance Criteria (Implied / Pre-established for Original Device):

While not explicitly stated as "acceptance criteria" in this specific document, the overall context of a 510(k) clearance implies that the device must demonstrate comparable performance to its predicate. The key focus of this Special 510(k) is to show that the software update does not adversely affect, and in fact, improves, the device's performance, particularly concerning the false negative C. tropicalis issue. The acceptance criterion for this update seems to be that the corrected software resolves the C. tropicalis issue and does not significantly degrade performance on other targets.

2. Sample Size Used for the Test Set and Data Provenance:

• Test Set Sample Size: The document refers to "reanalysis of the performance data (clinical and non-clinical studies) with the modified pouch module software." It explicitly mentions that the C. tropicalis issue was identified during testing of Maine Molecular Quality Controls, Inc. (MMQC) FilmArray BCID2 Control Panel M416. While the specific number of tests performed on this control or other test sets with the modified software is not given in this summary, the "reanalysis of the performance data" implies that the original clinical and non-clinical study data from the predicate device (K193519) was re-evaluated using the new software.
• Data Provenance: The issue was identified through testing with an external control material (C. tropicalis synthetic control material). The "reanalysis of the performance data (clinical and non-clinical studies)" implies that data from the original (K193519) approval was used, which would have primarily involved prospective clinical samples from various geographic locations. However, this specific document does not detail the provenance of the original clinical data. The new data generated to address the C. tropicalis issue was from synthetic control material.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts:

• This information is not provided in the given FDA summary. The document focuses on a software update to an already cleared device, addressing a technical issue related to melting temperature analysis. The ground truth for the original clinical studies would have been established through a combination of clinical methods (e.g., standard microbiological culture and identification, susceptibility testing), but those details are not part of this specific Special 510(k) summary.

4. Adjudication Method for the Test Set:

• This information is not provided in the given FDA summary. For the original clinical studies related to the predicate device, an adjudication method (e.g., expert panel review of discrepancies) would likely have been employed, but this specific document does not detail it.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance:

• This is not applicable. The device (BIOFIRE Blood Culture Identification 2 Panel) is a fully automated multiplex nucleic acid test. It does not involve human readers interpreting results in the same way an imaging AI device would. Therefore, an MRMC study with human readers and AI assistance is not relevant to this device.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done:

• Yes. The BIOFIRE BCID2 Panel operates as a standalone diagnostic system. The "pouch module software" automatically interprets the melt curve analysis and combines data with internal controls to generate a test result. The description explicitly states, "The BIOFIRE Software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel." There is no "human-in-the-loop" performance component for result generation, although clinical interpretation of results in conjunction with Gram stain and other findings is required.

7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.):

• For the specific issue addressed in this Special 510(k) regarding C. tropicalis, the ground truth was known composition of synthetic control material.
• For the broader re-analysis of "clinical and non-clinical studies" (referring to the original predicate data), the ground truth for microbial identification in blood cultures typically involves standard microbiological culture (subculturing) and definitive identification methods (e.g., biochemical tests, mass spectrometry, sequencing), often supplemented by clinical context. The document emphasizes that "Subculturing of positive blood cultures is necessary to recover organisms for susceptibility testing and epidemiological typing, to identify organisms in the blood culture that are not detected by the BIOFIRE BCID2 Panel, and for determination of species detected but not identified within complexes, groups, or genera by the BIOFIRE BCID2 Panel assays," highlighting the role of traditional microbiology as a gold standard.

8. The Sample Size for the Training Set:

• This information is not provided in the given FDA summary, as the nature of this submission is a software update to an existing device, not a de novo algorithm development. The "pouch module software" utilizes pre-defined melting temperature (Tm) ranges and analysis parameters. The "reanalysis to include data generated from synthetic control materials" effectively served to refine or "retrain" specific parameters (Tm ranges) within the existing analytical framework, but the original training set size for the entire panel's algorithm is not detailed here.

9. How the Ground Truth for the Training Set Was Established:

• This information is not provided in the given FDA summary. For the original development of the BIOFIRE BCID2 Panel, the ground truth for establishing the algorithms' detection parameters and Tm ranges would have been based on extensive analytical studies using well-characterized microbial isolates (strains with confirmed identities) at various concentrations, likely identified by definitive culture-based methods and sequencing. The update specifically states that "Data from the synthetic control material(s) had not been included in the establishment and validation of the assay Tm ranges used for analysis by the pouch module software," indicating that the "training" for the C. tropicalis Tm range was incomplete previously and was corrected by incorporating this synthetic control material data.

Ask a specific question about this device

The LIAISON PLEX Yeast Blood Culture (BCY) Assay is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on the LIAISON PLEX System for simultaneous detection and identification of multiple potentially pathogenic fungal organisms in positive blood culture. The LIAISON PLEX BCY Assay is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain fungal organisms as determined by Gram Stain. The LIAISON PLEX BCY Assay detects and identifies the following fungal organisms:

Candida albicans
Candida auris
Candida dubliniensis
Candida famata
Candida glabrata
Candida guilliermondii
Candida kefyr
Candida krusei
Candida lipolytica
Candida lusitaniae
Candida parapsilosis
Candida tropicalis
Candida haemulonii / duobushaemulonii
Cryptococcus neoformans / gattii

The detection and identification of specific fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from LIAISON PLEX BCY Assay are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment management decisions.

Negative results in the setting of a suspected bloodstream infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by LIAISON PLEX BCY Assay may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by LIAISON PLEX BCY Assay, susceptibility testing and differentiation of mixed growth) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.

The LIAISON PLEX "Yeast Blood Culture Assay (BCY Assay) is performed directly on blood culture media using blood culture bottles identified as positive by a continuous monitoring blood culture system, and which contain a fungal organism, as determined by a Gram stain. The system consists of an instrument, a single-use disposable test cartridge, and a transfer pipette. The user loads the sample into the sample port of the LIAISON PLEX Yeast Blood Culture Assay Cartridge. Next, the user sets up the sample order on the LIAISON PLEX System by first entering the sample information or scanning the barcode ID located on the sample tube, then scanning the barcode ID located on the test cartridge. Last, the user inserts the test cartridge into the processing module to initiate the test. The LIAISON PLEX System identifies the assay being run and automatically initiates the proper testing protocol to process the sample, analyze the data, and generate test results.

The LIAISON PLEX System automates the BCY Assay sample analysis through the following steps: a) Sample Preparation: Nucleic acid extraction via mechanical and chemical cell lysis and magnetic bead-based nucleic acid isolation; b) Amplification: Multiplex PCR based amplification of the extracted nucleic acid to generate target specific amplicons; c) Hybridization: Amplified DNA hybridizes to specific capture DNA arrayed on a glass slide in a microarray format and the bound target DNA, in turn, hybridizes with mediator and gold-nanoparticle probes; d) Signal Analysis: Gold nanoparticle probes bound specifically to target-containing spots in the microarray are silver-enhanced, and light scatter from the spots is measured and further analyzed to determine the presence (Detected) or absence (Not Detected) of a target.

Acceptance Criteria and Device Performance for LIAISON PLEX Yeast Blood Culture Assay

The LIAISON PLEX Yeast Blood Culture (BCY) Assay is a qualitative nucleic acid multiplex in vitro diagnostic test for the simultaneous detection and identification of multiple potentially pathogenic fungal organisms in positive blood cultures. The study summarized below aimed to demonstrate the device meets its acceptance criteria through analytical and clinical performance evaluations.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for the LIAISON PLEX BCY Assay clinical study were predefined as:

• Sensitivity: ≥ 90% for all targets
• Specificity: ≥ 95% for each target
• Failure rate: ≤ 15%

The clinical performance data, combining prospective and pre-selected specimens, is summarized in the table below, comparing it against the acceptance criteria.

Table 1: LIAISON PLEX BCY Assay Clinical Performance (Acceptance Criteria vs. Reported Performance)

NA = Not applicable, as there were no positive cases for this target in the combined prospective/pre-selected dataset to calculate sensitivity, or no negative cases to calculate specificity.

The reported performance demonstrates that the LIAISON PLEX BCY Assay met all defined acceptance criteria.

2. Sample Sizes Used for the Test Set and Data Provenance

The test set for the clinical performance evaluation utilized a combination of prospective, pre-selected, and contrived specimens:

• Prospective Specimens (Clinical Study): 69 unique specimens from four geographically diverse clinical sites in the United States. These were collected prospectively between June 2023 and October 2023. One initial specimen was excluded due to an inconclusive Gram stain result.
• Pre-selected Specimens (Clinical Study): 63 remnant, de-identified specimens sourced from 6 different sites/vendors in the United States. The data provenance is retrospective clinical specimens that were pre-characterized.
• Contrived Specimens: 829 specimens were artificially generated. These were blinded, randomized, and tested at all four testing sites during June 2023. These are contrived data.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

The provided document does not explicitly state the number of experts or their specific qualifications (e.g., radiologist with 10 years of experience) used to establish the ground truth for the test set.

However, the ground truth for the clinical study was established by comparing the LIAISON PLEX BCY Assay's performance against a Standard of Care culture followed by identification by Matrix Assisted Laser Desorption/Ionization coupled to time-of-flight Mass Spectrometry (MALDI-TOF MS) for all fungal targets. For the pre-selected specimens, it is mentioned that they were "characterized by an FDA cleared molecular assay prior to enrollment in the study." This implies that the ground truth relied on established laboratory methodologies and potentially an FDA-cleared reference method.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method like "2+1" or "3+1" using human experts for the clinical study. The comparison was made against a "Standard of Care culture followed by identification by MALDI-TOF MS" as the reference method, which serves as the definitive ground truth for the presence and identification of the fungal organisms. For the pre-selected specimens, an "FDA cleared molecular assay" was used for characterization prior to study entry. These are objective laboratory methods rather than subjective expert interpretations requiring adjudication.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not reported. This device is an in vitro diagnostic (IVD) assay designed for automated detection and identification of fungal organisms directly from blood culture samples. Its performance is evaluated against laboratory reference methods (culture and MALDI-TOF MS, or existing molecular assays), not against human reader interpretation of images or other subjective assessments. Therefore, there is no discussion of how human readers improve with or without AI assistance, as AI assistance is not part of the described use case for this IVD device.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, the performance presented for the LIAISON PLEX BCY Assay in the analytical and clinical performance sections is for the standalone algorithm/device performance directly on the sample, without human interpretation or intervention in the diagnostic output. The system automates sample analysis, including nucleic acid extraction, amplification, hybridization, and signal analysis, to generate detected or not detected results. The indication for use clearly states the results are "intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment management decisions," but this refers to the clinical application context, not a human-in-the-loop for the device's fundamental diagnostic accuracy calculation.

7. The Type of Ground Truth Used

The primary ground truth used for the clinical study was:

• Standard of Care culture followed by identification by Matrix Assisted Laser Desorption/Ionization coupled to time-of-flight Mass Spectrometry (MALDI-TOF MS). This is a laboratory-based, objective method for identifying microorganisms.
• For pre-selected specimens, ground truth was established by an previously FDA-cleared molecular assay.
• For contrived specimens, the ground truth was inherently known based on how the samples were prepared (spiking known organisms).

8. The Sample Size for the Training Set

The document does not report specific sample sizes for a separate training set for this device. The information provided heavily details analytical and clinical verification/validation (test set) studies. For diagnostic devices like this (nucleic acid assays), the development process typically involves extensive analytical characterization (Limit of Detection, Inclusivity, Exclusivity, Interference, Reproducibility) and then clinical validation. It is probable that internal development, optimization, and early verification studies used various sets of samples, but these are not explicitly termed "training sets" in the context of machine learning model training as one might expect for AI/ML-based diagnostic software.

9. How the Ground Truth for the Training Set Was Established

As no explicit "training set" is documented (in the context of AI/ML), there is no description of how ground truth was established for such a set. However, for the analytical studies and development of the assay, ground truth would have been established through well-characterized reference strains and clinical isolates, quantified using standard microbiology techniques (e.g., CFU/mL for Limit of Detection). These methods involve culturing, molecular characterization, and established laboratory practices to confirm the identity and concentration of the microorganisms.

Ask a specific question about this device

The GenMark ePlex® Blood Culture Identification Gram-Negative (BCID-GN) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous qualitative detection and identification of multiple potentially pathogenic gram-negative bacterial organisms and select determinants associated with antimicrobial resistance in positive blood culture. In addition, the ePlex BCID-GN Panel is capable of detecting several gram-positive bacteria (Pan Gram-Positive assay) and several Candida species (Pan Candida assay). The ePlex BCID-GN Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain gram-negative organism.

The following bacterial organisms and genes associated with antibiotic resistance are identified using the ePlex BCID-GN Panel: Acinetobacter baumannii, Bacteroides fragilis, Citrobacter sakazakii, Enterobacter cloacae complex, Enterobacter (non-cloacae complex), Escherichia coli, Fusobacterium necrophorum, Fusobacterium nucleatum, Haemophilus influenzae, Klebsiella oxytoca, Klebsiella pneumoniae group, Morganii, Neisseria meningitidis, Proteus, Proteus mirabilis, Pseudomonas aeruginosa, Salmonella, Serratia marcescens, Stenotrophomonas maltophilia, CTX-M (blaCTX-M), IMP (blaMP) , KPC (blaKPC) , NDM (blaNDM), OXA (blaOXA) (OXA-23 and OXA-48 groups only), and VIM (blaVIM).

The ePlex BCID-GN Panel contains assays for the detection of genetic determinants associated with resistance to antimicrobial agents including CTX-M(blaCTX-M), which is associated with resistance to extended spectrum betalactamase (ESBL)-mediated resistance to penicillins, cephalosporins, and monobactams, as well as OXA (blaOXA) (OXA-23 and OXA-48 groups only), KPC (blaKPC), and metallo-beta-lactamases IMP (blaIMP), and NDM (blaNDM), which is associated with carbapenemase-mediated resistance. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance assays do not indicate susceptibility, as there are multiple mechanisms of resistance in gramnegative bacteria.

The ePlex BCID-GN Panel also contains targets designed to detect a broad range of organisms with a potentially misleading Gram stain result or organisms that may be missed by Gram staining altogether, for example in the case of coinfections. These include a broad Pan Gram-Positive assay (which is designed to detect Bacillus cereus group, Bacillus subtilis group, Enterococcus, Staphylococus, and Streptococcus), as well as a Pan Candida assay, which is designed to detect four Candida species: Candida albicans, Candida krusei, and Candida parapsilosis.

The detection and identification of specific bacterial and fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-GN Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

Negative results in the setting of a suspected bloodstream infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-GN Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-GN Panel and for susceptibility testing, differentiation of mixed growth, and association of antimicrobial resistance marker genes to a specific organism) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.

The ePlex Blood Culture Identification Gram-Negative (BCID-GN) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.

Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.

During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.

The presented document is a 510(k) summary for the GenMark ePlex Blood Culture Identification Gram-Negative (BCID-GN) Panel, a qualitative nucleic acid multiplex in vitro diagnostic test. The study aims to demonstrate that the updated device (Subject Device) is substantially equivalent to its predicate device (original GenMark ePlex BCID-GN Panel, K182619). The data focuses on analytical and clinical performance.

Here's an analysis based on the provided text, addressing your specific points:

1. A table of acceptance criteria and the reported device performance:

The document doesn't explicitly state a single "acceptance criteria" table with pre-defined thresholds for all metrics (like sensitivity, specificity) against which the reported performance is directly compared in a summary table. However, it implicitly demonstrates acceptance by presenting individual performance metrics (Sensitivity/PPA and Specificity/NPA) for each target organism and resistance gene across different sample types (Prospective, Retrospective, Contrived, and Overall). The consistent high percentages for these metrics indicate that the device met the required performance for regulatory acceptance, even if the precise numerical cut-offs aren't explicitly stated in a singular table for all parameters.

Instead of a single "acceptance criteria" table, the document functions as a detailed report of performance against implicit acceptance criteria for in vitro diagnostic devices, which typically demand high sensitivity and specificity. The data tables already present the "reported device performance."

Example of reported device performance for a few key targets (extracted from Tables 7-34):

(Note: "Overall" for Pan targets combines Prospective, Retrospective, and Retrospective (Non-Intended Use), but excludes Contrived. The overall figures for other targets combine Prospective/Retrospective and Contrived samples as a whole.)

2. Sample sized used for the test set and the data provenance:

• Test Set Sample Size:

  • Clinical Samples: 349 prospective samples (167 fresh, 182 frozen) and 577 retrospective samples. Total clinical samples: 926.
  • Contrived Samples: 777 samples.
  • Additional Retrospective (Non-Intended Use) for Pan targets: 741 samples.
  • Total evaluable samples across studies: 349 (prospective) + 577 (retrospective) + 777 (contrived) + 741 (non-intended use for pan targets) = 2444 samples in total tested across various evaluations. The overall performance tables combine various subsets of these.

• Data Provenance:

  • Country of Origin: Not explicitly stated, but 7 clinical sites were involved in prospective collection (suggests multi-center, likely within the US given FDA submission).
  • Retrospective or Prospective: Both.
    • Prospective: 349 samples collected from June 2014 through July 2016 (frozen) and June through July 2018 (fresh).
    • Retrospective: 577 samples collected.
    • Contrived: Laboratory-generated samples.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

The document does not explicitly state the "number of experts" or their specific "qualifications" involved in establishing the ground truth. It refers to "standard laboratory procedures for identification of blood culture isolates, including traditional and automated identification methods, MALDI-TOF IVD, and microbiological and biochemical techniques" (Table 4). For antibiotic resistance genes, it uses "analytically validated qPCR amplification assays followed by bi-directional sequencing." This implies laboratory professionals with expertise in microbiology and molecular diagnostics perform these comparator methods, but specific numbers or individual qualifications are not detailed.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

The document does not describe an explicit "adjudication method" involving multiple human readers (e.g., 2+1 or 3+1). The ground truth is established through comparator methods as described above (standard laboratory procedures, PCR/sequencing). Any discrepancies between the device and these comparator methods are analyzed and explained (e.g., the detailed footnotes in the performance tables and the discussion regarding CTX-M false negatives). This is typical for in vitro diagnostic (IVD) device studies, where ground truth is often determined by a reference laboratory standard or follow-up confirmatory testing, rather than human expert consensus on image interpretation.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) test for direct detection of pathogens and resistance genes from blood cultures, not an "AI-assisted image interpretation" device to be used by human readers. Therefore, the concept of human readers improving with AI assistance is not applicable here.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

Yes, this study represents a standalone technical performance evaluation of the ePlex BCID-GN Panel device. The device itself performs the detection and identification, and its results are compared directly to the gold standard comparator methods. There is no "human-in-the-loop" performance element in the operation of this specific diagnostic test.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

The ground truth was established by a combination of:

• Standard Laboratory Procedures: including traditional and automated identification methods, MALDI-TOF IVD, and microbiological and biochemical techniques for organism identification.
• Molecular Confirmation: Analytically validated PCR assays followed by bi-directional sequencing for specific organism identifications (e.g., Acinetobacter baumannii, Candida parapsilosis) and for all antibiotic resistance genes (qPCR amplification followed by bi-directional sequencing).
• Additional Testing for Discrepancies: Further investigations (e.g., repeat extractions, qPCR testing from isolates, testing with FDA-cleared multiplex assays) were used to resolve discrepancies and confirm the true status of samples, as detailed in the footnotes for several performance tables (e.g., Table 28 for CTX-M).

This ground truth method is based on a hierarchy of established laboratory and molecular techniques rather than human expert consensus on interpretation.

8. The sample size for the training set:

The document does not explicitly mention a "training set" in the context of machine learning, as this is a molecular diagnostic device, not an AI/ML product. However, the development of such a device involves extensive analytical studies related to inclusivity (reactivity), exclusivity (specificity), and limit of detection (LoD), which are analogous to data used in the development or "training" phase.

• Analytical Reactivity (Inclusivity): Evaluated with a panel of 336 strains/isolates.
• Limit of Detection (LoD): Determined using quantified reference strains for each target.
• In silico analysis: Used for predicted reactivity of genus/group assays and resistance markers, involving evaluation of sequence data.

While not a "training set" in the AI sense, these analytical studies inform the design and performance characteristics of the diagnostic assays.

9. How the ground truth for the training set was established:

As noted above, there isn't a "training set" in the AI/ML sense. For the analytical studies that are foundational to the device's design (e.g., inclusivity, LoD):

• Ground truth for inclusivity (analytical reactivity): Established by using characterized strains/isolates with known identity. The strains' identities are determined by standard microbiological and molecular methods.
• Ground truth for LoD: Established by using quantified reference strains where the concentration (CFU/mL) of the organism is precisely known.
• Ground truth for in silico analysis: Based on existing genetic sequence data and bioinformatic analysis to predict reactivity, relying on established genetic databases and characterizations.

Ask a specific question about this device

The BioFire® Blood Culture Identification 2 (BCID2) Panel is a multiplexed nucleic acid test intended for use with FilmArray® 2.0 or FilmArray® Torch systems for the simultaneous qualitative detection and identification of multiple bacterial and yeast nucleic acids and select genetic determinants associated with antimicrobial resistance. The BioFire BCID2 Panel test is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system. Results are intended to be interpreted in conjunction with Gram stain results. The following organism types and subtypes are identified using the BioFire BCID2 Panel:

Gram Positive Bacteria

• Enterococcus faecalis
• · Staphylococcus spp.
• · Streptococcus spp.
• Enterococcus faecium
• Staphylococcus aureus
• · Streptococcus agalactiae (Group B)
• Listeria monocytogenes
• Staphylococcus epidermidis
• Streptococcus pneumoniae
• Staphylococcus lugdunensis
• · Streptococcus pyogenes (Group A)

Gram Negative Bacteria

• Acinetobacter calcoaceticus-baumannii complex
• · Enterobacterales
• · Bacteroides fragilis
• Enterobacter cloacae complex
• Haemophilus influenza
• · Escherichia coli
• · Neisseria meningitidis (encapsulated)
• · Klebsiella aerogenes
• · Pseudomonas aeruginosa
• · Klebsiella oxytoca
• · Stenotrophomonas maltophilia
• · Klebsiella pneumoniae group
• · Proteus spp.
• · Salmonella spp.
• · Serratia marcescens

Yeast

• · Candida albicans
• Candida krusei
• · Cryptococcus neoformans/gattii
• Candida auris
• · Candida parapsilosis
• · Candida tropicalis
• Candida glabrata

The BioFire BCID2 Panel contains assays for the detection of genetic determinants associated with resistance to methicillin (mecA/C and mecA/C in conjunction with MREJ, vancomycin (vanA and vanB), 0-lactams including penicillins, cephalosporins, monobactams, and carbapenems (blaCTX-M, blaKPC, blaNDM, blaOXA48-like, bla VIM) to aid in the identification of potentially antimicrobial-resistant organisms in positive blood culture samples. In addition, the panel includes an assay for the mobilized genetic determinant mcr-1, an emerging marker of public health importance. The animicrobial resistance gene or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene and marker assays do not indicate susceptibility, as multiple mechanisms of resistance to methicillin, vancomycin, B-lactams, and colistin exist.

Antimicrobial Resistance Genes

• CTX-M
• КРС
• · mecA/C
• NDM
• vanA/B
• · IMP
• mcr-1
• · mecA/C and MREJ (MRSA)
• OXA-48-like
• VIM

The BioFire BCID2 Panel is indicated as an aid in the diagnosis of bloodstream infection and results should be used in conjunction with other clinical and laboratory findings. Positive results do not rule out co-infection with organisms not included in the BioFire BCID2 Panel is not intended to monitor treatment for bloodstream infection.

Subculturing of positive blood cultures is necessary to recover organisms for susceptibility testing and epidemiological typing, to identify organisms in the blood culture that are not detected by the BioFire BCID2 Panel, and for determination of species detected but not identified within complexes, groups, or genera by the BioFire BCID2 Panel assays.

The BioFire Blood Culture Identification 2 (BCID2) Panel is designed to simultaneously identify 43 bacteria and yeast responsible for bloodstream infections, as well as select genetic determinants of antimicrobial resistance (see Table 1), in a timeframe(about an hour) that allows the test results to be used in determining appropriate patient treatment and management. The BioFire BCID2 Panel is performed directly on positive blood culture samples.

The BioFire BCID2 Panel is compatible with BioFire's PCR-based in vitro diagnostic FilmArray Torch systems for infectious disease testing, A specific software module (i.e., BioFire BCID2 Panel pouch module) is used to perform BioFire BCID2 Panel testing on these systems.

A test is initiated by loading Hydration Solution into one port of the FilmArray pouch and positive blood culture specimen mixed with the provided Sample Buffer into the other port of the BioFire BCID2 Panel pouch and placing it in a FilmArray instrument. The pouch contains all of the reagents required for specimen testing and analysis in a freeze-dried format: the addition of Hydration and Sample/Buffer Mix rehydrates the reagents. After the pouch is prepared, the FilmArray Software quides the user through the pouch into the instrument, scanning the pouch barcode, entering the sample identification, and initiating the run.

The FilmArray instruments contain coordinated systems of inflatable bladders and seal points, which act on the pouch to control the movement of liquid between the pouch blisters. When a bladder is inflated over a reagent blister, it forces liquid from the blister into connecting channels. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically-controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the PCR reactions and the melt curve analysis.

Nucleic acid extraction occurs within the FilmArray pouch using mechanical lysis followed by purification using standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, the FilmArray performs a nested multiplex PCR that is executed in two stages. During the first stage, the FilmArray performs a single, large volume, highly multiplexed reverse transcription PCR (rt-PCR) reaction. The products from first stage PCR are then diluted and combined with a fresh, primer-free master mix and a fluorescent double-stranded DNA binding dye (LC Green® Plus, BioFire Diagnostics). The solution is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The 2nd stage PCR, is is performed in singleplex fashion in each well of the conclusion of the 2nd stage PCR, the array is interrogated by melt curve analysis for the detection of signature amplicons denoting the presence of specific targets. A digital camera placed in front of the 2nd stage PCR captures fluorescent images of the PCR reactions and software interprets the data.

The FilmArray Software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel.

Here is a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally implied to be high sensitivity and specificity for each analyte. Rather than explicit criteria, the document presents the achieved performance.

2. Sample Sizes Used for the Test Set and Data Provenance

The "test set" for clinical performance was comprised of:

• Prospective Multi-center Study: 1093 residual PBC specimens, with 1074 evaluable for final analysis.
  • Data Provenance: Nine geographically distinct study sites (seven in the EU) participated. Data collected from October 2018 to May 2019. Specimens were a mix of fresh (1005) and frozen (69) for later testing.
• Archived Specimens: 427 frozen archived PBC specimens, with 395 evaluable. 370 contained confirmed analytes of interest.
  • Data Provenance: Collected from 12 external laboratories (countries not specified, but likely a mix given the mention of EU sites for prospective study). Retrospective.
• Seeded Blood Culture Specimens: 552 seeded specimens.
  • Data Provenance: Laboratory-generated.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document does not explicitly state the "number of experts" or their specific qualifications for establishing ground truth. However, it indicates the following methods were used for reference:

• Bacteria and Cryptococcus: Standard manual and automated microbiological/biochemical identification methods (performed by SOC labs and abstracted from medical charts).
• Candida species: SOC identification for genus level, followed by PCR & sequencing of isolates for species identification.
• AMR Genes:
  • Method 1: One PCR assay performed direct from PBC followed by sequencing of PCR amplicon.
  • Method 2: Commercially available FDA-cleared and CE-marked molecular IVD assays performed on PBC.
  • Method 3: PCR & sequencing for specific resistance gene from applicable cultured isolates.
  • Method 4: Phenotypic antimicrobial susceptibility testing (AST) methods on cultured isolates.

It can be inferred that trained laboratory personnel and potentially infectious disease or microbiology experts interpreted these results, consistent with standard clinical laboratory practices.

4. Adjudication Method for the Test Set

The document details investigations for discrepant results. For example:

• For Enterococcus faecalis FN specimens, additional molecular methods were used for re-testing.
• For Staphylococcus epidermidis FN specimens, additional molecular testing and sequencing were performed.
• For mecA/C and MREJ (MRSA) discrepancies, isolates were tested by PCR/sequencing and had phenotypic AST. Original Cepheid Xpert® MRSA/SA BC tests were also re-tested.

This suggests an internal adjudication process involving further molecular testing and phenotypic analysis to confirm the true status of discrepant samples, rather than a formal blinded 2+1 or 3+1 expert adjudication. Discrepant results were not simply resolved by majority vote but by further in-depth laboratory investigation.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No multi-reader multi-case (MRMC) comparative effectiveness study comparing human readers with AI assistance versus without AI assistance was mentioned or conducted. The BioFire BCID2 Panel is a standalone diagnostic device that provides qualitative detection and identification of microbial nucleic acids and resistance genes, not an assistive AI tool for human interpretation.

6. Standalone Performance

Yes, extensive standalone performance studies were conducted. The clinical performance tables (Tables 12-24) and the analytical performance sections (Limit of Detection, Analytical Reactivity (Inclusivity), Analytical Specificity (Cross-Reactivity and Exclusivity), Reproducibility, Interference) directly demonstrate the algorithm's performance without human intervention in result interpretation. The system automatically interprets results and generates reports.

7. Type of Ground Truth Used

The ground truth for the clinical studies was established using a combination of:

• Culture-based microbiology: Standard manual and automated microbiological/biochemical identification methods, followed by species identification where necessary (e.g., Candida species by PCR & sequencing).
• Molecular methods: PCR and sequencing of isolates or direct from positive blood cultures, and commercially available FDA-cleared/CE-marked molecular IVD assays for AMR genes.
• Phenotypic Antimicrobial Susceptibility Testing (AST): Used for AMR gene concordance, particularly for cases like mecA/C and MREJ (MRSA).

For seeded specimens, the "known analyte composition" served as ground truth.

8. Sample Size for the Training Set

The document does not explicitly state the sample size for a "training set" in the context of machine learning. The BioFire BCID2 Panel is a nucleic acid amplification test (PCR-based), where the "training" analogous to model development for AI would involve the design and optimization of primers and probes, and the setting of fluorescence thresholds for detection. This process is primarily based on analytical studies and known genetic sequences, not a labeled dataset like in typical AI development.

However, a large number of isolates were used in the Analytical Reactivity (Inclusivity) study (over 450 isolates) and Analytical Specificity (Cross-Reactivity and Exclusivity) study (many on-panel and off-panel organisms tested, some at high concentrations). These studies would contribute to the robust development and validation of the oligonucleotide sequences and parameters used in the panel tests.

9. How the Ground Truth for the Training Set Was Established

As explained above, for a nucleic acid amplification test like the BioFire BCID2 Panel, the "training set" doesn't strictly align with AI/ML terminology. The development of the assays (e.g., primer and probe design) relied on:

• In silico analysis of sequences: Public databases of genetic sequences were used to design assays that are inclusive of target strains and exclusive of non-target organisms.
• Testing of characterized isolates: Over 450 isolates (representing genetic, geographic, and temporal diversity) were tested at varying concentrations to ensure reliable detection. The "ground truth" for these isolates would have been their known identity (species/subspecies) and genetic markers (e.g., AMR genes) determined by established microbial identification techniques and sequencing (e.g., ATCC strains, clinical isolates with known characteristics).

This rigorous analytical evaluation forms the basis of the device's "knowledge" or "training" on targets and non-targets.

Ask a specific question about this device

The iCubate, Inc. iC-GN Assay™ for use on the iC-System™ is a qualitative, multiplexed, in vitro diagnostic test for the detection and identification of potentially pathogenic gram negative bacteria, which may cause bloodstream infection (BSI). The iC-GN Assay™ is performed directly on positive blood cultures, confirmed by Gram stain to contain gram negative bacilli. Cultures demonstrating mixed Gram stain results should not be tested on the assay. The iC-GN Assay™ is validated for use with select BACTEC™, BacT/ALERT® and VersaTREK® blood culture bottles. The iC-GN Assay™ is indicated for use in conjunction with other clinical and laboratory findings, such as culture, to aid in the diagnosis of bacterial bloodstream infections; however, it is not used to monitor bloodstream infections.

The iC-GN Assay™ detects target DNA and identifies the following:

Bacterial Genera and Species: Acinetobacter baumannii complex, Enterobacter cloacae complex, Escherichia coli, Klebsiella oxytoca, Klebsiella pneumoniae, Pseudomonas aeruginosa, Proteus species, Serratia marcescens

Resistance Markers: KPC (blaKPC)- associated with resistance to carbapenems, NDM (blaNDM)- associated with resistance to carbapenems, CTX-M group 1(blaCTX-M group 1)- associated with resistance to extended spectrum beta-lactams

In mixed growth, the iC-GN Assay™ does not specifically attribute detection of KPC, NDM, or CTX-M group 1 to a specific genera or species.

Sub-culturing of positive blood cultures is necessary to recover organisms for susceptibility testing, identification of organisms not detected by the iC-GN Assay™, differentiation of mixed growth, association of antimicrobial resistance marker genes to a specific organism, or for epidemiological typing.

The iC-GN Assay™ utilizes polymerase chain reaction (PCR) for the multiplex amplification of specific targets and detects the amplified targets with microarray hybridization. Targets are detected directly from patient positive blood cultures confirmed by Gram stain to contain gram negative bacilli. The iC-GN Assay utilizes proprietary ARM-PCR (Amplicon Rescued Multiplex PCR) technology allowing for multiple targets to be amplified in one reaction. Testing is done in a self-contained, automated, disposable cassette using the iCubate™ processor (iC-Processor™). After the reaction is complete, the cassette is read on the iCubate® reader (iC-Reader™). Results from the iC-Reader™ are interpreted by iC-Report™ software and a final report is displayed on the iMac® computer.

To operate, the user opens the iC-Cassette™ cap and pipettes an aliquot of the diluted positive blood culture sample into the sample/PCR well in the bottom well plate of the cassette. Once inoculated, the cassette cap is closed, and all extraction, amplification and detection processes are completed in the cassette, a closed system. Extraction, amplification and detection sequences are defined by an assay script controlled by the iC-Processor™.

The processing script is defined within a barcode label positioned on the top of each iC-Cassette™ which communicates with the iC-Processor™. To access and pierce the foilsealed reagent wells located in the bottom well plate of the cassette, the processor manipulates the cassette to move the cassette pipette horizontally and vertically. The script directs the transfer of reagents between the wells in the bottom well plate and finally to the array within the cassette. The iC-Processor™ is capable of processing four (4) iC-Cassettes™ with random access.

Once processing is complete, the cassette is manually transferred from the iC-Processor™ to the iC-Reader™ where the microarray within the cassette is read. The iC-Reader™ is capable of reading up to four (4) iC-Cassettes™ at one time. The results are interpreted via the iC-Report™ software and displayed for the user on the iMac®. Raw data and result interpretations are stored within the iMac®; raw data is accessible to iCubate® service personnel only and not to the end user.

When finished with a loaded iC-GN Cassette™, it should be disposed as biohazardous waste.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally implied by the "Percent Agreement" values (positive and negative) to reference methods. While specific numeric acceptance thresholds are not explicitly stated as "acceptance criteria," performance metrics of 95% or higher with tight confidence intervals are typically considered strong evidence of meeting performance expectations for such devices. For reproducibility and equivalency, "≥ 95% performance" is explicitly stated as the acceptance criteria.

2. Sample size used for the test set and the data provenance

• Method Comparison Study (Clinical Study):

  • Total specimens enrolled: 1107
  • Specimens included in performance analysis: 1002
    • Fresh prospective specimens: 976
    • Frozen prospective specimens (retrospectively tested): 26 (2.6%)
  • Contrived samples: 170
  • Data Provenance: Five geographically dispersed clinical sites in the U.S. (NY, WI, NM, FL, IN). The specimens were "leftover de-identified specimens from anaerobic blood culture bottles flagged as positive." This indicates a retrospective collection of clinical samples taken from patients, which were then de-identified and used for the study.

• Reproducibility Study:

  • Eighteen-organism panel tested in triplicate across five, non-consecutive days by two independent operators at each of three sites. This implies approximately 18 organisms * 3 replicates * 5 days * 2 operators * 3 sites = 1620 tests, but the table breaks down performance by target with denominators of ~90, suggesting a more specific number per target/concentration.

• Limit of Detection (LoD) Study:

  • Twenty-seven representative strains, a minimum of three per target. Tested in a minimum of twenty replicates (for confirmation phase) on each of three unique cassette lots.

• Inclusivity Study:

  • Eighty-two (82) representative strains, a minimum of ten strains for each target analyte. Each strain tested in triplicate.

• Exclusivity Study:

  • A total of 114 strains. Each strain tested in triplicate (retested in replicates of 3 or 10 for discordant results).

• Microbial Interference Study:

  • Sixty (60) gram negative exclusivity strains in combination with eight (8) iC-GN target organisms. Each combination tested in triplicate (retested in replicates of 3 or 10 for discordant results).

• Competitive Inhibition Study:

  • One target organism (low concentration) combined with a second target organism (high concentration). Each combination tested in triplicate (retested in replicates of 10 for false negatives).

• Interfering Substances Study:

  • Eight representative target organisms plus one non-target organism. Organism/interferent combination tested in triplicate (retested in replicates of 10 or 3 for discordant results).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The document does not specify the number or qualifications of experts used to establish the ground truth. It states that the ground truth for the clinical method comparison was based on:

• "reference culture followed by MALDI identification" for organism targets.
• "PCR amplification followed by confirmatory bidirectional sequencing" for resistance markers.
• "Phenotypic antimicrobial susceptibility testing (AST)" was also performed to identify samples requiring sequencing.

While these are standard laboratory methods, the involvement of specific "experts" (e.g., microbiologists, infectious disease specialists) beyond performing these standard laboratory procedures is not detailed.

4. Adjudication method for the test set

The document does not describe a formal adjudication method (like 2+1 or 3+1). For the method comparison study, discordant samples were "sequenced" (for resistance markers) or further analyzed by PCR/bi-directional sequencing for organism targets. For other performance studies (inclusivity, exclusivity, microbial interference), discordant results led to retesting in replicates (3 or 10). This indicates a re-testing approach for discrepancies rather than a multi-expert consensus.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. The iC-GN Assay is an in vitro diagnostic (IVD) device, specifically a molecular diagnostic test for identifying microorganisms and resistance markers from positive blood cultures. It does not involve "human readers" interpreting images or data in a way that would typically be evaluated in an MRMC study for AI assistance. Its performance is compared against laboratory reference methods.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

Yes, the studies presented evaluate the standalone performance of the iC-GN Assay. The assay is an automated system (iC-System) that detects and identifies targets directly from positive blood cultures using PCR and microarray hybridization. The results are interpreted by software (iC-Report™) and displayed for the user. The performance data (reproducibility, LoD, inclusivity, exclusivity, microbial interference, competitive inhibition, and method comparison) all reflect the device's inherent analytical and clinical performance without a human interpreting the primary result, but rather the system's output. The human role is operational (loading samples, reading the final report), not interpretive of the raw data.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used for the clinical method comparison study was established using:

• Reference culture followed by MALDI identification: This is a combination of microbiological culture methods and advanced biochemical analysis to identify bacterial species.
• PCR amplification followed by confirmatory bidirectional sequencing: This is a molecular method used to confirm the presence and identity of specific genes (e.g., resistance markers) or bacterial species through DNA sequencing.
• Phenotypic antimicrobial susceptibility testing (AST): Used to guide which resistance markers might need further sequencing for confirmation.

For other analytical studies (Inclusivity, Exclusivity, LoD, etc.), the ground truth was based on the known identity and concentration of the characterized bacterial strains used.

8. The sample size for the training set

The document does not provide details about a "training set" in the context of an algorithm or AI development. This device is a molecular diagnostic test, not an AI/ML-based diagnostic that would typically involve a separate training, validation, and test set for algorithm development. The studies described are performance validation studies for the finished medical device.

9. How the ground truth for the training set was established

As there's no explicit mention of a "training set" for an algorithm, the method for establishing its ground truth is not described. The ground truth for the verification and validation studies (as detailed in point 7) was established through standard microbiological and molecular laboratory techniques, which served as the comparator for assessing device performance.

Ask a specific question about this device

The GenMark ePlex® Blood Culture Identification Gram-Negative (BCID-GN) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous qualitative detection and identification of multiple potentially pathogenic gram-negative bacterial organisms and select determinants associated with antimicrobial resistance in positive blood culture. In addition, the ePlex BCID-GN Panel is capable of detecting several gram-positive bacteria (Pan Gram-Positive assay) and several Candida species (Pan Candida assay). The ePlex BCID-GN Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain gram-negative organism.

The following bacterial organisms and genes associated with antibiotic resistance are identified using the ePlex BCID-GN Panel: Acinetobacter baumannii, Bacteroides fragilis, Citrobacter, Cronobacter sakazakii. Enterobacter cloacae complex, Enterobacter (non-cloacae complex), Escherichia coli, Fusobacterium necrophorum, Fusobacterium nucleatum, Haemophilus influenzae, Klebsiella oxytoca, Klebsiella pneumoniae group, Morganella morganii, Neisseria meningitidis, Proteus, Proteus mirabilis, Pseudomonas aeruginosa, Salmonella, Serratia, Serratia marcescens, Stenotrophomas maltophilia, СТХ-М (blactх-м), IMP (blамм) , КРС (blakec) , NDM (bland), OXA (blaoxa) (OXA-23 and OXA-48 groups only), and VIM (blaviм).

The ePlex BCID-GN Panel contains assays for the detection of genetic determinants associated with resistance to antimicrobial agents including CTX-M(blactx.M), which is associated with resistance to extended spectrum beta-lactamase (ESBL)-mediated resistance to penicillins, cephalosporins and monobactams, as well as OXA (blaoxA) (OXA-23 and OXA-48 groups only), KPC (blakpc), and metallo-beta-lactamases IMP (blavM), VIM (blavM), and NDM (blaNDM), which is associated with carbapenemase-mediated resistance. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance assays do not indicate susceptibility, as there are multiple mechanisms of resistance in gram-negative bacteria.

The ePlex BCID-GN Panel also contains targets designed to detect a broad range of organisms with a potentially misleading Gram stain result or organisms that may be missed by Gram staining altogether, for example in the case of co-infections. These include a broad Pan Gram-Positive assay (which is designed to detect Bacillus cereus group, Bacillus subtilis group, Enterococcus, Staphylococcus, and Streptococcus), as well as a Pan Candida assay, which is designed to detect four Candida species: Candida albicans, Candida glabrata, Candida krusei, and Candida parapsilosis.

The detection and identification of specific bacterial and fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-GN Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

Negative results in the setting of a suspected bloodstream infection may be due to infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-GN Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-GN Panel and for susceptibility testing, differentiation of mixed growth, and association of antimicrobial resistance marker genes to a specific organism) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.

The ePlex Blood Culture Identification Gram-Negative (BCID-GN) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.

Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.

During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.

Here's a summary of the acceptance criteria and study information for the ePlex Blood Culture Identification Gram Negative (BCID-GN) Panel, extracted from the provided text:

Acceptance Criteria and Device Performance for ePlex Blood Culture Identification Gram Negative (BCID-GN) Panel

The ePlex BCID-GN Panel is a qualitative nucleic acid multiplex in vitro diagnostic test for the simultaneous qualitative detection and identification of multiple potentially pathogenic gram-negative bacterial organisms and select determinants associated with antimicrobial resistance in positive blood cultures. Performance characteristics were evaluated through clinical and analytical studies.

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are generally implied by the achieved performance metrics, which show high Positive Percent Agreement (PPA) for sensitivity and Negative Percent Agreement (NPA) for specificity. While explicit numerical acceptance thresholds are not directly stated as "acceptance criteria" in a dedicated section, the provided tables demonstrate the device's performance against its intended use. For brevity, the "Overall" performance (combining prospective, retrospective, and contrived samples where applicable) is used here to represent the device's reported performance against implied high accuracy criteria.

2. Sample Sizes Used for the Test Set and Data Provenance

The test set comprised a combination of clinical and contrived samples:

• Prospective Clinical Samples: 349 evaluable samples (167 fresh, 182 frozen). Patients were of all ages and genders. Collected at 7 clinical sites.
• Retrospective Clinical Samples: 577 evaluable samples. Collected at 10 clinical sites.
• Contrived Samples: 777 samples.

Data Provenance: The clinical samples (prospective and retrospective) were collected at multiple clinical sites, suggesting data from different countries/regions, but specific countries of origin are not detailed. Both prospective and retrospective collection methods were used.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not explicitly state the number of experts or their specific qualifications (e.g., radiologist with 10 years of experience) used to establish the ground truth. It refers to "standard laboratory procedures for identification of blood culture isolates, including traditional and automated identification methods, MALDI-TOF IVD, and microbiological and biochemical techniques" as the comparator method. For Acinetobacter baumannii and Candida parapsilosis, as well as antibiotic resistance genes, PCR assays followed by bi-directional sequencing were used for confirmation. This implies a reliance on established laboratory protocols and potentially expert interpretation within those processes, but specific expert count or detailed qualifications are not provided.

4. Adjudication Method for the Test Set

The document does not describe an explicit adjudication method (e.g., 2+1, 3+1). The "Comparator Method" served as the reference standard and was used to determine true positive (TP), false negative (FN), true negative (TN), and false positive (FP) results. Discrepancies were investigated using PCR/sequencing (as noted in footnotes for various tables), suggesting a re-evaluation process for conflicting results but not a formal multi-reader adjudication scheme for the initial ground truth establishment.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

No MRMC comparative effectiveness study involving human readers and AI assistance was performed or described in the provided text. The study focuses on the standalone performance of the ePlex BCID-GN Panel against laboratory reference methods.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, the study described is a standalone performance study of the ePlex BCID-GN Panel. The performance metrics (PPA and NPA) are calculated by comparing the device's output directly against the "Comparator Method" without human interpretation influencing the device's result.

7. The Type of Ground Truth Used

The ground truth was established using expert consensus methods based on standard laboratory procedures for organism identification and analytically validated molecular assays (qPCR followed by bi-directional sequencing) for confirming specific organisms and resistance genes. This includes:

• Traditional and automated identification methods
• MALDI-TOF IVD
• Microbiological and biochemical techniques
• PCR/sequencing for specific organisms and resistance markers.

8. The Sample Size for the Training Set

The document does not mention a distinct "training set" in the context of the clinical performance evaluation. The clinical studies (prospective, retrospective, and contrived samples) are described as evaluation/test sets. However, the "Analytical Reactivity (Inclusivity)" and "Limit of Detection (LoD)" studies (Tables 56 and 57) describe testing of various strains and isolates to establish the device's analytical performance and broad reactivity. These analytical studies are crucial for the development and "training" (in a broader sense of assay design and validation) of such molecular diagnostic panels.

Specifically:

• Limit of Detection (LoD): At least 20 replicates per target were tested for each condition using quantified reference strains (Table 56 lists 38 organisms/targets with specific strains).
• Analytical Reactivity (Inclusivity): A panel of 336 strains/isolates was evaluated (Table 57 provides a partial list of these validated strains).
• Predicted (in silico) Reactivity: Bioinformatic analysis covered many additional variants for genus/group assays and resistance markers (Tables 58-76).

9. How the Ground Truth for the Training Set Was Established

For the analytical "training" aspects:

• LoD: The ground truth for LoD was established by using quantified reference strains with known concentrations in simulated blood culture sample matrix. The lowest concentration detected in ≥95% of tested replicates was defined as the LoD.
• Analytical Reactivity (Inclusivity): The ground truth was established by testing well-characterized strains/isolates with known identities at specified concentrations, confirming their detection by the panel.
• Predicted (in silico) Reactivity: This involved bioinformatic analysis of existing sequence data for various organisms and resistance gene variants to predict the panel's ability to detect them. This is a computational method for establishing theoretical ground truth based on genetic sequences.

Ask a specific question about this device

The GenMark ePlex Blood Culture Identification Fungal Pathogen (BCID-FP) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous detection and identification of multiple potentially pathogenic fungal organisms in positive blood culture. The ePlex BCID-FP Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain fungal organism.

The following fungal organisms are identified using the ePlex BCID-FP Panel: Candida albicans, Candida auris, Candida dubliniensis, Candida famata, Candida glabrata, Candida guilliermondii, Candida kefyr, Candida krusei, Candida lusitaniae, Candida parapsilosis, Candida tropicalis, Cryptococcus gattii, Cryptococcus neoformans, Fusarium and Rhodotorula.

The detection and identification of specific fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-FP Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

Negative results in the setting of a suspected bloodstream infection may be due to infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-FP Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-FP Panel, susceptibility testing and differentiation of mixed growth) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.

The ePlex Blood Culture Identification Fungal Pathogen (BCID-FP) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.

Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.

During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.

This document describes the analytical and clinical performance of the GenMark ePlex Blood Culture Identification Fungal Pathogen (BCID-FP) Panel, an in vitro diagnostic test. The information provided is sufficient to extract the requested details about acceptance criteria and study proving the device meets them.

1. Table of acceptance criteria and reported device performance:

The document implicitly defines acceptance criteria through the reported performance characteristics. While no explicit "acceptance criteria" table is provided, the clinical performance (Sensitivity/PPA and Specificity/NPA) tables against a comparator method serve as the primary evidence of meeting performance expectations. Analytical performance characteristics also define a form of acceptance criteria (e.g., LOD, exclusivity).

Here’s a table summarizing the reported device performance for each target organism, which represents the device meeting its performance objectives. The device demonstrates high sensitivity (positive percent agreement) and specificity (negative percent agreement) across various sample types (prospective, retrospective, and contrived).

2. Sample size used for the test set and the data provenance:

The test set for evaluating clinical performance consisted of:

• Prospective Samples: 21 evaluable samples (11 fresh, 10 frozen) collected at 6 clinical sites. These samples are from patients of all ages and genders. Collection dates are specified from May 2015 through July 2016 (frozen) and July through August 2018 (fresh). The country of origin is not explicitly stated but implied to be the US given the FDA submission. This data is prospective.
• Retrospective Samples: 120 samples collected from 9 sites. This data is retrospective.
• Contrived Samples: 725 evaluable samples prepared by spiking isolates into blood culture bottles. These are laboratory-prepared samples.

The total number of samples evaluated for clinical performance was 866 (11 fresh prospective + 10 frozen prospective + 120 retrospective + 725 contrived).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

The document does not specify the number of experts or their qualifications (e.g., radiologists with 10 years of experience) used to establish the ground truth. Instead, it relies on standard laboratory procedures and analytically validated PCR assays followed by bi-directional sequencing as the comparator methods (ground truth). This implies that the ground truth was established through a combination of traditional microbiological methods and molecular techniques, not through expert consensus reading of images or other subjective assessments.

4. Adjudication method for the test set:

The document describes the "comparator method" as the gold standard. For specific targets like Candida auris, Fusarium, Rhodotorula, and to confirm Candida parapsilosis, PCR/sequencing was used to determine the presence or absence of the organism, effectively acting as an adjudication step for these cases. For other organisms, standard laboratory procedures (culture, MALDI-TOF IVD, etc.) defined the ground truth. There is no mention of a traditional reader adjudication process (e.g., 2+1 or 3+1) as would be common in image-based AI studies, as this is a molecular diagnostic test. For discrepant results (e.g., section "Co-detections in Clinical Samples"), PCR/sequencing was used to investigate.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:

No, an MRMC comparative effectiveness study was not done. This type of study is typically performed for AI-assisted diagnostic tools that involve human interpretation of images. The ePlex BCID-FP Panel is an in vitro diagnostic (IVD) test that directly detects and identifies genetic material, so human readers are not involved in its direct interpretation in the way they would be in an AI imaging study. Therefore, there is no effect size of how much human readers improve with AI vs without AI assistance.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

Yes, the primary clinical performance evaluation is a standalone performance of the algorithm (the ePlex BCID-FP Panel) against a defined ground truth (comparator methods). The reported sensitivity/PPA and specificity/NPA values are purely the device's performance.

7. The type of ground truth used:

The ground truth for the clinical performance evaluation was established using:

• Standard laboratory procedures: This includes traditional and automated culture, MALDI-TOF IVD (Matrix-Assisted Laser Desorption/Ionization Time-Of-Flight Mass Spectrometry), and other microbiological and biochemical techniques.
• Analytically validated PCR assays followed by bi-directional sequencing: This advanced molecular method was used for specific targets (Candida auris, Fusarium, Rhodotorula) and to confirm certain identifications (Candida parapsilosis).

This represents a combination of expert consensus (through standard lab practices) and molecular outcomes data.

8. The sample size for the training set:

The document does not explicitly state the sample size for a "training set" in the context of machine learning. As this is a molecular diagnostic assay using nucleic acid hybridization and PCR, not a machine learning algorithm that learns from data in the same way, the concept of a distinct 'training set' for the device's core functionality specification might not apply directly in the conventional AI sense. The development of such assays involves analytical validation using numerous strains and concentrations (analytical reactivity, LOD, exclusivity), which implicitly serve as a form of "training" or optimization data during product development, but this is distinct from machine learning model training. The provided data focuses on the performance evaluation (test set) for regulatory approval.

9. How the ground truth for the training set was established:

Given that there isn't a "training set" in the typical machine learning sense, the way "ground truth" would be established for the development of such a molecular assay would involve:

• Known Reference Strains: Use of well-characterized microbial strains (e.g., ATCC, CBS, CDC strains) with confirmed identities. These are used in analytical studies like Limit of Detection (LOD) and Analytical Reactivity (Inclusivity), as well as Competitive Inhibition studies. Table 21 ("Contrived Sample Summary") and Table 27 ("Analytical Reactivity (Inclusivity) Results") list numerous specific strains and their origins (e.g., ATCC, CBS, NCPF, CDC#) used in testing.
• Sequencing and Phenotypic Characterization: During the assay's development, target sequences would be determined through genome sequencing, and phenotypic characteristics would be confirmed through established microbiological methods.

Therefore, the "ground truth" during device development (analogous to training/development data in AI) relies on well-characterized laboratory standards and molecular gold standards.

Ask a specific question about this device

The GenMark ePlex Blood Culture Identification Gram-Positive (BCID-GP) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous qualitative detection and identification of multiple potentially pathogenic gram-positive bacterial organisms and select determinants associated with antimicrobial resistance in positive blood culture. In addition, the ePlex BCID-GP Panel is capable of detecting a wide variety of gram-negative bacteria (Pan Gram-Negative assay) and several Candida species (Pan Candida assay). The ePlex BCID-GP Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain gram-positive organism.

The following bacterial organisms and genes associated with antibiotic resistance are identified using the ePlex BCID-GP Panel: Bacillus cereus group, Bacillus subtilis group, Corynebacterium, Cutibacterium acnes (Propionibacterium acnes), Enterococcus, Enterococcus faecalis, Enterococcus faecium, Lactobacillus, Listeria monocytogenes, Micrococcus, Staphylococcus, Staphylococcus aureus, Staphylococcus epidermidis, Staphylococcus lugdunensis, Streptococcus agalactiae (GBS), Streptococcus anginosus group, Streptococcus pneumoniae, Streptococcus pyogenes (GAS), mecA, mecC, vanA and vanB.

The ePlex BCID-GP Panel contains assays for the detection of genetic determinants associated with resistance to methicillin (mecA and mecC) to aid in the identification of potentially antimicrobial resistant organisms in positive blood culture samples. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease.

The ePlex BCID-GP Panel also contains targets designed to detect a broad range of organisms with a potentially misleading Gram stain result or organisms that may be missed by Gram staining altogether, for example in the case of co-infections. These include a broad Pan Gram-Negative assay as well as a Pan Candida assay, which is designed to detect four of the most prevalent Candida species: Candida albicans, Candida glabrata, Candida krusei and Candida parapsilosis.

The detection and identification of specific bacterial and fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-GP Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.

Negative results in the setting of a suspected bloodstream infection may be due to infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-GP Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-GP Panel and for susceptibility testing. differentiation of mixed growth and association of antimicrobial resistance marker genes to a specific organism) and clinical presentation must be taken into consideration in the final diagnosis of blood stream infection.

The ePlex Blood Culture Identification Gram-Positive (BCID-GP) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.

Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.

During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.

Here's an analysis of the provided text to extract information about the device's acceptance criteria and the study proving it meets them.

The document is a 510(k) Premarket Notification from the FDA regarding the GenMark ePlex Blood Culture Identification Panel - Gram Positive (BCID-GP) Panel (K181663). The core of the performance data revolves around clinical performance (sensitivity/PPA and specificity/NPA) against comparator methods, and analytical performance (Limit of Detection, Inclusivity, Exclusivity, Reproducibility, Interfering Substances, Competitive Inhibition).

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the clinical and analytical performance targets the device aimed to meet, typically demonstrating non-inferiority or substantial equivalence to a predicate device and adequate analytical performance. The tables provided (Table 6 through Table 30 for clinical performance, and Table 61 onwards for reproducibility) present the device's reported performance. A formal table of acceptance criteria with corresponding results isn't explicitly laid out with a "criteria" column alongside "performance" in the document, but we can infer the criteria from the achieved results and the FDA clearance itself, which implies acceptable performance.

For this device, the primary acceptance criteria would be high Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) with established comparator methods for all targeted organisms and resistance genes, along with robust analytical performance.

Here's a summary of the reported performance, reflecting the device meeting implicit acceptance criteria for clinical accuracy:

Table 1: Derived Acceptance Criteria and Reported Device Performance (Selected Examples)

Detailed Information about the Study:

1. A table of acceptance criteria and the reported device performance

   • See the table above for inferred criteria and reported performance based on clinical PPA/NPA. The acceptance criteria are not explicitly stated as quantitative thresholds in the provided text, but the successful clearance of the device implies that the observed performance was deemed acceptable by the FDA.

2. Sample sizes used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

   • Test Set (Clinical Study):
     • Prospective Samples: Total 711 evaluable samples. Collected from 7 clinical sites. 312 were tested fresh (Jan-Feb 2018), and 399 were frozen (June 2014-July 2016).
     • Retrospective Samples: Total 586 evaluable samples. Collected to supplement low prevalence targets.
     • Combined Clinical Samples: 1297 (711 prospective + 586 retrospective).
     • Contrived Samples: 565 samples prepared by spiking isolates into blood culture bottles.
   • Data Provenance: The document states "A prospective, multicenter clinical study was conducted at 7 clinical sites." While specific countries are not mentioned, FDA clearances typically involve studies predominantly conducted in the USA or with data acceptable to US regulatory standards. The terms "multicenter" and "clinical sites" imply different geographical locations although not explicitly named.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

   • The ground truth for the clinical test set was established using "Standard laboratory procedures" including traditional and automated culture, MALDI-TOF IVD, and microbiological and biochemical techniques for organism identification. Specific PCR assays followed by bi-directional sequencing (or 16S sequencing for Corynebacterium, Staphylococcus epidermidis, Candida parapsilosis) were used for confirmation where necessary. For antibiotic resistance genes (mecA, mecC, vanA, vanB), ground truth was established using analytically validated qPCR amplification assays followed by bi-directional sequencing.
   • The document does not specify the number of experts or their qualifications (e.g., board-certified microbiologists, medical technologists with X years of experience) who performed these standard laboratory procedures or interpreted the sequencing results to establish ground truth.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

   • The document describes a comparison between the ePlex BCID-GP Panel results and "comparator method(s) results." For discrepant results (e.g., false positives, false negatives), additional analytically validated PCR assays and sequencing were used for confirmation (e.g., for Corynebacterium, S. epidermidis, Pan Candida, Staphylococcus for mecA, and Enterococcus for vanA/vanB).
   • This is a form of adjudication, where discrepant results are further investigated. However, it's not a multi-reader adjudication method (like 2+1, 3+1) involving multiple human readers interpreting results, as this device performs automated detection. Instead, it's a discrepancy resolution method using a higher-level, analytically validated diagnostic test (PCR/sequencing) as the tie-breaker/confirmatory method.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

   • No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic test for identifying microorganisms and resistance markers, not an AI-assisted diagnostic imaging or interpretation tool for human readers. Its performance is evaluated against laboratory standard reference methods, not against human reader performance or improvement with AI assistance.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

   • Yes, the primary clinical and analytical performance studies are standalone (algorithm only) performance. The ePlex BCID-GP Panel is an automated test on the ePlex Instrument. Its results are generated by the device's assay and software, and then compared directly to the ground truth established by comparator laboratory methods. There is no explicit human-in-the-loop component described for its diagnostic performance evaluation, although clinical interpretation of the results by medical professionals is clearly indicated as necessary for patient management decisions (as stated in the Indications for Use).

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

   • The ground truth for the clinical study was based on "Standard laboratory procedures" for organism identification (culture, MALDI-TOF IVD, microbiological/biochemical techniques) and analytically validated PCR assays with bi-directional sequencing for confirmation of specific organisms and all resistance genes. This would fall under laboratory reference standard/molecular confirmation. Clinical outcomes data was not primarily used for ground truth.

8. The sample size for the training set

   • The document does not specify a separate "training set" size for the development of the ePlex BCID-GP Panel's algorithm. For in vitro diagnostic devices like this one, the development process might involve internal validation and optimization, but the regulatory submission focuses on the performance of the final, locked version of the device using a clinically representative "test set." The text primarily describes the clinical evaluation (prospective, retrospective, contrived samples) which serves as the independent test set for regulatory submission.

9. How the ground truth for the training set was established

   • As a distinct "training set" is not explicitly mentioned as per typical machine learning contexts, the document doesn't detail how ground truth for such a set was established. It's plausible that internal development and optimization would have used similar laboratory reference methods as those used for the clinical validation.

Ask a specific question about this device