(209 days)
The GenMark ePlex® Blood Culture Identification Gram-Negative (BCID-GN) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous qualitative detection and identification of multiple potentially pathogenic gram-negative bacterial organisms and select determinants associated with antimicrobial resistance in positive blood culture. In addition, the ePlex BCID-GN Panel is capable of detecting several gram-positive bacteria (Pan Gram-Positive assay) and several Candida species (Pan Candida assay). The ePlex BCID-GN Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain gram-negative organism.
The following bacterial organisms and genes associated with antibiotic resistance are identified using the ePlex BCID-GN Panel: Acinetobacter baumannii, Bacteroides fragilis, Citrobacter sakazakii, Enterobacter cloacae complex, Enterobacter (non-cloacae complex), Escherichia coli, Fusobacterium necrophorum, Fusobacterium nucleatum, Haemophilus influenzae, Klebsiella oxytoca, Klebsiella pneumoniae group, Morganii, Neisseria meningitidis, Proteus, Proteus mirabilis, Pseudomonas aeruginosa, Salmonella, Serratia marcescens, Stenotrophomonas maltophilia, CTX-M (blaCTX-M), IMP (blaMP) , KPC (blaKPC) , NDM (blaNDM), OXA (blaOXA) (OXA-23 and OXA-48 groups only), and VIM (blaVIM).
The ePlex BCID-GN Panel contains assays for the detection of genetic determinants associated with resistance to antimicrobial agents including CTX-M(blaCTX-M), which is associated with resistance to extended spectrum betalactamase (ESBL)-mediated resistance to penicillins, cephalosporins, and monobactams, as well as OXA (blaOXA) (OXA-23 and OXA-48 groups only), KPC (blaKPC), and metallo-beta-lactamases IMP (blaIMP), and NDM (blaNDM), which is associated with carbapenemase-mediated resistance. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance assays do not indicate susceptibility, as there are multiple mechanisms of resistance in gramnegative bacteria.
The ePlex BCID-GN Panel also contains targets designed to detect a broad range of organisms with a potentially misleading Gram stain result or organisms that may be missed by Gram staining altogether, for example in the case of coinfections. These include a broad Pan Gram-Positive assay (which is designed to detect Bacillus cereus group, Bacillus subtilis group, Enterococcus, Staphylococus, and Streptococcus), as well as a Pan Candida assay, which is designed to detect four Candida species: Candida albicans, Candida krusei, and Candida parapsilosis.
The detection and identification of specific bacterial and fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-GN Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.
Negative results in the setting of a suspected bloodstream infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-GN Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-GN Panel and for susceptibility testing, differentiation of mixed growth, and association of antimicrobial resistance marker genes to a specific organism) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.
The ePlex Blood Culture Identification Gram-Negative (BCID-GN) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.
Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.
During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.
The presented document is a 510(k) summary for the GenMark ePlex Blood Culture Identification Gram-Negative (BCID-GN) Panel, a qualitative nucleic acid multiplex in vitro diagnostic test. The study aims to demonstrate that the updated device (Subject Device) is substantially equivalent to its predicate device (original GenMark ePlex BCID-GN Panel, K182619). The data focuses on analytical and clinical performance.
Here's an analysis based on the provided text, addressing your specific points:
1. A table of acceptance criteria and the reported device performance:
The document doesn't explicitly state a single "acceptance criteria" table with pre-defined thresholds for all metrics (like sensitivity, specificity) against which the reported performance is directly compared in a summary table. However, it implicitly demonstrates acceptance by presenting individual performance metrics (Sensitivity/PPA and Specificity/NPA) for each target organism and resistance gene across different sample types (Prospective, Retrospective, Contrived, and Overall). The consistent high percentages for these metrics indicate that the device met the required performance for regulatory acceptance, even if the precise numerical cut-offs aren't explicitly stated in a singular table for all parameters.
Instead of a single "acceptance criteria" table, the document functions as a detailed report of performance against implicit acceptance criteria for in vitro diagnostic devices, which typically demand high sensitivity and specificity. The data tables already present the "reported device performance."
Example of reported device performance for a few key targets (extracted from Tables 7-34):
| Target | Sample Type | Sensitivity/PPA % (95% CI) | Specificity/NPA % (95% CI) |
|---|---|---|---|
| Acinetobacter baumannii | Overall | 100 (95.1-100) | 99.9 (99.7-100) |
| Bacteroides fragilis | Overall | 95.6 (87.8-98.5) | 99.9 (99.6-100) |
| Escherichia coli | Overall | 96.9 (94.4-98.3) | 99.8 (99.4-99.9) |
| CTX-M | Overall | 93.1 (88.1-96.1) | 100 (99.7-100) |
| KPC | Overall | 98.1 (89.9-99.7) | 99.9 (99.6-100) |
| Pan Candida | Overall | 62.5 (30.6-86.3) | 99.7 (99.4-99.9) |
| Pan Gram-Positive | Overall | 78.2 (67.8-85.9) | 97.9 (95.6-99.0) |
(Note: "Overall" for Pan targets combines Prospective, Retrospective, and Retrospective (Non-Intended Use), but excludes Contrived. The overall figures for other targets combine Prospective/Retrospective and Contrived samples as a whole.)
2. Sample sized used for the test set and the data provenance:
-
Test Set Sample Size:
- Clinical Samples: 349 prospective samples (167 fresh, 182 frozen) and 577 retrospective samples. Total clinical samples: 926.
- Contrived Samples: 777 samples.
- Additional Retrospective (Non-Intended Use) for Pan targets: 741 samples.
- Total evaluable samples across studies: 349 (prospective) + 577 (retrospective) + 777 (contrived) + 741 (non-intended use for pan targets) = 2444 samples in total tested across various evaluations. The overall performance tables combine various subsets of these.
-
Data Provenance:
- Country of Origin: Not explicitly stated, but 7 clinical sites were involved in prospective collection (suggests multi-center, likely within the US given FDA submission).
- Retrospective or Prospective: Both.
- Prospective: 349 samples collected from June 2014 through July 2016 (frozen) and June through July 2018 (fresh).
- Retrospective: 577 samples collected.
- Contrived: Laboratory-generated samples.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
The document does not explicitly state the "number of experts" or their specific "qualifications" involved in establishing the ground truth. It refers to "standard laboratory procedures for identification of blood culture isolates, including traditional and automated identification methods, MALDI-TOF IVD, and microbiological and biochemical techniques" (Table 4). For antibiotic resistance genes, it uses "analytically validated qPCR amplification assays followed by bi-directional sequencing." This implies laboratory professionals with expertise in microbiology and molecular diagnostics perform these comparator methods, but specific numbers or individual qualifications are not detailed.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
The document does not describe an explicit "adjudication method" involving multiple human readers (e.g., 2+1 or 3+1). The ground truth is established through comparator methods as described above (standard laboratory procedures, PCR/sequencing). Any discrepancies between the device and these comparator methods are analyzed and explained (e.g., the detailed footnotes in the performance tables and the discussion regarding CTX-M false negatives). This is typical for in vitro diagnostic (IVD) device studies, where ground truth is often determined by a reference laboratory standard or follow-up confirmatory testing, rather than human expert consensus on image interpretation.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
No, an MRMC comparative effectiveness study was not done. This device is an in vitro diagnostic (IVD) test for direct detection of pathogens and resistance genes from blood cultures, not an "AI-assisted image interpretation" device to be used by human readers. Therefore, the concept of human readers improving with AI assistance is not applicable here.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
Yes, this study represents a standalone technical performance evaluation of the ePlex BCID-GN Panel device. The device itself performs the detection and identification, and its results are compared directly to the gold standard comparator methods. There is no "human-in-the-loop" performance element in the operation of this specific diagnostic test.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc):
The ground truth was established by a combination of:
- Standard Laboratory Procedures: including traditional and automated identification methods, MALDI-TOF IVD, and microbiological and biochemical techniques for organism identification.
- Molecular Confirmation: Analytically validated PCR assays followed by bi-directional sequencing for specific organism identifications (e.g., Acinetobacter baumannii, Candida parapsilosis) and for all antibiotic resistance genes (qPCR amplification followed by bi-directional sequencing).
- Additional Testing for Discrepancies: Further investigations (e.g., repeat extractions, qPCR testing from isolates, testing with FDA-cleared multiplex assays) were used to resolve discrepancies and confirm the true status of samples, as detailed in the footnotes for several performance tables (e.g., Table 28 for CTX-M).
This ground truth method is based on a hierarchy of established laboratory and molecular techniques rather than human expert consensus on interpretation.
8. The sample size for the training set:
The document does not explicitly mention a "training set" in the context of machine learning, as this is a molecular diagnostic device, not an AI/ML product. However, the development of such a device involves extensive analytical studies related to inclusivity (reactivity), exclusivity (specificity), and limit of detection (LoD), which are analogous to data used in the development or "training" phase.
- Analytical Reactivity (Inclusivity): Evaluated with a panel of 336 strains/isolates.
- Limit of Detection (LoD): Determined using quantified reference strains for each target.
- In silico analysis: Used for predicted reactivity of genus/group assays and resistance markers, involving evaluation of sequence data.
While not a "training set" in the AI sense, these analytical studies inform the design and performance characteristics of the diagnostic assays.
9. How the ground truth for the training set was established:
As noted above, there isn't a "training set" in the AI/ML sense. For the analytical studies that are foundational to the device's design (e.g., inclusivity, LoD):
- Ground truth for inclusivity (analytical reactivity): Established by using characterized strains/isolates with known identity. The strains' identities are determined by standard microbiological and molecular methods.
- Ground truth for LoD: Established by using quantified reference strains where the concentration (CFU/mL) of the organism is precisely known.
- Ground truth for in silico analysis: Based on existing genetic sequence data and bioinformatic analysis to predict reactivity, relying on established genetic databases and characterizations.
§ 866.3365 Multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures.
(a)
Identification. A multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures is a qualitative in vitro device intended to simultaneously detect and identify microorganism nucleic acids from blood cultures that test positive by Gram stain or other microbiological stains. The device detects specific nucleic acid sequences for microorganism identification as well as for antimicrobial resistance. This device aids in the diagnosis of bloodstream infections when used in conjunction with other clinical and laboratory findings. However, the device does not replace traditional methods for culture and susceptibility testing.(b)
Classification. Class II (special controls). The special control for this device is FDA's guideline document entitled “Class II Special Controls Guideline: Multiplex Nucleic Acid Assay for Identification of Microorganisms and Resistance Markers from Positive Blood Cultures.” For availability of the guideline document, see § 866.1(e).