(85 days)
The GenMark ePlex Blood Culture Identification Fungal Pathogen (BCID-FP) Panel is a qualitative nucleic acid multiplex in vitro diagnostic test intended for use on GenMark's ePlex Instrument for simultaneous detection and identification of multiple potentially pathogenic fungal organisms in positive blood culture. The ePlex BCID-FP Panel is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system and which contain fungal organism.
The following fungal organisms are identified using the ePlex BCID-FP Panel: Candida albicans, Candida auris, Candida dubliniensis, Candida famata, Candida glabrata, Candida guilliermondii, Candida kefyr, Candida krusei, Candida lusitaniae, Candida parapsilosis, Candida tropicalis, Cryptococcus gattii, Cryptococcus neoformans, Fusarium and Rhodotorula.
The detection and identification of specific fungal nucleic acids from individuals exhibiting signs and/or symptoms of bloodstream infection aids in the diagnosis of bloodstream infection when used in conjunction with other clinical information. The results from the ePlex BCID-FP Panel are intended to be interpreted in conjunction with Gram stain results and should not be used as the sole basis for diagnosis, treatment, or other patient management decisions.
Negative results in the setting of a suspected bloodstream infection may be due to infection with pathogens that are not detected by this test. Positive results do not rule out co-infection with other organisms; the organism(s) detected by the ePlex BCID-FP Panel may not be the definite cause of disease. Additional laboratory testing (e.g. sub-culturing of positive blood cultures for identification of organisms not detected by ePlex BCID-FP Panel, susceptibility testing and differentiation of mixed growth) and clinical presentation must be taken into consideration in the final diagnosis of bloodstream infection.
The ePlex Blood Culture Identification Fungal Pathogen (BCID-FP) Panel is based on the principles of competitive nucleic acid hybridization using a sandwich assay format, wherein a single-stranded target binds concurrently to a sequence-specific solution-phase signal probe and a solid-phase electrode-bound capture probe. The test employs nucleic acid extraction, target amplification via polymerase chain reaction (PCR) or reverse transcription PCR (RT-PCR) and hybridization of target DNA. In the process, the double-stranded PCR amplicons are digested with exonuclease to generate single-stranded DNA suitable for hybridization.
Nucleic acid extraction from biological samples occurs within the cartridge via cell lysis, nucleic acid capture onto magnetic beads, and release for amplification. The nucleic acid extraction is processed through microfluidic liquid handling. Once the nucleic acid targets are captured and inhibitors are washed away, the magnetic particles are delivered to the electrowetting environment on the printed circuit board (PCB) and the targets are eluted from the particles and amplified.
During hybridization, the single-stranded target DNA binds to a complementary, single-stranded capture probe immobilized on the working gold electrode surface. Single-stranded signal probes (labeled with electrochemically active ferrocenes) bind to specific target sequence / region adjacent to the capture probe. Simultaneous hybridization of target to signal probes and capture probe is detected by alternating current voltammetry (ACV). Each working electrode on the array contains specific capture probes, and sequential analysis of each electrode allows detection of multiple analyte targets.
This document describes the analytical and clinical performance of the GenMark ePlex Blood Culture Identification Fungal Pathogen (BCID-FP) Panel, an in vitro diagnostic test. The information provided is sufficient to extract the requested details about acceptance criteria and study proving the device meets them.
1. Table of acceptance criteria and reported device performance:
The document implicitly defines acceptance criteria through the reported performance characteristics. While no explicit "acceptance criteria" table is provided, the clinical performance (Sensitivity/PPA and Specificity/NPA) tables against a comparator method serve as the primary evidence of meeting performance expectations. Analytical performance characteristics also define a form of acceptance criteria (e.g., LOD, exclusivity).
Here’s a table summarizing the reported device performance for each target organism, which represents the device meeting its performance objectives. The device demonstrates high sensitivity (positive percent agreement) and specificity (negative percent agreement) across various sample types (prospective, retrospective, and contrived).
| Target | Sample Type | Sensitivity/PPA % (95% CI) | Specificity/NPA % (95% CI) |
|---|---|---|---|
| Candida albicans | Overall | 97.1 (89.9-99.2) | 99.9 (99.3-100) |
| Candida auris | Overall | 100 (92.7-100) | 100 (99.5-100) |
| Candida dubliniensis | Overall | 100 (93.1-100) | 100 (99.5-100) |
| Candida famata | Overall | 100 (93.0-100) | 100 (99.5-100) |
| Candida glabrata | Overall | 98.3 (91.1-99.7) | 99.8 (99.1-99.9) |
| Candida guilliermondii | Overall | 98.0 (89.5-99.6) | 100 (99.5-100) |
| Candida kefyr | Overall | 100 (93.0-100) | 99.8 (99.1-99.9) |
| Candida krusei | Overall | 100 (92.9-100) | 100 (99.5-100) |
| Candida lusitaniae | Overall | 98.0 (89.3-99.6) | 99.9 (99.3-100) |
| Candida parapsilosis | Overall | 98.3 (91.1-99.7) | 99.9 (99.3-100) |
| Candida tropicalis | Overall | 100 (92.9-100) | 99.9 (99.3-100) |
| Cryptococcus gattii | Overall | 100 (92.9-100) | 100 (99.5-100) |
| Cryptococcus neoformans | Overall | 100 (93.7-100) | 100 (99.5-100) |
| Fusarium | Overall | 98.6 (92.3-99.7) | 100 (99.5-100) |
| Rhodotorula | Overall | 96.2 (87.0-98.9) | 99.9 (99.3-100) |
2. Sample size used for the test set and the data provenance:
The test set for evaluating clinical performance consisted of:
- Prospective Samples: 21 evaluable samples (11 fresh, 10 frozen) collected at 6 clinical sites. These samples are from patients of all ages and genders. Collection dates are specified from May 2015 through July 2016 (frozen) and July through August 2018 (fresh). The country of origin is not explicitly stated but implied to be the US given the FDA submission. This data is prospective.
- Retrospective Samples: 120 samples collected from 9 sites. This data is retrospective.
- Contrived Samples: 725 evaluable samples prepared by spiking isolates into blood culture bottles. These are laboratory-prepared samples.
The total number of samples evaluated for clinical performance was 866 (11 fresh prospective + 10 frozen prospective + 120 retrospective + 725 contrived).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
The document does not specify the number of experts or their qualifications (e.g., radiologists with 10 years of experience) used to establish the ground truth. Instead, it relies on standard laboratory procedures and analytically validated PCR assays followed by bi-directional sequencing as the comparator methods (ground truth). This implies that the ground truth was established through a combination of traditional microbiological methods and molecular techniques, not through expert consensus reading of images or other subjective assessments.
4. Adjudication method for the test set:
The document describes the "comparator method" as the gold standard. For specific targets like Candida auris, Fusarium, Rhodotorula, and to confirm Candida parapsilosis, PCR/sequencing was used to determine the presence or absence of the organism, effectively acting as an adjudication step for these cases. For other organisms, standard laboratory procedures (culture, MALDI-TOF IVD, etc.) defined the ground truth. There is no mention of a traditional reader adjudication process (e.g., 2+1 or 3+1) as would be common in image-based AI studies, as this is a molecular diagnostic test. For discrepant results (e.g., section "Co-detections in Clinical Samples"), PCR/sequencing was used to investigate.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:
No, an MRMC comparative effectiveness study was not done. This type of study is typically performed for AI-assisted diagnostic tools that involve human interpretation of images. The ePlex BCID-FP Panel is an in vitro diagnostic (IVD) test that directly detects and identifies genetic material, so human readers are not involved in its direct interpretation in the way they would be in an AI imaging study. Therefore, there is no effect size of how much human readers improve with AI vs without AI assistance.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
Yes, the primary clinical performance evaluation is a standalone performance of the algorithm (the ePlex BCID-FP Panel) against a defined ground truth (comparator methods). The reported sensitivity/PPA and specificity/NPA values are purely the device's performance.
7. The type of ground truth used:
The ground truth for the clinical performance evaluation was established using:
- Standard laboratory procedures: This includes traditional and automated culture, MALDI-TOF IVD (Matrix-Assisted Laser Desorption/Ionization Time-Of-Flight Mass Spectrometry), and other microbiological and biochemical techniques.
- Analytically validated PCR assays followed by bi-directional sequencing: This advanced molecular method was used for specific targets (Candida auris, Fusarium, Rhodotorula) and to confirm certain identifications (Candida parapsilosis).
This represents a combination of expert consensus (through standard lab practices) and molecular outcomes data.
8. The sample size for the training set:
The document does not explicitly state the sample size for a "training set" in the context of machine learning. As this is a molecular diagnostic assay using nucleic acid hybridization and PCR, not a machine learning algorithm that learns from data in the same way, the concept of a distinct 'training set' for the device's core functionality specification might not apply directly in the conventional AI sense. The development of such assays involves analytical validation using numerous strains and concentrations (analytical reactivity, LOD, exclusivity), which implicitly serve as a form of "training" or optimization data during product development, but this is distinct from machine learning model training. The provided data focuses on the performance evaluation (test set) for regulatory approval.
9. How the ground truth for the training set was established:
Given that there isn't a "training set" in the typical machine learning sense, the way "ground truth" would be established for the development of such a molecular assay would involve:
- Known Reference Strains: Use of well-characterized microbial strains (e.g., ATCC, CBS, CDC strains) with confirmed identities. These are used in analytical studies like Limit of Detection (LOD) and Analytical Reactivity (Inclusivity), as well as Competitive Inhibition studies. Table 21 ("Contrived Sample Summary") and Table 27 ("Analytical Reactivity (Inclusivity) Results") list numerous specific strains and their origins (e.g., ATCC, CBS, NCPF, CDC#) used in testing.
- Sequencing and Phenotypic Characterization: During the assay's development, target sequences would be determined through genome sequencing, and phenotypic characteristics would be confirmed through established microbiological methods.
Therefore, the "ground truth" during device development (analogous to training/development data in AI) relies on well-characterized laboratory standards and molecular gold standards.
§ 866.3365 Multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures.
(a)
Identification. A multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures is a qualitative in vitro device intended to simultaneously detect and identify microorganism nucleic acids from blood cultures that test positive by Gram stain or other microbiological stains. The device detects specific nucleic acid sequences for microorganism identification as well as for antimicrobial resistance. This device aids in the diagnosis of bloodstream infections when used in conjunction with other clinical and laboratory findings. However, the device does not replace traditional methods for culture and susceptibility testing.(b)
Classification. Class II (special controls). The special control for this device is FDA's guideline document entitled “Class II Special Controls Guideline: Multiplex Nucleic Acid Assay for Identification of Microorganisms and Resistance Markers from Positive Blood Cultures.” For availability of the guideline document, see § 866.1(e).