(80 days)
The FilmArray Blood Culture Identification (BCID) Panel is a qualitative multiplexed nucleic acid-based in vitro diagnostic test intended for use with the FilmArray Instrument. The FilmArray BCID Panel is capable of simultaneous detection and identification of multiple bacterial and yeast nucleic acids and select genetic determinants of antimicrobial resistance. The BCID assay is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system that demonstrates the presence of organisms as determined by Gram stain.
The following gram-positive bacteria, gram-negative bacteria, and yeast are identified using the FilmArray BCID Panel: Enterococci, Listeria monocytogenes, commonly encountered Staphylococci (including specific differentiation of Staphylococcus aureus), commonly encountered Streptococci (with specific differentiation of Streptococcus agalactiae, Streptococcus pneumoniae, and Streptococcus pyogenes), Acinetobacter baumannii, commonly encountered Enterobacteriaceae (including specific differentiation of the Enterobacter cloacae complex, Escherichia coli, Klebsiella oxytoca, Klebsiella pneumoniae, Proteus, and Serratia marcescens), Haemophilus influenzae, Neisseria meningitidis (encapsulated), Pseudomonas aeruginosa, Candida albicans, Candida glabrata, Candida krusei, Candida parapsilosis, and Candida tropicalis.
The FilmArray BCID Panel also contains assays for the detection of genetic determinants of resistance to methicillin (mecA), vancomycin (vanA and vanB), and carbapenems (blaKPC) to aid in the identification of potentially antimicrobial resistant organisms in positive blood culture samples. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene assays do not indicate susceptibility, as multiple mechanisms of resistance to methicillin, vancomycin, and carbapenems exist.
FilmArray BCID is indicated as an aid in the diagnosis of specific agents of bacteremia and fungemia and results should be used in conjunction with other clinical and laboratory findings. Positive FilmArray results do not rule out co-infection with organisms not included in the FilmArray BCID Panel. FilmArray BCID is not intended to monitor treatment for bacteremia or fungemia.
Subculturing of positive blood cultures is necessary to recover organisms for susceptibility testing and epidemiological typing, to identify organisms in the blood culture that are not detected by the FilmArray BCID Panel, and for species determination of some Staphylococci, Enterococci, Streptococci, and Enterobacteriaceae that are not specifically identified by the FilmArray BCID Panel assays.
The FilmArray Blood Culture Identification (BCID) Panel is a multiplex nucleic acid test designed to be used with the FilmArray Instrument. The FilmArray BCID pouch contains freeze-dried reagents to perform nucleic acid purification and nested, multiplex PCR with DNA melt analysis. The FilmArray Blood Culture Identification (BCID) Panel simultaneously tests a single positive blood culture sample to provide results for 24 different organisms and organism groups that cause bloodstream infections and three genetic markers that are known to confer antimicrobial resistance (see Table 1).
A test is initiated by loading Hydration Solution and a positive blood culture sample mixed with the provided Sample Buffer into the FilmArray BCID pouch. The pouch contains all of the reagents required for specimen testing and analysis in a freeze-dried format; the addition of Hydration Solution and Sample/Buffer Mix rehydrates the reagents. After the pouch is prepared, the FilmArray Software guides the user though the steps of placing the pouch into the instrument, scanning the pouch barcode, entering the sample identification, and initiating the run.
The FilmArray Instrument contains a coordinated system of inflatable bladders and seal points, which act on the pouch to control the movement of liquid between the pouch blisters. When a bladder is inflated over a reagent blister. it forces liquid from the blister into connecting channels. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the pouch to drive the PCR reactions and the melt curve analysis.
Nucleic acid extraction occurs within the FilmArray pouch using mechanical lysis and standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, the FilmArray performs a nested multiplex PCR that is exceuted in two stages. During the first stage, the FilmArray performs a single, large volume, highly multiplexed PCR reaction which includes all primers of the outer primer sets. The products from first stage PCR are then diluted and combined with a fresh, primer-free master mix and a fluorescent double stranded DNA binding dye (LC Green Plus-, BioFire Diagnostics). The solution is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The 200 stage PCR, or nested PCR, is performed in singleplex fashion in each well of the array. At the conclusion of the 2" stage PCR, the array is interrogated by melt curve analysis for the detection of signature amplicons denoting the presence of specific targets. A digital camera placed in front of the 2nd stage PCR captures fluorescent images of the PCR reactions and software interprets the data.
The FilmArray Software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism and antimicrobial resistance gene on the panel.
Here's a breakdown of the acceptance criteria and study information for the FilmArray Blood Culture Identification (BCID) Panel, based on the provided text:
FilmArray Blood Culture Identification (BCID) Panel
1. Table of Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state quantitative acceptance criteria in a dedicated section. However, the "Clinical Performance" and "Reproducibility" sections present performance metrics (sensitivity, specificity, positive percent agreement, negative percent agreement, and percent agreement with expected results) that act as the de facto acceptance criteria. I'll present the overall clinical performance from Tables 6, 7, 8, 9, and 10 and the overall reproducibility from Tables 59 and 60.
Overall Clinical Performance (Aggregated from Prospective and Seeded Arms)
| Analyte Group | Performance Metric | Value | 95% Confidence Interval |
|---|---|---|---|
| Gram-Positive Organisms (Overall) | |||
| Enterococcus | Sensitivity/PPA | 97.7% | 93.4-99.5% |
| Specificity/NPA | 99.8% | 99.5-99.9% | |
| Listeria monocytogenes | Sensitivity/PPA | 100% | 90.3-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Staphylococcus | Sensitivity/PPA | 96.5% | 95.0-97.7% |
| Specificity/NPA | 99.1% | 98.5-99.6% | |
| Staphylococcus aureus | Sensitivity/PPA | 98.4% | 96.1-99.6% |
| Specificity/NPA | 99.8% | 99.5-99.9% | |
| Streptococcus | Sensitivity/PPA | 97.5% | 94.3-99.2% |
| Specificity/NPA | 99.8% | 99.4-99.9% | |
| Streptococcus agalactiae | Sensitivity/PPA | 100% | 90.3-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Streptococcus pneumoniae | Sensitivity/PPA | 97.3% | 85.8-99.9% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Streptococcus pyogenes | Sensitivity/PPA | 100% | 90.7-100% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Gram-Negative Organisms (Overall) | |||
| Acinetobacter baumannii | Sensitivity/PPA | 100% | 93.0-100% |
| Specificity/NPA | 99.8% | 99.5-99.9% | |
| Enterobacteriaceae | Sensitivity/PPA | 98.4% | 96.9-99.3% |
| Specificity/NPA | 99.8% | 99.4-99.9% | |
| Enterobacter cloacae complex | Sensitivity/PPA | 97.4% | 86.5-99.9% |
| Specificity/NPA | 99.9% | 99.6-100% | |
| Escherichia coli | Sensitivity/PPA | 98% | 94.4-99.6% |
| Specificity/NPA | 99.8% | 99.4-99.9% | |
| Klebsiella oxytoca | Sensitivity/PPA | 92.2% | 82.7-97.4% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Klebsiella pneumoniae | Sensitivity/PPA | 97.1% | 91.9-99.4% |
| Specificity/NPA | 99.6% | 99.2-99.8% | |
| Proteus | Sensitivity/PPA | 100% | 91.0-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Serratia marcescens | Sensitivity/PPA | 98.7% | 93.0-100% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Haemophilus influenzae | Sensitivity/PPA | 100% | 91.8-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Neisseria meningitidis | Sensitivity/PPA | 100% | 90.3-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Pseudomonas aeruginosa | Sensitivity/PPA | 98.1% | 89.7-100% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Yeast (Overall) | |||
| Candida albicans | Sensitivity/PPA | 100% | 94.4-100% |
| Specificity/NPA | 99.8% | 99.5-99.9% | |
| Candida glabrata | Sensitivity/PPA | 100% | 92.7-100% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Candida krusei | Sensitivity/PPA | 100% | 90.5-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Candida parapsilosis | Sensitivity/PPA | 96.7% | 88.7-99.6% |
| Specificity/NPA | 99.9% | 99.7-100% | |
| Candida tropicalis | Sensitivity/PPA | 100% | 91.0-100% |
| Specificity/NPA | 100% | 99.8-100% | |
| Antimicrobial Resistance Genes (PCR/Sequencing Direct from Blood Culture) | |||
| mecA (All Staphylococcus) | Sensitivity/PPA | 98.4% | 96.8-99.3% |
| Specificity/NPA | 98.3% | 96.0-99.4% | |
| vanA/B (Enterococcus) | Sensitivity/PPA | 100% | 94.4-100% |
| Specificity/NPA | 100% | 94.6-100% | |
| KPC (Enterobacteriaceae & others) | Sensitivity/PPA | 100% | 91.0-100% |
| Specificity/NPA | 100% | 99.3-100% | |
| Antimicrobial Resistance Genes (PCR/Sequencing of Cultured Isolates) | |||
| mecA (All Staphylococcus) | Positive Percent Agreement | 98.9% | 97.5-99.6% |
| Negative Percent Agreement | 87.9% | 83.9-91.3% | |
| vanA/B (Enterococcus) | Positive Percent Agreement | 100% | 94.0-100% |
| Negative Percent Agreement | 94.4% | 86.2-98.4% | |
| KPC (Enterobacteriaceae & others) | Positive Percent Agreement | 100% | 91.0-100% |
| Negative Percent Agreement | 100% | 99.3-100% |
Overall Reproducibility (All Sites, Across All Analytes Tested)
| Performance Metric | Value | 95% Confidence Interval |
|---|---|---|
| Organism Assays (% Agreement) | ~100% | (Min: 98.0%, Max: 100%) |
| Antimicrobial Resistance Assays (% Agreement) | ~100% | (Min: 98.0%, Max: 100%) |
2. Sample Sizes and Data Provenance for the Test Set:
- Total Test Set Sample Size: 2207 blood cultures
- Prospective Arm: 1568 specimens
- 67 specimens excluded (e.g., >8 hours past positivity, incomplete reference data, previous specimen from same subject).
- Tested either fresh (821 specimens) or frozen (747 specimens).
- Seeded Arm: 639 specimens
- 77 cultures excluded (e.g., >8 hours past positivity, not positive by automated blood culture, contaminated).
- Tested either fresh (419 specimens) or frozen (220 specimens).
- Prospective Arm: 1568 specimens
- Data Provenance: The study was a "two-armed clinical study" conducted at eight U.S. clinical sites. This indicates the data is prospective for the clinical arm and retrospective/contrived for the seeded arm (as isolates were seeded into blood culture bottles).
3. Number of Experts and Qualifications for Ground Truth - Test Set:
The document does not explicitly state the number of experts used to establish the ground truth for the test set, nor their specific qualifications (e.g., "radiologist with 10 years of experience").
Instead, the ground truth for organism detection was established using "Standard manual and automated microbiological/biochemical identification methods." For Acinetobacter baumannii, this was supplemented with "16S PCR with bi-directional sequencing."
For antimicrobial resistance genes, the ground truth involved "PCR with bi-directional sequencing for specific resistance gene - direct from blood culture" or "PCR with bi-directional sequencing for specific resistance gene from - appropriate cultured isolates." Additionally, "Standard manual and automated phenotypic antimicrobial susceptibility testing of appropriate cultured isolates" was performed for informational purposes.
While these are standard laboratory methods typically performed by trained microbiologists or laboratory technicians, the number and specific qualifications of the experts overseeing or performing these reference methods are not detailed.
4. Adjudication Method for the Test Set:
The document does not explicitly describe an adjudication method (like 2+1 or 3+1) for discrepant results in the test set.
However, the detailed tables of results often include footnotes explaining discrepancies. For instance, for Acinetobacter baumannii, isolates misidentified by phenotypic methods were subjected to 16S PCR and bi-directional sequencing for definitive characterization. Similarly, for other organisms and resistance genes, bidirectional sequencing was used to investigate false positive and false negative results. This implies an investigative process to reconcile differences, rather than a formal, predefined expert adjudication panel for every case.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
A Multi-Reader Multi-Case (MRMC) comparative effectiveness study was NOT done for this device. The study evaluated the standalone performance of the FilmArray BCID Panel against reference laboratory methods, not its impact on human reader performance or diagnostic accuracy with AI assistance.
6. Standalone Performance Study:
Yes, a standalone study was performed. The entire "Clinical Performance" section (Tables 6-10) reports the performance of the FilmArray BCID Panel as a standalone diagnostic device. The results (sensitivity, specificity, PPA, NPA) are presented for the algorithm/device only, without human-in-the-loop interaction, compared against established laboratory reference methods. The "Reproducibility" section also evaluates the device's technical consistency in a standalone capacity.
7. Type of Ground Truth Used:
The ground truth primarily used was:
- Expert Consensus/Reference Laboratory Methods:
- Standard manual and automated microbiological/biochemical identification methods for all organism detections.
- 16S PCR with bi-directional sequencing for A. baumannii speciation.
- PCR with bi-directional sequencing for antimicrobial resistance gene detection (from blood culture or cultured isolates).
- Pathology: Not explicitly mentioned as a primary ground truth for organism identification in this context, which focuses on molecular/microbial identification.
- Outcomes Data: Not used as ground truth for diagnostic accuracy in this study.
8. Sample Size for the Training Set:
The document does not explicitly specify a sample size for a "training set." The studies primarily focus on clinical validation (prospective and seeded arms) and analytical validation (inclusivity, exclusivity, reproducibility). While the device's algorithm undoubtedly relies on an underlying model, information regarding its specific training data and size is not provided in this regulatory submission summary. The inclusivity studies involve testing a "diverse collection of 303 isolates," which could be considered part of the data informing the assay's design and optimization, but it's not explicitly labeled as a "training set" in the context of a machine learning model.
9. How the Ground Truth for the Training Set Was Established:
As the document does not explicitly identify a "training set" in the context of a machine learning model with corresponding ground truth establishment, this information is not provided. The inclusivity and exclusivity studies, which evaluate the assay's breadth and specificity, likely contribute to the "training" or development of the assay's detection capabilities. The ground truth for these analytical studies was established by known characteristics of the isolates (e.g., ATCC strains, clinical isolates with confirmed identity), and confirmed through standard microbiological identification and sequencing methods.
§ 866.3365 Multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures.
(a)
Identification. A multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures is a qualitative in vitro device intended to simultaneously detect and identify microorganism nucleic acids from blood cultures that test positive by Gram stain or other microbiological stains. The device detects specific nucleic acid sequences for microorganism identification as well as for antimicrobial resistance. This device aids in the diagnosis of bloodstream infections when used in conjunction with other clinical and laboratory findings. However, the device does not replace traditional methods for culture and susceptibility testing.(b)
Classification. Class II (special controls). The special control for this device is FDA's guideline document entitled “Class II Special Controls Guideline: Multiplex Nucleic Acid Assay for Identification of Microorganisms and Resistance Markers from Positive Blood Cultures.” For availability of the guideline document, see § 866.1(e).