(87 days)
The FilmArray Blood Culture Identification (BCID) Panel is a qualitative multiplexed nucleic acid-based in vitro diagnostic test intended for use with FilmArray systems. The FilmArray BCID Panel is capable of simultaneous detection and identification of multiple bacterial and yeast nucleic acids and select genetic determinants of antimicrobial resistance. The BCID assay is performed directly on blood culture samples identified as positive by a continuous monitoring blood culture system that demonstrate the presence of organisms as determined by Gram stain.
The following gram-positive bacteria, gram-negative bacteria, and yeast are identified using the FilmArray BCID Panel: Enterococci, Listeria monocytogenes, Staphylococci (including specific differentiation of Staphylococcus aureus), Streptococci (with specific differentiation of Streptococcus agalactiae, Streptococcus pneumoniae, and Streptococcus pyogenes), Acinetobacter baumannii, Enterobacteriaceae (including specific differentiation of the Enterobacter cloacae complex, Escherichia coli, Klebsiella oxytoca, Klebsiella pneumoniae, Proteus, and Serratia marcescens), Haemophilus influenzae, Neisseria meningitidis (encapsulated), Pseudomonas aeruginosa, Candida albicans, Candida glabrata, Candida krusei, Candida parapsilosis, and Candida tropicalis.
The FilmArray BCID Panel also contains assays for the detection of genetic determinants of resistance to methicillin (mecA), vancomycin (vanA and vanB), and carbapenems (blakpr) to aid in the identification of potentially antimicrobial resistant organisms in positive blood culture samples. The antimicrobial resistance gene detected may or may not be associated with the agent responsible for disease. Negative results for these select antimicrobial resistance gene assays do not indicate susceptibility, as multiple mechanisms of resistance to methicillin, vancomycin, and carbapenems exist.
FilmArray BCID is indicated as an aid in the diagnosis of specific agents of bacteremia and fungemia and results should be used in conjunction with other clinical and laboratory findings. Positive FilmArray results do not rule out co-infection with organisms not included in the FilmArray BCID Panel. FilmArray BCID is not intended to monitor treatment for bacteremia or fungemia.
Subculturing of positive blood cultures is necessary to recover organisms for susceptibility testing and epidemiological typing, to identify organisms in the blood culture that are not detected by the FilmArray BCID Panel, and for species determination of some Staphylococci, Enterococci, Streptococci, and Enterobacteriaceae that are not specifically identified by the FilmArray BCID Panel assays.
The FilmArray Blood Culture Identification (BCID) Panel is a multiplex nucleic acid test designed to be used with a FilmArray system. The FilmArray BCID pouch contains freeze-dried reagents to perform nucleic acid purification and nested, multiplex PCR with DNA melt analysis. The FilmArray Blood Culture Identification (BCID) Panel simultaneously tests a single positive blood culture sample to provide results for 24 different organisms and organism groups that cause bloodstream infections and three genetic markers that are known to confer antimicrobial resistance (see Table 1).
A test is initiated by loading Hydration Solution and a positive blood culture sample mixed with the provided Sample Buffer into the FilmArray BCID pouch. The pouch contains all of the reagents required for specimen testing and analysis in a freeze-dried format; the addition of Hydration Solution and sample Buffer Mix rehydrates the reagents. After the pouch is prepared, the software guides the user though the steps of placing the pouch into the instrument, scanning the pouch barcode, entering the sample identification, and initiating the run.
The instrument contains a coordinated system of inflatable bladders and seal points, which act on the pouch to control the movement of liquid between the pouch blisters. When a bladder is inflated over a reagent blister, it forces liquid from the blister into connecting channels. Alternatively, when a seal is placed over a connecting channel it acts as a valve to open or close a channel. In addition, electronically controlled pneumatic pistons are positioned over multiple plungers in order to deliver the rehydrated reagents into the blisters at the appropriate times. Two Peltier devices control heating and cooling of the pouch to drive the PCR reactions and the melt curve analysis.
Nucleic acid extraction occurs within the FilmArray pouch using mechanical lysis and standard magnetic bead technology. After extracting and purifying nucleic acids from the unprocessed sample, a nested multiplex PCR is executed in two stages. During the first stage, a single, large volume, highly multiplexed PCR reaction which includes all primers of the outer primer sets, is performed. The products from first stage PCR are then diluted and combined with a fresh, primer-free master mix and a fluorescent double stranded DNA binding dye (LC Green" Plus, BioFire Defense, LLC). The solution is then distributed to each well of the array. Array wells contain sets of primers designed specifically to amplify sequences internal to the PCR products generated during the first stage PCR reaction. The 2nd stage PCR, or nested PCR, is performed in singleplex fashion in each well of the array. At the conclusion of the 21th stage PCR, the array is interrogated by melt curve analysis for the detection of signature amplicons denoting the presence of specific targets. A digital camera placed in front of the array captures fluorescent images of the PCR reactions and software interprets the data.
The software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel.
Here's an analysis of the provided text regarding the acceptance criteria and study for the FilmArray Blood Culture Identification (BCID) Panel:
The document describes a 510(k) premarket notification for the FilmArray BCID Panel for use with the FilmArray 2.0 system, claiming substantial equivalence to a previously cleared device (K130914). The studies discussed are primarily focused on demonstrating this equivalence rather than establishing wholly new acceptance criteria for the device itself. The acceptance criteria essentially align with the performance of the predicate device.
1. A table of acceptance criteria and the reported device performance
The acceptance criteria are implicitly based on demonstrating comparable performance to the predicate FilmArray BCID Panel on the original FilmArray system. The reported performance for the new device (FilmArray BCID Panel on FilmArray 2.0) is effectively the validation that it meets these implicit acceptance criteria.
| Acceptance Criteria (Implicit) | Reported Device Performance (FilmArray 2.0 with both loading procedures) |
|---|---|
| Maintain overall PPA (Positive Percent Agreement) ≥ 95.3% as seen with the predicate. | Overall PPA ≥ 99.1% (Lower bound 95% CI ≥ 95.3%) |
| Maintain overall NPA (Negative Percent Agreement) ≥ 99.7% as seen with the predicate. | Overall NPA ≥ 99.9% (Lower bound 95% CI ≥ 99.7%) |
| Achieve 100% agreement for most analytes across various comparisons (system, loading tools). | 100% concordance observed for most analytes across all comparisons. Occasional discrepancies attributed to analyte levels well below typical positive blood culture levels. |
| Maintain a Tm (melting temperature) variation of ±0.5°C or less compared to the predicate system. | Mean Tm values for all FilmArray BCID assays on modified configurations were ±0.5°C or less compared to the current configuration. |
| Reproducibility for positive and negative results with expected agreement. | High reproducibility (100% or very close to 100%) for majority of analytes across sites and loading procedures. (See Table 6 for detailed percentages per analyte). Lower bounds of 95% CI for overall concordance were consistently high (e.g., 96.0%-100%, 92.2%-99.7%). |
| Tm reproducibility (within-site/system and overall) within acceptable standard deviations. | Tm standard deviations were consistently low, generally ±0.1 to ±0.4 degrees Celsius, indicating high reproducibility. (See Table 7 for detailed values). |
2. Sample sized used for the test set and the data provenance
- Clinical Performance Study Test Set:
- Sample Size: 100 specimens (plus 2 additional runs for the modified system, totaling 102 runs for the current system and 202 for the modified system). These were selected such that each analyte (and antibiotic resistance marker) was represented 3-5 times.
- Data Provenance: Retrospective. Specimens were "previously obtained during the FilmArray BCID prospective clinical evaluation." Seeded blood cultures (remnant from the prospective BCID clinical study) were used for rare BCID analytes. The country of origin is not explicitly stated, but the FDA submission and company location (Salt Lake City, UT, USA) suggest it is likely US-based data.
- Low Analyte (Titration) Study Test Set:
- Sample Size: Not explicitly stated as a single number, but involves dilution series of samples for various analytes. Each test level for each organism was tested with 5 replicates (e.g., 5/5 (100%) in Table 4). The total number of tests for each analyte across the dilution series is 20 (5 replicates x 4 dilutions). With ~30 analytes, this implies hundreds of individual tests.
- Data Provenance: This appears to be an analytical study using contrived samples (dilution series of samples containing a mix of BCID analytes), not directly clinical specimens. Country of origin not specified, but likely internal lab data.
- Reproducibility Study Test Set:
- Sample Size: For each analyte, at each test level (Positive/Negative), 30 tests were performed at each of 3 sites, for a total of 90 data points per analyte/test level/loading procedure combination. This was then done for both syringe and injection vial loading procedures, effectively 180 data points per analyte/test level. There were 6 organisms with 2 test levels each for the most detailed analysis (positive/negative), and several other analytes with only negative test levels contributing to overall statistics.
- Data Provenance: Contrived blood culture samples spiked with various concentrations of BCID organisms, tested across three different test sites (likely in the US based on the submission).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not mention the use of experts to establish ground truth for the test sets. For the clinical performance study, the ground truth for previously collected clinical specimens would have been established by standard clinical microbiology methods. For the seeded blood cultures and contrived samples, the ground truth is inherently known by the study design (i.e., what organisms were spiked into the samples). This is a common approach for in vitro diagnostic (IVD) device studies.
4. Adjudication method for the test set
Not applicable, as ground truth was established by microbiological culture or by design (for contrived samples), not by expert opinion requiring adjudication.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is an in vitro diagnostic (IVD) device, not an AI-assisted diagnostic imaging system that uses human readers. The "reader" here is the instrument and its software, and the comparison is between two versions of the instrument/software system, and different loading methods.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, the device (FilmArray BCID Panel) performs as a standalone algorithm/instrument system to detect and identify nucleic acids. There is no human interpretation of raw data; "The software automatically interprets the results of each DNA melt curve analysis and combines the data with the results of the internal pouch controls to provide a test result for each organism on the panel."
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
- Clinical Performance Study: Ground truth for patient specimens would have been established by conventional microbiology methods (culture, identification, susceptibility testing). For seeded blood cultures, the ground truth was based on the known spiked organisms.
- Low Analyte Study: Ground truth was based on the known composition and concentration of the spiked organisms in the contrived samples.
- Reproducibility Study: Ground truth was based on the known composition and expected detection of the spiked organisms in the contrived samples.
8. The sample size for the training set
The document does not provide information about a separate "training set" in the context of device development or any machine learning approach. This is an IVD device based on molecular biology principles (PCR and melt analysis), not a machine learning algorithm that typically requires explicit training datasets. The development and optimization of the assays would have involved extensive R&D, but not in the "training set" sense of AI/ML.
9. How the ground truth for the training set was established
As noted in point 8, the concept of a "training set" in the AI/ML sense is not directly applicable to this type of IVD device. The ground truth for developing and validating the PCR assays would involve standard molecular biology and microbiology techniques to confirm the presence and identity of target organisms and resistance genes.
§ 866.3365 Multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures.
(a)
Identification. A multiplex nucleic acid assay for identification of microorganisms and resistance markers from positive blood cultures is a qualitative in vitro device intended to simultaneously detect and identify microorganism nucleic acids from blood cultures that test positive by Gram stain or other microbiological stains. The device detects specific nucleic acid sequences for microorganism identification as well as for antimicrobial resistance. This device aids in the diagnosis of bloodstream infections when used in conjunction with other clinical and laboratory findings. However, the device does not replace traditional methods for culture and susceptibility testing.(b)
Classification. Class II (special controls). The special control for this device is FDA's guideline document entitled “Class II Special Controls Guideline: Multiplex Nucleic Acid Assay for Identification of Microorganisms and Resistance Markers from Positive Blood Cultures.” For availability of the guideline document, see § 866.1(e).