Search Results

MDx-Chex for BCID2 is intended for use as an external positive and negative assayed control to monitor the performance of the qualitative detection of yeast, Gram positive and Gram negative bacteria, as vell as snciated antimicrobial resistance genes, by the BioFire FilmArray Blood Culture Identification 2 (BCID2) Panel on FilmArray systems. Control 1 - GN: Gram negative bacteria: Acinetobacter colcacer in complex, Baceronides fragilis, Enterobacter cloacae complex, Escherichia coli, Klebsiella oxytoca, Klebsiella oxytoca, Klebsiella pneumats Jray, Proteus spp., Salmonella spp., Serratia marcescens, Heisseria vivoca, Neisseria meningitides, Poeudnonas aeruginosa, Stenotrophomonas matophilia; antimicrobial resistance genes: KPC, CTX-M, IMP, NDM, OXA-Ike, VIM, mcr- 1. Control 2 - GPY: Gram positive bacteria: Enterococcus face: 11, 27712-10, nr., iUDM, U.Steria monocytogenes, Staphylococcus aureus, Staphylococcus epidermidis, Staphylococcus lugdumsis, Streptococcus agalactiae, Streptococcus pneumonia, Streptococcus pyogenes; yeast: Candida dhicans, Candida glabrata, Candida krusei, Candida parapsilosis, Candida tropicalis, Cryptococcus neoformans/gatit; animicrobial resistance genes mecA/C and MREJ, vanA/B. This product is not intended to replace manufacturer controls provided with the device.

MDx-Chex for BCID2 Control 1 - GN is positive for certain pathogens and antimicrobial resistance genes in the FilmArray BCID 2 panel and negative for those contained in MDx-Chex for BCID2 Control 2 - GPY. MDx-Chex for BCID2 Control 2 - GPY is positive for the remaining pathogen and antimicrobial resistance genes and negative for those present in MDx-Chex for BCID2 Control 1 - GN (see Table 1 below). Each control mix also contains and controls for blood and blood culture media components that have been identified as PCR inhibitors namely hemoglobin, leukocyte DNA, and anticoagulants. MDx-Chex for BCID2 is a quality control containing stabilized blood components, blood culture media components, and inactivated microorganisms resulting in a full-process, cellular-based control for the BioFire BCID2 Panel. Use of full-process cellular controls are necessary to evaluate the entire analytical process, including sample lysis, nucleic acid isolation and purification, amplification, detection, and analysis, as well as the impact of PCR inhibitors and preanalytical variables. Routine use of full process quality controls can help identify variations in the test system that can lead to incorrect results.

This document describes the performance of a quality control material (MDx-Chex for BCID2) for a qualitative detection assay (BioFire FilmArray Blood Culture Identification 2 (BCID2) Panel). Therefore, the "acceptance criteria" and "device performance" are related to the ability of this quality control material to consistently produce expected positive and negative results when tested with the BCID2 panel, along with its stability under various conditions.

The provided document describes a series of performance studies for the MDx-Chex for BCID2 device. Since this is a quality control material and not a diagnostic device that interprets clinical images or data, many of the typical acceptance criteria and study design elements of AI/ML-based diagnostic devices (e.g., number of experts, adjudication methods, MRMC studies) are not applicable here.

Here's an analysis based on the information provided, focusing on the relevant criteria for a quality control material:

1. Table of Acceptance Criteria and the Reported Device Performance:

The acceptance criterion across all relevant studies (Multi-Site Precision, Single-Site Precision, Lot-to-Lot Reproducibility, Closed-Vial Stability, Open-Vial Stability, Shipping Stability) is "overall positive and negative percent agreement of ≥ 95%" (except for Lot-to-Lot Reproducibility which states "≥ 90%").

2. Sample size used for the test set and the data provenance:

• Multi-Site Precision (Reproducibility):
  • Sample Size: 10 samples per control level (20 samples total per lot) were tested over 10 days at each of 4 sites. Three lots were used, totaling 240 expected results for positive agreement (4 sites * 20 samples/lot * 3 lots). The narrative indicates "230/240" for observed positive results and "239/240" for negative results (which means 240 expected negative results reported).
  • Data Provenance: Prospective. Collected internally at Streck (La Vista, NE) and externally at UNMC (Omaha, NE), Children's Hospital and Medical Center (Omaha, NE), and Mary Lanning Healthcare (Hastings, NE). All locations are in the USA.
• Single-Site Precision (Repeatability):
  • Sample Size: 20 samples per lot per level were tested by four operators over 20 non-consecutive days. Three lots were used. This totals 120 expected results for positive agreement (20 samples/lot/level * 2 levels * 3 lots, if pooling levels/lots for the single site precision table, or 2023 = 120 positive AND 120 negative). The table shows 114/120 for positive and 120/120 for negative.
  • Data Provenance: Prospective. Collected at Streck (La Vista, NE), USA.
• Lot-to-Lot Reproducibility:
  • Sample Size: 6 samples per control lot and level (12 total samples per lot). Three lots were tested. A total of 36 complete runs were generated. The tables show 12 expected results per lot for positive and negative agreements (12 expected positive results + 12 expected negative results, repeated for 3 lots).
  • Data Provenance: Prospective. Likely collected at Streck (La Vista, NE), USA, as it states "Data from the first 6 timepoints in the repeatability study above were also used".
• Closed-Vial Stability:
  • Sample Size: 10 samples per control lot and level. Three lots. Tested at Day 0 (baseline) and at least Day 61. For Day 0, 120 tubes were completed. For Day 61+, 60 tubes per storage condition (2-8°C and 20-25°C) across the three lots were tested.
  • Data Provenance: Prospective. Likely collected at Streck (La Vista, NE), USA.
• Open-Vial Stability:
  • Sample Size: Same as Closed-Vial Stability, 10 samples per lot per level, three lots, same time points.
  • Data Provenance: Prospective. Likely collected at Streck (La Vista, NE), USA.
• Matrix Effect:
  • Sample Size: Triplicate tests (3/3) for each matrix type (MDx-Chex positive, Clinical positive, MDx-Chex negative, Clinical negative).
  • Data Provenance: Prospective. Likely collected at Streck (La Vista, NE), USA.
• Shipping Stability:
  • Sample Size: Two sets of samples from one control lot were used. Ten samples of Control 1 and Control 2 were tested for each temperature profile (Summer, Winter). This results in 20 expected positive results and 20 expected negative results per temperature profile.
  • Data Provenance: Prospective. Likely collected at Streck (La Vista, NE), USA.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

Not applicable. This is a quality control material where the "ground truth" is the expected positive or negative result for the specific analytes contained or not contained within the control sample, based on its formulation. The performance is assessed by how consistently the QC material produces these known results when run on the target diagnostic system (BioFire FilmArray BCID2 Panel).

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

Not applicable. As noted above, the "ground truth" is inherent to the formulation of the quality control material itself (i.e., it is designed to be positive for certain targets and negative for others). Where initial false results occurred, the "correct results upon a single retest" are noted, which implies a re-run of the test and confirms the expected outcome for the control. This is a standard practice in QC testing to ensure an isolated error vs. a systemic issue.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

Not applicable. This is not an AI/ML diagnostic device for human interpretation. It's a quality control material for an automated molecular diagnostic assay.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

The device (MDx-Chex for BCID2) is a physical control material. Its "standalone" performance is assessed by how accurately the BioFire FilmArray BCID2 Panel (the primary device it controls) identifies the known positive and negative components within the MDx-Chex material. The studies listed demonstrate this "standalone" performance of the control material with the primary diagnostic system. There is no "algorithm" per se being evaluated in terms of its diagnostic accuracy on clinical cases.

7. The type of ground truth used:

• Formulation-based Ground Truth: The ground truth for this quality control material is established by its known composition. The MDx-Chex for BCID2 Controls are manufactured to contain specific inactivated microorganisms and antimicrobial resistance genes (for positive results) or to be free of certain targets (for negative results), as detailed in Table 1 (pages 4-5).
• For the "Matrix Effect" study, "inactivated Streptococcus pneumoniae was spiked into the control matrix" and "into a BD BACTEC Plus Aerobic/F culture bottle with negative whole blood to simulate a clinical samples". The expected results are thus inherently known based on the spiking.

8. The sample size for the training set:

Not applicable in the conventional sense of training an AI/ML model for diagnostic purposes. This is a manufactured chemical/biological control product. Its "training" equivalent would be the R&D and manufacturing processes to ensure its consistent composition and stability. However, if looking for manufacturing lots/batches rather than a statistical training set: Most studies used three separately manufactured lots of the control material (Lot 20363, Lot 20366, Lot 21129) to demonstrate reproducibility and stability.

9. How the ground truth for the training set was established:

Not applicable as it's not an AI/ML model with a discrete training set. The "ground truth" for the characteristics of the product itself would be established through its design specifications, formulation, and in-house characterization during manufacturing.

Ask a specific question about this device