(90 days)
MDx-Chex™ for BC-GN is intended for use as an external positive and negative assayed control to monitor the performance of the qualitative detection of Gram-Negative bacteria and associated antimicrobial resistance genes, by the Luminex VERIGENE® Gram-Negative Blood Culture Nucleic Acid Test (BC-GN) on Luminex VERIGENE® systems. The MDx-Chex™ for BC-GN Positive and Negative Controls are composed of a buffered solution with stabilized erythrocytes and leukocytes in a matrix of blood culture media components. Positive Control: Gram-negative bacteria: Escherichia coli, Klebsiella pneumoniae, Klebsiella oxytoca, Pseudomonas aeruginosa, Species: Acinetobacter spe., Citrobacter spp., Enterobacter spp., Proteus spp.; antimicrobial resistance genes: CTX-M, IMP, KPC, NDM, OXA and VIM. Negative Control: buffered solution only. This product is not intended to replace manufacturer controls provided with the device.
MDx-Chex™ for BC-GN is a quality control kit consisting of positive and negative controls for the Luminex VERIGENE® Gram-Negative Blood Culture Test (BC-GN). The MDx-Chex™ for BC-GN Positive Control is positive for pathogens and resistance mechanisms in the VERIGENE BC-GN test (See Table 1). The MDx-Chex™ for BC-GN Negative Control is negative for pathogens and resistance mechanisms in the VERIGENE BC-GN test. Each control mix also controls for blood and blood culture media components that have been identified is inhibitors to DNA hybridization assays, namely hemoglobin, leukocyte DNA, and anticoagulants.
The MDx-Chex™ for BC-GN quality control kit contains stabilized blood components, blood culture media components, and inactivated, intact microorganisms resulting in a full-process, cellular-based control for the Luminex VERIGENE BC-GN panel. Use of full-process cellular controls are necessary to evaluate the entire analytical process, including sample lysis, nucleic acid isolation, DNA hybridization detection, and analysis, as well as the impact of inhibitors present in blood culture samples and preanalytical variables. Routine use of full process quality controls can help identify variations in the test system that can lead to incorrect results.
The provided document describes the performance of the MDx-Chex™ for BC-GN device, a quality control material, rather than an AI/ML medical device for diagnosis or prediction. Therefore, many of the requested categories (e.g., number of experts, adjudication method, MRMC comparative effectiveness, standalone performance, training set details) are not applicable to this type of device and study.
However, I can extract the acceptance criteria and reported device performance from the provided text for the relevant studies.
Acceptance Criteria and Device Performance
The general acceptance criterion for all studies (Multi-Site Precision, Single-Site Precision, Lot-to-Lot Reproducibility, Closed-Vial Stability, and Shipping Stability) was ≥ 90% agreement with expected results. For Matrix Effect studies, the acceptance criteria was also ≥ 90% agreement for positive detection of analyte for positive controls and ≥ 90% agreement for negative detection of analyte for negative controls.
1. Table of Acceptance Criteria and the Reported Device Performance
| Study | Acceptance Criteria (Positive Percent Agreement - PPA) | Reported Device Performance (PPA) | Acceptance Criteria (Negative Percent Agreement - NPA) | Reported Device Performance (NPA) |
|---|---|---|---|---|
| Multi-Site Precision | ≥ 90% | 99% (89/90) | ≥ 90% | 100% (90/90) |
| Single-Site Precision (Repeatability) | ≥ 90% | 98% (59/60) | ≥ 90% | 100% (60/60) |
| Lot-to-Lot Reproducibility (per lot) | ≥ 90% | 100% (10/10) for all 3 lots | ≥ 90% | 100% (10/10) for all 3 lots |
| Within-Run Precision (per lot) | ≥ 90% | 100% (10/10) for lot 22343 | ≥ 90% | 100% (10/10) for lot 22343 |
| Closed-Vial Stability (Overall) | ≥ 90% | 100% (60/60) at Day 0, 100% (60/60) at Day 61+ (2-8°C & 20-25°C) | ≥ 90% | 100% (60/60) at Day 0, 100% (60/60) at Day 61+ (2-8°C & 20-25°C) |
| Closed-Vial Stability (Per Lot, Day 61+, 20-25°C) | ≥ 90% | 100% (20/20) for lot 22343, 95% (19/20) for lot 22353, 90% (18/20) for lot 22355 | ≥ 90% | 100% (20/20) for all 3 lots |
| Shipping Stability | ≥ 90% | 100% (20/20) for Summer & Winter (both storage temp) | ≥ 90% | 100% (20/20) for Summer & Winter (both storage temp) |
| Matrix Effect (Positive Control) | ≥ 90% | 100% (3/3) | N/A | N/A |
| Matrix Effect (Negative Control) | N/A | N/A | ≥ 90% | 100% (3/3) |
2. Sample sizes used for the test set and the data provenance
-
Multi-Site Precision:
- Sample Size: 10 positive control samples and 10 negative control samples for each of 3 MDx-Chex™ for BC-GN lots, tested across 3 sites, for a total of 30 samples per control type per lot. This resulted in 90 runs per control type (positive/negative) and 180 total runs for data analysis.
- Data Provenance: Not explicitly stated, but implies a prospective study design conducted by the manufacturer (Streck) for regulatory submission.
-
Single-Site Precision (Repeatability):
- Sample Size: 20 samples per control type (positive and negative) for each of 3 MDx-Chex™ for BC-GN lots, tested over 20 days. This resulted in 120 runs (20 runs per control type per lot) for data analysis.
- Data Provenance: Not explicitly stated, but implies a prospective study design conducted by the manufacturer (Streck).
-
Lot-to-Lot Reproducibility:
- Sample Size: 10 positive and 10 negative control tubes per MDx-Chex™ for BC-GN lot (3 lots), resulting in 30 data points per control type (60 total data points).
- Data Provenance: Not explicitly stated, but implies a prospective study design conducted by the manufacturer (Streck).
-
Within-Run Precision:
- Sample Size: 10 tests for each positive and negative control tube from one MDx-Chex™ for BC-GN lot (total of 20 tests).
- Data Provenance: This data was sourced from the Day 60 (2C) closed-vial stability data.
-
Closed-Vial Stability and Shipping Stability:
- Sample Size: 20 positive and 20 negative control samples per MDx-Chex™ for BC-GN lot (3 lots), collected at different timepoints and stored at different temperatures. For shipping: one lot (RPL #22355) with 20 samples per control type for each simulated shipping profile.
- Data Provenance: Not explicitly stated, but implies a prospective study design conducted by the manufacturer (Streck).
-
Matrix Effect:
- Sample Size: Simulated positive MDx-Chex™ for BC-GN matrix (triplicate), simulated positive clinical sample (triplicate), simulated negative MDx-Chex™ for BC-GN matrix (triplicate), simulated negative clinical sample (triplicate). Total of 12 tests.
- Data Provenance: Not explicitly stated, but implies a prospective study design conducted by the manufacturer (Streck).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
Not applicable. This device is a quality control material for an in vitro diagnostic test. The "ground truth" is defined by the expected performance of the control material (i.e., whether it should be detected as positive or negative for specific pathogens/genes by the Luminex VERIGENE® BC-GN system). This "ground truth" is inherent to the control material's formulation and its intended reactivity with the target diagnostic system, not established by human experts.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
Not applicable. As a quality control material, the "ground truth" is binary (positive/negative for specific targets) and is determined by the composition of the control and the design of the diagnostic test it monitors. There is no human adjudication process involved.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Not applicable. This is not an AI/ML diagnostic device, nor does it involve human readers or cases. It is a quality control material for an automated molecular diagnostic test.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
The performance detailed is that of the quality control material when used "standalone" with the Luminex VERIGENE® Gram-Negative Blood Culture Nucleic Acid Test (BC-GN) on Luminex VERIGENE® systems. The "algorithm" in this context refers to the Luminex VERIGENE® system itself, and the studies assess how well the control material performs within that system as expected. The control itself does not have an "algorithm."
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The "ground truth" for these studies is the expected reactivity of the quality control material with the Luminex VERIGENE® BC-GN system.
- Positive Controls: Expected to be "Detected" for specific Gram-negative bacteria and antimicrobial resistance genes.
- Negative Controls: Expected to be "Not Detected" (buffered solution only).
This ground truth is based on the known composition of the MDx-Chex™ for BC-GN control material.
8. The sample size for the training set
Not applicable. This device is a quality control material, not an AI/ML model that requires training data.
9. How the ground truth for the training set was established
Not applicable. See point 8.
§ 866.3920 Assayed quality control material for clinical microbiology assays.
(a)
Identification. An assayed quality control material for clinical microbiology assays is a device indicated for use in a test system to estimate test precision or to detect systematic analytical deviations that may arise from reagent or analytical instrument variation. This type of device consists of single or multiple microbiological analytes intended for use with either qualitative or quantitative assays.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Premarket notification submissions must include detailed device description documentation and information concerning the composition of the quality control material, including, as appropriate:
(i) Analyte concentration;
(ii) Expected values;
(iii) Analyte source;
(iv) Base matrix;
(v) Added components;
(vi) Safety and handling information; and
(vii) Detailed instructions for use.
(2) Premarket notification submissions must include detailed documentation, including line data as well as detailed study protocols and a statistical analysis plan used to establish performance, including:
(i) Description of the process for value assignment and validation.
(ii) Description of the protocol(s) used to establish stability.
(iii) Line data establishing precision/reproducibility.
(iv) Where applicable, assessment of matrix effects and any significant differences between the quality control material and typical patient samples in terms of conditions known to cause analytical error or affect assay performance.
(v) Where applicable, identify or define traceability or relationship to a domestic or international standard reference material and/or method.
(vi) Where applicable, detailed documentation related to studies for surrogate controls.
(3) Premarket notification submissions must include an adequate mitigation (e.g., real-time stability program) to the risk of false results due to potential modifications to the assays specified in the device's 21 CFR 809.10 compliant labeling.
(4) Your 21 CFR 809.10 compliant labeling must include the following:
(i) The intended use of your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following:
(A) Assayed control material analyte(s);
(B) Whether the material is intended for quantitative or qualitative assays;
(C) Stating if the material is a surrogate control; and
(D) The system(s), instrument(s), or test(s) for which the quality control material is intended.
(ii) The intended use in your 21 CFR 809.10(a)(2) and (b)(2) compliant labeling must include the following statement: “This product is not intended to replace manufacturer controls provided with the device.”
(iii) A limiting statement that reads “Quality control materials should be used in accordance with local, state, federal regulations, and accreditation requirements.”