(171 days)
The BD MAX Check-Points CPO Assay performed on the BD MAX System is a qualitative, automated in vitro diagnostic real-time PCR test designed for the detection and differentiation of the carbapenemase genes blakes, blayble blaymoblance and black-48, that are associated with carbapenem non-susceptibility in Gram-negative bacteria. The assay does not distinguish between the blay™ and bland genes.
The BD MAX Check-Points CPO Assay is intended as an aid to infection control in the detection of carbapenem-non-susceptible bacteria that colonize patients in healthcare settings. The BD MAX Check-Points CPO Assay is not intended to guide or monitor treatment for carbapenem-non-susceptible bacterial infections. A negative BD MAX Check-Points CPO Assay result does not preclude the presence of other resistance mechanisms.
Testing is performed on rectal swabs from patients at risk for intestinal colonisation with carbapenem nonsusceptible bacteria. This test is intended for use in conjunction with clinical presentation, laboratory findings, and epidemiological information. Results of this test should not be used as the sole basis for patient management decisions. Concomitant cultures are necessary to recover organisms for epidemiological typing, antimicrobial susceptibility testing, and for further confirmatory bacterial identification.
The BD MAX Check-Points CPO Assay detects the presence of carbapenemase genes in Gram-negative bacteria and includes an internal Sample Processing Control.
Rectal swab specimens are collected from patients using ESwab. After sampling they are transported to the laboratory in the Amies transport media of the ESwab. The ESwab is vortexed and a 50 µl aliquot is transferred to the Sample Buffer Tube using a pipette with disposable filter tip. The Sample Buffer Tube is closed with a septum cap and vortexed. Once the worklist is generated and the clinical specimen is loaded on the BD MAX system, along with a BD MAX Check-Points CPO Reagent Strip and BD MAX PCR Cartridge, the run is started and no further operator intervention is required. The BD MAX System automates sample preparation, including target organism lysis, DNA extraction and concentration, reagent rehydration, target nucleic acid sequence amplification and detection using real-time PCR. The interpretation of the signal is performed automatically by the BD MAX System. The assay also includes a Sample Processing Control that is provided in the Extraction Tube and subjected to extraction, concentration and amplification steps. The Sample Processing Control monitors for the presence of potential inhibitory substances as well as system or reagent failures.
Following enzymatic cell lysis at an elevated temperature, the released nucleic acids are captured on magnetic affinity beads. The beads, with the bound nucleic acids, are washed and the nucleic acids are eluted. Eluted DNA is neutralized and transferred to the Master Mix Tube to rehydrate the PCR reagents. After rehydration, the BD MAX System dispenses a fixed volume of PCR-ready solution into the BD MAX PCR Cartridge. Microvalves in the BD MAX PCR Cartridge are sealed by the system prior to initiating PCR to contain the amplification mixture thus preventing evaporation and contamination. The amplified DNA targets are detected using hydrolysis (TaqMan®) probes, labeled at one end with a fluorescent reporter dve (fluorophore) and at the other end with a quencher moiety. Probes labeled with different fluorophores are used to detect amplicons for the carbapenemase genes KPC, VIM, OXA-48, NDM, IMP and the Sample Processing Control in five different optical channels of the BD MAX System.
The VIM and IMP genes are combined in one optical channel of the BD MAX system, all other genes have a separate optical channel. When the probes are in their native state, the fluorescence of the fluorophore is quenched due to its proximity to the quencher. However, in the presence of target DNA, the probes hybridize to their complementary sequences and are hydrolyzed by the 5'-3' exonuclease activity of the DNA polymerase as it synthesizes the nascent strand along the DNA template. As a result, the fluorophores are separated from the quencher molecules and fluorescence is emitted. The BD MAX System monitors these signals at each cvcle and interprets the data at the end of the program to report the final results.
The provided document describes the performance characteristics of the BD MAX Check-Points CPO Assay. This device is a qualitative, automated in vitro diagnostic real-time PCR test for the detection and differentiation of specific carbapenemase genes (blaKPC, blaVIM/blaIMP, blaOXA-48, and blaNDM).
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance:
The document doesn't explicitly state "acceptance criteria" in a separate section with specific numerical thresholds for sensitivity and specificity. However, based on the presentation of "Performance Results" with 95% Confidence Intervals (CI), it's implied that the device is deemed acceptable if these performance metrics are sufficiently high and demonstrate clinical utility. The general expectation for diagnostic assays is high sensitivity and specificity.
Since direct acceptance criteria are not provided, I will present the reported performance, which implicitly met the FDA's requirements for substantial equivalence.
| Performance Metric (Gene) | Acceptance Criteria (Implied) | Reported Device Performance (Prospective Study, MacConkey + chromID PCR/Sequencing) | Reported Device Performance (Contrived Study, PCR/Sequencing) |
|---|---|---|---|
| KPC Sensitivity/PPA | High (e.g., >85-90%) | 88.2% (73.4-95.3% CI) | 100% (88.6-100% CI) |
| KPC Specificity/NPA | High (e.g., >95-98%) | 99.8% (99.4-99.9% CI) | 100% (97.3-100% CI) |
| VIM/IMP Sensitivity/PPA | High (e.g., >85-90%) | 75.0% (40.9-92.9% CI) | 96.2% (87.0-98.9% CI) |
| VIM/IMP Specificity/NPA | High (e.g., >95-98%) | 99.2% (98.7-99.6% CI) | 100% (96.7-100% CI) |
| OXA-48 Sensitivity/PPA | High (e.g., >85-90%) | 96.2% (81.1-99.3% CI) | 100% (88.6-100% CI) |
| OXA-48 Specificity/NPA | High (e.g., >95-98%) | 99.0% (98.3-99.4% CI) | 100% (97.3-100% CI) |
| NDM Sensitivity/PPA | High (e.g., >85-90%) | 100% (20.7-100% CI) | 100% (88.6-100% CI) |
| NDM Specificity/NPA | High (e.g., >95-98%) | 99.9% (99.6-100% CI) | 100% (97.3-100% CI) |
| Non-Reportable Rate | Low (e.g., 90-95%) | Ranged from 97.5% - 100% | N/A |
| Inter-lot Reproducibility | High (e.g., >90-95%) | Ranged from 99.2% - 100% | N/A |
| Intra-lab Reproducibility | High (e.g., >90-95%) | Ranged from 97.9% - 100% | N/A |
| Carry-over Contamination | No positive calls from negative specimens after high-load positives | No positive calls in 166 negative specimens | N/A |
| Mixed Infection/Competitive Interference | No interference observed | No interference in 13 mixed specimens | N/A |
2. Sample Sizes Used for the Test Set and Data Provenance:
-
Test Set Size:
- Prospective Specimens: 1486 initially collected, 1473 eligible for clinical evaluation after excluding unreportable ones.
- Contrived Specimens: 166.
- Analytical Inclusivity: 93 strains (containing 97 target genes).
- Analytical Specificity: 26 organisms tested.
- Interfering Substances: 29 substances tested.
- Reproducibility Studies (Inter-lab, Inter-lot, Intra-lab): Varied sample sizes per gene and concentration level, typically involving 60-120 replicates for positives and 72-100 for negatives per study type.
- Carry-over Contamination: 166 negative specimens.
- Mixed Infection/Competitive Interference: 13 specimens.
-
Data Provenance:
- Clinical (Prospective) Study: Samples collected from "five (5) geographically diverse clinical centers" as part of routine patient care. This indicates prospective collection from real-world clinical settings, likely within the country of the clinical centers (not explicitly stated, but generally implied to be within the jurisdiction of the submitting company or where clinical trials are feasible).
- Contrived Study: Specimens were prepared by spiking "well-characterized strains into unique negative rectal swab matrix." This is a laboratory-based study design.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:
- The document does not mention the number or qualifications of experts involved in establishing the ground truth for the clinical specimens.
- The ground truth for the clinical (prospective) study was established by a "reference method" consisting of:
- Bacterial culture for recovery of non-susceptible isolates from rectal swab specimens.
- Followed by detection of antibiotic resistance genes by PCR and sequencing.
- Bacterial culture protocol included both direct plating and overnight enrichment.
- For contrived specimens, ground truth was also established by PCR and sequencing of the spiked strains.
- For analytical inclusivity, strains were either from "public collections and well-characterized clinical isolates," implying previous expert characterization.
4. Adjudication Method for the Test Set:
- The document does not describe an adjudication method involving multiple experts for the clinical test set results.
- The comparison is between the BD MAX device results and the reference method (culture + PCR/sequencing), which serves as the established ground truth. Discrepancies would typically be reviewed by laboratory personnel, but a formal multi-expert adjudication process is not detailed.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, and the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done.
- This device is an automated in vitro diagnostic real-time PCR test, not an AI-assisted imaging or diagnostic tool that involves human readers interpreting results with or without AI assistance. The results are interpreted automatically by the BD MAX System.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done:
- Yes, this entire study is a standalone performance evaluation of the algorithm/device itself.
- The "BD MAX System automates sample preparation, including target organism lysis, DNA extraction and concentration, reagent rehydration, target nucleic acid sequence amplification and detection using real-time PCR."
- "The interpretation of the signal is performed automatically by the BD MAX System."
- The performance tables directly compare the BD MAX Check-Points CPO (the device's output) to the gold standard reference method.
7. The type of ground truth used:
- Expert Consensus + Laboratory Reference Method for Clinical Specimens: For prospective clinical specimens, the ground truth was established by a bacterial culture-based reference method including PCR and sequencing. While not explicitly "expert consensus" in the sense of multiple independent interpretations, the reliance on established laboratory techniques (culture, PCR, sequencing) for genetic characterization is a form of highly reliable, validated ground truth in molecular diagnostics.
- Molecular Characterization for Contrived Specimens: For contrived specimens, the ground truth was based on the known molecular characteristics of the spiked strains, confirmed by PCR and sequencing.
- Known Strain Characteristics for Analytical Studies: For inclusivity, specificity, LoD, and reproducibility studies, the ground truth was based on the known genetic profile and concentration of the bacterial strains used, confirmed by standard molecular methods (e.g., PCR and sequencing for gene presence).
8. The sample size for the training set:
- The document does not provide information on a specific training set size because this is a molecular diagnostic assay, not a machine learning model that undergoes a distinct "training" phase with labeled data in the same way.
- The development of such an assay involves extensive research and development to design primers and probes, optimize reaction conditions, and establish thresholds, but this is a different process than training an AI algorithm on a dataset. The analytical studies (LoD, inclusivity, specificity) demonstrate the robustness and limits of the assay design.
9. How the ground truth for the training set was established:
- As mentioned above, there isn't a "training set" in the AI/ML sense for this type of device.
- The design of the assay (primers, probes) would have been based on established scientific knowledge of carbapenemase gene sequences and validated against known reference strains with confirmed gene presence/absence through sequencing and other molecular characterization techniques. This iterative process of design and validation ensures the assay's ability to detect the target genes accurately.
§ 866.1640 Antimicrobial susceptibility test powder.
(a)
Identification. An antimicrobial susceptibility test powder is a device that consists of an antimicrobial drug powder packaged in vials in specified amounts and intended for use in clinical laboratories for determining in vitro susceptibility of bacterial pathogens to these therapeutic agents. Test results are used to determine the antimicrobial agent of choice in the treatment of bacterial diseases.(b)
Classification. Class II (performance standards).