(84 days)
The Xpert® MRSA NxG Assay, performed on the GeneXpert® Instrument Systems, is a qualitative in vitro diagnostic test intended for the detection of methicillin-resistant Staphylococcus aureus (MRSA) DNA directly from nasal swabs in patients at risk for nasal colonization. The test utilizes automated real-time polymerase chain reaction (PCR) for the amplification of MRSA-specific DNA targets and fluorogenic target-specific hybridization probes for the real-time detection of the amplified DNA. The Xpert MRSA NxG Assay is intended to aid in the prevention and control of MRSA infections in healthcare settings. The Xpert MRSA NxG Assay is not intended to diagnose, guide, or monitor treatment for MRSA infections, or provide results of susceptibility to methicillin. A negative result does not preclude MRSA nasal colonization. Concomitant cultures are necessary to recover organisms for epidemiological typing or for further susceptibility testing.
The Xpert MRSA NxG Assay is an automated real-time polymerase chain reaction (PCR) in vitro diagnostic test for qualitative detection of methicillin-resistant Staphylococcus aureus (MRSA) directly from nasal swab specimens of patients at risk for colonization with MRSA in a healthcare setting. The Xpert MRSA NxG Assay is performed on the Cepheid GeneXpert® Instrument Systems (GeneXpert Dx, GeneXpert Infinity-48, GeneXpert Infinity-48s, and GeneXpert Infinity-80 systems). The GeneXpert Instrument System platform automates sample preparation, amplification and real-time detection. The GeneXpert Instrument Systems require the use of single-use, disposable cartridges (the Xpert MRSA NxG cartridges) that hold the PCR reagents and host the PCR process. Because the cartridges are self-contained and specimens never come into contact with working parts of the instrument modules, cross-contamination between samples is minimized. The Xpert MRSA NxG Assay includes reagents for the detection of target MRSA DNA. The primers and probes in the Xpert MRSA NxG Assay detect proprietary sequences for the gene for methicillin/oxacillin resistance (mecA and mecC), and staphylococcal cassette chromosome mec (SCCmec), which is inserted into the SA chromosomal attB site. A Sample Processing Control (SPC) and a Probe Check Control (PCC) are internal controls utilized by the GeneXpert Instrument System platform. The SPC is present to control for adequate processing of the target bacteria and to monitor for the presence of inhibitor(s) in the PCR assay to avoid false-negative results. The Probe Check Control verifies reagent rehydration, real-time PCR tube filling in the cartridge, probe integrity, and dye stability. The single-use, multi-chambered fluidic cartridges are designed to complete sample preparation and real-time PCR for the detection of genomic DNA from methicillinresistant Staphylococcus aureus (MRSA) in 70 minutes or less. The GeneXpert Instrument Systems, comprised of the GeneXpert Dx Systems and the GeneXpert Infinity Systems, have 1 to 80 randomly accessible modules, depending upon the instrument, that are each capable of performing separate sample preparation and real-time PCR and RT-PCR tests. Each module contains a syringe drive for dispensing fluids (i.e., the syringe drive activates the plunger that works in concert with the rotary valve in the cartridge to move fluids between chambers), an ultrasonic horn for lysing cells or spores, and a proprietary I-CORE® thermocycler for performing real-time PCR and RT-PCR and detection. Nasal swab specimens are collected using the Cepheid Sample Collection Device and are transported to the laboratory or designated GeneXpert testing area. The nasal swab specimen is placed in a tube containing 2 mL of Elution Reagent. Following a brief vortexing of the sample, the entire contents of the Elution Reagent tube are transferred to the sample chamber of the Xpert MRSA NxG cartridge using a transfer pipette. For nasal swab specimens that are collected using Copan Liquid Amies Elution Swab (ESwab) collection and transport system, 300 uL of liquid sample is transferred to a tube containing 2 mL of Elution Reagent. Following a brief vortexing of the sample, the entire contents of the Elution Reagent tube are transferred to the sample chamber of the Xpert MRSA NxG cartridge using a transfer pipette. The user initiates a test from the system user interface and places the cartridge into the GeneXpert instrument platform, which performs hands-off real-time, multiplex PCR for detection of DNA. The results are automatically generated at the end of the process in a report that can be viewed and printed.
Here's a breakdown of the acceptance criteria and the study details for the Xpert MRSA NxG device, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" for the clinical performance in a numerical format that would be directly comparable to the predicate device. Instead, it aims to demonstrate "substantial equivalence" to the predicate device (BD MAX™ MRSA XT assay). The primary performance metrics for this type of in vitro diagnostic test are analytical sensitivity (Limit of Detection), clinical sensitivity, and clinical specificity. I've extracted the relevant performance metrics from the clinical studies to serve as the reported device performance.
| Performance Metric | Acceptance Criteria (Implied for Substantial Equivalence to Predicate) | Reported Device Performance (Xpert MRSA NxG Assay) | Comments |
|---|---|---|---|
| Analytical Studies | |||
| Limit of Detection (LoD) | Not explicitly stated (demonstrated reproducibility at 95% confidence) | Rayon Swab: 95% positive at 302 CFU/swab | The study determined the lowest concentration of MRSA that can be reproducibly distinguished from negative samples 95% of the time with 95% confidence. This is a measure of analytical sensitivity. |
| ESwab: 95% positive at 812 CFU/swab | |||
| Analytical Specificity (Cross-reactivity) | 100% specificity for tested non-MRSA organisms | 100% for 152 potentially cross-reactive microorganisms and human cells | No cross-reactivity with methicillin-susceptible Staphylococcus aureus (MSSA), coagulase-negative staphylococci, or other nasal flora. |
| Inclusivity | 100% detection of diverse MRSA strains | 100% (all 196 MRSA strains correctly detected) | Assessed with a wide range of MRSA strains, including various SCCmec types and known USA types. |
| Microbial Interference | No interference or cross-reactivity observed | No interference or cross-reactivity observed | Tested with 9 common commensal bacteria found in nasal cavities. |
| Interfering Substances | No false negatives; delays in Ct values acceptable if results remain correct | Identified 3 substances causing Ct delays, but no false negatives. | Addressed as limitations in the package insert. |
| Carry-Over Contamination | 100% prevention of crossover contamination | 100% (no carry-over in 42 negative samples) | Demonstrated in a study with high positive and negative samples. |
| Clinical Studies | |||
| Clinical Sensitivity | Substantially equivalent to predicate device | Rayon Swab: 91.0% (95% CI: 84.6-94.9) | Based on two prospective, multi-site studies against reference culture and susceptibility testing. |
| ESwab: 92.9% (95% CI: 86.0-96.5) | |||
| Combined: 91.8% (95% CI: 87.4-94.8) | |||
| Clinical Specificity | Substantially equivalent to predicate device | Rayon Swab: 96.9% (95% CI: 95.7-97.8) | |
| ESwab: 97.6% (95% CI: 96.2-98.5) | |||
| Combined: 97.2% (95% CI: 96.3-97.9) | |||
| Positive Predictive Value (PPV) | Not explicitly stated (reported for context) | Rayon Swab: 78.7% (95% CI: 71.3-84.7) | |
| ESwab: 83.5% (95% CI: 75.4-89.3) | |||
| Combined: 80.8% (95% CI: 75.5-85.2) | |||
| Negative Predictive Value (NPV) | Not explicitly stated (reported for context) | Rayon Swab: 98.9% (95% CI: 98.0-99.4) | |
| ESwab: 99.1% (95% CI: 98.1-99.5) | |||
| Combined: 98.9% (95% CI: 98.3-99.3) | |||
| Reproducibility | High agreement across sites, days, lots, operators | 98.6-100% agreement for low-positive and negative samples | Demonstrated excellent reproducibility in % agreement and low variability in Ct values. |
2. Sample Sizes Used for the Test Set and Data Provenance
- Clinical Test Sets:
- Rayon Swab Study: 1103 eligible nasal swab specimens.
- ESwab Study: 846 eligible nasal swab specimens.
- Combined: 1949 total specimens.
- Data Provenance: The clinical studies were prospective, multi-site investigational studies.
- The Rayon Swab study involved "eight investigational sites within the US and outside of the US."
- The ESwab study involved "six investigational sites within the US."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
The ground truth for the clinical test set was established using a reference culture and susceptibility testing method, not expert consensus in the traditional sense of medical image interpretation (e.g., radiologists). The document does not specify the number or qualifications of laboratory personnel performing these culture and susceptibility tests. However, the FDA's acceptance of this method implies that these are standard, qualified laboratory procedures.
4. Adjudication Method (for the test set)
The adjudication method was based on the reference culture and susceptibility testing.
- A specimen was considered positive for MRSA if the presence of MRSA was confirmed in either direct culture on MRSA selective chromogenic medium or via an enriched culture (Trypticase Soy Broth (TSB) with 6.5% Sodium Chloride followed by subculture on Blood Agar (BA) and MRSA selective chromogenic medium).
- Identification of presumptive S. aureus colonies was confirmed with Gram stain, catalase, and coagulase testing.
- MRSA confirmation was done by susceptibility testing with a Cefoxitin disk (30µg).
- For discrepancies (e.g., false positives/negatives by the device), repeat subculture of the enrichment broth was performed, as noted in the footnotes to Tables 8-7 and 8-8. This acts as a form of "adjudication" for discrepant results.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not performed. This device is an automated, standalone diagnostic assay (real-time PCR) for detecting MRSA DNA. It does not involve human "readers" in the interpretation of results in a way that would be typical for image-based AI tools. Therefore, there's no "human improvement with AI vs without AI assistance" to report.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance evaluation was conducted. The Xpert MRSA NxG Assay is an automated qualitative in vitro diagnostic test, and its performance (sensitivity, specificity) was directly compared against the reference method without human interpretation of the assay's output itself. The device itself performs sample preparation, amplification, and real-time detection, automatically generating results.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth used was laboratory culture and susceptibility testing, specifically:
- Direct culture on MRSA selective chromogenic medium.
- Enriched culture (TSB with 6.5% NaCl, followed by subculture on BA and MRSA selective chromogenic medium).
- Confirmation of S. aureus by Gram stain, catalase, and coagulase testing.
- Confirmation of MRSA by Cefoxitin disk susceptibility testing.
8. The Sample Size for the Training Set
The document does not explicitly mention a "training set" in the context of machine learning model development. This is an in vitro diagnostic device based on PCR technology, which typically does not involve machine learning training in the same way an AI imaging model would. The "training" here would refer to the development and optimization of the PCR primers and probes, and the cartridge and instrument system, which is part of the product development process rather than a distinct "training set" for an algorithm.
However, if we consider "training" in the sense of analytical development, the following data points could be relevant to the development and optimization of the assay's internal logic and performance parameters:
- Analytical Sensitivity (LoD) Study: Used 13 individual MRSA strains (representing various SCCmec types).
- Analytical Specificity (Cross-reactivity) Study: Tested 152 potentially cross-reactive microorganisms and human cells.
- Analytical Reactivity (Inclusivity) Study: Evaluated 196 methicillin-resistant Staphylococcus aureus strains.
These studies technically inform and confirm the performance of the developed assay, which is analogous to a validation set in traditional software, but not a training set for an AI/ML algorithm.
9. How the Ground Truth for the Training Set Was Established
As noted in point 8, the concept of a "training set" in the context of an AI/ML algorithm, along with its ground truth, is not applicable here. The assay's analytical parameters (e.g., LoD, specificity, inclusivity) were established through rigorous laboratory testing using well-characterized microbial strains and samples, where the "ground truth" for each analytical study (e.g., presence/absence of a specific MRSA strain, concentration, presence of interfering substance) was defined by standard microbiological and chemical methods.
§ 866.1640 Antimicrobial susceptibility test powder.
(a)
Identification. An antimicrobial susceptibility test powder is a device that consists of an antimicrobial drug powder packaged in vials in specified amounts and intended for use in clinical laboratories for determining in vitro susceptibility of bacterial pathogens to these therapeutic agents. Test results are used to determine the antimicrobial agent of choice in the treatment of bacterial diseases.(b)
Classification. Class II (performance standards).