(79 days)
NG-Test CARBA 5 is an in vitro rapid and visual multiplex immunochromatographic assay for the qualitative detection and differentiation of five common carbapenemases (KPC, OXA-48-like, VIM, IMP and NDM) from carbapenem non-susceptible pure bacterial colonies when grown on the following media:
- 5% sheep blood agar or MacConkey agar (16-24 hours) for testing Enterobacteriaceae and Pseudomonas aeruginosa
- HardyCHROM™ CRE agar (18-24 hours) for testing E. coli and KES (Klebsiella aerogenes, . Klebsiella oxytoca, Klebsiella pneumoniae, Enterobacter cloacae complex, and Serratia marcescens).
The NG-Test CARBA 5 is intended as an aid for infection control in the detection of carbapenemase-producing Enterobacteriaceae and Pseudomonas aeruginosa in healthcare settings. NG-Test CARBA 5 is not intended to guide or monitor treatment for carbapenem non-susceptible bacterial infections. A positive or negative NG-Test CARBA 5 test result does not rule out the presence of other mechanisms of antibiotic resistance. NG-Test CARBA 5 should be used in conjunction with other laboratory tests including phenotypic antimicrobial susceptibility testing.
NG-Test CARBA 5 is an in vitro rapid and visual multiplex immunochromatographic assay that detects one or more of the five common types of carbapenemase enzymes (KPC (K), OXA-48-like (O), IMP (I), VIM (V), NDM (N)) in bacterial colonies. Liquid extraction buffer is used as a cell lysing solution when mixed with colonies. Monoclonal antibodies that individually recognize each of the five carbapenemases are immobilized on a nitrocellulose membrane. Free monoclonal antibodies are present in the sample pad and labelled with colloidal gold. Upon addition of colonies mixed with extraction buffer to the sample pad, the capillary action of the nitrocellulose draws the sample the mobile antibodies and immobile antibodies on the test strip. The immobilized control antibodies capture any mobile antibodies that run through the sample pad and nitrocellulose without binding to other test lines. A positive result occurs when a red line appears on the control region and one or more lines appear in the test regions (K. O. V. I. or N) and indicates that the sample contains one or more carbapenemases. A negative result occurs when only the control line is observed and indicates that the sample does not contain any of the five carbapenemases. If the control line does not appear, the test result is invalid. The sample pad, and nitrocellulose strip are contained within a plastic cassette. After addition of colonies in extraction buffer to the sample port, a result can be read after 15 minutes.
The NG-Test CARBA 5 is an in vitro rapid and visual multiplex immunochromatographic assay designed to detect five common carbapenemase enzymes (KPC, OXA-48-like, IMP, VIM, NDM) in bacterial colonies. The device's performance was evaluated through a multi-center study to establish its acceptance criteria.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the NG-Test CARBA 5 are based on Positive Percent Agreement (PPA) and Negative Percent Agreement (NPA) compared to a composite reference method. The reported device performance met or exceeded these criteria for both Enterobacteriaceae and P. aeruginosa across different culture media.
Table 1: Acceptance Criteria and Reported Device Performance
| Category / Organism Group | Metric | Acceptance Criteria (Implicit) | Reported Device Performance (95% CI) |
|---|---|---|---|
| Enterobacteriaceae (Agar: 5% Sheep Blood or MacConkey) | Overall PPA | High agreement (generally ≥90%) | 100.0% (97.6% - 100.0%) |
| Overall NPA | High agreement (generally ≥90%) | 95.5% (88.9% - 98.2%) | |
| After Discrepant Analysis | Overall PPA | 100.0% (97.7% - 100.0%) | |
| Overall NPA | 100.0% (95.6% - 100.0%) | ||
| P. aeruginosa (Agar: 5% Sheep Blood or MacConkey) | Overall PPA | High agreement (generally ≥90%) | 100.0% (77.2% - 100.0%) |
| Overall NPA | High agreement (generally ≥90%) | 94.6% (85.4% - 98.2%) | |
| After Discrepant Analysis | Overall PPA | 100.0% (80.6% - 100%) | |
| Overall NPA | 100.0% (93.2% - 100%) | ||
| E. coli, KES (Agar: HardyCHROM™ CRE, Raw Stool) | Overall PPA | High agreement (generally ≥90%) | 100.0% (97.4% - 100.0%) |
| Overall NPA | High agreement (generally ≥90%) | 90.2% (77.5% - 96.1%) | |
| After Discrepant Analysis | Overall PPA | 100.0% (Likely same as without disp. analysis) | |
| Overall NPA | 100.0% (90.6% - 100.0%) | ||
| E. coli, KES (Agar: HardyCHROM™ CRE, C&S Cary Blair Stool) | Overall PPA | High agreement (generally ≥90%) | 100.0% (97.3% - 100.0%) |
| Overall NPA | High agreement (generally ≥90%) | 90.2% (77.5% - 96.1%) | |
| After Discrepant Analysis | Overall PPA | 100.0% (97.4% - 100.0%) | |
| Overall NPA | 100.0% (Likely same as without disp. analysis) |
(Note: "Implicit" acceptance criteria are derived from the fact that the device was found substantially equivalent, meaning it met FDA's expectations for performance for this class of device. Exact pre-defined numerical thresholds for acceptance were not explicitly stated in the provided text, but the reported performance generally reflects high agreement to the reference methods.)
2. Sample Size Used for the Test Set and Data Provenance
- Agar: 5% Sheep Blood or MacConkey agar:
- Total organisms initially tested: 310
- Organisms included in analysis: 309 (1 Pseudomonas species other than P. aeruginosa excluded)
- Enterobacteriaceae: 240 isolates (yielding 244 results due to co-production of two carbapenemases in four isolates).
- P. aeruginosa: 69 isolates.
- Provenance: Prospectively-collected and stock bacterial isolates from three geographically diverse hospitals.
- Agar: HardyCHROM™ CRE agar (from stool specimens):
- Total organisms initially enrolled: 186
- Organisms included in analysis: 185 (1 organism unavailable)
- Raw Stool: 180 organisms (184 target results) for E. coli and KES (Klebsiella aerogenes, K. oxytoca, K. pneumoniae, Enterobacter cloacae complex, and Serratia marcescens).
- C&S Cary Blair Stool: 178 organisms (182 target results) for E. coli and KES.
- Provenance: This data was collected internally using the same bacterial isolates (from blood and MacConkey agar study) cultured on HardyCHROM™ CRE after being seeded in raw and C&S Cary Blair stool.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not explicitly state the number or qualifications of experts (e.g., radiologists, microbiologists) who established the ground truth. However, the ground truth was established through a "composite reference method" combining:
- An FDA-cleared device: Xpert Carba-R by Cepheid (PCR for KPC, OXA-48 or 181, IMP, VIM, NDM). This is a molecular method, so human expertise here would primarily be in running the test and interpreting its output according to established protocols.
- Modified carbapenem inactivation method (mCIM) and EDTA carbapenemase inactivation method (eCIM) as described by CLSI M100, S29. These are phenotypic tests requiring skilled microbiologists for execution and interpretation.
- Antibiotic susceptibility testing results to ertapenem, imipenem, and meropenem. This also requires expertise in microbiology and adherence to CLSI guidelines.
- Identity and susceptibility of organisms were confirmed using FDA-cleared ID and AST systems. This implies expert oversight or use of automated systems with established performance.
For discrepant analysis, alternative PCR assays and bidirectional sequencing were used, which would involve highly specialized molecular microbiologists.
4. Adjudication Method for the Test Set
The study employed discrepant analysis to resolve inconsistencies between the NG-Test CARBA 5 results and the initial composite reference method results. This means that when the device result did not agree with the initial ground truth, further, more definitive tests (like alternative PCR assays and bidirectional sequencing) were performed. The results from these additional, often more gold-standard methods, were then used to adjudicate the true status of the isolate, effectively updating the ground truth for those specific cases. This "2+1" or "3+1" type of adjudication (NG-Test vs. Composite Reference, then a tie-breaker/definitive test) was used to ensure the most accurate ground truth for performance calculation.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
The provided text does not describe a multi-reader multi-case (MRMC) comparative effectiveness study involving human readers with and without AI assistance (as the NG-Test CARBA 5 is an immunochromatographic assay, not an AI-driven image analysis tool requiring human interpretation assistance). The device provides a visual, rapid result, and its performance is evaluated against laboratory gold standards, not through human reader performance improvement.
6. Standalone Performance Study
Yes, the performance study directly evaluates the standalone performance of the NG-Test CARBA 5 device. The provided tables and metrics (PPA, NPA) reflect the algorithm's performance (in this case, the immunochromatographic assay's ability to detect carbapenemases) without human intervention in the result determination process beyond the initial preparation and visual reading of the test strip. The operators were blinded to the expected result during setup and interpretation, indicating an objective assessment of the device's inherent performance.
7. Type of Ground Truth Used
The ground truth used was a composite reference method consisting of:
- Molecular (PCR): Xpert Carba-R (FDA-cleared device).
- Phenotypic (Enzymatic Activity): Modified Carbapenem Inactivation Method (mCIM) and EDTA Carbapenemase Inactivation Method (eCIM) as per CLSI guidelines.
- Phenotypic (Susceptibility Testing): Antibiotic susceptibility testing results (ertapenem, imipenem, meropenem).
- Discrepant Analysis: For conflicting results, more definitive molecular techniques (alternative PCR and bidirectional sequencing) were employed to establish the definitive truth.
This multifaceted approach to ground truth ensures robust and accurate classification of carbapenemase production.
8. Sample Size for the Training Set
The provided document does not mention a training set or its sample size. This is typical for in vitro diagnostic (IVD) devices like the NG-Test CARBA 5 which are based on biochemical or immunological principles, rather than machine learning algorithms that require explicit training data. The "analytical reactivity" section describes testing against a panel of 92 characterized strains, which could be considered an internal validation or analytical performance assessment, but not a "training set" in the machine learning sense.
9. How the Ground Truth for the Training Set Was Established
As no training set (in the context of machine learning) is described, there is no information on how its ground truth was established. The "Analytical Reactivity" study used 92 strains "characterized to have a target carbapenemase," implying that their carbapenemase status was already established by other reference methods prior to this specific evaluation. This characterization would have been done using methods similar to the composite reference described for the main performance study (e.g., PCR, sequencing, phenotypic tests).
§ 866.1640 Antimicrobial susceptibility test powder.
(a)
Identification. An antimicrobial susceptibility test powder is a device that consists of an antimicrobial drug powder packaged in vials in specified amounts and intended for use in clinical laboratories for determining in vitro susceptibility of bacterial pathogens to these therapeutic agents. Test results are used to determine the antimicrobial agent of choice in the treatment of bacterial diseases.(b)
Classification. Class II (performance standards).