(215 days)
HardyCHROM™ MRSA is a selective and differential chromogenic medium recommended for the qualitative detection of nasal colonization by methicillin-resistant Staphylococcus aureus (MRSA) to aid in the prevention and control of MRSA infections in health care settings. The test is performed on anterior nares swabs from patients and healthcare workers to screen for MRSA colonization. HardyCHROM™ MRSA is not intended to diagnose MRSA infection nor to guide or monitor therapy for MRSA infections. Concomitant cultures are necessary to recover organisms for susceptibility testing or epidemiological typing. A negative result does not preclude MRSA nasal colonization.
HardyCHROM™ MRSA is translucent and light amber in color and contains chromogens that release chromophores when cleaved by enzymes produced by methicillin resistant Staphylococcus aureus (MRSA) strains. Based on colony color, HardyCHROM™M MRSA allows for the reliable detection of most MRSA from clinical specimens in less than 24 hours based on the appearance of a pink/magenta colored colony. Non-MRSA strains are either inhibited by the addition of selective agents or utilize different chromogenic substrates in the media to produce different colored colonies. If none of the substrates are utilized, natural or white colored colonies will be present. If plates are negative for growth after 24 hours, it is recommended to re-incubate for an additional 24 hours.
Here's an analysis of the HardyCHROM™ MRSA study, focusing on acceptance criteria and performance data:
Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria as numerical thresholds for agreement or other metrics. Instead, the "Summary of Performance Data" sections describe the results and, for some interference studies, mention that "The sensitivity and specificity criteria of >98% were met." For the clinical reproducibility study, a "reproducibility rate of 100% for both inter-lot and overall testing intervals was achieved."
Based on the clinical study results, the performance of HardyCHROM™ MRSA is compared against "Traditional Culture" and a "Commercial Chromogenic MRSA Medium." We can infer the desired performance to be high agreement rates with these established methods.
Inferred Acceptance Criteria from Performance Data and Stated Goals:
| Metric | Acceptance Criteria (inferred) | Reported Device Performance (HardyCHROM™ MRSA) | Study/Section Proving Criterion Met |
|---|---|---|---|
| Interference Studies | |||
| Nasal Sprays - Sensitivity | >98% | >98% | Interference Study Results - Nasal Sprays (Page 1) |
| Nasal Sprays - Specificity | >98% | >98% | Interference Study Results - Nasal Sprays (Page 1) |
| Human Blood - Recovery Rate | 100% recovery (implied, no inhibition) | 100% recovery for all strains | Interference Study Results - Human Blood (Page 1) |
| Mucin - Inhibition | No inhibitory effect on growth/performance (implied) | No discernible differences in performance; MSSA inhibited | Interference Study Results - Mucin (Page 1) |
| Transport Media - Inhibition | No inhibitory effect on growth/performance (implied) | 100% recovery with no inhibition for all transport media tested | Interference Study Results - Transport Media (Page 2) |
| Cross-Reactivity Study | |||
| Inter-lot Reproducibility | >99% | >99% | Cross-Reactivity Study (Page 2) |
| Overall Testing Intervals Reproducibility | >99% | >99% | Cross-Reactivity Study (Page 2) |
| Clinical Performance (vs. Traditional Culture) | |||
| Positive Percent Agreement (PPA) at 24 hrs | High agreement with traditional culture (implied) | 93.3% (95% CI 89.6-98.4%) | Table 1: HardyCHROM™ MRSA vs. Traditional Culture 24 hours (Page 5) |
| Negative Percent Agreement (NPA) at 24 hrs | High agreement with traditional culture (implied) | 99.7% (95% CI 98.9 - 100%) | Table 1: HardyCHROM™ MRSA vs. Traditional Culture 24 hours (Page 5) |
| Positive Percent Agreement (PPA) at 48 hrs | High agreement with traditional culture (implied) | 97.0% (95% CI 89.6-98.5%) | Table 2: HardyCHROM™ MRSA vs. Traditional Culture 48 hours (Page 5) |
| Negative Percent Agreement (NPA) at 48 hrs | High agreement with traditional culture (implied) | 99.7% (95% CI 98.9 - 100%) | Table 2: HardyCHROM™ MRSA vs. Traditional Culture 48 hours (Page 5) |
| Clinical Performance (vs. Commercial Chromogenic MRSA Medium) | |||
| Positive Percent Agreement (PPA) at 24 hrs | High agreement with commercial medium (implied) | 98.3% (95% CI 97.5 – 99.3%) | Table 3: HardyCHROM™ MRSA vs. Commercial Chromogenic MRSA Medium 24 hours (Page 5) |
| Negative Percent Agreement (NPA) at 24 hrs | High agreement with commercial medium (implied) | 97.5% (95% CI 98.9 – 100%) | Table 3: HardyCHROM™ MRSA vs. Commercial Chromogenic MRSA Medium 24 hours (Page 5) |
| Positive Percent Agreement (PPA) at 48 hrs | High agreement with commercial medium (implied) | 98.4% (95% CI 97.5 - 99.3%) | Table 4: HardyCHROM™ MRSA vs. Commercial Chromogenic MRSA Medium 48 hours (Page 6) |
| Negative Percent Agreement (NPA) at 48 hrs | High agreement with commercial medium (implied) | 96.9% (95% CI 89.6-98.4%) | Table 4: HardyCHROM™ MRSA vs. Commercial Chromogenic MRSA Medium 48 hours (Page 6) |
| Reproducibility Testing (Clinical Lab Setting) | |||
| Inter-lot Reproducibility | 100% (explicitly stated) | 100% | Reproducibility Testing in Clinical Laboratory Setting (Page 8-9) |
| Overall Testing Intervals Reproducibility | 100% (explicitly stated) | 100% | Reproducibility Testing in Clinical Laboratory Setting (Page 8-9) |
| Sensitivity of MRSA strains | 100% (implied from study results) | 100% for MRSA strains tested | Reproducibility Testing in Clinical Laboratory Setting (Page 9) |
| Specificity of MSSA strains | 100% (implied from study results, "no growth") | 100% for MSSA strains (no growth) | Reproducibility Testing in Clinical Laboratory Setting (Page 9) |
| Agreement with PBP2', Cefoxitin, Oxacillin | Highly correlated with standard methods (implied high agreement) | PPA 24hr: 93.3%, PPA 48hr: 97.0% | |
| NPA 24hr: 99.7%, NPA 48hr: 99.7% | Tables 6-11 (Pages 7-8) |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Clinical Test Set: 443 samples were evaluated in the clinical study. These were surveillance specimens of anterior nasal swab samples.
- Data Provenance: The clinical study data was prospective and collected from three geographically diverse hospitals and clinics in the USA.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The traditional culture method served as the primary reference standard for establishing ground truth, using standard laboratory procedures.
- The "experts" in this context are the trained laboratory personnel at the three clinical study sites who performed and interpreted:
- Traditional culture (Trypticase Soy Agar with 5% Sheep Blood).
- StaphTEX latex agglutination test for suspected S. aureus colonies.
- PBP2' test for penicillin-binding protein 2a.
- Antibiotic disk diffusion methods (oxacillin 1µg, cefoxitin 30µg) interpreted according to CLSI guidelines (M100-S20).
- The document does not specify the number of individuals who performed these interpretations at each site, nor their specific qualifications (e.g., "clinical microbiologist with X years of experience"). It implies standard laboratory practice and qualified personnel.
4. Adjudication Method for the Test Set
The document details a form of discrepant analysis for the clinical study:
- Discrepant results between HardyCHROM™ MRSA and traditional culture (e.g., MRSA detected by one method but not the other) were confirmed as MRSA positive by cefoxitin disk diffusion. This acts as an adjudication step to determine the true MRSA status for discordant cases.
- Similarly, for comparisons with the commercial chromogenic MRSA medium, discrepant results were also confirmed as MRSA positive by cefoxitin disk diffusion.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. This device is a culture medium, not an AI or imaging diagnostic tool that would typically involve human readers interpreting results with or without AI assistance. The performance is assessed on the ability of the medium to directly grow and differentiate MRSA.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
The concept of "standalone performance" as typically applied to AI algorithms (algorithm only without human-in-the-loop) does not directly apply to this device. HardyCHROM™ MRSA is a diagnostic culture medium that requires laboratory processing and visual interpretation of colony color. The performance metrics presented (percent agreement, reproducibility, sensitivity, specificity) reflect the "standalone" diagnostic capability of the medium when correctly processed and interpreted by a trained professional (implying standard lab practice). It's not an algorithm that operates independently without human intervention or interpretation of its output.
7. The Type of Ground Truth Used (Expert Consensus, Pathology, Outcomes Data, etc.)
The primary ground truth for the clinical study was established by traditional culture methods combined with confirmatory phenotypic tests:
- Traditional culture on Trypticase Soy Agar with 5% Sheep Blood.
- Confirmation of S. aureus with StaphTEX latex agglutination.
- Confirmation of methicillin resistance (MRSA) using:
- PBP2' testing (for penicillin-binding protein 2a).
- Antibiotic disk diffusion (oxacillin 1µg and cefoxitin 30µg), interpreted according to CLSI guidelines.
- For discrepant results, cefoxitin disk diffusion was used as the confirmatory standard.
This is a form of expert-validated phenotypic characterization.
8. The Sample Size for the Training Set
The document does not describe a clear "training set" in the context of an AI algorithm. Instead, it details pre-clinical laboratory studies that would serve a similar purpose of developing and optimizing the medium's performance before large-scale clinical trials. These studies involved:
- Interference studies: ATCC® control strains of MRSA (ATCC® 33591, ATCC® 43300) and ten clinical well-characterized clinical MRSA isolates. The mucin study also included ten well-characterized clinical MSSA isolates.
- Cross-reactivity study: 121 strains of common nasal flora, including MRSA, MSSA, and 20 different species of coagulase-negative staphylococci.
- Recovery Study (LoD): Staphylococcus aureus ATCC® 43300 and Staphylococcus aureus ATCC® 33591.
- Mixed Infection Study: S. aureus MSSA ATCC® 25923 and E. coli ATCC® 25922 mixed with MRSA strains (ATCC® 33591 and 43300).
- Reproducibility Study with Characterized MRSA & MSSA strains: Twenty well-characterized test strains (fifteen MRSA strains including various PFGE types, and five MSSA strains).
Therefore, there isn't a single "training set" sample size, but rather multiple smaller sets of characterized bacterial strains used in various pre-clinical evaluations.
9. How the Ground Truth for the Training Set Was Established
The ground truth for the strains used in the pre-clinical (development/training-like) studies was established through:
- ATCC® Control Strains: These are standardized, well-characterized strains with known properties.
- Clinical Well-Characterized Isolates: These were obtained from private culture collections and were characterized by:
- Coagulase positive testing.
- PBP2' testing (for Methicillin Resistance).
- Cefoxitin (30 ug) testing (for Methicillin Resistance).
- For the reproducibility study, some MRSA strains had their DNA restriction patterns determined by pulsed-field gel electrophoresis (PFGE).
This indicates a robust, multi-faceted approach to characterizing the strains used in the early stages of device development and validation.
§ 866.1700 Culture medium for antimicrobial susceptibility tests.
(a)
Identification. A culture medium for antimicrobial susceptibility tests is a device intended for medical purposes that consists of any medium capable of supporting the growth of many of the bacterial pathogens that are subject to antimicrobial susceptibility tests. The medium should be free of components known to be antagonistic to the common agents for which susceptibility tests are performed in the treatment of disease.(b)
Classification. Class II (performance standards).