(90 days)
The Sensititre 20 - 24-hour Haemophilus influenzae/Streptococcus pneumoniae MIC or Breakpoint Susceptibility System is an in vitro diagnositic product for clinical susceptibility testing of fastidious isolates. This 510(k) is for Ceftolozane-tazobactam in the dilution range of 0.03/4 - 64/4 µg/ml for testing H. influenzae on the Sensitite 20 - 24-hour Haemophilus influenzae/Streptococcus pneumoniae MIC or Breakpoint Susceptibility System. Ceftolozane-tazobactam has been shown to be active both clinically and in vitro against the following organisms according to the FDA drug label: Haemophilus influenzae
Not Found
This FDA 510(k) clearance letter pertains to an in vitro diagnostic (IVD) device, specifically an antimicrobial susceptibility testing system. The questions you've posed (acceptance criteria, study design, ground truth, MRMC, etc.) are highly relevant to the evaluation of AI/ML-driven medical devices, particularly those that interpret images or other complex data to provide a diagnosis or assist in clinical decision-making.
However, the provided text does not contain the detailed study information required to answer your questions. This document is a clearance letter, stating that the Sensititre 20-24 hour Haemophilus influenzae/Streptococcus pneumoniae MIC or Breakpoint Susceptibility System with Ceftolozane-tazobactam has been found substantially equivalent to a predicate device.
Here's why the information is not present in this document:
- Type of Device: This is a microbiology susceptibility testing system, not an AI/ML image interpretation device. While it's an IVD, its function is to determine the minimum inhibitory concentration (MIC) or breakpoint susceptibility of an antibiotic against specific bacteria. This is typically done through a standardized laboratory process (e.g., microdilution) where the growth or inhibition of bacteria is observed, not through complex image analysis or AI algorithms in the way you're inferring.
- 510(k) Process: For traditional IVD devices like this, substantial equivalence is often demonstrated by showing the new device performs comparably to a legally marketed predicate device using established laboratory methods. The detailed performance data, including acceptance criteria and study results, would be submitted in the 510(k) application itself, but not typically summarized in the public clearance letter.
- Lack of AI/ML Component: There is no mention of AI or machine learning in the device description. Therefore, concepts like "human readers," "AI assistance," "effect size," "standalone algorithm," "training set," or "adjudication methods" for image interpretation are not applicable to the Sensititre system as described here.
To answer your questions accurately, one would need access to the actual 510(k) submission (K232310) which is often not publicly available in its entirety, or more specifically, the performance data section of that submission.
For illustrative purposes, if this were an AI/ML imaging device, a typical response addressing your questions would look like this (but again, the provided text does not support this):
Hypothetical Example (if this were an AI/ML imaging device and the information was available):
(Disclaimer: The following is entirely hypothetical and fabricated to demonstrate how your questions would be answered if the provided document were about an AI/ML medical device and contained the necessary study details. This information is NOT present in the provided FDA 510(k) clearance letter.)
The device is an AI-powered system designed to assist radiologists in detecting subtle findings on chest X-rays.
1. Table of Acceptance Criteria and Reported Device Performance
| Metric | Acceptance Criteria | Reported Device Performance (%) (95% CI) |
|---|---|---|
| Sensitivity (Standalone) | ≥ 90% for detecting actionable findings | 92.5% (91.1-93.8) |
| Specificity (Standalone) | ≥ 80% for non-actionable findings | 83.2% (82.0-84.3) |
| F-score (Standalone) | ≥ 0.85 | 0.88 |
| Reader Sensitivity (AI-assisted) | ≥ 5% improvement over unaided reading | 7.2% improvement (Statistically significant) |
| Reader False Positives (AI-assisted) | No significant increase over unaided reading | -1.5% change (Not statistically significant) |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: 5,000 anonymized chest X-ray images (DICOM format).
- Data Provenance: Retrospective study. Data collected from 10 different hospitals across the United States (70%), Canada (15%), and the UK (15%). Images covered a range of scanner types and patient demographics.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
- Number of Experts: 3 independent expert radiologists.
- Qualifications: All were board-certified radiologists with subspecialty training in thoracic imaging and a minimum of 10 years of post-fellowship clinical experience reading chest X-rays.
4. Adjudication Method for the Test Set
- Adjudication Method: 2+1. Two independent expert radiologists initially reviewed each case. If their initial assessments disagreed, a third, senior expert radiologist reviewed the case to reach a consensus for the final ground truth label.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size of Human Readers Improving with AI vs. Without AI Assistance
- MRMC Study: Yes, an MRMC study was conducted.
- Effect Size:
- Sensitivity: Human readers demonstrated a statistically significant increase in sensitivity of 7.2 percentage points (e.g., from 78.0% unaided to 85.2% with AI assistance) for detecting actionable findings.
- Specificity: No statistically significant change in specificity was observed, indicating the AI assistance did not lead to a significant increase in false positives.
- Reading Time: Average reading time per case decreased by 15% with AI assistance.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Standalone Performance: Yes, standalone performance was evaluated against the ground truth.
- Sensitivity: 92.5%
- Specificity: 83.2%
- AUC: 0.94 (Receiver Operating Characteristic Area Under the Curve)
7. The Type of Ground Truth Used
- Ground Truth Type: Expert consensus of 3 board-certified thoracic radiologists, further supported by correlation with available clinical outcomes data (e.g., follow-up imaging, pathology reports, surgical findings) where applicable and available in the retrospective dataset.
8. The Sample Size for the Training Set
- Training Set Sample Size: 50,000 anonymized chest X-ray images.
9. How the Ground Truth for the Training Set Was Established
- Ground Truth for Training:
- Initial labeling was performed by a team of trained clinical annotators.
- A subset of 10% of the training data was reviewed and verified by a single board-certified radiologist to ensure labeling consistency and accuracy.
- For particularly challenging or ambiguous cases during annotation, an internal consensus review by two radiologists was performed.
- A semi-supervised learning approach was employed where initial labels were used for training, and the model's performance on a validation set guided further expert review and refinement of labels for challenging cases.
§ 866.1640 Antimicrobial susceptibility test powder.
(a)
Identification. An antimicrobial susceptibility test powder is a device that consists of an antimicrobial drug powder packaged in vials in specified amounts and intended for use in clinical laboratories for determining in vitro susceptibility of bacterial pathogens to these therapeutic agents. Test results are used to determine the antimicrobial agent of choice in the treatment of bacterial diseases.(b)
Classification. Class II (performance standards).