Search Results
Found 1 results
510(k) Data Aggregation
(70 days)
Annalise Enterprise CXR Triage Pneumothorax
The device is designed to aid the clinical assessment of adult chest x-ray cases with features suggestive of pneumothorax and tension pneumothorax in the medical care environment. The device analyses cases using an artificial intelligence algorithm to identify findings. It makes case-level output available to a PACS for worklist prioritization or triage. The device is intended to be used by trained clinicians who are qualified to interpret chest X-rays as part of their scope of practice. The device is not intended to direct attention to specific portions of an image or to anomalies other than pneumothorax and tension pneumothorax. Its results are not intended to be used on a standalone basis for clinical decision making nor it is intended to rule out pneumothorax or tension pneumothorax, or otherwise preclude clinical assessment of X-ray cases.
Annalise Enterprise CXR Triage Pneumothorax is a software workflow tool that interfaces with RIS and PACS to obtain chest x-ray images to process. The artificial intelligence algorithm within the device uses deep learning techniques to identify the presence of pneumothorax and tension pneumothorax. The AI algorithm used in the device is a convolutional network trained using deep learning techniques. The AI results output from the device are sent to the reporting worklist software to enable AI assisted triage of the reporting worklist. The exact functionality available depends on the worklist software being used (RIS, PACS). This triage functionality uses the findings detected in each study by the AI model to provide information into the worklist software enabling the prioritization of the reporting worklist. Each organization can specify which findings will result in triage and the priority of each finding. It is important to note that the device will never decrease a study's existing priority in the worklist. This ensures that worklist items will never have their priorities downgraded by the AI software.
Acceptance Criteria and Device Performance Study
The Annalise Enterprise CXR Triage Pneumothorax device was evaluated through two core studies: a Standalone Performance Assessment (accuracy) and a Triage Effectiveness Assessment (turn-around time).
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are derived from the requirements set forth in Special Controls per 21CFR892.2080 for product code QFM, which include an AUC > 0.95 and sensitivity and specificity > 80%. The reported device performance is based on the "default balanced sensitivity and specificity" operating point.
| Metric | Acceptance Criteria (Product Code QFM) | Reported Device Performance (Pneumothorax) | Reported Device Performance (Tension Pneumothorax) |
|---|---|---|---|
| AUC | > 0.95 | 0.979 (95% CI: 0.970-0.986) | 0.988 (95% CI: 0.981-0.993) |
| Sensitivity | > 80% | 93.9% (95% CI: 91.8-96.1) | 94.3% (95% CI: 90.2, 98.4) |
| Specificity | > 80% | 92.2% (95% CI: 89.9-94.4) | 95.8% (95% CI: 94.3, 97.1) |
No specific acceptance criteria for triage effectiveness (turn-around time) were explicitly stated, but the reported performance was deemed "substantially equivalent to the total performance time published for the predicate device."
2. Sample Size for Test Set and Data Provenance
- Sample Size: 949 CXR cases from unique adult patients.
- Positive pneumothorax: n=413 (including tension pneumothorax n=123 as a subset)
- Negative pneumothorax: n=536
- Data Provenance: The test dataset was acquired retrospectively and consisted of eligible chest x-ray cases collected consecutively from 4 US hospital network sites. No data had previously been collected from these sites for training or testing of the device's AI algorithm, ensuring independence from the training dataset.
3. Number of Experts and Qualifications for Ground Truth
- Number of Experts: At least two American Board of Radiology (ABR)-certified radiologists for initial annotation, with a third radiologist for disagreement resolution.
- Qualifications of Experts: All radiologists were ABR-certified and protocol-trained.
4. Adjudication Method for Test Set
The adjudication method used was a 2+1 consensus method. Each deidentified CXR case was annotated in a blinded fashion by at least two ABR-certified and protocol-trained radiologists. In the event of disagreement between the first two radiologists, a third ground truther determined the consensus.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
The provided text does not mention a Multi-Reader Multi-Case (MRMC) comparative effectiveness study being conducted to assess the effect size of how much human readers improve with AI vs. without AI assistance. The study focuses on the standalone performance of the AI device and its triage effectiveness.
6. Standalone (Algorithm Only) Performance
Yes, a standalone (algorithm only without human-in-the-loop performance) study was done. The "Standalone Performance Assessment (accuracy)" directly evaluates the device's ability to identify pneumothorax and tension pneumothorax without human intervention. The results for AUC, sensitivity, and specificity presented above are from this standalone assessment.
7. Type of Ground Truth Used for Test Set
The ground truth used for the test set was expert consensus. This was established by at least two (and potentially three in case of disagreement) ABR-certified radiologists reviewing and annotating the deidentified CXR cases.
8. Sample Size for Training Set
The training dataset included over 1,500,000 chest x-ray images.
9. How Ground Truth for Training Set Was Established
Cases in the training dataset were labeled by at least three qualified radiologists for the presence or absence of radiographic features suggestive of pneumothorax or tension pneumothorax.
Ask a specific question about this device
Page 1 of 1