Search Results

Lung AI software device is a Computer-Aided Detection (CADe) tool designed to assist in the detection of consolidation/atelectasis and pleural effusion during the review of lung ultrasound scans.

The software is an adjunctive tool to alert users to the presence of regions of interest (ROI) with consolidation/atelectasis and pleural effusion within the analyzed lung ultrasound cine clip.

Lung AI is intended to be used on images collected from the PLAPS point, in accordance with the BLUE protocol.

The intended users are healthcare professionals who are trained and qualified in performing lung ultrasound and routinely perform lung ultrasounds as part of their current practice in a point-of-care environment—namely Emergency Departments (EDs). The device was not designed and tested with use environments representing EMTs and military medics.

Lung AI is not intended for clinical diagnosis and does not replace the healthcare provider's judgment or other diagnostic tests in the standard care for lung ultrasound findings. All cases where a Chest CT scan and/or Chest X-ray is part of the standard of care should undergo these imaging procedures, irrespective of the device output.

The software is indicated for adults only.

Lung AI is a Computer-Aided Detection (CADe) tool designed to assist in the analysis of lung ultrasound images by suggesting the presence of consolidation/atelectasis and pleural effusion in a scan. This adjunctive tool is intended to aid users to detect the presence of regions of interest (ROI) with consolidation/atelectasis and pleural effusion. However, the device does not provide a diagnosis for any disease nor replace any diagnostic testing in the standard of care.

The lung AI module processes Ultrasound cine clips and flags any evidence of pleural effusion and/or consolidation/atelectasis present without aggregating data across regions or making any patient-level decisions. For positive cases, a single ROI per clip from a frame with the largest pleural effusion (or consolidation/atelectasis) is generated as part of the device output. Moreover, the ROI output is for visualization only and should not be relied on for precise anatomical localization. The final decision regarding the overall assessment of the information from all regions/clips remains the responsibility of the user. Lung AI is intended to be used on clips collected only from the PLAPS point, in accordance with the BLUE protocol.

Lung AI is developed as a module to be integrated by another computer programmer into their legally marketed ultrasound imaging device. The software integrates with third-party ultrasound imaging devices and functions as a post-processing tool. The software does not include a built-in viewer; instead, it works within the existing third-party device interface.

Lung AI is validated to meet applicable safety and efficacy requirements and to be generalizable to image data sourced from ultrasound transducers of a specific frequency range.

The device is intended to be used on images of adult patients undergoing point-of-care (POC) lung ultrasound scans in the emergency departments due to suspicion of pleural effusion and/or consolidation/atelectasis. It is important to note that patient management decisions should not be made solely on the results of the Lung AI analysis.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided FDA 510(k) clearance letter for Lung AI (LAI001).

Acceptance Criteria and Device Performance for Lung AI (LAI001)

The Lung AI (LAI001) device underwent both standalone performance evaluation and a multi-reader, multi-case (MRMC) study to demonstrate its safety and effectiveness.

1. Table of Acceptance Criteria and Reported Device Performance

The document specifies performance metrics based on the standalone evaluation (sensitivity and specificity for detection and localization) and the MRMC study (AUC, sensitivity, and specificity for human reader performance with and without AI assistance). The acceptance criteria for the MRMC study are explicitly stated as an improvement of at least 2% in overall reader performance (AUC-ROC).

Standalone Performance Metrics (Derived from "Summary of Lung AI performance" and "Summary of Lung AI localization performance")

Note: Specific numerical acceptance criteria for standalone performance are not explicitly stated in the document, but the reported values demonstrated meeting the required performance for FDA clearance.

MRMC Study Acceptance Criteria and Reported Device Performance

2. Sample Size and Data Provenance for Test Set

• Sample Size for Standalone Test Set: 465 lung scans from 359 unique patients.

• Data Provenance: Retrospectively collected from 6 imaging centers in the U.S. and Canada, with more than 50% of the data coming from U.S. centers. The dataset was enriched with abnormal cases (at least 30% abnormal per center) and included diverse demographic variables (gender, age 21-96, ethnicity).

• Sample Size for MRMC Test Set: 322 unique patients (cases). Each of the 6 readers analyzed 748 cases per reading period, for a total of 4488 cases overall.

3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set)

• Number of Experts: Two US board-certified experts initially, with a third expert for adjudication.
• Qualifications of Experts: Experienced in point-of-care ultrasound, reading lung ultrasound scans, and diagnostic radiology.

4. Adjudication Method for Test Set

• Method: In cases of disagreement between the first two experts, a third expert provided adjudication. This is a "2+1" (primary readers + adjudicator) method.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was it done?: Yes, an MRMC study was conducted.
• Effect Size of Improvement:
  • Pleural Effusion:
    • AUC improved by 0.035 (ΔAUC-PLEFF = 0.035) when aided by the device.
    • Sensitivity improved by 0.18 (ΔSe-PLEFF = 0.18) when aided by the device.
    • Specificity slightly decreased by -0.03 when aided by the device.
  • Consolidation/Atelectasis:
    • AUC improved by 0.028 (ΔAUC-CONS = 0.028) when aided by the device.
    • Sensitivity improved by 0.16 (ΔSp-CONS = 0.16) when aided by the device.
    • Specificity slightly decreased by -0.008 when aided by the device.

6. Standalone (Algorithm Only) Performance Study

• Was it done?: Yes, the "Bench Testing" section describes a standalone performance evaluation.

7. Type of Ground Truth Used

• Type of Ground Truth: Expert consensus (established by two US board-certified experts with a third adjudicator) for the presence/absence of consolidation/atelectasis and pleural effusion per cine clip. They also provided bounding box annotations for localization ground truth.

8. Sample Size for Training Set

• Sample Size: 3,453 ultrasound cine clips from 1,036 patients.

9. How Ground Truth for Training Set Was Established

• The document states that the underlying deep learning models were "trained on a diverse dataset of 3,453 ultrasound cine clips from 1,036 patients." While it doesn't explicitly detail the process for establishing ground truth for the training set, it can be inferred that a similar expert review process, likely involving radiologists or expert sonographers, was used, as is standard practice for supervised deep learning in medical imaging. The clinical confounders mentioned (Pneumonia, Pulmonary Embolism, CHF, Tamponade, Covid19, ARDS, COPD) suggest a robust labeling process to differentiate findings.

Ask a specific question about this device