Search Results

Lung AI software device is a Computer-Aided Detection (CADe) tool designed to assist in the detection of consolidation/atelectasis and pleural effusion during the review of lung ultrasound scans.

The software is an adjunctive tool to alert users to the presence of regions of interest (ROI) with consolidation/atelectasis and pleural effusion within the analyzed lung ultrasound cine clip.

Lung AI is intended to be used on images collected from the PLAPS point, in accordance with the BLUE protocol.

The intended users are healthcare professionals who are trained and qualified in performing lung ultrasound and routinely perform lung ultrasounds as part of their current practice in a point-of-care environment—namely Emergency Departments (EDs). The device was not designed and tested with use environments representing EMTs and military medics.

Lung AI is not intended for clinical diagnosis and does not replace the healthcare provider's judgment or other diagnostic tests in the standard care for lung ultrasound findings. All cases where a Chest CT scan and/or Chest X-ray is part of the standard of care should undergo these imaging procedures, irrespective of the device output.

The software is indicated for adults only.

Lung AI is a Computer-Aided Detection (CADe) tool designed to assist in the analysis of lung ultrasound images by suggesting the presence of consolidation/atelectasis and pleural effusion in a scan. This adjunctive tool is intended to aid users to detect the presence of regions of interest (ROI) with consolidation/atelectasis and pleural effusion. However, the device does not provide a diagnosis for any disease nor replace any diagnostic testing in the standard of care.

The lung AI module processes Ultrasound cine clips and flags any evidence of pleural effusion and/or consolidation/atelectasis present without aggregating data across regions or making any patient-level decisions. For positive cases, a single ROI per clip from a frame with the largest pleural effusion (or consolidation/atelectasis) is generated as part of the device output. Moreover, the ROI output is for visualization only and should not be relied on for precise anatomical localization. The final decision regarding the overall assessment of the information from all regions/clips remains the responsibility of the user. Lung AI is intended to be used on clips collected only from the PLAPS point, in accordance with the BLUE protocol.

Lung AI is developed as a module to be integrated by another computer programmer into their legally marketed ultrasound imaging device. The software integrates with third-party ultrasound imaging devices and functions as a post-processing tool. The software does not include a built-in viewer; instead, it works within the existing third-party device interface.

Lung AI is validated to meet applicable safety and efficacy requirements and to be generalizable to image data sourced from ultrasound transducers of a specific frequency range.

The device is intended to be used on images of adult patients undergoing point-of-care (POC) lung ultrasound scans in the emergency departments due to suspicion of pleural effusion and/or consolidation/atelectasis. It is important to note that patient management decisions should not be made solely on the results of the Lung AI analysis.

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided FDA 510(k) clearance letter for Lung AI (LAI001).

Acceptance Criteria and Device Performance for Lung AI (LAI001)

The Lung AI (LAI001) device underwent both standalone performance evaluation and a multi-reader, multi-case (MRMC) study to demonstrate its safety and effectiveness.

1. Table of Acceptance Criteria and Reported Device Performance

The document specifies performance metrics based on the standalone evaluation (sensitivity and specificity for detection and localization) and the MRMC study (AUC, sensitivity, and specificity for human reader performance with and without AI assistance). The acceptance criteria for the MRMC study are explicitly stated as an improvement of at least 2% in overall reader performance (AUC-ROC).

Standalone Performance Metrics (Derived from "Summary of Lung AI performance" and "Summary of Lung AI localization performance")

Note: Specific numerical acceptance criteria for standalone performance are not explicitly stated in the document, but the reported values demonstrated meeting the required performance for FDA clearance.

MRMC Study Acceptance Criteria and Reported Device Performance

2. Sample Size and Data Provenance for Test Set

• Sample Size for Standalone Test Set: 465 lung scans from 359 unique patients.

• Data Provenance: Retrospectively collected from 6 imaging centers in the U.S. and Canada, with more than 50% of the data coming from U.S. centers. The dataset was enriched with abnormal cases (at least 30% abnormal per center) and included diverse demographic variables (gender, age 21-96, ethnicity).

• Sample Size for MRMC Test Set: 322 unique patients (cases). Each of the 6 readers analyzed 748 cases per reading period, for a total of 4488 cases overall.

3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set)

• Number of Experts: Two US board-certified experts initially, with a third expert for adjudication.
• Qualifications of Experts: Experienced in point-of-care ultrasound, reading lung ultrasound scans, and diagnostic radiology.

4. Adjudication Method for Test Set

• Method: In cases of disagreement between the first two experts, a third expert provided adjudication. This is a "2+1" (primary readers + adjudicator) method.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was it done?: Yes, an MRMC study was conducted.
• Effect Size of Improvement:
  • Pleural Effusion:
    • AUC improved by 0.035 (ΔAUC-PLEFF = 0.035) when aided by the device.
    • Sensitivity improved by 0.18 (ΔSe-PLEFF = 0.18) when aided by the device.
    • Specificity slightly decreased by -0.03 when aided by the device.
  • Consolidation/Atelectasis:
    • AUC improved by 0.028 (ΔAUC-CONS = 0.028) when aided by the device.
    • Sensitivity improved by 0.16 (ΔSp-CONS = 0.16) when aided by the device.
    • Specificity slightly decreased by -0.008 when aided by the device.

6. Standalone (Algorithm Only) Performance Study

• Was it done?: Yes, the "Bench Testing" section describes a standalone performance evaluation.

7. Type of Ground Truth Used

• Type of Ground Truth: Expert consensus (established by two US board-certified experts with a third adjudicator) for the presence/absence of consolidation/atelectasis and pleural effusion per cine clip. They also provided bounding box annotations for localization ground truth.

8. Sample Size for Training Set

• Sample Size: 3,453 ultrasound cine clips from 1,036 patients.

9. How Ground Truth for Training Set Was Established

• The document states that the underlying deep learning models were "trained on a diverse dataset of 3,453 ultrasound cine clips from 1,036 patients." While it doesn't explicitly detail the process for establishing ground truth for the training set, it can be inferred that a similar expert review process, likely involving radiologists or expert sonographers, was used, as is standard practice for supervised deep learning in medical imaging. The clinical confounders mentioned (Pneumonia, Pulmonary Embolism, CHF, Tamponade, Covid19, ARDS, COPD) suggest a robust labeling process to differentiate findings.

Ask a specific question about this device

Strain AI is intended for noninvasive processing of cardiac ultrasound images to provide measurements of global longitudinal strain of adult patients with suspected disease.

Exo's Strain Al is a software as a medical device (SaMD), intended as an aid in diagnostic analysis of echocardiography data. It specifically measures the global longitudinal strain (GLS) from apical 4-chamber (A4C) cardiac ultrasound images.

This software is developed as a module to be integrated by another computer programmer into their legally marketed ultrasound imaging device.

The software does not have a built-in viewer; instead, it integrates into a third-party ultrasound imaging device. The software functions as a post-processing tool, analyzing images after they are acquired. End-users have the option to accept or reject the provided measurements.

Strain Al takes as input image data and provides as an output a quantitative measurement of the global longitudinal strain (GLS) from apical 4-chamber (A4C) cardiac ultrasound images. It is important to note that patient management decisions should not be made solely on the results of the Strain AI analysis.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

Note: The document states that the performance was "successfully evaluated" and "consistent among clinically meaningful subgroups," and the reported ICC and RMSD values contribute to this conclusion, suggesting they met internal acceptance criteria. Formal numerical thresholds for acceptance are not explicitly listed in this summary.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Test Set: Not explicitly stated as a single number. The document mentions "test data encompassing diverse demographic variables, including gender, age (ranging from 21 to 96), and ethnicity."
• Data Provenance:
  • Country of Origin: Not specified.
  • Retrospective or Prospective: Not explicitly stated, however, the phrase "images acquired during a routine clinical practice" could suggest retrospective use of existing clinical data or prospective collection within a routine clinical setting. It's not definitively clear from the text.
  • Specifics: Data was collected from "multiple clinical sites in metropolitan cities with diverse racial patient populations."

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

• Number of Experts: Not explicitly stated.
• Qualifications of Experts: Not explicitly stated.

The document indicates that "The ground truth (reference data) was obtained using the reference device." This implies that the ground truth was established by the output of the reference device (Us2.v2) rather than direct expert interpretation of the images.

4. Adjudication Method for the Test Set

• Adjudication Method: Not applicable, as the ground truth was derived from the output of a reference device (Us2.v2), not from multiple expert adjudications.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was an MRMC study done? No. The study focused on the standalone performance of the Strain AI device against a reference device, not on how human readers' performance might improve with AI assistance.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

• Was a standalone study done? Yes. The study evaluated the Strain AI's performance by comparing its output (GLS measurements) directly to the ground truth established by a reference device, without explicit human interaction or modification of the AI's results during the performance assessment. The device is described as "a software as a medical device (SaMD)" that "functions as a post-processing tool, analyzing images after they are acquired." While "End-users have the option to accept or reject the provided measurements," the performance evaluation itself appears to be a direct comparison of the AI's output.

7. Type of Ground Truth Used

• Type of Ground Truth: The ground truth (reference data) was established using the reference device, Us2.v2 (K233676). This indicates a "device-based" or "software-based" ground truth, where the output of another legally marketed and classified device serves as the standard for comparison.

8. Sample Size for the Training Set

• Sample Size for Training Set: Not explicitly stated. The document mentions "The test data was entirely separated from the training/validation datasets acquired from independent clinical sites."

9. How the Ground Truth for the Training Set Was Established

• How Ground Truth for Training Set Was Established: Not explicitly stated. The document only mentions that the AI algorithms are "trained with clinical data." It does not detail the specific method used to establish the ground truth for this training data.

Ask a specific question about this device

The AI Platform is intended for noninvasive processing of ultrasound images to detect, measure, and calculate relevant medical parameters of structures and function of adult patients with suspected disease. The device is intended to be used on images from adult patients.

Exo Al Platform is a software as a medical device (SaMD) that helps qualified users with image-based assessment of ultrasound examinations in adult patients. It is designed to simplify workflow by helping trained healthcare providers evaluate, quantify, and generate reports for ultrasound images. The device is intended to generate images and a report that can be reviewed in a typical standard of care setting.

Al Platform takes as an input imported Digital Imaging and Communications in Medicine (DICOM) images from ultrasound scanners of a specific range and allows users to detect, measure, and calculate relevant medical parameters of structures and function of patients with suspected disease. It provides users with a specific toolset for viewing ultrasound images of the lung and heart, placing landmarks, and creating reports.

Key features of the software are

• LVEF AI: an Al-assisted tool for quantification of ejection on cardiac ultrasound images.
• . Lung Al: an Al-assisted tool to suggest presence of lung structures and artifacts on ultrasound images.

Exo Al Platform does not perform any function that could not be accomplished by a trained user manually. It's important to note that patient management decisions should not be made solely on the results of the Al Platform analysis.

Acceptance Criteria & Device Performance Study for Exo AI Platform (AIP001)

The Exo AI Platform (AIP001) is a software as a medical device (SaMD) intended for noninvasive processing of ultrasound images to detect, measure, and calculate relevant medical parameters of structures and function in adult patients with suspected disease. This document outlines the acceptance criteria and the studies performed to demonstrate the device meets these criteria for both its cardiac (LVEF AI) and lung (Lung AI) functionalities.

1. Table of Acceptance Criteria and Reported Device Performance

For Cardiac Ultrasound (LVEF AI - Ejection Fraction Measurement):

For Lung Ultrasound (Lung AI - A-lines and B-lines detection/quantification):

(Note: Specific quantitative targets for "High" ICC and "Low" RMSD/Kappa are not explicitly stated in the provided text, but the reported values demonstrate strong performance in common clinical contexts for these metrics.)

2. Sample Size Used for the Test Set and Data Provenance

• LVEF AI (Cardiac Function): 151 subjects
• Lung AI (Lung Function): 125 subjects

Data Provenance: The data was acquired during routine clinical practice from multiple clinical sites in metropolitan cities, ensuring diverse racial patient populations. The data encompassed diverse demographic variables, including gender, age (20-96 years), and BMI (15.3-52.8). The images were acquired from both cart-based and portable ultrasound devices. The test data was explicitly stated to be entirely separated from the training/validation datasets and not used for any part of the training. This suggests a retrospective collection of data designed for independent validation. The countries of origin are not specified, but "metropolitan cities with diverse racial patient populations" implies a multi-site, potentially multi-national, collection or at least a highly diverse domestic setting.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• LVEF AI (Ejection Fraction): The ground truth was obtained as the average ejection fraction measurement of three experts.
• Lung AI (A-line Presence): The ground truth was determined by consensus of two or more experts.
• Lung AI (B-line Counts): The ground truth was determined as the average of B-line counts from three experts.

Qualifications of Experts: The document does not explicitly state the specific qualifications of these experts (e.g., number of years of experience, specific board certifications like radiologist or cardiologist). However, the context of "routine clinical practice" and "experts" implies highly qualified medical professionals experienced in interpreting cardiac and lung ultrasound images.

4. Adjudication Method for the Test Set

• LVEF AI (Ejection Fraction): The adjudication method for the reference data (ground truth) was established by taking the average ejection fraction measurement of three experts. This implies a method akin to "average of multiple readers."
• Lung AI (A-line Presence): The adjudication method for the ground truth was determined by consensus of two or more experts. This suggests a qualitative agreement, where at least two experts had to concur.
• Lung AI (B-line Counts): The adjudication method for the ground truth was established by taking the average of B-line counts from three experts. (Similar to LVEF AI).

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done or reported in the provided text. The performance assessment focused on the standalone performance of the AI tool against expert-established ground truth, not on how human readers' performance improved with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance study was done. The reported ICC, RMSD, and Kappa values directly assess the AI Platform's accuracy and reliability in generating measurements and detections independently, against the expert-derived ground truth. The statement that "Exo AI Platform does not perform any function that could not be accomplished by a trained user manually" also reinforces its role as an automated tool, evaluated on its own.

7. The Type of Ground Truth Used

The type of ground truth used was expert consensus / expert measurement.

• For Cardiac Ejection Fraction: Average measurements from three experts.
• For Lung A-line Presence: Consensus of two or more experts.
• For Lung B-line Counts: Average counts from three experts.

8. The Sample Size for the Training Set

The sample size for the training set is not specified in the provided text. The document only explicitly mentions that the test data was entirely separated from the training/validation datasets.

9. How the Ground Truth for the Training Set Was Established

The document does not specify how the ground truth for the training set was established. It only mentions that the AI algorithms (Deep Convolutional Neural Networks) were "trained with clinical data."

Ask a specific question about this device

Bladder Al uses machine-learning techniques to aid in the quantification of bladder volume from ultrasound images. The device is intended to be used on images of patients aged two years or older.

Bladder Al is a standalone software as a medical device (SaMD) that helps qualified users with image-based assessment of bladder ultrasound images in patients aged 2 or older. It is designed to simplify workflow by helping trained healthcare providers evaluate, quantify, and generate reports for bladder ultrasound images.

Bladder Al takes as an input imported Digital Imaging and Communications in Medicine (DICOM) images from ultrasound scanners and allows users to measure bladder volumes of a single frame and multi-frame ultrasound images, as well as create and finalize examination reports. It provides users with a specific toolset for viewing ultrasound images of the bladder, placing landmarks, and creating reports.

Key features of the software are

• ML-based semi-automatic landmark placements
• Bladder dimension and volume measurements
• Report generation

Here's a summary of the acceptance criteria and the study proving the device meets them, based on the provided text:

Bladder AI (AIBV01) Performance Study Summary

1. Acceptance Criteria and Reported Device Performance:

The document doesn't explicitly state "acceptance criteria" with numerical thresholds directly. However, it demonstrates performance by reporting accuracy and reliability metrics that would implicitly serve as the criteria for clearance. The study's conclusion that "the algorithm performance is reliable and accurate compared to expert clinician" and that "the results support the generalizability of the Bladder Al across the intended patient population" suggests these metrics met the internal or regulatory expectations.

2. Sample Size and Data Provenance for the Test Set:

• Sample Size: 122 subjects.
• Data Provenance: A diverse collection of clinical sites in metropolitan cities, chosen to provide a broad range of demographic variables (ethnicity, gender, age 2 to 95 years old). The document does not specify the country of origin, but "metropolitan cities" implies a broad geographic reach within a developed context. The test data was retrospective as it consists of "images acquired from cart-based and portable ultrasound devices."

3. Number of Experts and Qualifications for Ground Truth Establishment (Test Set):

• Number of Experts: Three expert clinicians.
• Qualifications: The document states they were "expert clinicians." Specific experience levels (e.g., "radiologist with 10 years of experience") are not provided.

4. Adjudication Method for the Test Set:

• Adjudication Method: The ground truth for bladder volume was obtained as the average bladder volume measurement among three expert clinicians. This suggests a form of consensus or averaging method, rather than a 2+1 or 3+1 rule for disagreement.

5. MRMC Comparative Effectiveness Study:

• Was an MRMC study done? No. The study assessed the standalone performance of the Bladder AI compared to expert consensus. There is no mention of human readers using the AI for assistance and comparing their performance with and without AI.
• Effect Size of Human Readers Improvement with AI vs. without AI assistance: Not applicable, as no MRMC study was conducted.

6. Standalone Performance:

• Was standalone (algorithm only without human-in-the-loop performance) done? Yes. The performance metrics (Mean volume difference, ICC) are for the Bladder AI's measurements compared directly to the expert-established ground truth value.

7. Type of Ground Truth Used:

• Type of Ground Truth: Expert consensus. Specifically, the "average bladder volume measurement among three expert clinicians."

8. Sample Size for the Training Set:

• The document implies a training set was used, stating "Training and validation datasets have been selected and maintained to be appropriately independent of one another." However, the specific sample size for the training set is not provided in the excerpt.

9. How the Ground Truth for the Training Set was Established:

• The document states that the "Training and validation datasets have been selected and maintained to be appropriately independent of one another." Similar to the training set sample size, the method for establishing ground truth for the training set is not explicitly detailed in the provided text. It is reasonable to infer it would involve expert review, but the specifics are not elucidated.

Ask a specific question about this device