(92 days)
The AI Platform is intended for noninvasive processing of ultrasound images to detect, measure, and calculate relevant medical parameters of structures and function of adult patients with suspected disease. The device is intended to be used on images from adult patients.
Exo Al Platform is a software as a medical device (SaMD) that helps qualified users with image-based assessment of ultrasound examinations in adult patients. It is designed to simplify workflow by helping trained healthcare providers evaluate, quantify, and generate reports for ultrasound images. The device is intended to generate images and a report that can be reviewed in a typical standard of care setting.
Al Platform takes as an input imported Digital Imaging and Communications in Medicine (DICOM) images from ultrasound scanners of a specific range and allows users to detect, measure, and calculate relevant medical parameters of structures and function of patients with suspected disease. It provides users with a specific toolset for viewing ultrasound images of the lung and heart, placing landmarks, and creating reports.
Key features of the software are
- LVEF AI: an Al-assisted tool for quantification of ejection on cardiac ultrasound images.
- . Lung Al: an Al-assisted tool to suggest presence of lung structures and artifacts on ultrasound images.
Exo Al Platform does not perform any function that could not be accomplished by a trained user manually. It's important to note that patient management decisions should not be made solely on the results of the Al Platform analysis.
Acceptance Criteria & Device Performance Study for Exo AI Platform (AIP001)
The Exo AI Platform (AIP001) is a software as a medical device (SaMD) intended for noninvasive processing of ultrasound images to detect, measure, and calculate relevant medical parameters of structures and function in adult patients with suspected disease. This document outlines the acceptance criteria and the studies performed to demonstrate the device meets these criteria for both its cardiac (LVEF AI) and lung (Lung AI) functionalities.
1. Table of Acceptance Criteria and Reported Device Performance
For Cardiac Ultrasound (LVEF AI - Ejection Fraction Measurement):
| Acceptance Criteria (Performance Metric) | Target | Reported Device Performance (95% CI) |
|---|---|---|
| Ejection Fraction Parasternal Long-axis | ||
| - Intraclass Correlation Coefficient (ICC) | High | 0.93 (0.89 - 0.96) |
| - Root Mean Square Difference (RMSD) | Low | 6.12 (5.30 - 8.36) |
| Ejection Fraction Apical Biplane | ||
| - Intraclass Correlation Coefficient (ICC) | High | 0.95 (0.90 - 0.98) |
| - Root Mean Square Difference (RMSD) | Low | 4.81 (3.99 - 7.25) |
| Ejection Fraction Apical (AP4) Single Plane | ||
| - Intraclass Correlation Coefficient (ICC) | High | 0.92 (0.88 - 0.95) |
| - Root Mean Square Difference (RMSD) | Low | 6.06 (5.27 - 8.20) |
| Ejection Fraction Apical (AP2) Single Plane | ||
| - Intraclass Correlation Coefficient (ICC) | High | 0.92 (0.87 - 0.95) |
| - Root Mean Square Difference (RMSD) | Low | 6.25 (5.33 - 8.82) |
| Overall Ejection Fraction Measurement (All Views) | ||
| - Intraclass Correlation Coefficient (ICC) | High | 0.93 (0.91 - 0.95) |
| - Root Mean Square Difference (RMSD) | Low | 5.90 (5.35 - 7.23) |
For Lung Ultrasound (Lung AI - A-lines and B-lines detection/quantification):
| Acceptance Criteria (Performance Metric) | Target | Reported Device Performance |
|---|---|---|
| A-lines Presence (Agreement) | High | Kappa = 0.84 |
| B-lines Counts (Reliability) | High | ICC = 0.97 |
(Note: Specific quantitative targets for "High" ICC and "Low" RMSD/Kappa are not explicitly stated in the provided text, but the reported values demonstrate strong performance in common clinical contexts for these metrics.)
2. Sample Size Used for the Test Set and Data Provenance
- LVEF AI (Cardiac Function): 151 subjects
- Lung AI (Lung Function): 125 subjects
Data Provenance: The data was acquired during routine clinical practice from multiple clinical sites in metropolitan cities, ensuring diverse racial patient populations. The data encompassed diverse demographic variables, including gender, age (20-96 years), and BMI (15.3-52.8). The images were acquired from both cart-based and portable ultrasound devices. The test data was explicitly stated to be entirely separated from the training/validation datasets and not used for any part of the training. This suggests a retrospective collection of data designed for independent validation. The countries of origin are not specified, but "metropolitan cities with diverse racial patient populations" implies a multi-site, potentially multi-national, collection or at least a highly diverse domestic setting.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- LVEF AI (Ejection Fraction): The ground truth was obtained as the average ejection fraction measurement of three experts.
- Lung AI (A-line Presence): The ground truth was determined by consensus of two or more experts.
- Lung AI (B-line Counts): The ground truth was determined as the average of B-line counts from three experts.
Qualifications of Experts: The document does not explicitly state the specific qualifications of these experts (e.g., number of years of experience, specific board certifications like radiologist or cardiologist). However, the context of "routine clinical practice" and "experts" implies highly qualified medical professionals experienced in interpreting cardiac and lung ultrasound images.
4. Adjudication Method for the Test Set
- LVEF AI (Ejection Fraction): The adjudication method for the reference data (ground truth) was established by taking the average ejection fraction measurement of three experts. This implies a method akin to "average of multiple readers."
- Lung AI (A-line Presence): The adjudication method for the ground truth was determined by consensus of two or more experts. This suggests a qualitative agreement, where at least two experts had to concur.
- Lung AI (B-line Counts): The adjudication method for the ground truth was established by taking the average of B-line counts from three experts. (Similar to LVEF AI).
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done
No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done or reported in the provided text. The performance assessment focused on the standalone performance of the AI tool against expert-established ground truth, not on how human readers' performance improved with AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance study was done. The reported ICC, RMSD, and Kappa values directly assess the AI Platform's accuracy and reliability in generating measurements and detections independently, against the expert-derived ground truth. The statement that "Exo AI Platform does not perform any function that could not be accomplished by a trained user manually" also reinforces its role as an automated tool, evaluated on its own.
7. The Type of Ground Truth Used
The type of ground truth used was expert consensus / expert measurement.
- For Cardiac Ejection Fraction: Average measurements from three experts.
- For Lung A-line Presence: Consensus of two or more experts.
- For Lung B-line Counts: Average counts from three experts.
8. The Sample Size for the Training Set
The sample size for the training set is not specified in the provided text. The document only explicitly mentions that the test data was entirely separated from the training/validation datasets.
9. How the Ground Truth for the Training Set Was Established
The document does not specify how the ground truth for the training set was established. It only mentions that the AI algorithms (Deep Convolutional Neural Networks) were "trained with clinical data."
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).