(184 days)
The Bay Labs, Inc. EchoMD Automated Ejection Fraction software is used to process previously acquired transthoracic cardiac ultrasound images, to store images, and to manipulate and make measurements on images using a personal computer or a compatible DICOM-compliant PACS system in order to provide automated estimation of left ventricular ejection. This measurement can be used to assist the clinician in a cardiac evaluation.
The EchoMD Automated Ejection Fraction Software is indicated for use in adult patients.
The EchoMD Automated Ejection (AutoEF) software applies machine learning algorithms to process echocardiography images in order to calculate left ventricular ejection fraction. The software operates in between the DICOM source and the DICOM destination. EchoMD AutoEF performs left ventricular ejection measurements using both the apical four chamber and apical two chamber cardiac ultrasound views.
The software selects the image clips to be used, performs the AutoEF calculation, and forwards the results to a destination PACS server for clinician viewing. The output of the Ejection Fraction estimate stated as a percentage, which is displayed via the destination PACS system. The software applies machine learning algorithms to assess image quality and provides this information as qualitative and quantitative user feedback. By automating the estimation of ejection fraction, the EchoMD software is designed to streamline the clinician's calculation of the measurement.
Here's a breakdown of the acceptance criteria and study details for the Bay Labs, Inc. EchoMD Automated Ejection Fraction Software, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Root Mean Square Deviation (RMSD) below a set threshold as compared to the reference ground truth EF. | 8.290% RMSD |
1. Sample Size and Data Provenance
- Test Set Sample Size: Over 300 previously-acquired studies.
- Data Provenance: Retrospective, non-interventional validation study. The country of origin is not explicitly stated, but given the FDA submission, it's highly likely to be within the US or compliant with US standards.
2. Number of Experts and Qualifications
- Number of Experts: Not explicitly stated for the primary validation study's ground truth establishment. However, an additional study was performed "with a different set of cardiologists" on a subset of the validation patient studies. The specific number of cardiologists for this additional study is also not provided.
- Qualifications of Experts: The experts involved in the "additional study" were "cardiologists." No further details on their experience (e.g., years of experience) are given.
3. Adjudication Method for the Test Set
- The text does not describe an explicit adjudication method (e.g., 2+1, 3+1) for establishing the ground truth EF for the test set. The ground truth was established by "the biplane method of disks ejection fraction reported." It implies that the reported EF values were considered the reference.
4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Was an MRMC study done? The text states, "An additional study was performed with a different set of cardiologists on a subset of the validation patient studies to further demonstrate the generalizability of the software." This suggests a form of multi-reader study, but it's not explicitly framed as an MRMC comparative effectiveness study in the sense of comparing human readers with AI vs. without AI assistance. The focus seems to be on verifying the generalizability of the software's performance against different cardiologists.
- Effect Size: No effect size or improvement for human readers with AI assistance versus without AI assistance is reported. The study's purpose was to demonstrate the generalizability of the software's performance, not to quantify human reader improvement.
5. Standalone (Algorithm Only) Performance
- Was a standalone performance study done? Yes. The primary endpoint measured the EchoMD AutoEF ejection fraction measurements compared to the biplane method ejection fraction, with a root mean square deviation calculated. This directly assesses the algorithm's performance independent of a human-in-the-loop scenario.
- Performance Metric: Root Mean Square Deviation (RMSD).
- Performance Result: 8.290% RMSD (p-value 0.00052).
6. Type of Ground Truth Used
- Ground Truth: "the biplane method of disks ejection fraction reported." This is a clinical standard for Left Ventricular Ejection Fraction (LVEF) measurement based on echocardiography. While it relies on expert interpretation and measurement, it is not explicitly described as "expert consensus" in the sense of multiple independent readers reaching a consensus. It's more aligned with a established clinical measurement method.
7. Sample Size for the Training Set
- The sample size for the training set is not provided in the document. The text only states that "Test datasets were strictly segregated from algorithm training datasets."
8. How Ground Truth for Training Set was Established
- The method for establishing the ground truth for the training set is not specified. The document only mentions that the software "applies machine learning algorithms to process echocardiography images."
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).