(81 days)
EchoGo Core is intended to be used for quantification and reporting of results of cardiovascular function to support physician diagnosis. EchoGo Core is indicated for use in adult populations.
EchoGo Core 2.0 is a software application manufactured by Ultromics to provide a report of left ventricular cardiac function, in the form of secondary capture DICOM files and/or as a structured DICOM report, to aid interpreting physicians with diagnostic decision-making process. EchoGo Core 2.0 applies to ultrasound images of the heart (echocardiograms).
EchoGo Core 2.0 utilizes artificial intelligence (AI) for the operator-assisted automatic quantification of commonly measured echocardiographic metrics. Independent training, test and validation datasets were used for training and performance assessment of the device.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria were formalized in terms of Root Mean Square (RMS) error against reference values generated using the comparator device, TomTec Arena TTA2. The text states that "the acceptance criteria were formalized such that EchoGo Core 2.0 would produce measures of left ventricular (LV) length, volume at end diastole (ED), and end systole (ES), ejection fraction (EF), stroke volume, cardiac output, global longitudinal strain (GLS) and segmental longitudinal strain (SLS) with an RMS error below a set, pre-determined threshold." While the exact pre-determined thresholds for acceptance are not explicitly listed in numerical values, the "Performance against the comparator device is summarised as follows," implying these are the reported performance values that met the (unspecified) acceptance thresholds.
| Left Ventricular Metric | Root Mean Square Error (% RMS) |
|---|---|
| Length | 3.06 - 4.59 |
| Volume at End Diastole and End Systole | 8.57 - 16.59 |
| Ejection Fraction | 6.69 - 8.50 |
| Stroke Volume | 10.57 - 13.68 |
| Global Longitudinal Strain | 3.36 - 4.79 |
| Systolic Segmental Longitudinal Strain | 5.51 - 9.98 |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: 214 previously unseen studies.
- Data Provenance:
- Country of Origin: Not explicitly stated, but the submission is from Ultromics Limited in the United Kingdom.
- Retrospective or Prospective: Retrospective. The study was described as a "formal retrospective, non-interventional validation study."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The ground truth for the test set was established using a comparator device, TomTec Arena TTA2 (K150122), not human experts providing a direct ground truth. The acceptance criteria were formalized in terms of Root Mean Square (RMS) error against reference values generated using the comparator device. There is no mention of experts establishing ground truth for the test set; instead, the comparator device served as the reference.
4. Adjudication Method for the Test Set
Not applicable, as the ground truth was established by a comparator device rather than human interpretation requiring adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, and the Effect Size
No, an MRMC comparative effectiveness study where human readers improve with AI vs. without AI assistance was not conducted or reported. The study focused on the device's performance against a comparator device.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) was Done
The primary performance study described seems to align with a standalone assessment against a comparator device. The text states, "EchoGo Core 2.0 would produce measures... with an RMS error below a set, pre-determined threshold" and "Performance against the comparator device is summarised as follows." This suggests the algorithm's output was directly compared to the output of the TomTec Arena TTA2.
However, it's crucial to note the device description also states: "EchoGo Core 2.0 requires an operator at key steps to confirm or relabel automatically labeled acquisition views (if required) and approve the left ventricle segmentations (contours) proposed by the AI." And "The operator will review the report produced and may be asked to approve cautions that are added to the report." This indicates that while the core performance metrics were evaluated in an algorithm-centric manner against a comparator, the device's intended use involves a "human-in-the-loop" for confirmation/approval steps. The reported performance metrics (RMS error) are likely for the algorithm's core measurements before potential human override, as the study compared them to an automated comparator device.
7. The Type of Ground Truth Used
The ground truth was established by comparison to a legally marketed predicate device (TomTec Arena TTA2, K150122). Values generated by the TomTec Arena TTA2 served as the reference values for calculating RMS error.
8. The Sample Size for the Training Set
The sample size for the training set is not specified in the provided text. The text only mentions that "Independent training, test and validation datasets were used for training and performance assessment of the device" and that "Test datasets were strictly segregated from algorithm training datasets."
9. How the Ground Truth for the Training Set was Established
The method for establishing ground truth for the training set is not explicitly detailed in the provided text. It only states that "Independent training, test and validation datasets were used for training and performance assessment of the device."
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).