(144 days)
LVivo platform is intended for non-invasive processing of ultrasound images to detect, measure, and calculate relevant medical parameters of structures and function of patients with suspected disease. In addition, it has the ability to provide Quality Score feedback.
The LVivo platform is a software system for automated analysis of ultrasound examinations. Automated analysis of echocardiographic examinations is done using DICOM movies. The LVivo platform supports global and segmental evaluation of the left ventricle (LV) of the heart. The global LV function is evaluated from two of the apical views: four-chamber (4CH) and two-chamber (2CH) by ejection fraction (EF). The segmental LV function is done from three apical views 4CH, 2CH and three chamber (3CH) and supports wall motion evaluation and strain. LVivo CE-EF (Contrast EF) extends the current toolbox of the LVivo platform by providing the ability to process Ultrasonic DICOM images which acquire by Ultrasound Equipment in which the patient was prescribed a contrast agent. In addition to the LV analysis, the cardiology toolbox includes a module for automated evaluation of the Right Ventricular function. The LVivo platform includes one additional non-cardiac module for the measurement of the bladder volume.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Parameter | Acceptance Criteria (Implied by Study Design) | Reported Device Performance |
|---|---|---|
| EDV 4CH Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.96, CI [0.94, 0.97] |
| EDV 4CH Bland-Altman Mean Difference (Bias) | Close to 0 (implicitly, within acceptable clinical limits) | -2.68 ml |
| EDV 4CH Bland-Altman Limits of Agreement (LOA) | Narrow range (implicitly, clinically acceptable spread) | (-36.02, 30.66) |
| ESV 4CH Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.98, CI [0.97, 0.99] |
| ESV 4CH Bland-Altman Mean Difference (Bias) | Close to 0 | -3.87 ml |
| ESV 4CH Bland-Altman Limits of Agreement (LOA) | Narrow range | (-25.58, 17.82) |
| EF 4CH Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.95, CI [0.93, 0.97] |
| EF 4CH Bland-Altman Mean Difference (Bias) | Close to 0 | 1.26% points |
| EF 4CH Bland-Altman Limits of Agreement (LOA) | Narrow range | (-8.42, 10.96) |
| EDV 2CH Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.96, CI [0.94, 0.97] |
| EDV 2CH Bland-Altman Mean Difference (Bias) | Close to 0 | -5.69 ml |
| EDV 2CH Bland-Altman Limits of Agreement (LOA) | Narrow range | (-36.02, 24.44) |
| ESV 2CH Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.98, CI [0.97, 0.99] |
| ESV 2CH Bland-Altman Mean Difference (Bias) | Close to 0 | -3.87 ml |
| ESV 2CH Bland-Altman Limits of Agreement (LOA) | Narrow range | (-25.58, 17.82) |
| EF 2CH Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.93, CI [0.90, 0.95] |
| EF 2CH Bland-Altman Mean Difference (Bias) | Close to 0 | -0.54% points |
| EF 2CH Bland-Altman Limits of Agreement (LOA) | Narrow range | (-12.18, 11.1) |
| BP EDV Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.98, CI [0.97, 0.99] |
| BP EDV Bland-Altman Mean Difference (Bias) | Close to 0 | -4.1 ml |
| BP EDV Bland-Altman Limits of Agreement (LOA) | Narrow range | (-29.04, 20.84) |
| BP ESV Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.98, CI [0.97, 0.99] |
| BP ESV Bland-Altman Mean Difference (Bias) | Close to 0 | -2.77 ml |
| BP ESV Bland-Altman Limits of Agreement (LOA) | Narrow range | (-19.67, 14.13) |
| BP EF Correlation (Automated vs. Manual) | High correlation (implicitly, r > 0.90) | r = 0.96, CI [0.94, 0.97] |
| BP EF Bland-Altman Mean Difference (Bias) | Close to 0 | 0.39% points |
| BP EF Bland-Altman Limits of Agreement (LOA) | Narrow range | (-8.26, 9.05) |
| Automatic Processing Rate | High rate (implicitly, > 80%) | 90% (91/101 exams) |
Study Information
-
Sample size used for the test set and the data provenance:
- Sample Size: 101 patient exams.
- Data Provenance: The data was collected from multiple sources: Beth Israel, Soroka, Hadassah (presumably Israel, based on the manufacturer's location), and UCMC (likely a US center given the race information collection). The text indicates that 69 of the 101 patients were collected from the US. The study is retrospective, as it refers to collected "patient exams."
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: 3 sonographers.
- Qualifications: The text explicitly states "3 sonographers" were used. While their specific years of experience are not mentioned, the implication is that they are qualified medical professionals capable of performing and interpreting ultrasound measurements. The subsequent review by a cardiologist also suggests oversight and validation of their measurements.
-
Adjudication method for the test set:
- The ground truth was "comprised of the measurements by the 3 sonographers." This suggests a consensus or averaging approach among the three sonographers. It further states, "No further changes by the cardiologist to the measurements were needed following the cardiologist's review," implying the cardiologist reviewed and implicitly approved the sonographers' consensus measurements. This could be interpreted as a form of expert consensus adjudication, where the sonographers' measurements formed the basis, and a cardiologist provided final validation.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance was not described. This study directly compared the algorithm's performance to human (sonographer) ground truth measurements rather than assessing human reader improvement with AI assistance.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance evaluation of the algorithm was done. The study compares the "automated results" of the LVivo Software Application to the established "manual measurements" (ground truth). The fact that the algorithm "processed automatically 91/101 (90%) of the exams" and these automated results were compared to the manually derived ground truth confirms a standalone evaluation.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The ground truth used was expert consensus based on "manual measurements" performed by "3 sonographers," which were then reviewed and confirmed by a cardiologist.
-
The sample size for the training set:
- The document does not provide the sample size for the training set. It only discusses the performance evaluation using the test set of 101 patient exams.
-
How the ground truth for the training set was established:
- The document does not provide any information on how the ground truth for the training set was established. The focus of this submission is on the performance evaluation of the final device using a designated test set.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).