(245 days)
Indications for use of the product are quantification and reporting of cardiovascular, fetal, and abdominal structures and function of patients with suspected disease to support the physician in the diagnosis.
The purpose of this Traditional 510(k) Pre-market Notification is to introduce the SWM, 3D Auto TV and 3D Auto CFQ software applications as well as compatibility of VeriSight ICE / Pro ICE Probe data with the subject device Ultrasound Workspace Version 6.0.
The semi-automated Segmental Wall Motion feature (SWM) evaluates the segmental (regional) function of the left ventricle (LV) from adult TTE echo examinations. It performs border detection and tracking to identify each of the LV seqments, provides segmental wall motion scores for each segments of the LV by using machine learning algorithms and calculates an overall wall motion score index (WMSI) as the average of the segmental scores.
3D Auto TV software enables semi-automated quantification of the tricuspid valve. At a high level, this is accomplished through automatically derived measurements from a segmented model of the tricuspid valve annulus formed by the software through model-based segmentation of the acquired ultrasound images.
3D Auto CFQ provides semi-automated quantification of Mitral Requrgitation (MR) volume and peak flow rate based on 3D color flow images. This application uses a known fluid dynamic model of flow that is adapted to the acquired color information. This allows quantitative assessment of mitral valve leakage during systole. The derived result supports the assessment of mitral regurgitation volume and peak flow rate.
Data Compatibility of the VeriSight ICE / Pro ICE Probe, transducers cleared for the EPIQ Series Diagnostic Ultrasound System (K202216), will be introduced for Ultrasound Workspace 6.0.
General software architecture of the previously cleared version TOMTEC-ARENA remains unchanged. Two new clinical application packages will be introduced with UWS6.0: 3D Auto TV and 3D Auto CFQ. An existing feature AutoStrain Left Ventricle (AutoStrain LV) gains additional functionality by integration of Segmental Wall Motion (SWM) feature. The module using AutoStrain LV together with SWM is named 2D Auto LV.
Here's a breakdown of the acceptance criteria and study information for the Philips Ultrasound Workspace (UWS 6.0) based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Feature/Metric | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Segmental Wall Motion (SWM) | ||
| Pearson's correlation coefficient for WMSI (compared to LVivo SWM) | Lower Confidence Bound > 0.8 | 0.957 (95%CI 0.933, 0.972). Met acceptance criteria. |
| 3D Auto TV | ||
| Limits of Agreement (LoA) for annulus size (distance) (compared to 4D Cardio-View) | Within ± 46% | LoA for annulus size within acceptance criteria. |
| Limits of Agreement (LoA) for annulus shape (circumference) (compared to 4D Cardio-View) | Within ± 52% | LoA for annulus shape within acceptance criteria. |
| Relative bias for distance (size) (compared to inter-observer variability) | Within +/- 17.37% | Met. |
| Relative bias for circumference (shape) (compared to inter-observer variability) | Within +/- 23.68% | Met. |
| Mean relative error of measurement primitives on in-silico phantoms | Within +/- 1% | Met. |
| Limits of Agreement for measurement primitives on in-silico phantoms | Within +/- 5% | Met. |
| 3D Auto CFQ | ||
| Maximum allowable difference (Δ) for regurgitant volume (compared to CMR) | 61.6 mL | Lower end of 95% Cl for LoA was -58.37, upper end of 95% Cl for LoA was 34.18. Met acceptance criteria. |
| Mean difference (bias) for regurgitant volume (compared to CMR) | Within +/- 19.2 mL | Met. |
| Pearson's correlation for peak regurgitant flow (compared to 2D PISA) | Upper and lower bounds of 95% confidence interval > 0.8 | Exceeded acceptance criteria. Met acceptance criteria. |
2. Sample Size and Data Provenance
-
Segmental Wall Motion (SWM):
- Test Set Sample Size: Not explicitly stated as a number, but the study involved "subjects referred for clinical TTE exam."
- Data Provenance: Retrospective, from transthoracic (TTE) ultrasound clips. Country of origin not specified.
-
3D Auto TV:
- Test Set Sample Size: Not explicitly stated as a number, but involved "subjects whose clips contributed to the study." These subjects "represented a broad range of demographics, body habitus, and their severity of tricuspid regurgitation."
- Data Provenance: Not explicitly stated as retrospective or prospective, but used "transesophageal echocardiography (TEE) cardiac clips." Country of origin not specified.
-
3D Auto CFQ:
- Test Set Sample Size: Not explicitly stated as a number, but involved "the same subjects" for comparisons between 3D Auto CFQ and 2D PISA.
- Data Provenance: Not explicitly stated as retrospective or prospective. Used "acquired 3D color flow images" and compared to Cardiac Magnetic Resonance Imaging (CMR). Country of origin not specified.
3. Number of Experts and Qualifications for Test Set Ground Truth
- Segmental Wall Motion (SWM): Not applicable for SWM's primary validation as it was compared against another software application (LVivo SWM) as ground truth.
- 3D Auto TV: 3 clinical experts (reviewers). Qualifications for these experts are not explicitly stated beyond "clinical experts (reviewers)."
- 3D Auto CFQ: Not applicable for 3D Auto CFQ's primary validation as it was compared against CMR as ground truth, and for peak flow, against 2D PISA methodology.
4. Adjudication Method for the Test Set
- Segmental Wall Motion (SWM): Not applicable, as the comparison was algorithm-to-algorithm.
- 3D Auto TV: No explicit mention of an adjudication method among the three experts. The text states "results compared to manual measurements by the same reviewers performed within 4D Cardio-View application, used as a ground truth." This implies individual expert measurements were used for comparison, but not necessarily adjudicated for consensus beyond the individual performing the manual measurement.
- 3D Auto CFQ: Not applicable, as the comparison was against objective measures (CMR) and an accepted methodology (2D PISA).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No MRMC comparative effectiveness study is mentioned that measures the effect size of how much human readers improve with AI vs without AI assistance. The studies performed were for standalone performance (SWM, 3D Auto TV, 3D Auto CFQ algorithms) compared to established ground truths or other methods. A summative evaluation for 2D Auto LV, including SWM, indicated that "16 target users completed the critical tasks... with a success rate of 97.7%," but this is a user experience/usability study, not a comparative effectiveness study measuring improved human performance with AI.
6. Standalone (Algorithm Only) Performance Study
- Yes, standalone performance studies were done for each new feature:
- SWM: The SWM algorithm's performance was evaluated against the LVivo SWM application ("ground truth"). This is a standalone algorithm-to-algorithm comparison.
- 3D Auto TV: The 3D Auto TV software's performance was evaluated by comparing its derived measurements to manual measurements performed by clinical experts using the 4D Cardio-View application ("ground truth"). Although experts performed the "ground truth" measurements, the focus was on the algorithm's accuracy relative to these manual measurements.
- 3D Auto CFQ: The 3D Auto CFQ software's performance was evaluated by comparing its regurgitant volume output to Cardiac Magnetic Resonance Imaging (CMR) ("ground truth") and its peak regurgitant flow output to the 2D PISA methodology. These are standalone evaluations of the algorithm's outputs.
7. Type of Ground Truth Used
- Segmental Wall Motion (SWM): Ground truth was established by comparing the SWM algorithm's output to the LVivo SWM (DiA Imaging Analysis) application. This is an algorithm-to-algorithm comparison where the comparator software is considered the ground truth.
- 3D Auto TV: Ground truth was established by manual measurements performed by clinical experts within the 4D Cardio-View application. This represents expert-derived measurements from a reference software application. In-silico phantoms with known dimensions were also used for accuracy and precision of underlying measurement primitives.
- 3D Auto CFQ:
- For regurgitant volume: Cardiac Magnetic Resonance Imaging (CMR) regurgitant volume (RVol) was used. This is considered an outcomes data/reference standard.
- For peak regurgitant flow: 2D PISA methodology was used. This is an expert-derived methodology/reference method.
8. Sample Size for the Training Set
- The document does not explicitly state the sample sizes used for training the machine learning algorithms for SWM, 3D Auto TV, or 3D Auto CFQ. It only refers to the retrospective nature of some of the test data.
9. How the Ground Truth for the Training Set Was Established
- The document does not explicitly describe how the ground truth for the training set was established for any of the machine learning algorithms. It mentions the "use of machine learning algorithms" but focuses on the validation (test set) ground truth.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).