(59 days)
The intended use of the 5000 Compact series ultrasound systems is diagnostic ultrasound imaging and fluid flow analysis of the human body with the following Indications for Use:Abdominal, Cardiac Adult, Cardiac Pediatric, Carotid, Cerebral Vascular, Cephalic (Adult),Cephalic (Neonatal), Fetal Echo, Fetal/Obstetric, Gynecological, Intraoperative (Vascular), Lung, Musculoskeletal (Conventional), Musculoskeletal (Superficial), Ophthalmic, Pediatric, Peripheral Vessel, Small Parts, Transesophageal (Cardiac), Transrectal, Transvaginal, and Urology.
The purpose of this Traditional 510(k) Pre-Market Notification is to introduce the Auto Measure Artificial Intelligence-Machine Learning software feature onto the 5000 Compact Series Ultrasound Systems.
The Auto Measure feature utilizes machine learning to provide a subset of semi-automated and editable measures during an echocardiography or when reviewing an already acquired echocardiography. When Auto Measure Version 2 is enabled, the healthcare professional performs an echocardiography with a workflow that provides the user with a semi-automated measurement that can be edited, accepted, or rejected.
Philips has designed Auto Measure as a "locked" algorithm prior to marketing. As defined by FDA in the discussion paper Proposed Requlatory Framework for Modifications to AI/ML Based Software as a Medical Device (SaMD) published April 2, 2019, this "locked" algorithm provides the same result each time the same input is applied to it and does not change with use.
The Auto Measure software feature does not introduce new modes, presets, measurements, or system components (e.g. transducers) to the Philips 5000 Compact Series Ultrasound Systems K222648.
No hardware changes to the 5000 Compact Series Ultrasound Systems K222648 are required when using the Auto Measure feature, and existing, commercialized Philips transducers are used for the Auto Measure feature.
5000 Compact Series Ultrasound Systems are part of the VM platform product family, The Auto Measure Version 1 feature was originally cleared (K211597) on EPIQ and Affiniti models running software version 9.0 (VM9.0). The Auto Measure feature is also available to all software releases following VM9.0.
Since the initial Auto Measure feature initial clearance (Version 1.0), a subset of integrated measurement detectors has been trained with additional training data in the Auto Measure feature (Version 2) which is scope of this submission
Auto Measure for this premarket notification utilizes the same software version platform VM as the Reference Device, Affiniti Diagnostic Ultrasound Systems K211597.
This document describes the Philips 5000 Compact Series Ultrasound Systems with the new Auto Measure Version 2 AI-Machine Learning software feature. The information provided focuses on the performance data to demonstrate substantial equivalence.
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for each measurement detector are based on the Checked Limits of Agreement (LoAs), derived from clinical literature on human interobserver variability. The reported device performance is the Measured Limits of Agreement (LoA) between the detector's prediction and the manual ground truth. Each detector met its specified acceptance criteria.
| Parameter | Detector | N (Sample Size) | Checker Limits of Agreement (LoAs) | Measured Limits of Agreement (LoA) (detector prediction vs manual ground truth) |
|---|---|---|---|---|
| Ao Sinus Diameter | DETECTOR_ID_BMOD E_AO_AOSV | 308 | [-35.0%, 35.0%] | [-11.0022%, 11.2816%] |
| Ao STJ Diameter | DETECTOR_ID_BMOD E_AO_AOSTJ | 301 | [-35.0%, 35.0%] | [-11.4181%, 13.1139%] |
| Asc Ao Diameter | DETECTOR_ID_BMOD E_AO_AOASC | 204 | [-35.0%, 35.0%] | [-14.7794%, 15.9598%] |
| IVSd | DETECTOR_ID_BMOD E_LV_LVDISTANCE_S AME_LINE | 305 | [-35.0%, 35.0%] | [-33.2114%, 28.1632%] |
| LVIDd | DETECTOR_ID_BMOD E_LV_LVDISTANCE_S AME_LINE | 457 | [-35.0%, 35.0%] | [-14.1564%, 12.4223%] |
| LVIDs | DETECTOR_ID_BMOD E_LV_LVID_ES | 469 | [-35.0%, 35.0%] | [-23.4752%, 26.0242%] |
| LVOT Diameter | DETECTOR_ID_BMOD E_LV_LVOT | 453 | [-35.0%, 35.0%] | [-17.729%, 16.1588%] |
| LVPWd | DETECTOR_ID_BMOD E_LV_LVDISTANCE_S AME_LINE | 305 | [-35.0%, 35.0%] | [-33.1364%, 29.9544%] |
| RV Base | DETECTOR_ID_BMOD E_RV_RVD_BASE | 302 | [-35.0%, 35.0%] | [-16.1373%, 25.9079%] |
| RV Mid | DETECTOR_ID_BMOD E_RV_RVD_MID | 243 | [-35.0%, 35.0%] | [-25.0913%, 30.2573%] |
| RV Length | DETECTOR_ID_BMOD E_RV_RVL | 117 | [-35.0%, 35.0%] | [-14.6089%, 13.3871%] |
| TV Annulus | DETECTOR_ID_BMOD E_RV_TVANN | 53 | [-35.0%, 35.0%] | [-19.3628%, 18.0347%] |
| MV Decel. Time | DETECTOR_ID_DOPPLER_MV_DECEL_E_DURATION | 136 | [-25.0%, 25.0%] | [-23.5717%, 22.8591%] |
| MV Peak A Vel | DETECTOR_ID_DOPPLER_MV_VMAX_A_VELOCITY | 229 | [-24.0%, 24.0%] | [-12.3694%, 15.2363%] |
| MV Peak E Vel | DETECTOR_ID_DOPPLER_MV_VMAX_E_VELOCITY | 136 | [-24.0%, 24.0%] | [-9.2081%, 9.1223%] |
| AV VTI | DETECTOR_ID_DOPPLER_AV_VTI | 247 | [-22.0%, 22.0%] | [-21.5393%, 19.8925%] |
| LVOT VTI | DETECTOR_ID_DOPPLER_LVOT_VTI | 234 | [-22.0%, 22.0%] | [-17.267%, 17.2093%] |
| PV VTI | DETECTOR_ID_DOPPLER_PV_VTI | 66 | [-22.0%, 22.0%] | [-20.6140%, 21.0567%] |
2. Sample Sizes Used for the Test Set and Data Provenance
The test set consisted of a total of 500 studies, with one study per subject. This included 200 known normal subjects and 300 patients with a confirmed pathology.
The data provenance is described as:
- Anonymized transthoracic echocardiography DICOM data and metadata.
- Data recorded by multiple sonographers and physicians qualified in echocardiography.
- Large sample of adults of various ethnicities.
- Echocardiographic recordings acquired according to guideline-standard echocardiographic procedures between 2010 and 2021.
- Data collected from 20 centers and 16 countries.
The number of studies used for validation (N) per specific detector varied, as not all measurements are performed in all studies, ranging from 53 to 469 as shown in the table above.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The ground truth for the test set was established by clinical experts during routine care. While the exact number is not specified for the test set's ground truth, it is stated that "All detector models were trained from the ground up... Training of the detectors was based on manual measurements performed by human experts done for diagnostic purpose." This implies the ground truth for both training and testing was established by qualified human experts performing manual measurements for diagnostic purposes. The data was collected by "multiple sonographers and physicians qualified in echocardiography."
4. Adjudication Method for the Test Set
The adjudication method is not explicitly described as a formal 'X+Y' consensus method. The document states that the Auto Measure results were "compared to ground truth measurements established by clinical experts during routine care." This suggests that the established clinical measurements served as the ground truth directly for comparison, rather than a separate adjudication process of multiple readers for the test set.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size of How Much Human Readers Improve with AI vs. Without AI Assistance
A Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly detailed in this summary. The study focuses on evaluating the standalone performance of the Auto Measure feature against expert manual measurements, using human-human interobserver variability data from clinical studies as a benchmark for acceptance criteria, not for direct comparison of human performance with and without AI assistance. The document mentions that the feature is a "workflow improvement" and that "the operator is responsible for the final result and has to apply manual edits to the automated output whenever required."
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, a standalone performance study was done. The "Detector performance results" table directly compares the "detector prediction" against "manual ground truth." The text further clarifies: "Auto Measure analyzed a set of new, previously images, and its automated results were compared to ground truth measurements established by clinical experts during routine care. Both the manual and automated measurements were performed on the same images without any adjustments to the software's output or the clinical data used as ground truth." This indicates a direct evaluation of the algorithm's output without human intervention in the Auto Measure readings during the test phase.
7. The Type of Ground Truth Used
The ground truth used was expert consensus / manual measurements performed by clinical experts during routine care for diagnostic purposes. These manual measurements are considered the "current gold standard" against which the Auto Measure output is compared.
8. The Sample Size for the Training Set
The total training pool comprises more than 6000 studies.
9. How the Ground Truth for the Training Set Was Established
The ground truth for the training set was established through manual measurements performed by human experts for diagnostic purposes. The data was collected by "qualified medical [personnel] in echocardiography using TTA software (versions TTA2.31.00 and TTA2.50.00) or various Philips ultrasound systems annotation and following established guidelines (Mitchel et al 2019)." These data were then used for training and tuning in a cross-validation framework.
§ 892.1550 Ultrasonic pulsed doppler imaging system.
(a)
Identification. An ultrasonic pulsed doppler imaging system is a device that combines the features of continuous wave doppler-effect technology with pulsed-echo effect technology and is intended to determine stationary body tissue characteristics, such as depth or location of tissue interfaces or dynamic tissue characteristics such as velocity of blood or tissue motion. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.