(195 days)
The AAA Model is a software application designed to view and quantify 3D image data acquired by Philips diagnostic ultrasound systems for use in measuring the anterior diameter of abdominal aortic aneurysms. Optionally, lateral diameter, maximum diameter, and partial volume of an abdominal aortic aneurysm can also be provided. It is intended to be used by trained and qualified healthcare professionals in clinical point-of-care facilities.
AAA Model for QLAB Advanced Quantification Software is a software application designed for structural measurements of an Abdominal Aortic Aneurism (AAA), including volume measurement and diameter measurements. AAA Model is designed to assist in monitoring a previously diagnosed Abdominal Aortic Aneurisms in two ways:
- to follow the anteroposterior (AP) maximum diameter for a Native AAA, and
- to follow the anteroposterior (AP) maximum diameter for a post-surgical AAA.
Philips QLAB Advanced Quantification Software (QLAB) is designed to view and quantify image data acquired on Philips ultrasound systems. QLAB is available either as a stand-alone product that can function on a standard PC, a dedicated workstation, and on-board Philips' ultrasound systems. AAA Model is compatible with the Philips EPIQ Diagnostic Ultrasound System.
Here's a summary of the acceptance criteria and study details for the Philips AAA Model, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
| Test Type | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Bench Testing | ||
| Measurement Accuracy Verification (Volume) | ± 9% accuracy for volume measurement, with 90% confidence interval. | "All measurements met the acceptance criteria." |
| Measurement Accuracy Verification (Diameter) | ± 5% accuracy for all diameter measurements (AP/LAT/MAD), with 90% confidence interval. | "All measurements met the acceptance criteria." |
| Clinical Testing | ||
| Native AAA Clinical Evaluation | At least 80% of all cases in which Philips AAA Model provides maximum AP diameter results that match standard of care 2D ultrasound measurements to within ± 10%. | "Result met the target successful rate." (This implies the 80% with ±10% agreement was achieved). |
| Post-EVAR AAA Clinical Evaluation | At least 80% of all cases in which Philips AAA Model provides maximum AP diameter results that match standard of care 2D ultrasound measurements to within ± 10%. | "Result met the target successful rate." (This implies the 80% with ±10% agreement was achieved). |
2. Sample Sizes Used for the Test Set and Data Provenance
- Bench Testing:
- Sample Size: 7 samples were used for each measurement (volume, AP diameter, LAT diameter, MAD diameter).
- Data Provenance: The data was generated using "EPIQ diagnostic ultrasound system...to take 3D ultrasound images of 3 phantoms with various diameter." This indicates a controlled, simulated environment (phantom study).
- Clinical Testing:
- Native AAA:
- Sample Size: 91 datasets (from an initial 129 gathered) met the inclusion criteria.
- Data Provenance: "129 Native AAA datasets were gathered from one hospital in Copenhagen, Denmark. ... All patients age > 18 years old." This indicates prospective or retrospective clinical data from Denmark.
- Post-EVAR AAA:
- Sample Size: 45 datasets (from an initial 77 gathered) met the inclusion criteria.
- Data Provenance: "77 Post-EVAR AAA datasets were gathered from one hospital in Copenhagen, Denmark. ... All patients age > 18 years old." This indicates prospective or retrospective clinical data from Denmark.
- Native AAA:
3. Number of Experts Used to Establish Ground Truth and Qualifications
The document does not explicitly state the number of experts used to establish ground truth or their specific qualifications (e.g., "radiologist with 10 years of experience").
For clinical testing, the "standard of care 2D ultrasound measurement" is used as the ground truth. It is implied that these measurements would have been performed by qualified healthcare professionals (e.g., sonographers or physicians) at the hospital in Copenhagen, Denmark, as part of routine clinical practice.
4. Adjudication Method for the Test Set
The document does not explicitly describe a formal adjudication method (like 2+1 or 3+1) for establishing ground truth in either the bench testing or clinical testing.
- For bench testing, the ground truth is derived from the known dimensions of the phantoms.
- For clinical testing, the "standard of care 2D ultrasound measurement" is considered the reference. It is not specified if multiple readers were used for these reference measurements or if any discrepancies were adjudicated.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a multi-reader multi-case (MRMC) comparative effectiveness study was not done. The study compared the device's measurements to a "standard of care 2D ultrasound measurement," but it was not designed to evaluate the improvement of human readers with AI assistance versus without. The device itself is described as "not a computer-assisted detection (CADe) device, and does not use artificial intelligence or machine learning."
6. Standalone Performance Study
Yes, a standalone performance study was done. The device's performance was evaluated by directly comparing its measurements (AP diameter, volume, etc.) against established ground truth (phantom measurements for bench testing, standard of care 2D ultrasound measurements for clinical testing). The description implies that the AAA Model was used as an independent tool to quantify the images.
7. Type of Ground Truth Used
- Bench Testing: Physical dimensions of phantoms (a known, manufactured standard).
- Clinical Testing: "Standard of care 2D ultrasound measurement" for abdominal aortic aneurysm AP diameter.
8. Sample Size for the Training Set
The document does not provide information regarding a separate training set or its sample size. The device is explicitly stated as "not a computer-assisted detection (CADe) device, and does not use artificial intelligence or machine learning," which suggests it may not have a traditional 'training set' in the context of machine learning. Its functionality seems to be based on pre-programmed algorithms for structural measurements rather than learned patterns from a training dataset.
9. How Ground Truth for the Training Set Was Established
As noted in point 8, the document does not mention a training set for the AAA Model, given its description as not using AI/ML. Therefore, the method for establishing ground truth for a training set is not applicable or provided.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).