(316 days)
WebPAX is intended for use in the communication and storage of medical images. WebPAX is also intended for use as a comprehensive solution to view, optimize, and post-process diagnostic medical images as an aid to physicians and other healthcare professionals in the evaluation of digital imaging examinations.
Due to special customer requirements based on the imaging modality and clinical focus, WebPAX can be configured with different combinations of clinical applications which are intended to assist the physician in diagnosis or treatment planning. This includes commercially available post-processing techniques such as multi-planar reconstruction (MPR).
Lossy compressed mammographic images and digitized film screen images must not be reviewed for primary image interpretations. Mammographic images may only be interpreted using cleared monitors intended for mammography display. MPR is not intended for mammography use.
Not intended for diagnostic use on mobile devices.
WebPAX consists of a server, typically located in a hospital data center, and one or more client systems used by technicians and physicians. On the server side, WebPAX is a software-only solution that is installed on customer-supplied hardware, such as dedicated computer server or virtual machine. The server-side software communicates with other DICOM-compliant systems and with end-users via standard Intranet and/or Internet mechanisms. On the client side, technicians and physicians access the WebPAX server using standard Internet web browsers, such as Google Chrome and Internet explorer.
WebPAX provides users with the ability to manage, store, and interpret digital medical images, including the following application specific modules:
- . Echocardiography Workstation
- Cardiovascular MRI Workstation
- 3D Post Processing (Multi-Planar Reconstruction)
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Software Verification | WebPAX software meets all requirements described in the Software Requirement Specification. Tested on Google Chrome and Microsoft Internet Explorer 11. All tests passed. |
| Software Validation - Measurement Accuracy | Accuracy for "WebPAX Measurement Accuracy" (Basic measurements, Echocardiography measurements, Cardiovascular MR measurements) assessed by direct comparison to measurements made using the Predicate Device on the exact same DICOM dataset, image, and location(s). WebPAX met the Acceptance Criteria that accuracy be greater than 98%. |
| Reader Study - 3D Post Processing | Performance of WebPAX's 3D post processing capability (Multi-Planar Reconstruction) is equivalent to that of the predicate Siemens Leonardo Workstation. All scores for WebPAX were at least a 1 (Excellent) or a 2 (Very good). The performance of WebPAX was comparable to that of the predicate Siemens device. |
| Functionality Comparison | Compared to predicate Siemens Leonardo: - Basic PACS Functions: Yes (DICOM standard) - Generic Tools (Measurements): Yes (DICOM standard, Basic Geometry) - Specialty Tools: Echocardiography: Yes (DICOM standard; predicate does not offer; testing demonstrated performance met established acceptance criteria) - Specialty Tools: Cardiovascular MRI: Yes (ROI Area times Slice Thickness, ROI Grayscale times Scale Factor, ROI Grayscale versus time) - 3D Post Processing (MPR): Yes (comparable performance demonstrated in reader study) |
Study Details
1. Sample sizes used for the test set and the data provenance:
- Software Validation Report (Measurement Accuracy): The exact number of DICOM datasets/images used for testing measurement accuracy is not explicitly stated. It mentions "exact same DICOM dataset, image, and location(s) within the image" but not the total count.
- Reader Study Report (3D Post Processing): Six DICOM datasets representing typical clinical scenarios were used.
- Data Provenance: Not explicitly stated for either test. The "typical clinical scenarios" for the reader study imply a retrospective collection of real patient data, but the specific country of origin or whether it was retrospective/prospective is not provided.
2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Reader Study Report (3D Post Processing): Three board-certified physicians were used. Their qualification is stated as "who routinely read cardiovascular MRI studies."
- Software Validation Report (Measurement Accuracy): For measurement accuracy, the "ground truth" was established by measurements from the predicate device. No human experts were involved in establishing ground truth for this specific test; it was a direct device-to-device comparison.
3. Adjudication method for the test set:
- Reader Study Report (3D Post Processing): The text mentions that "All scores for WebPAX were at least a 1 (Excellent) or a 2 (Very good)." This suggests a scoring system, but it does not describe an explicit adjudication method (e.g., 2+1, 3+1 consensus). It implies an independent rating by each physician from which the reported scores were derived and compared.
- Software Validation Report (Measurement Accuracy): No adjudication method was applicable as the comparison was directly against the predicate device's output.
4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:
- Yes, a multi-reader multi-case (MRMC) comparative effectiveness study was done for the 3D post-processing capability (referred to as a "Reader Study Report").
- Effect size of how much human readers improve with AI vs without AI assistance: This study was designed to demonstrate equivalence of the WebPAX's 3D post-processing capability to the predicate device, not to show improvement of human readers with AI assistance. The study assessed the performance of the device's output as judged by human readers, rather than the readers' performance with and without the device. The conclusion was "The performance of WebPAX was comparable to that of the predicate Siemens device."
5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, the "Software Validation Report" for measurement accuracy serves as a standalone performance assessment. The WebPAX's measurement functionalities (Basic measurements, Echocardiography measurements, Cardiovascular MR measurements) were directly compared against the predicate device's measurements for the same DICOM data, without human interpretation in the loop for this specific test.
6. The type of ground truth used:
- Software Validation Report (Measurement Accuracy): The ground truth was effectively established by the measurements provided by the predicate device.
- Reader Study Report (3D Post Processing): The ground truth was expert consensus/evaluation (based on the physicians' scores of 'Excellent' or 'Very Good' for the WebPAX output). While not explicit "ground truth" in the sense of a definitive diagnosis, their assessment of the post-processing quality acted as the reference.
7. The sample size for the training set:
- The document does not provide information regarding the sample size for any training set. This device appears to be a PACS system with post-processing capabilities, not necessarily a deep learning AI model that requires a distinct "training set." The performance tests focused on verification, validation of measurements, and a reader study comparing it to a predicate.
8. How the ground truth for the training set was established:
- As no information on a training set is provided, this question cannot be answered from the document.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).