(59 days)
qp-Prostate is an image processing software package to be used by trained professionals, including radiologists specialized in prostate imaging, urologists and oncologists. The software runs on a standard "off-the-shelf" workstation and can be used to perform image viewing, processing and analysis of prostate MR images. Data and images are acquired through DICOM compliant imaging devices and modalities. Patient management decisions should not be based solely on the results of qp-Prostate. qp-Prostate does not perform a diagnostic function, but instead allows the users to visualize and analyze DICOM data.
qp-Prostate is a medical image viewing, processing and analyzing software package for use by a trained user or healthcare professional, including radiologists specialized in prostate imaging, urologists and oncologists. These prostate MR images, when interpreted by a trained physician, may yield clinically useful information.
qp-Prostate consists of a modular platform based on a plug-in software architecture. Apparent Diffusion Coefficient (ADC) post-processing and Perfusion - Pharmacokinetics post-processing (PKM) are embedded into the platform as plug-ins to allow prostate imaging quantitative analysis.
The platform runs as a client-server model that requires a high-performance computer installed by QUIBIM inside the hospital or medical clinic network. The server communicates with the Picture Archiving and Communication System (PACS) through DICOM protocol, qp-Prostate is accessible through the web browser (Google Chrome or Mozilla Firefox) of any standard "off-the-shelf" computer connected to the hospital/center network.
The main features of the software are:
- Query/Retrieve interaction with PACS;
- Apparent Diffusion Coefficient (ADC) post-processing (MR imaging);
- Perfusion Pharmacokinetics (PKM) post-processing (MR imaging);
- DICOM viewer; and
- Structured reporting.
The software provides MR imaging analysis plug-ins to objectively measure different functional properties in prostate images.
Here's a breakdown of the acceptance criteria and study details for the qp-Prostate device, based on the provided document:
Acceptance Criteria and Device Performance
The document does not explicitly present a table of "acceptance criteria" with numerical targets and reported performance. Instead, it describes performance testing conducted to demonstrate functional equivalence and safety and effectiveness compared to a predicate device. The performance data generally aims to show the device functions as intended and is comparable to the predicate.
The closest we can get to a "table of acceptance criteria and reported device performance" is by looking at the types of tests done and the general conclusions.
Implicit Acceptance Criteria (Inferred from testing and purpose):
- Accuracy of Diffusion-ADC and Perfusion-Pharmacokinetics (PKM) calculations: The device should accurately compute these parameters.
- Accuracy of algorithmic functions: The Spatial Smoothing, Registration, Automated Prostate Segmentation, Motion Correction, and automated AIF selection algorithms should perform correctly.
- Equivalence to Predicate Device: The performance of qp-Prostate should be comparable to the Olea Sphere v3.0, especially in quantitative outputs.
- Functionality as intended: All listed features (Query/Retrieve, DICOM viewer, Structured reporting, etc.) should work correctly.
Reported Device Performance (from the document):
- Diffusion-ADC and Perfusion-Pharmacokinetics (PKM) analysis modules: Evaluated using QIBA's Digital Reference Objects (DROs), including noise modeling, for technical performance. The document concludes that the "tests results demonstrate that qp-Prostate functioned as intended."
- Algorithmic functions (Spatial Smoothing, Registration, Automated Prostate Segmentation, Motion Correction, AIF selection): Tested using a dataset of prostate clinical cases. The document states "performance testing with prostate MR cases" was conducted.
- Comparison to Predicate Device: Performed using 157 clinical cases, demonstrating that the device is "as safe and effective as its predicate device, without introducing new questions of safety and efficacy."
- Overall Conclusion: "Performance data demonstrate that qp-Prostate is as safe and effective as the OLEA Sphere v3.0. Thus, gp-Prostate is substantially equivalent."
Since no specific numerical thresholds for accuracy or performance metrics are provided in the document, a quantitative table cannot be generated. The acceptance is based on successful completion of the described performance tests and demonstrating substantial equivalence.
Study Details
Here's the detailed information regarding the studies:
1. Sample sized used for the test set and the data provenance:
- Digital Reference Object (DRO) Analysis (Bench Testing):
- Sample Size: Not explicitly quantified as "sample size" in terms of number of patient cases, but rather refers to universally accepted digital reference objects from QIBA.
- Provenance: These are synthetic or standardized digital objects designed for technical performance evaluation, not originating from specific patients or countries.
- Clinical Testing of Algorithms (Motion Correction, Registration, Spatial Smoothing, AIF Selection, Prostate Segmentation):
- Motion Correction algorithm: 155 DCE-MR and DWI-MR prostate sequences from 155 different patients.
- Registration algorithm: 112 T2-Weighted MR, DCE-MR, and 108 DWI-MR prostate sequences from different patients.
- Spatial Smoothing algorithm: 51 transverse T2-weighted, DCE-MR, and DWI-MR prostate sequences from 51 different patients.
- AIF selection algorithm: 242 DCE-MR prostate sequences from 242 different patients.
- Prostate Segmentation algorithm: 243 transverse T2-weighted MR prostate sequences from 243 different patients.
- Provenance for these clinical cases: The document states they were "acquired from different patients in different machines with multiple acquisition protocols" and from "different three major MRI vendors: Siemens, GE and Philips, magnetic field of 3T and 1.5T cases." There is no explicit mention of the country of origin or whether the data was retrospective or prospective, though "clinical cases" used for validation often imply retrospective use of existing data.
- Comparison to Predicate Device:
- Sample Size: 157 T2-weighted MR, DCE-MR, and 141 DWI-MR prostate sequences from different patients.
- Provenance: Similar to the algorithmic clinical testing, from "different patients in different machines with multiple acquisition protocols" and from "different three major MRI vendors: Siemens, GE and Philips, magnetic field of 3T and 1.5T cases."
2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document does not explicitly state the number of experts used or their specific qualifications (e.g., years of experience as radiologists) for establishing ground truth for the clinical test sets.
- For the Digital Reference Object (DRO) analysis, the "ground truth" is inherent to the design of the DROs themselves (proposed by QIBA), rather than established by human experts for each test.
3. Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- The document does not specify any adjudication method used for establishing ground truth or for resolving discrepancies in the test sets.
4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study assessing human reader improvement with AI assistance was not performed or reported in this document. The study compared the device's technical performance and its output with that of a predicate device, not with human readers or human readers assisted by AI.
5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, the performance testing described is primarily standalone. The "bench testing" with DROs and the "clinical testing" of the algorithms (Motion Correction, Registration, etc.) evaluate the device's inherent performance without a human-in-the-loop scenario. The comparison to the predicate device also assesses the algorithm's output directly against the predicate's output. The device itself is described as "an image processing software package to be used by trained professionals" and "does not perform a diagnostic function, but instead allows the users to visualize and analyze DICOM data," which points to it being a tool that supports human interpretation rather than replacing it.
6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Bench Testing (DWI & DCE): Digital Reference Objects (DROs) proposed by QIBA, which serve as a synthetic, known ground truth for quantitative accuracy.
- Clinical Testing of Algorithms & Comparison to Predicate: The document does not explicitly define the ground truth for these clinical performance tests. Given that it's comparing the outputs of image processing algorithms, the ground truth would likely involve:
- Reference standard values: For quantitative parameters like ADC, Ktrans, kep, ve, it would likely involve comparing the device's calculated values against a recognized reference standard or the predicate's output considered as a benchmark.
- Visual assessment/Expert review: For the performance of segmentation, registration, and motion correction, expert radiologists would visually assess the accuracy of the algorithm's output to determine if it "functioned as intended."
- The statement "comparison against the predicate device" implies the predicate's output is used as a reference point.
7. The sample size for the training set:
- The document does not provide any information on the sample size used for the training set of the algorithms. It focuses solely on the validation and verification performed post-development.
8. How the ground truth for the training set was established:
- Since no information on the training set or its sample size is provided, there is also no information on how the ground truth for the training set was established.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).