(61 days)
RTapp™ is a stand-alone software that provides a means for comparison of imaging data that is DICOM compliant. It allows the registration and display of medical images as an aid during use by radiation oncology.
RTapp™ v2.0 is a stand-alone software medical device. RTapp analyzes and visualizes the day-to-day variations in a radiation therapy patient's individual anatomical structures and the effect of those changes on the treatment dose; it is an aid during use by radiation oncology.
The RTapp software:
- Automatically queries and retrieves treatment plan data and images from any DICOM compliant equipment.
- Automatically processes all patient's treatment fractions, flagging and presenting an advance warning of treatment plans at risk with an email notification.
- Monitors and evaluates treatment plan performance in real time by using the Plan Performance Dose Volume Histogram (DVH). The DVH projects the amount of dose to be delivered.
- Displays Deformable Image Registration contours, cross correlation metrics and flagging of large deformations.
- Projects when Organs At Risk will exceed dose constraints.
- Dose estimation
- Generates reports as PDF with images and graphs.
The provided document is a 510(k) Premarket Notification from the U.S. FDA for the RTapp™ v2.0 device. This document details the device's indications for use, its comparison to a predicate device, and the general software verification and validation testing performed. However, it does not contain specific acceptance criteria tables or detailed study results (like sample sizes, expert qualifications, adjudication methods, MRMC studies, standalone performance data, or detailed ground truth methodologies) that would typically be associated with a comprehensive study proving a device meets specific performance criteria.
The document states:
- "Software verification and validation testing was conducted, and documentation was provided as recommended by FDA's guidance, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices. The RTapp software is considered a "Major" level of concern."
- "Non-clinical, verification and validation (V&V) performance testing was conducted on anonymized, retrospective, clinically-relevant data using computer systems for single and multi-users that met minimum configuration specifications in order to test the device design requirements and user needs."
- "RTapp™ v2.0, which includes some off-the-shelf software, underwent V&V testing and regression analysis of manual test suites interfacing with external DICOM servers."
- "The following features for comparing DICOM-compliant images were tested: automated and manual workflows, image visualization, alignment, settings for configuring the dashboard and visual displays, user interface, importing and exporting DICOM data, processing and loading time, dose volume histogram graphs, and reports."
- "All test protocols passed, and acceptance criteria were met indicating successful verification and validation of the subject device and substantial equivalence to the predicate device."
This information confirms that V&V testing was performed and acceptance criteria were met, but the specific details of these criteria and the associated study data are not provided in this summary document. The document focuses on demonstrating substantial equivalence to a predicate device (RTapp™ v1.0) rather than presenting a detailed clinical performance study with the metrics you've requested.
Therefore,Based on the provided FDA 510(k) summary for RTapp™ v2.0, the document does not contain the detailed information necessary to fully answer your request regarding specific acceptance criteria metrics, study sample sizes, expert qualifications, or detailed ground truth methodologies.
The document states that "All test protocols passed, and acceptance criteria were met indicating successful verification and validation of the subject device and substantial equivalence to the predicate device." However, it does not disclose the specific numerical acceptance criteria or the raw performance data against those criteria.
Here's what can be inferred or stated based on the provided text, and what information is missing:
1. A table of acceptance criteria and the reported device performance:
- Acceptance Criteria: Not explicitly stated in numerical form. The document generally refers to successful verification and validation of device features related to image comparison, registration, display, workflow, data handling, and reporting.
- Reported Device Performance: Not numerically reported. The document only states that "All test protocols passed, and acceptance criteria were met."
2. Sample size used for the test set and the data provenance:
- Sample Size: Not specified.
- Data Provenance: "Anonymized, retrospective, clinically-relevant data." The country of origin is not specified.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Not specified.
- Qualifications of Experts: Not specified.
- The document implies that the ground truth for evaluating features like deformable image registration accuracy or dose estimation would rely on established radiation oncology principles and possibly comparison to the predicate device's known performance.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- Not specified.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- Answer: No, an MRMC comparative effectiveness study is not mentioned. The device, RTapp™ v2.0, is described as an "aid" for radiation oncology professionals and is not a diagnostic AI intended to replace or significantly augment human perception in a way that typically warrants an MRMC study for improved reader performance. Its function is comparison of DICOM-compliant imaging data, registration, and display, with features like flagging at-risk plans and projecting dose constraints. The focus is on functionality and substantial equivalence to a predicate, not on improving human reader diagnostic accuracy.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Answer: Yes, or at least implied. The V&V testing likely included automated tests of the software's functionality and accuracy in its core tasks (e.g., image registration, dose volume histogram generation, data import/export), indicating standalone performance assessment. The device is a "stand-alone software."
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- "Clinically-relevant data" implies that the ground truth would be based on established clinical benchmarks or calculations for image registration, dose distribution, and other parameters that RTapp™ v2.0 processes. It's not explicitly stated if this involved new expert consensus readings or if it relied on the inherent properties of the DICOM data as processed by accepted clinical standards. Given it's an "aid" for comparison of imaging data for radiation oncology, the ground truth would likely relate to the accuracy of its data analysis and visualization capabilities compared to expected clinical outcomes or calculations.
8. The sample size for the training set:
- Not specified. The document refers to V&V testing on "anonymized, retrospective, clinically-relevant data" for the test set. It does not explicitly mention a separate "training set" or how a machine learning model, if any, was trained, as the device's core technology is described as "Deformable Image Registration using an "Optical-Flow" algorithm."
9. How the ground truth for the training set was established:
- Not specified. (See point 8).
In summary, this 510(k) notification confirms that the required software verification and validation testing was performed and met acceptance criteria, demonstrating substantial equivalence to the predicate. However, it does not provide the detailed study specifics or quantitative performance metrics typically found in a clinical performance study summary.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).