(76 days)
The RAPTOR® Radiation Treatment Planning System is intended to be used by radiation oncology professionals for clinical review and judgement of radiation treatment plans and dose estimates based on computation, output and displays. The goal of the system is to produce consistent results using well-documented algorithms. It will provide outputs for radiation delivery equipment including documented and released multileaf collimators [MLCs] for planning, optimization, delivery and reporting of treatments. The device provides output data in the form of displays, hardcopy prints and / or plots, set-up instructions and other control information to guide the physician and / or other competent health care professional in selecting the optimum patient treatment plan.
The RAPTOR® Treatment Planning System is used to generate computation, display, evaluation and output radiation dose estimates for clinical review and judgement prior to treating the patient. It uses well-documented algorithms to provide output data in the form of displays, hardcopy prints, plots and treatment instructions to guide the radiation oncology professional and / or physician to select the optimum treatment plan. The MLC module is used to generate plans and instructions for documented and released multileaf collimators.
The RAPTOR Treatment Planning System is a collection of software modules that execute known and documented algorithms to produce radiation dose estimations. All data is user controlled and is a table look-up formation is presented graphically on CRT screens and hardcopy reports. Various models are available based on the specific features desired by the customer (e.q., asymmetric jaws, electron pencil beam calculations, etc.) to best meet their clinical needs. The software is designed to run on a PC platform utilizing the Microsoft Windows NT® operating system. The RAPTOR Treatment Planning System includes modules for the development of instructions and controls for multileaf collimators. It is designed to upgradable in both hardware and software features. All dates are four digit numbers so the system is year 2000 compliant.
This looks like a 510(k) premarket notification for a medical device called the RAPTOR® Treatment Planning System. Unfortunately, the provided document does not contain the detailed information required to answer your request about acceptance criteria and the comprehensive study that proves the device meets those criteria.
Here's why and what information is missing:
- No specific acceptance criteria table: The document states that the system "has a product specification, which was used as the basis for the development of verification and validation plans, tests and acceptance criteria." However, it does not explicitly list these acceptance criteria.
- No detailed study results or performance metrics: While it mentions "Validation and verification testing... demonstrate that the software is safe and effective" and that it "performs and is equivalent to the predicate systems," it does not provide any quantitative performance data (e.g., accuracy percentages, precision values, specific metrics like sensitivity, specificity, AUC) or a comparison against predefined acceptance thresholds.
- Missing information on sample size, data provenance, ground truth, experts, MRMC studies, or standalone performance.
Based on the provided text, I cannot complete the table or answer the specific questions about the study design and results. The document focuses on the regulatory submission process, the device's intended use, and its general technological characteristics, rather than the detailed validation study methodology and outcomes.
However, I can extract the following relevant information:
1. A table of acceptance criteria and the reported device performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Not explicitly stated in the provided document. The document mentions "product specification, which was used as the basis for the development of verification and validation plans, tests and acceptance criteria." | "Validation and verification testing of the RAPTOR® Treatment Planning System demonstrate that the software is safe and effective. Test completion shows that the device performs and is equivalent to the predicate systems for those features for which this submission is being made as well as overall performance." |
Missing Information:
- Specific quantitative acceptance criteria (e.g., accuracy within X%, dose calculation deviation within Y%).
- Quantitative results from the verification and validation tests (e.g., actual measured accuracies, deviations, etc.) that demonstrate the device met these criteria.
2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
Missing Information:
- Sample size (number of cases/patients used) for the test set.
- Data provenance (e.g., country of origin, retrospective/prospective nature of the data).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
Missing Information:
- Number of experts.
- Qualifications of experts. This document is about a treatment planning system, so "ground truth" would likely relate to the accuracy of dose calculations or treatment plan parameters, which might be compared against established physics models, phantom measurements, or potentially consensus among medical physicists/radiation oncologists.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
Missing Information:
- Adjudication method for establishing ground truth or resolving discrepancies.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Missing Information:
- There is no indication of an MRMC comparative effectiveness study being performed. This device is a treatment planning system, not an AI-assisted diagnostic tool, so such a study might not be directly applicable in the typical sense of "human readers."
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Missing Information:
- While the validation and verification tests would inherently be "standalone" in evaluating the software's calculations, the specific results and metrics of this standalone performance are not provided. The device focuses on "producing radiation dose estimations" and "output and displays," which implies an algorithm-only component.
7. The type of ground truth used (expert concensus, pathology, outcomes data, etc)
Missing Information:
- The document does not specify the method for establishing "ground truth," which for a radiation therapy planning system would likely involve physical measurements in phantoms, comparison to established analytical models, or potentially simulations.
8. The sample size for the training set
Missing Information:
- The document does not mention a training set, as it describes a system involving "known and documented algorithms" and "table look-up." This suggests a more deterministic, physics-based approach rather than a machine learning model that requires a separate training set.
9. How the ground truth for the training set was established
Missing Information:
- As there's no mention of a training set, this information is not applicable.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.