(56 days)
The XiO RTP System is used to create treatment plans for any cancer patient for whom external beam radiation therapy or brachytherapy has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.
The XiO Radiation Treatment Planning system accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume, which is the structure to be treated, and critical structures, or organs-at-risk, to which radiation dose must be limited. Based on the dose prescribed, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The XiO system produces a display of radiation dose distribution within the patient, indicating doses to the target volume and critical structures. Appropriate clinical personnel select the plan that they believe most effectively maximizes dose to the target volume while minimizing dose to critical structures. The parameters of the plan are output in hard-copy format for later reference placed in the patient file. This Premarket Notification addresses the addition of the Proton Spot Scanning. XiO provides the user with the ability to choose between multiple dose calculation algorithms, selecting the algorithm most appropriate for the given clinical scenario.
The provided K102216 submission for the XiO RTP System with Proton Spot Scanning focuses on the safety and effectiveness of a radiation treatment planning system. Therefore, the "acceptance criteria" and "device performance" in this context refer to the accuracy of the dose calculation algorithm and the successful execution of verification tests, rather than typical clinical performance metrics like sensitivity, specificity, or AUC which are common for diagnostic AI devices.
Here's an analysis of the acceptance criteria and study information provided:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria Category | Specific Criteria / Test Type | Reported Device Performance |
|---|---|---|
| Dose Calculation Accuracy | Comparison of calculated vs. measured doses. | Algorithm testing performed to ensure dose calculation accuracy. (Implies successful comparison, though specific metrics not detailed.) |
| System Functionality | Verification tests (Pass/Fail requirements for system working as designed). | XiO successfully passed verification testing. |
| Clinical Suitability | Clinically oriented validation test cases, executed in-house. | Product deemed fit for clinical use. |
2. Sample Size Used for the Test Set and Data Provenance
The document states:
- "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy."
- "Clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."
This indicates the test sets were synthetically created or derived from experimental measurements in a lab setting (for algorithm performance) and internal validation cases rather than patient data.
- Sample Size: Not explicitly stated for either the algorithm testing or the clinically oriented validation test cases. It is implied there were sufficient cases to validate the algorithms and system functionality.
- Data Provenance: The data for algorithm testing would likely be from physical measurements in a lab (e.g., phantom studies) against which the calculated doses are compared. The "clinically oriented validation test cases" were "written and executed in-house" by the manufacturer (CMS customer support personnel), suggesting simulated clinical scenarios or predefined test inputs mirroring real-world conditions, rather than primary patient data.
- Retrospective or Prospective: Both types of testing (algorithm and validation test cases) are described as retrospective analyses or internal validation exercises on predefined scenarios/data, not prospective studies on real patients.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- Number of Experts: Not explicitly stated.
- Qualifications of Experts: The ground truth for dose calculation accuracy would be established by dosimetrists or medical physicists who perform the physical measurements of radiation dose distributions in a lab setting. The "clinically oriented validation test cases" were executed by "CMS customer support personnel," which might include individuals with dosimetric or clinical application knowledge, but their specific qualifications are not detailed beyond "customer support personnel." Given the "Major Level of Concern" for this device, a qualified medical physicist would likely have overseen or been involved in the interpretation of algorithm accuracy.
4. Adjudication Method for the Test Set
Not applicable in the conventional sense for a typical AI diagnostic device. The "ground truth" for this device is the measured physical dose or the correct output based on system specifications for verification tests. Discrepancies would be resolved by re-measurement, re-analysis, or debugging, not by expert consensus adjudication of human interpretation.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size
No. An MRMC study is not relevant for this type of device (a radiation treatment planning system). The device assists human readers (dosimetrists/medical physicists) in planning treatments but does not present images for interpretation in a diagnostic context. Its primary function is calculation and display of dose distributions.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, in essence. The "algorithm testing" performed to compare calculated against measured doses is a standalone performance assessment of the core dose calculation engine. This evaluates the algorithm's accuracy independent of a human user's input or interpretation after the calculation.
7. The Type of Ground Truth Used
- Algorithm Testing: Measured physical dose distributions (e.g., from phantom studies, ion chamber measurements, film dosimetry). This is a form of empirical measurement/experimental data.
- Verification Tests: The expected correct system behavior and output as defined by the system's design specifications. This can be considered definitive system specification ground truth.
- Clinically Oriented Validation Test Cases: Predefined correct treatment plans or expected outcomes based on established clinical practice and physics principles. This combines elements of expert consensus (on what constitutes a correct plan) and physics-based ground truth.
8. The Sample Size for the Training Set
Not applicable. This document describes a traditional software upgrade to a radiation treatment planning system, not a machine learning or AI algorithm that requires a "training set" in the common sense. The "training" of such a system involves the development and calibration of physics-based dose calculation algorithms against physical models and experimental data, not supervised learning from a dataset of labeled clinical cases.
9. How the Ground Truth for the Training Set Was Established
Not applicable. As noted above, this is not an AI/ML device that uses a "training set" in the context of supervised learning. The underlying physics models and algorithms are developed based on established scientific principles, physical measurements, and mathematical formulations, which constitute their "ground truth" or foundational knowledge.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.