Search Results

The XiO RTP System is used to create treatment plans for any cancer patient for whom external beam radiation therapy or brachytherapy has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

The XiO Radiation Treatment Planning system accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume, which is the structure to be treated, and critical structures, or organs-at-risk, to which radiation dose must be limited. Based on the dose prescribed, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The XiO system produces a display of radiation dose distribution within the patient, indicating doses to the target volume and critical structures. Appropriate clinical personnel select the plan that they believe most effectively maximizes dose to the target volume while minimizing dose to critical structures. The parameters of the plan are output in hard-copy format for later reference placed in the patient file. This Premarket Notification addresses the addition of the Proton Spot Scanning. XiO provides the user with the ability to choose between multiple dose calculation algorithms, selecting the algorithm most appropriate for the given clinical scenario.

The provided K102216 submission for the XiO RTP System with Proton Spot Scanning focuses on the safety and effectiveness of a radiation treatment planning system. Therefore, the "acceptance criteria" and "device performance" in this context refer to the accuracy of the dose calculation algorithm and the successful execution of verification tests, rather than typical clinical performance metrics like sensitivity, specificity, or AUC which are common for diagnostic AI devices.

Here's an analysis of the acceptance criteria and study information provided:

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

The document states:

• "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy."
• "Clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

This indicates the test sets were synthetically created or derived from experimental measurements in a lab setting (for algorithm performance) and internal validation cases rather than patient data.

• Sample Size: Not explicitly stated for either the algorithm testing or the clinically oriented validation test cases. It is implied there were sufficient cases to validate the algorithms and system functionality.
• Data Provenance: The data for algorithm testing would likely be from physical measurements in a lab (e.g., phantom studies) against which the calculated doses are compared. The "clinically oriented validation test cases" were "written and executed in-house" by the manufacturer (CMS customer support personnel), suggesting simulated clinical scenarios or predefined test inputs mirroring real-world conditions, rather than primary patient data.
• Retrospective or Prospective: Both types of testing (algorithm and validation test cases) are described as retrospective analyses or internal validation exercises on predefined scenarios/data, not prospective studies on real patients.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• Number of Experts: Not explicitly stated.
• Qualifications of Experts: The ground truth for dose calculation accuracy would be established by dosimetrists or medical physicists who perform the physical measurements of radiation dose distributions in a lab setting. The "clinically oriented validation test cases" were executed by "CMS customer support personnel," which might include individuals with dosimetric or clinical application knowledge, but their specific qualifications are not detailed beyond "customer support personnel." Given the "Major Level of Concern" for this device, a qualified medical physicist would likely have overseen or been involved in the interpretation of algorithm accuracy.

4. Adjudication Method for the Test Set

Not applicable in the conventional sense for a typical AI diagnostic device. The "ground truth" for this device is the measured physical dose or the correct output based on system specifications for verification tests. Discrepancies would be resolved by re-measurement, re-analysis, or debugging, not by expert consensus adjudication of human interpretation.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size

No. An MRMC study is not relevant for this type of device (a radiation treatment planning system). The device assists human readers (dosimetrists/medical physicists) in planning treatments but does not present images for interpretation in a diagnostic context. Its primary function is calculation and display of dose distributions.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, in essence. The "algorithm testing" performed to compare calculated against measured doses is a standalone performance assessment of the core dose calculation engine. This evaluates the algorithm's accuracy independent of a human user's input or interpretation after the calculation.

7. The Type of Ground Truth Used

• Algorithm Testing: Measured physical dose distributions (e.g., from phantom studies, ion chamber measurements, film dosimetry). This is a form of empirical measurement/experimental data.
• Verification Tests: The expected correct system behavior and output as defined by the system's design specifications. This can be considered definitive system specification ground truth.
• Clinically Oriented Validation Test Cases: Predefined correct treatment plans or expected outcomes based on established clinical practice and physics principles. This combines elements of expert consensus (on what constitutes a correct plan) and physics-based ground truth.

8. The Sample Size for the Training Set

Not applicable. This document describes a traditional software upgrade to a radiation treatment planning system, not a machine learning or AI algorithm that requires a "training set" in the common sense. The "training" of such a system involves the development and calibration of physics-based dose calculation algorithms against physical models and experimental data, not supervised learning from a dataset of labeled clinical cases.

9. How the Ground Truth for the Training Set Was Established

Not applicable. As noted above, this is not an AI/ML device that uses a "training set" in the context of supervised learning. The underlying physics models and algorithms are developed based on established scientific principles, physical measurements, and mathematical formulations, which constitute their "ground truth" or foundational knowledge.

Ask a specific question about this device

The XiO RTP System is used to create treatment plans for any cancer patient for whom external beam radiation therapy or brachytherapy has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

The XiO Radiation Treatment Planning system accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume, which is the structure to be treated, and critical structures, or organs-atrisk, to which radiation dose must be limited.

Based on the dose prescribed, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The XiO system produces a display of radiation dose distribution within the patient, indicating doses to the target volume and critical structures. Appropriate clinical personnel select the plan that they believe most effectively maximizes dose to the target volume while minimizing dose to critical structures. The parameters of the plan are output in hard-copy format for later reference placed in the patient file.

This Premarket Notification addresses the addition of the Electron Monte Carlo dose calculation algorithm. XiO provides the user with the ability to choose between multiple dose calculation algorithms, selecting the algorithm most appropriate for the given clinical scenario. More accurate dose computation increases the probability that disease will be effectively treated and decreases the probability of undesirable side effects. No algorithm produces a perfectly accurate description of dose distribution; all algorithms have limitations, which are generally well understood and documented in scientific literature.

The addition of the Monte Carlo dose calculation algorithm gives users a new option for electron treatment plans. The algorithm represents the state of the art in radiation treatment planning and is widely recognized as the most accurate method currently available for computing the dose delivered by a beam of high-energy electrons.

The provided text describes the XiO RTP System with the addition of an Electron Monte Carlo dose calculation algorithm. While it mentions verification testing and algorithmic accuracy, it does not explicitly define acceptance criteria in a quantitative table or detail a study that directly proves the device meets such criteria through a clinical or reader study.

Here's a breakdown of the available information against your requested points:

1. Table of Acceptance Criteria and Reported Device Performance

The document does not provide a formal table of acceptance criteria or specific quantitative performance metrics beyond stating that XiO "successfully passed verification testing" and that algorithmic testing was performed to "ensure dose calculation accuracy." No specific numerical targets for accuracy or precision are given.

2. Sample Size Used for the Test Set and Data Provenance

The document states:

• "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy."
• "In addition, clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

The specific sample size for this "algorithm testing" or these "validation test cases" is not mentioned.
The data provenance (e.g., country of origin, retrospective or prospective) for this testing is not specified. It's implied to be internal data ("in-house by CMS customer support personnel") rather than real-world patient data from specific countries.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• Number of Experts: The document does not specify a number of experts involved in establishing ground truth for the test set. It mentions "qualified clinicians" review plans and "CMS customer support personnel" executed validation test cases, but it doesn't detail their role in establishing ground truth for an independent test set.
• Qualifications of Experts: It refers to "qualified clinicians" and "Dosimetrist or Medical Physicist" as typical users of the system. However, it does not specify the qualifications (e.g., years of experience, board certification) of individuals who might have established ground truth for testing.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for a test set. The testing mentioned appears to be primarily algorithm verification against measured doses and internal validation cases, not a process involving multiple human reviewers resolving discrepancies.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study was done. The document explicitly states: "Clinical trials were not performed as part of the development of this product." It focuses on algorithm accuracy and internal validation.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

Yes, a standalone study was done, but not in a formal clinical sense. The "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy" is a standalone evaluation of the algorithm's performance against a physical standard. The "clinically oriented validation test cases" also represent an evaluation of the algorithm's output without direct human-in-the-loop performance measurement that would typically be seen in a reader study.

7. Type of Ground Truth Used

The primary type of "ground truth" implied for the algorithm testing is measured doses. This means physical measurements of radiation dose distribution, likely obtained from phantoms or experimental setups, were used as the reference standard against which the Monte Carlo algorithm's calculated doses were compared. The "clinically oriented validation test cases" likely involved scenarios with expected or known outcomes based on physics principles or established clinical practices, rather than pathology or patient outcomes data.

8. Sample Size for the Training Set

The document does not mention a discrete training set sample size. The Monte Carlo algorithm is a physics-based simulation method rather than a machine learning model that would typically have a "training set" in the conventional sense. Its development would involve calibrating physical parameters and validating the underlying physics models, rather than training on a dataset of examples.

9. How the Ground Truth for the Training Set Was Established

As the Monte Carlo algorithm is not a typical machine learning model trained on a "training set" of data, the concept of establishing ground truth for a training set in this context is not applicable or described. The "ground truth" for the development of such an algorithm would be based on fundamental physics principles, experimentally validated cross-sections, and particle interaction data, rather than annotated clinical cases.

Ask a specific question about this device