(14 days)
RayStation is a software system designed for treatment planning and analysis of radiation therapy.
The treatment plans provide treatment unit set-up parameters and estimates of dose distributions expected during the proposed treatment, and may be used to administer treatments after review and approval by the intended user.
The system functionality can be configured based on user needs.
The intended users of RayStation shall be clinically qualified radiation therapy staff trained in using the system.
RayStation is a treatment planning system, i.e. a software program for planning and analysis of radiation therapy plans. Typically, a treatment plan is created by importing patient images obtained from a CT scanner, defining regions of interest either manually or semi-automatically, deciding on a treatment setup and objectives, optimizing the treatment parameters, comparing rival plans to find the best compromise, computing the clinical dose distribution, approving the plan and exporting it.
The provided document is a 510(k) summary for RayStation 1.0, a radiation treatment planning system. It describes the device, its intended use, and its substantial equivalence to predicate devices. However, it does not contain detailed acceptance criteria and a study proving the device meets those criteria in the typical format of a device performance study with quantitative metrics like sensitivity, specificity, or AUC, and associated confidence intervals.
Instead, the closest information regarding "acceptance criteria" and "proof" of meeting them is found in section 5.11:
"The dose algorithm in both RayStation 1.0 and RayAutoplan 1.0 is the same. This is supported by the dose algorithm accuracy testing, which has used the same test specification 1.0 as was previously used for Ray Autoplan 1.0. The tests include dose calculation for a wide variety of field geometries, treatment units, treatment setups and patient positions, including different dose grid resolution settings. All tests were run successfully for RayStation 1.0."
Based on this, here's an attempt to answer your questions, highlighting where information is not present in the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Dose algorithm accuracy for a wide variety of field geometries. | The dose algorithm accuracy testing, using the same test specification as Ray Autoplan 1.0, included tests for "a wide variety of field geometries." All tests were run successfully for RayStation 1.0. (No specific quantitative performance metrics like percentage error or gamma index are provided in the summary, just a qualitative "successfully.") |
| Dose algorithm accuracy for various treatment units. | The dose algorithm accuracy testing included tests for "treatment units." All tests were run successfully for RayStation 1.0. |
| Dose algorithm accuracy for diverse treatment setups. | The dose algorithm accuracy testing included tests for "treatment setups." All tests were run successfully for RayStation 1.0. |
| Dose algorithm accuracy for different patient positions. | The dose algorithm accuracy testing included tests for "patient positions." All tests were run successfully for RayStation 1.0. |
| Dose algorithm accuracy for different dose grid resolutions. | The dose algorithm accuracy testing included tests for "different dose grid resolution settings." All tests were run successfully for RayStation 1.0. |
| Functional workflow (e.g., import, ROI creation, plan generation). | The detailed workflow steps (import data, define ROIs, create plan, optimize, review, export) imply successful execution and system response as described (e.g., "The system imports the data and checks consistency," "The system adds the ROIs," "The system generates a deliverable step-and-shoot plan"). No quantitative performance metrics are provided. |
| Device functions as described for treatment planning and analysis. | The device's description and intended use imply that it performs as a treatment planning system producing set-up parameters and dose distributions. The 510(k) clearance indicates the FDA found it substantially equivalent to predicates for this purpose. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: Not explicitly stated. The document refers to "a wide variety of field geometries, treatment units, treatment setups and patient positions, including different dose grid resolution settings" within the dose algorithm accuracy testing. It does not quantify the number of cases or specific configurations tested.
- Data Provenance: Not explicitly stated. Given it's a software system for radiation treatment planning, the "data" would likely be simulated or phantom-based data used for dose calculation comparison, rather than patient data in the traditional sense of clinical studies. It's likely software-generated or phantom-measurement driven.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
Not applicable/Not mentioned. For a dose calculation algorithm, "ground truth" is typically established by physical measurements (e.g., ionization chambers, film dosimetry in phantoms) or highly-validated reference dose calculation algorithms, rather than by expert human consensus on image interpretation. The document does not specify how the "test specification" for dose algorithm accuracy was developed or validated, nor if expert review was part of establishing comparison standards.
4. Adjudication Method for the Test Set
Not applicable/Not mentioned. Adjudication methods like 2+1 or 3+1 typically refer to human expert disagreement resolution in diagnostic tasks. For software validation (especially a dose calculation engine), "adjudication" would refer to how discrepancies between the software's output and the reference truth were handled, but this is not detailed. The phrase "All tests were run successfully" suggests a pass/fail criterion was met.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done
No. This type of study (MRMC) is typically performed for diagnostic devices where human readers interpret images. RayStation is described as a treatment planning system, where the primary function is to calculate and visualize dose distributions, and assist in planning. There is no mention of a human-in-the-loop diagnostic task where AI improves reader performance.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, implicitly. The "dose algorithm accuracy testing" described in section 5.11 is a standalone performance evaluation of the core dose calculation engine. The statement "All tests were run successfully for RayStation 1.0" indicates this standalone algorithm performance met the established (though unspecified) criteria.
7. The Type of Ground Truth Used
The ground truth for the dose algorithm accuracy testing would most likely be established by:
- Physical measurements: Using phantoms and dosimetric equipment (e.g., ion chambers, film, diode arrays) to measure actual dose distributions in a controlled environment.
- Reference dose calculations: Comparing RayStation's calculated dose to a highly validated, clinically accepted "gold standard" dose calculation engine or analytical solutions for simple geometries.
The document does not explicitly state which method was used, but these are standard practices for validating such systems. It does not refer to expert consensus, pathology, or outcomes data.
8. The Sample Size for the Training Set
Not applicable/Not mentioned. RayStation 1.0 is described as a treatment planning system with a dose calculation algorithm (collapsed cone). This type of algorithm is based on physical models of radiation interaction and transport, not typically a machine learning algorithm that requires a "training set" in the conventional sense. Therefore, the concept of a "training set" for an AI algorithm doesn't directly apply here.
9. How the Ground Truth for the Training Set was Established
Not applicable, as there's no indication of a machine learning-based "training set." If the system incorporated any machine learning components (which is not explicitly stated for RayStation 1.0 in this document, particularly not for its core dose calculation), then the ground truth for that specific component's training would need to be addressed. However, based solely on the provided text, this question is not relevant.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.