(57 days)
The Accuray Precision™ Treatment Planning System is indicated for creation and assessment of external photon beam irradiation treatment plans for radiation therapy, stereotactic radiotherapy or stereotactic radiosurgery for lesions, tumors and conditions anywhere in the body when radiation treatment is indicated. A treatment plan provides an estimate of the dose distribution and the parameters utilized by the radiation delivery system. Plans must be reviewed and approved by qualified medical practitioners prior to delivery.
The Accuray Precision™ Treatment Planning System is intended to be used by physicians, medical physicists, and dosimetrists to generate radiation therapy, stereotactic radiotherapy or stereotactic radiosurgery treatment plans. Plans may be created with the Accuray Precision™ treatment planning system for delivery using Intensity Modulated Radiation Therapy (IMRT) or 3-D Conformal Radiation Therapy (3DCRT) techniques.
The users will be able to create a plan that satisfies established clinical objectives. For stereotactic radiosurgery, the plan will generally involve delivering a tumoricidal dose to target tissue, while minimizing dose to other tissues. For radiation therapy and stereotactic radiotherapy, the plan will generally involve delivering a damaging dose to diseased tissue at a level that allows healthy tissue in the target volume to recover, while also minimizing dose to tissue outside the target volume.
The treatment plan with dose distributions and complete delivered dose value along with the input data will be available through a user display or printed report for user review and evaluation against the treatment prescription and established physics models. The treatment plan will then be saved by the user, approved by the qualified medical practitioner, and subsequently delivered by the treatment delivery system.
The Accuray Precision™ Treatment Planning System is a radiation therapy planning system used for creation and assessment of treatment plans for delivery by radiation therapy, stereotactic radiotherapy or stereotactic radiosurgery treatment systems.
The radiation treatment plan is developed by using diagnostic CT (primary image series) and/or secondary images (MR and PET) of the patient previously acquired prior to treatment and saved into the central database server (iDMS™) Data Management System). The images are imported from the database server into the Accuray Precision™ System and registered/ fused for dose calculation and to accurately define/contour regions of interest (target) and the surrounding critical anatomical structures to avoid.
The user (dosimetrist/ medical physician) then specifies the treatment delivery machine (CyberKnife® or TomoTherapy® systems), treatment delivery mode (e.g. 3DCRT, IMRT) and the patient alignment at the treatment machine. This is followed by specifying the radiation dose criteria for the identified regions of interest, as well as the number of fractions over which this dose is to be administered. After the relevant data has been entered, the user initiates the treatment plan calculation or optimization process. When the treatment plan has been calculated, the user may refine the plan with adjustments to the regions of interest, avoidance structures, and dose criteria and re-optimize the plan.
Once an optimized treatment plan is produced that meets the requirements of the intended therapy, the prescribing physician approves the plan for delivery and the plan is saved on the database server as a treatment delivery plan. The plan reports are also printed for the patient record.
This document describes the Accuray Precision™ Treatment Planning System. It appears to be a 510(k) summary submitted to the FDA. The information provided heavily emphasizes comparison to a previously cleared predicate device rather than detailing extensive standalone performance studies for the modified device.
Here's an analysis of the provided text with respect to your questions:
1. A table of acceptance criteria and the reported device performance
The document does not explicitly present a table of acceptance criteria with corresponding device performance values in a quantifiable manner typical for assessing diagnostic or prognostic AI/ML devices. Instead, it focuses on demonstrating equivalence to a predicate device.
The "Performance" section states: "The performance test data for subject device, Accuray Precision™ Treatment Planning System with modifications, confirms that the user will be able to create, save, review and modify treatment plans with the same or higher level of quality as compared to the treatment plans created using the predicate device... The results from testing included in the premarket notification demonstrate that the performance characteristics of the subject device are equivalent to the predicate device."
The table provided under "Technological Characteristics" compares the subject device to the predicate device, highlighting modifications rather than laying out specific performance metrics.
Implicit Acceptance Criteria (based on the provided text's focus):
- Treatment Plan Quality: "same or higher level of quality as compared to the treatment plans created using the predicate device."
- Equivalence: Performance characteristics are "equivalent to the predicate device."
- Safety and Efficacy: Modifications "do not raise new questions of safety or efficacy and the subject device is as safe and as effective as the predicate device."
Reported Device Performance:
The document broadly states that the performance test data confirms the user can create, save, review, and modify treatment plans with the same or higher quality compared to the predicate device, and that its performance characteristics are equivalent.
Specific quantitative metrics (e.g., dose calculation accuracy within X%, contouring agreement with Y%) are not provided in this summary document. The "Dosimetry Tests" mentioned for the predicate are "Absolute Dose and End-to-End (E2E)" and "Gamma index," and the subject device is stated to have "Same as predicate," implying these tests were used to demonstrate equivalence, but the results are not detailed here.
2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
This information is not explicitly provided in the given 510(k) summary. The document mentions "performance test data" but does not specify the sample size of treatment plans or patient cases used for testing, nor the provenance (country, retrospective/prospective nature) of the data.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
This information is not provided in the given 510(k) summary. Ground truth establishment, if involving expert review, is not detailed. The indication for use states that "Plans must be reviewed and approved by qualified medical practitioners prior to delivery," and the intended use specifies "physicians, medical physicists, and dosimetrists" as users. This implies expert involvement in the clinical workflow, but not necessarily in the validation test set's ground truth creation for regulatory submission.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
This information is not provided in the given 510(k) summary.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
This type of study is not mentioned in the document. The device is a "Treatment Planning System," which is a tool used by medical professionals, not explicitly an AI-assisted diagnostic or decision-support system in the sense of image interpretation for diagnosis. The study described focuses on the system's ability to generate treatment plans equivalent to a predicate, not on how human readers' performance improves with or without the system's immediate "assistance" in an MRMC setting.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
The context suggests that the "performance test data" likely evaluated the algorithm's output (treatment plans, dose calculations) in a standalone fashion against expected or established benchmarks (implicitly validated through the predicate device). However, the document doesn't explicitly separate "algorithm only" performance versus system performance with human interaction during the testing phase. The system's purpose is to be used by medical professionals, so "human-in-the-loop" is inherent to its intended use and evaluation. The statement "The user will be able to create, save, review and modify treatment plans..." implies the complete system with user interaction was evaluated.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The document does not explicitly state the type of ground truth used for validating the treatment plan quality. For treatment planning systems, ground truth often involves:
- Physics models/calculations: Comparing computed dose distributions against theoretical phantom measurements or highly accurate dose engines.
- Clinical expert review/consensus: Having qualified medical physicists and physicians review the generated plans for clinical acceptability, often against established treatment protocols or expert-generated "ideal" plans.
- Phantom measurements: Physical measurements (e.g., using ion chambers, film, or arrays) within phantoms to verify dose delivery accuracy.
The mention of "Dosimetry Tests" (Absolute Dose, End-to-End, Gamma index) strongly suggests that physical phantom measurements and comparisons against established physics models were a primary component of the ground truth for dose calculation accuracy.
8. The sample size for the training set
This information is not provided in the 510(k) summary. Medical charged-particle radiation therapy systems (and their planning software) typically rely on complex physics models and algorithms, not necessarily on "training sets" in the machine learning sense for their core dose calculation engine, though components might use data-driven approaches. If any such algorithms were used, their training details are not disclosed here.
9. How the ground truth for the training set was established
As the sample size and nature of a "training set" are not discussed, the method for establishing its ground truth is also not provided.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.