(139 days)
Oncospace is used to configure and review radiotherapy treatment with malignant or benign disease in the prostate, head, and neck regions. It allows for set up of radiotherapy treatment protocols, association of a potential treatment plan with the protocol(s), submission of a dose prescription and achievable dosimetric goals to a treatment planning system, and review of the treatment plan. It is intended for use by qualified, trained radiation therapy professionals (such as medical physicists, oncologists, and dosimetrists). This device is for prescription use by order of a physician.
The Oncospace software supports radiation oncologists and medical dosimetrists during radiotherapy treatment planning. The software includes locked machine learning algorithms. During treatment planning, the Oncospace software works in conjunction with, and does not replace, a treatment planning system (TPS). The Oncospace software is intended to augment the treatment planning process by: allowing the radiation oncologist to select and customize a treatment planning protocol that includes dose prescription (number of fractions, dose normalization), a delivery method (beam type and geometry), and protocol-based dosimetric goals/objectives for treatment targets, and organs at risk (OAR); predicting dosimetric goals/objectives for OARs based on patient-specific anatomical geometry; automating the initiation of plan optimization on a TPS by supplying the dose prescription, delivery method, protocol-based target objectives, and predicted OAR objectives; providing a user interface for plan evaluation against protocol-based and predicted goals. Diagnosis and treatment decisions occur prior to treatment planning and do not involve Oncospace. Decisions involving Oncospace are restricted to setting of dosimetric goals for use during plan optimization and plan evaluation. Human judgement continues to be applied in accepting these goals and updating them as necessary during the iterative beam optimization process. Human judgement is also still applied as in standard practice during plan quality assessment; the protocol-based OAR goals as the primary means of plan assessment, with the role of the predicted goals being to provide additional information as to whether dose to an OAR may be able to be further lowered. When Oncospace is used in conjunction with a TPS, the user retains full control of the TPS, including finalization of the treatment plan created for the patient. Oncospace also does not interface with the treatment machines. The risk to patient safety is lower than a TPS since it only informs the treatment plan, does not allow region of interest editing, does not make treatment decisions, and does not interface directly with the treatment machine or any record and verify system. Oncospace's OAR dose prediction approach, and the use of predictions in end-to-end treatment planning workflow, has been tested for use with a variety of cancer treatment plans. These included a wide range of target and OAR geometries, prescriptions and boost strategies (sequential and simultaneous delivery). Validity has thus been demonstrated for the range of prediction model input features encountered in the test cases. This range is representative of the diversity of the same feature types (describing target-OAR proximity, target and OAR shapes, sizes, etc.) encountered across all cancer sites. Given that the same feature types will be used in OAR dose prediction models trained for all sites, the modeling approach validated here is not cancer site specific, but rather is designed to predict OAR DVHs based on impactful features common to all sites. The software is designed to be used in the context of all forms of intensity-modulated photon beam radiotherapy. The planning objectives themselves are intended to be TPS-independent: these are instead dependent on the degree of organ sparing possible given the beam modality and range of delivery techniques for plans in the database. To facilitate streamlined transmission of DICOM files and plan parameters Oncospace includes scripts using the treatment planning system's scripting language (for example, Pinnacle).
Here's a breakdown of the acceptance criteria and study details for the Oncospace device, based on the provided document:
Acceptance Criteria and Device Performance
The document describes the validation testing as demonstrating "non-inferiority of mean organ-at-risk (OAR) dose sparing" and maintaining "target coverage." While specific numerical acceptance criteria are not explicitly presented in a table format in the provided text, the reported performance is detailed against the implicit goal of non-inferiority.
Table of Acceptance Criteria (Implicit) and Reported Device Performance
| Acceptance Criteria (Implicit) | Reported Device Performance |
|---|---|
| Non-inferiority of mean OAR dose sparing (vs. traditional planning workflow) for prostate cancers. Non-inferiority margin: 10 Gy (originally estimated). | The trial demonstrated non-inferiority of mean OAR dose to 5 Gy for prostate. Oncospace plan mean dose was statistically significantly lower for 2 OARs for prostate (indicating superior sparing). No statistically significant differences in mean dose for the remaining 5 OARs for prostate. |
| Non-inferiority of mean OAR dose sparing (vs. traditional planning workflow) for head & neck cancers. Non-inferiority margin: 10 Gy (originally estimated). | The trial demonstrated non-inferiority of mean OAR dose to 8 Gy for head & neck. Oncospace plan mean dose was statistically significantly lower for 1 OAR for head & neck (indicating superior sparing). No statistically significant differences in mean dose for the remaining 26 OARs for head & neck. |
| Maintenance of target coverage (vs. traditional planning workflow). | No statistically significant difference in target coverage between clinical plans and plans created with use of the Oncospace system. |
| Reduction or maintenance of the number of plan optimizations required to arrive at a clinically acceptable plan. | Out of all the plans tested, no plan required more optimizations using Oncospace versus using traditional radiation treatment planning clinical workflow. The average number of iterations was reduced by 77% using Oncospace. |
| All system requirements and acceptance criteria for clinical, standard user interface, and cybersecurity. | The verification tests met all system requirements and acceptance criteria. (Specific details of these criteria are not provided for inclusion in the table, but the document states they were met). |
Study Details
-
Sample sizes used for the test set and the data provenance:
- Prostate cancer: 13 retrospective, heterogenous, traditionally planned radiation treatment plans.
- Head & neck cancer: 19 retrospective, heterogenous, traditionally planned radiation treatment plans.
- Data Provenance: Retrospective clinical data from an unspecified country (likely USA, given the FDA submission, but not explicitly stated).
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document implies that "traditional radiation treatment plans" served as the reference for comparison, effectively the "ground truth" for OAR sparing and target coverage.
- The "qualified, trained radiation therapy professionals (such as medical physicists, oncologists, and dosimetrists)" are mentioned as the intended users and presumably responsible for creating and validating these traditional plans, thus establishing their clinical acceptability. However, the exact number of experts involved in establishing the ground truth for this specific test set, or their specific qualifications, is not explicitly stated.
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- The document does not describe an explicit adjudication method for the test set. The ground truth ("traditional radiation treatment plans") is assumed to be clinically established by the "qualified, trained radiation therapy professionals" who created them. The comparison is between the performance of Oncospace-assisted plans and these established traditional plans, rather than a re-adjudication of the traditional plans themselves.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- This was not a MRMC comparative effectiveness study in the typical sense of evaluating human reader performance with and without AI assistance on diagnostic tasks.
- The study compared the dosimetric outcomes and planning efficiency of plans generated with Oncospace assistance versus traditionally planned cases.
- The "improvement" is in the efficiency of the planning process (77% reduction in iterations) and the non-inferiority (and sometimes superiority) of OAR sparing with Oncospace, when compared to traditional planning methods. It's a system-level comparison rather than a human-reader performance comparison directly.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- No, this was not a standalone algorithm-only study. Oncospace "does not replace, a treatment planning system (TPS)" and is explicitly designed to "augment the treatment planning process." The study compared "plans created with use of the Oncospace system" against "traditional plans," implying human-in-the-loop for both scenarios (one assisted by Oncospace, the other by traditional workflow).
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- The ground truth for the dosimetric comparisons (OAR doses and target coverage) were the retrospective, traditionally planned radiation treatment plans which were clinically established as acceptable. This implicitly represents expert consensus through the standard clinical planning process.
- The ground truth for planning efficiency was the number of optimizations required for the traditional plans, compared to those with Oncospace.
-
The sample size for the training set:
- Prostate cancer: 1336 treatment plans were used for model development and internal validation (split 80/20).
- Head & neck cancer: 796 treatment plans were used for model development and internal validation (split 80/20).
-
How the ground truth for the training set was established:
- The document states that the training data consisted of "treatment plans." These would typically be clinically approved and delivered (or deliverable) radiation treatment plans, implying that their dosimetric characteristics (including OAR doses and target coverage) were established as acceptable by qualified medical professionals (radiation oncologists, medical physicists, dosimetrists) during their routine clinical use. The "gold-standard treatment plans" are mentioned in a different context (dose objective comparison), but this concept likely extends to the training data as well.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.