(109 days)
Varian Treatment is designed to assist the operator of a radiation therapy device by retrieving treatment plans from the Varian System Database for ARIA® Radiation Therapy Management, by providing accurate treatment setups, by monitoring setup parameters, by preventing treatment when machine parameters are out of conformance to treatment plan parameters, and by sending the treatment history for recording to the Varian System Database.
Varian Treatment (VTx) is a treatment delivery management application that performs an interface role between the ARIA® Oncology Information System for Radiation Oncology (ARIA OIS RO); ARIA® Radiation Therapy Management System (ARIA RTM), and an external radiation therapy delivery system (e.g., medical linear accelerator). The contents of this submission are applicable to version 13.0 of Varian Treatment.
The device does not contain any biologics or drug components. The device is not intended for single use. No parts of the system are provided sterile or are intended for sterilization. There are no patient-contacting materials. Varian Treatment is operated by radiation therapy therapists in accordance with the prescription of a radiation oncologist and under the general supervision of a chief therapist and medical physicist. Healthcare/treatment facilities are the intended environment of use.
The associated hardware for Varian Treatment consists of:
- Single workstation
- Two 27-inch monitors
- Mouse
- Keyboard
- In-Room Keypad
- MICAP
- KVM Switch
- Display Port Switcher
- Connecting Cables (Network, Serial, and Fiber)
- Barcode Reader (Optional Accessory)
The VTx workstation is the computer that hosts the system software described in this section. This workstation uses a mouse and keyboard for the operator's control in the treatment control room. Varian Treatment is connected over the institution's corporate network to the Varian system database via MICAP (firewall). Varian Treatment uses an inroom monitor and kevpad to allow the operator certain functions from the treatment room. Barcode readers are connected to the VTx workstation for use with either PAVS or VVS when licensed (optional); these barcode readers are used for patient and accessory verification. The Display Port Switcher allows for switching between the In-Room monitor application and the Varian Treatment main application from the treatment room when PAVS or VVS is licensed. The KVM Switch and other Connecting Cables are used in connecting the in-room monitor to the VTx workstation.
The main software application for Varian Treatment provides functionality to load a patient session from the Varian System database, validates patient data, sends data to the treatment control system for the linear accelerator, verifies actual parameters versus planned parameters, receives treatment history and saves it to the Varian System database. VTx Administration is the administrative application for the main application, allowing a user to view, modify, and save configuration settings, user and group management, and group rights management for the VTx application. The Multivendor DICOM Service (MVDS) application acts between the main application and the external vendor's imaging system in supported, compatible devices. The VTx system software operates on a Microsofte Windows® operating system on the workstation.
The provided document is a 510(k) premarket notification for a medical device called Varian Treatment (VTx), which is software designed to manage radiation therapy treatment delivery. The document outlines the device's indications for use, comparison to a predicate device, and performance data used to establish substantial equivalence.
However, the document does not contain the specific information requested about acceptance criteria, detailed study results for device performance (especially in a clinical or multi-reader multi-case setting), sample sizes for test/training sets, expert qualifications for ground truth establishment, or adjudication methods. The performance data section primarily discusses non-clinical tests like software verification and validation, with explicit statements that no animal studies or clinical tests were included.
Therefore, many of the requested fields cannot be filled from the provided text.
Here's what can be extracted and what cannot:
1. A table of acceptance criteria and the reported device performance
The document does not provide a table with explicit acceptance criteria (e.g., minimum accuracy, sensitivity, specificity) and corresponding reported device performance in a quantitative sense as might be expected for an AI/ML diagnostic system. Instead, it states that "The result of the software verification and validation testing was that the product conformed to the defined user needs, intended uses, and safety requirements." This is qualitative and refers to general software functionality rather than specific performance metrics against a clinical ground truth.
2. Sample sized used for the test set and the data provenance
The document states that "Software verification and validation testing were conducted and documentation was provided." It does not specify the sample size of data used for this testing, nor does it mention the data provenance (e.g., country of origin, retrospective/prospective). Since no clinical studies were performed, there isn't a "test set" in the sense of patient data.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
Not applicable. The ground truth for software verification and validation tests would be the expected output or behavior of the software based on its design specifications, not expert consensus on medical images or diagnoses. No clinical test set requiring expert ground truth was mentioned.
4. Adjudication method for the test set
Not applicable. No clinical test set, therefore no adjudication method for establishing ground truth.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
The document explicitly states: "No animal studies or clinical tests have been included in this submission in support of the substantial equivalence determination." Therefore, no MRMC study was performed.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
The software (VTx) is described as "designed to assist the operator of a radiation therapy device" and "performs an interface role." It is inherently a human-in-the-loop system (an assistant tool for operators), not a standalone diagnostic algorithm. The performance evaluation focused on its functional correctness and safety as a software component, not its standalone diagnostic or treatment effectiveness.
7. The type of ground truth used
For the software verification and validation, the ground truth was the defined user needs, intended uses, and safety requirements of the software. This is a functional truth, not a clinical truth based on pathology, expert consensus, or outcomes data.
8. The sample size for the training set
Not applicable. This device is described as software that manages treatment delivery, verifies parameters, and sends data. It's not an AI/ML model that would typically have a distinct "training set" in the sense of learning from large datasets of clinical images or patient outcomes. The software development process would involve coding and internal testing, not machine learning training.
9. How the ground truth for the training set was established
Not applicable, as there's no mention of a traditional "training set" for an AI/ML model.
Summary Table of Available Information based on the provided document:
| Acceptance Criteria Aspect | Information from provided document |
|---|---|
| 1. Acceptance Criteria & Reported Device Performance | Acceptance Criteria: "conformed to the defined user needs, intended uses, and safety requirements." |
| Reported Performance: "The result of the software verification and validation testing was that the product conformed to the defined user needs, intended uses, and safety requirements." (Qualitative statement, no quantitative metrics provided like accuracy, sensitivity, specificity, etc.) | |
| 2. Test Set Sample Size & Data Provenance | Sample Size: Not specified. |
| Data Provenance: Not specified. No clinical test set mentioned. The testing was for software verification and validation. | |
| 3. Experts for Ground Truth & Qualifications | Not applicable. No clinical test set requiring expert ground truth. |
| 4. Adjudication Method for Test Set | Not applicable. |
| 5. MRMC Comparative Effectiveness Study Performed? Effect Size? | No. "No animal studies or clinical tests have been included." |
| 6. Standalone Performance Study Performed? | Not applicable. This is a human-in-the-loop assistance device, not a standalone diagnostic algorithm. Performance focused on functional correctness as software. |
| 7. Type of Ground Truth Used | Defined user needs, intended uses, and safety requirements of the software. |
| 8. Training Set Sample Size | Not applicable. No AI/ML training set mentioned. |
| 9. How Training Set Ground Truth Established | Not applicable. |
In conclusion, the provided FDA 510(k) document for Varian Treatment (VTx) focuses on demonstrating substantial equivalence primarily through non-clinical software verification and validation. It explicitly states that no animal or clinical studies were performed. Therefore, the detailed performance metrics, test/training set descriptions, and human expert involvement typically associated with AI/ML device evaluations (as implied by the requested questions) are not present in this document. The "proof" relies on the software meeting its specified design requirements and being functionally equivalent to its predicate.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.