(115 days)
The intended purpose of the Diamond Edition of Setup 2000 Version 3.0 software is to provide an accurate and convenient method of independently checking, on a point dose basis, the treatment planning calculations of the primary radiation treatment-planning (RTP) computer. The primary RTP computer is capable of importing CT images of the patient, identifying target volumes, superimposing the treatment beams and optimizing the weighting and modulation of the beam to produce an acceptable radiation distribution within the patient. The Diamond Edition of Setup 2000 Version 3.0 calculates point doses under specified conditions communicated to the software through operator interaction (keyboard and mouse) or through file import.
The Diamond Edition of Setup 2000 Version 3.0 software program is designed to operate in a Microsoft Windows environment either on a sinqle PC or on a workstation with the treatment machine and patient data shared on a server. This software does not control any device that delivers radiation to the patient. This software does not communicate directly with the treatment machine, the primary treatment planning (RTP) computer or the record and verify system. This software does have the ability to transfer treatment data files in a standardized format (e.g. RTP Connectivity Protocol by IMPAC) from the primary treatment planning computer or R&V system to facilitate the entry of the data to the software and the transfer of data to the R&V system for electronic charting. The software calculates the treatment machine monitor units (MU or time as appropriate) or dose to validate the values calculated by the RTP computer or to provide the settings for simple treatments not requiring the RTP plan prescribed by the physician. The software provides these calculations using methods and models published in the professional literature, the specific treatment information provided by the user and the values in a treatment machine database of measured physical parameters entered and validated by the medical physicist. The results of calculations are documented in printed or electronic reports to be placed in the patient's chart.
This document does not contain explicit acceptance criteria tables or detailed study results proving the device meets specific performance metrics in the way a typical medical device validation study report would. Instead, it relies on historical performance, comparison to predicate devices, and internal testing processes.
However, based on the provided text, we can infer some aspects of intended performance and the general approach to demonstrating safety and effectiveness.
Here's an attempt to structure the information based on the prompt's requirements, with explicit statements where information is present and noting where it's absent:
Acceptance Criteria and Device Performance Study for K&S Associates, Inc. Diamond Edition of Setup 2000 Version 3.0
1. Table of Acceptance Criteria and Reported Device Performance:
| Acceptance Criteria (Inferred from Text) | Reported Device Performance (Inferred/Stated) |
|---|---|
| Accuracy of MU/Dose Calculation | "The software calculates the treatment machine monitor units (MU or time as appropriate) or dose to validate the values calculated by the RTP computer or to provide the settings for simple treatments." |
| "Diamond produces more accurate IMRT dose results through the application of more recent and more sophisticated calculation models." | |
| "The intended purpose of the Diamond Edition of Setup 2000 Version 3.0 software is to provide an accurate and convenient method of independently checking, on a point dose basis, the treatment planning calculations of the primary radiation treatment-planning (RTP) computer." | |
| "We require this software to be accurate..." | |
| No specific quantifiable accuracy metric (e.g., % deviation) is provided in this document. | |
| Reliability/Dependability | "a simple, reliable design" |
| "We require this software to be accurate, reliable, dependable and efficient..." | |
| "The DOS Setup program has been in use by hundreds of highly qualified professionals across the US since 1984 without a single reported incidence of calculation error." (Referencing an earlier version, implying similar reliability for the current version). | |
| Safety - Mitigation of Hazards | "Risk and hazard analysis provided in this submission demonstrate the traceability... that every conceivable hazard to the patient and the integrity of the data files has been mitigated to the fullest extent possible." |
| "a substantial system of internal validation checks and pre-calculation validation provide a substantial Error Avoidance System (EAS)." | |
| Effectiveness - Quality Assurance Tool | "powerful and effective quality assurance tool for the analysis and verification of radiation therapy treatment plans..." |
| "promotes better patient care by bridging the ever increasing gap between the sophisticated calculation capability of the modern treatment planning computers and the otherwise limited calculational resources..." | |
| Substantial Equivalence to Predicate Devices | Numerous points listed in "Intended use and Substantial Equivalence" section (e.g., allows MU/dose calculation, uses similar calculation methods, uses treatment machine database, stores calculations, generates reports). |
| Error Avoidance | "reduces operator errors through the use of active internal validation of user inputs and other machine limitations" |
| "Diamond provides a substantial Error Avoidance System (EAS) to minimize operator errors." | |
| Compatibility with Modern RT Features (Asymmetric jaws, Dynamic wedging, MLC, IMRT) | "Have the ability to perform calculations with modern treatment machine options such collimators with asymmetric jaws, dynamic wedging and multi-leaf collimators (MLC)." |
| "Have the ability to import treatment information files for intensity modulated radiation therapy (IMRT) having multiple segments with MLC settings and calculate the point dose at specified points of interest." |
2. Sample Size Used for the Test Set and Data Provenance:
- Sample Size: Not explicitly stated as a number of cases or patients.
- Data Provenance: The document mentions "a complex clinical environment at institutions such as the MD Anderson Cancer Center" for testing, implying real-world patient data (or simulated clinical scenarios based on real complexity). It also refers to "a network of selected knowledgeable and qualified users" testing the software. It is prospective for these specific tests as they were done during development, but the data itself could be retrospective patient data used for simulation/testing, or prospective phantom measurements.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:
- Number of Experts: Unspecified, but refers to "a network of selected knowledgeable and qualified users" and "qualified health professionals" (e.g., medical physicists).
- Qualifications: "knowledgeable and qualified users," "medical physicist and other qualified health professionals." For earlier versions, it mentions "hundreds of highly qualified professionals across the US," and their own staff consisting of "medical physics consulting organization" members. No specific years of experience are listed.
4. Adjudication Method for the Test Set:
- Adjudication Method: Not explicitly stated. The testing process involved "alpha tested (System Tests and Validation Tests) at the K&S facility" and then "a network of selected knowledgeable and qualified users" and "QED consultants and institutions such as the MD Anderson Cancer Center test the software." This implies independent testing and review, but a formal adjudication process (e.g., 2+1, 3+1 consensus) is not detailed.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- No, an MRMC comparative effectiveness study was not explicitly described. The document focuses on the software's standalone capability to assist human users (medical physicists) in quality assurance by providing independent checks, rather than comparing human performance with and without the AI assistance in a formal MRMC study. The "effect size of how much human readers improve with AI vs without AI assistance" is therefore not reported.
6. Standalone (Algorithm Only Without Human-in-the-Loop) Performance:
- Yes, a standalone performance assessment was done implicitly. The software calculates MU or dose to "validate the values calculated by the RTP computer" or to "provide the settings for simple treatments." This implies the algorithm's direct output (calculated dose/MU) is compared against other calculations or established benchmarks. The "System Tests and Validation Tests" as well as the independent testing by external users would have evaluated the raw computational accuracy of the software itself.
7. Type of Ground Truth Used:
- Ground Truth: The ground truth for dose/MU calculations would be derived from:
- Published methods and models in professional literature: The software explicitly states it "provides these calculations using methods and models published in the professional literature."
- Measured physical parameters in a treatment machine database: "values in a treatment machine database of measured physical parameters entered and validated by the medical physicist."
- Comparison to primary RTP computer calculations: The software's purpose is to "independently checking, on a point dose basis, the treatment planning calculations of the primary radiation treatment-planning (RTP) computer."
- Diode/TLD measurements: The software has the "ability to calculate and report Diode/TLD measurements," and "there is an additional requirement to perform measurements with ion chambers, TLD, film or other appropriate devices" for IMRT, suggesting these physical measurements serve as ground truth or validation.
8. Sample Size for the Training Set:
- Not applicable / Not explicitly stated. This is a calculation software based on physics models and measured machine data, not a machine learning model that requires a "training set" in the conventional sense. Its "training" or configuration involves entering and validating the physics parameters for specific treatment machines by a medical physicist.
9. How the Ground Truth for the Training Set Was Established:
- Not applicable. As above, it's not a machine learning model with a training set. The accuracy of its underlying models and machine parameters are "validated by the medical physicist" using established measurement protocols and published literature.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.