(266 days)
The SOFTDISO system is a standalone software solution designed to be used by medical physicists, radiation oncologists, and dosimetrists to have an overview of the treatment plans delivered from Treatment Systems (LINAC Systems using high energy x-rays) to the patient. This solution is to be used as a quality control check purposes only.
SOFTDISO does not provide any conclusions or policy for interpretation of the results based on the comparison results data. The output from SOFTDISO cannot be and must not be used for changing the planning or treatment strategy or as a means of proving the effectiveness of the quality control process/chain during treatment. It is the physician's responsibility to verify the working of the quality control process/chain and the correctness of the dose delivered.
SOFTDISO software permit an IN-VIVO dosimetric analysis (IVD) in patient for 3DCRT, IMRT and VMAT beams using a Si-EPID portal systems.
The dosimetric analysis consists of two tests elaboration, dose reconstruction in the isocenter point and EPID images y-analysis. In case of out of tolerance levels, the system is capable of underline the right type of quality control useful to remove the causes of discrepancy.
SOFTDISO software is based on the usage of generalized correlation factors between transit signals measured by EPID and doses measured in water equivalent solid phantom, along the beam central axis.
SOFTDISO software uses generalized functions taken from measurements of 70 beams of Varian, Elekta and Siemens linacs. So only a small set of dosimetric measurements must be performed by the user for the software commissioning, and some of those measurements are already performed during the linac's beams calibration. In particular some of the measurements that the user performs include calculating the beam dose in cGy/UM in reference conditions (field 10.10 cm2 at drif =10 cm depth), the beam quality indicator (TPR20, 10) and the attenuation factor WF for wedged beams. The user also performs a measure of the EPID signal in reference condition (field 10.10 cm2 at SED distance) for EPID calibration.
SOFTDISO software can use information coming from record and verify systems through DICOM and DICOM-RT protocols. DICOM protocol is used for transferring images from CT and from EPID and the DICOM-RT protocol is used for transferring information related to the TPS system.
The provided document describes the K170345 510(k) submission for the SOFTDISO device. Here's a breakdown of the acceptance criteria and the study that proves the device meets those criteria, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| R ratios for 3DCRT within ± 5% | All 192 tests for 3DCRT were within ± 5% |
| Gamma analysis (y%) for 3DCRT ≥ 95% | All 192 tests for 3DCRT were ≥ 95% |
| Gamma analysis (ymean) for 3DCRT ≤ 0.3 | All 192 tests for 3DCRT were ≤ 0.3 |
| R ratios for IMRT within ± 5% | All 48 tests for IMRT were within ± 5% |
| Gamma analysis (y%) for IMRT ≥ 95% | All 48 tests for IMRT were ≥ 95% |
| Gamma analysis (ymean) for IMRT ≤ 0.3 | All 48 tests for IMRT were ≤ 0.3 |
| R ratios for VMAT within ± 5% | All 60 tests for VMAT were within ± 5% |
| Gamma analysis (y%) for VMAT ≥ 95% | All 60 tests for VMAT were ≥ 95% |
| Gamma analysis (ymean) for VMAT ≤ 0.3 | All 60 tests for VMAT were ≤ 0.3 |
| Ease of setup and use | Indicated SOFTDISO was easy to setup and use |
| Average analysis time per patient | Approximately 2.5 minutes per patient |
| Ability to identify errors during treatment (e.g., patient setup, laser misalignment, TPS beam-implementation, patient morphological changes) | Was able to identify errors that occur during treatment (e.g., patient setup, laser misalignment, TPS beam-implementation) and patient morphological changes (e.g., tumor shrinkage, gas pockets, loss of patient weight) |
Study Details
2. Sample size used for the test set and the data provenance
- External Validation (Dosimetry Accuracy):
- 3DCRT: 192 tests
- IMRT: 48 tests
- VMAT: 60 tests
- Data Provenance: Conducted at "three different clinical sites" for 3DCRT and IMRT, and "two different clinical sites" for VMAT. The document does not specify the country of origin, but "clinical sites" suggests real-world patient or phantom data. This appears to be prospective data collection as part of validation, though the exact nature (e.g., real patient treatments vs. carefully controlled phantom experiments) is not fully detailed.
- Clinical Validation (Usability and Error Detection):
- Usability: 1287 treatments
- Error Detection: 823 patient data (340 3DCRT, 483 VMAT treatments)
- Data Provenance: The document states "Clinical validation was performed by applying in-vivo dosimetry checks on 1287 treatments" and "performed on 823 patient data." This strongly suggests retrospective or prospective use of real patient data from clinical settings. The country of origin is not specified.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not explicitly state the number of experts used to establish the ground truth or their specific qualifications for the test set. The Indications for Use state that "It is the physician's responsibility to verify the working of the quality control process/chain and the correctness of the dose delivered," implying that medical physicists, radiation oncologists, and dosimetrists are involved in the overall quality control process. However, for the validation studies themselves, the method for establishing ground truth and the experts involved are not detailed.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
The document does not describe any adjudication method used for the test set.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
A multi-reader multi-case (MRMC) comparative effectiveness study was not described. The validation focused on the performance of the device itself against established dosimetric tolerances and its ability to identify errors, rather than directly comparing human reader performance with and without the device.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, a standalone performance evaluation was performed as part of the "External validation to validate the accuracy of dosimetry procedure." This involved evaluating the R ratios and gamma analysis indexes of the SOFTDISO software against preset tolerance levels (e.g., ± 5% for R ratios, ≥ 95% for γ%, ≤ 0.3 for γmean). This directly assesses the algorithm's accuracy in comparing planned and measured doses.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth appears to be based on dosimetric comparison against established tolerance levels.
- For the R ratios, the implicit ground truth is the "planned dose" from the Treatment Planning System (TPS) and the "measured dose" from the EPID, with the expectation that these should match within a certain tolerance.
- For the gamma analysis, the ground truth is again the comparison between a "reference EPID image" (likely related to the planned dose distribution) and a "current one," with deviation measured against industry-standard gamma criteria.
- For the clinical validation identifying errors, the ground truth for "errors" (patient setup, laser misalignment, TPS beam-implementation, patient morphological changes) would likely be established through independent clinical assessment or follow-up by the medical physicists and oncologists at the sites. The document doesn't detail how these "errors" were definitively confirmed as ground truth for the study.
8. The sample size for the training set
The document states, "SOFTDISO software uses generalized functions taken from measurements of 70 beams of Varian, Elekta and Siemens linacs." This indicates that these 70 beams likely formed part of the training or commissioning data used to develop these generalized functions within the software, which then allows for site-specific commissioning with a smaller dataset. The exact nature of this "training set" for a machine learning context is not detailed, but the 70 beams are the most relevant number provided for the model's underlying data.
9. How the ground truth for the training set was established
For the "generalized functions taken from measurements of 70 beams," the ground truth was established through direct physical measurements. The text mentions:
- "measurements of 70 beams of Varian, Elekta and Siemens linacs."
- "Small set of dosimetric measurements must be performed by the user for the software commissioning, and some of those measurements are already performed during the linac's beams calibration."
- Specifically highlights measurements for "calculating the beam dose in cGy/UM in reference conditions (field 10.10 cm2 at drif =10 cm depth), the beam quality indicator (TPR20, 10) and the attenuation factor WF for wedged beams," and "a measure of the EPID signal in reference condition (field 10.10 cm2 at SED distance) for EPID calibration."
This indicates that the ground truth for the "training" data (the 70 beams and
user-performed commissioning measurements) was established through physical dosimetric measurements performed by qualified personnel (likely medical physicists) in a controlled environment.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.