(29 days)
SciMoCa is a software product intended to provide quality assurance of a radiotherapy dose calculated by a treatment planning system by allowing a clinician to re-calculate the dose with an independent dose calculation algorithm and compare the two doses.
SciMoCa is not a treatment planning system or a radiation delivery device. It is to be used only by trained radiation oncology personnel for quality assurance purposes.
SciMoCa is a standalone software product that allows clinicians to perform quality assurance of a radiotherapy treatment dose generated by a treatment planning system via recalculation of the dose with an independent Monte Carlo dose calculation algorithm. SciMoCa is implemented in a service-oriented client-server architecture that allows one or more clients to communicate calculation requests to a central dose calculation server. SciMoCa is not a treatment planning system or a radiation delivery device. It is to be used only by trained radiation oncology personnel for quality assurance purposes.
The provided text does not contain detailed information about the acceptance criteria and the comprehensive study that proves the device meets those criteria. It mentions "Validation testing involved simulated clinical workflows and environment, as well as algorithm testing" and that the product "passed verification testing and was deemed safe and effective for its intended use." However, it lacks specific metrics, study designs, or methodologies that would allow for a complete answer to your detailed questions.
Therefore, many of the requested fields cannot be filled from the provided text.
Here's an attempt to answer based on the limited information available:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Not specified | "Passed verification testing and was deemed safe and effective for its intended use." |
| Not specified | "The product was deemed fit for clinical use." |
| Not specified | "Performed to specifications and works as designed." |
2. Sample size used for the test set and the data provenance
- Sample Size: Not specified. The document mentions "Over 700 test procedures were executed," but this refers to the number of test procedures, not the number of distinct cases or data samples in a test set.
- Data Provenance: Not specified. There is no information about country of origin or whether the data was retrospective or prospective.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- Number of Experts: Not specified.
- Qualifications of Experts: Not specified.
4. Adjudication method for the test set
- Adjudication Method: Not specified.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
There is no mention of an MRMC study or any assessment of human reader improvement with or without AI assistance. The device, SciMoCa, is described as an "independent dose calculation algorithm" for quality assurance, not as an AI-assisted diagnostic tool for human readers.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, based on the description, the primary validation would likely be standalone performance. The device's function is to "re-calculate the dose with an independent dose calculation algorithm and compare the two doses." This implies the algorithm's output is compared against a reference (the treatment planning system's calculated dose), which is a form of standalone evaluation. However, specific performance metrics for this standalone performance (e.g., accuracy, deviation limits) are not provided in the text.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The ground truth for dose calculation in radiotherapy QA typically involves comparing the device's recalculated dose against an established reference dose (e.g., from a validated treatment planning system or a benchmark phantom calculation). The text states SciMoCa "allows a clinician to re-calculate the dose with an independent dose calculation algorithm and compare the two doses," implying the "ground truth" or reference is the dose from the primary treatment planning system. However, the exact methodology for establishing this comparison and what constitutes "ground truth" for the validation itself is not detailed.
8. The sample size for the training set
- Training Set Sample Size: Not specified. The document refers to "algorithm testing" but provides no details on training data.
9. How the ground truth for the training set was established
- Ground Truth Establishment for Training Set: Not specified.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.