(248 days)
ArcherQA is a software product intended to provide quality assurance of a radiation treatment plan generated by a commercial treatment planning system (TPS) by allowing a clinician to re-calculate dose parameters with dose calculation algorithms independent from the commercial TPS and compare the two set of dose information. ArcherQA is not a TPS or a radiation delivery device. It is to be used only by trained radiation oncology personnel for quality assurance purposes.
ArcherQA (v1.0) is a standalone software product used within a radiation therapy clinic for quality assurance ( Q A ) and treatment plan verification via recalculation of the dose with a GPU-based independent Monte Carlo dose calculation algorithm. ArcherQA is neither a radiation delivery device (e.g. a linear accelerator) nor a Treatment Planning System (TPS). ArcherQA cannot design or transmit instructions to a delivery device or control any other medical device. ArcherQA is an analysis tool meant solely for QA purposes when used by trained medical professionals. Being a software-only QA tool, ArcherQA never comes into contact with patients. ArcherQA performs dose calculation verifications for radiation treatment plans by independently calculating radiation dose. Using both algorithm and hardware-enhanced Monte Carlo methods, ArcherQA can verify the final dose for radiotherapy plans. The calculation is based on read-in treatment plans that are initially calculated by a TPS. ArcherQA supports photon and electron dose calculation. The treatment modalities that can be evaluated by ArcherQA include: three-dimensional conformal radiotherapy (3D CRT), intensity-modulated radiotherapy (IMRT), volumetric modulated arc therapy (VMAT), stereotactic radiosurgery (SRS), stereotactic body radiotherapy (SBRT). ArcherQA also performs dose delivery OA by using the measurement data recorded in a linear accelerator's delivery log files to compare with calculated the delivery dose. This is presented to the end user in a software component of ArcherQA called Fraction Check.
Here's an analysis of the provided text to extract the acceptance criteria and study details for ArcherOA (V1.0):
1. Table of Acceptance Criteria and Reported Device Performance
The provided FDA letter and 510(k) summary do not explicitly state quantitative acceptance criteria or a direct comparison table with specific performance metrics (e.g., accuracy percentages, gamma pass rates with thresholds). Instead, the performance is described in a more general qualitative manner, focusing on agreement with a predicate device.
Table 1: ArcherQA Performance Claims
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Performs to specifications and works as designed. | "Test results demonstrate conformance to applicable requirements and specifications." |
| Provides clinical dose verification in radiotherapy equivalent to predicate devices. | "Bench testing against the predicate device SciMoCa with 30 randomly selected samples from available database demonstrates good agreement, proving that both devices can be used as equivalent third-party independent software for clinical dose verification in radiotherapy." |
| As safe, as effective, and performs as well as predicate devices. | "ArcherQA is believed to be substantially equivalent to predicate devices in terms of its indications for use, technical characteristics, and overall performance." and "The information provided in this submission indicates the subject device is as safe, is as effective, and performs as well as predicate devices." |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 30 randomly selected samples.
- Data Provenance: "From available database." The document does not specify the country of origin of this data, nor does it explicitly state whether it was retrospective or prospective. Given it was "available," it is highly likely to be retrospective data.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not specify the number of experts or their qualifications used to establish ground truth for the test set. The comparison was made against a predicate device (SciMoCa) which itself performs dose calculations. The assumption is that the SciMoCa calculations served as a reference or a "truth" proxy for comparison.
4. Adjudication Method for the Test Set
The document describes "bench testing against the predicate device SciMoCa" and demonstrating "good agreement." This implies a direct comparison method rather than an expert adjudication process for the test set ground truth. The predicate device's output was the reference point.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not conducted. The device (ArcherQA) is a standalone software for quality assurance and dose calculation verification; it does not involve human readers directly improving their performance with or without AI assistance in the way an MRMC study typically assesses. Its purpose is to verify the output of a commercial Treatment Planning System (TPS).
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance evaluation was done. The "bench testing against the predicate device SciMoCa" directly assesses the algorithm's performance in calculating dose parameters independently, without human intervention in the calculation process.
7. The Type of Ground Truth Used
The "ground truth" for the test set was implicitly the dose calculations generated by the predicate device, SciMoCa. The study aimed to show "good agreement" between ArcherQA's calculations and SciMoCa's calculations.
8. The Sample Size for the Training Set
The document does not provide information regarding the sample size used for any training set. It focuses solely on the validation against the predicate device. As a dose calculation verification software, it might not have a "training set" in the traditional machine learning sense that predicts an outcome, but rather relies on established physics models (Monte Carlo).
9. How the Ground Truth for the Training Set Was Established
Since information about a training set is not provided, details on how its ground truth was established are also not available in the supplied text.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.