(14 days)
In a radiation therapy department, a course of patient treatment is prescribed by a written directive from a radiation oncologist. Certain parameters from this directive are used by the medical dosimetrist and/or medical physicist along with x-rays or CT scans where applicable to plan a course of treatment. Typically a radiation treatment planning system is used to properly define the precise location and dose distribution for the radiation therapy. As part of the planning process monitor unit settings and dose to a volume or specific anatomical points will be generated.
The MuCheck - Monitor Unit/Dose validation Software verifies the monitor unit or dose calculated by the primary treatment planning system. MuCheck can also be used to calculate the monitor or dose for simple plans that do not require the use of the primary treatment planning system.
The intended use of the MuCheck software has been extended by the addition of an optional module called BrachyCheck. Brachycheck is used to verify brachytherapy procedures(implants using radioactive sources) performed by a brachytherapy treatment planning system. As part of the planning process, dwell positions will be generated and dose contributions from these dwell positions to specified anatomical points will be calculated.
The BrachyCheck module (device under review) verifies the dose calculated to the specified points by the primary treatment planning system. It serves as quality assurance as part of good treatment protocol to have a second means to verify the accuracy of the primary calculation.
The BrachyCheck Dose Validation module is an optional software module that is part of the MuCheck software. MuCheck is designed to operate on a personal computer in a Windows environment. It has been designed to operate either in a stand alone mode independent of any brachytherapy treatment planning system or to import plans from a brachytherapy treatment planning system. It does not connect to or control any radiation hardware device. BrachyCheck performs dose calculations to verify the dose calculated by the primary radiation brachytherapy treatment planning system.
The provided text describes the MuCheck - Monitor Unit/Dose Validation Program, with a specific focus on the BrachyCheck Dose Validation module. The information provided outlines the regulatory submission for premarket notification (510(k)) and details about its intended use and a summary of safety and effectiveness.
Here's an analysis of the acceptance criteria and study information, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Feature/Aspect | Acceptance Criteria (Explicitly Stated or Inferred) | Reported Device Performance |
|---|---|---|
| Primary Function | Verify dose calculated by primary brachytherapy treatment planning system. | BrachyCheck performs dose calculations to verify the dose calculated by the primary radiation brachytherapy treatment planning system. The non-clinical tests were conducted using a brachytherapy treatment planning system or hand calculations and BrachyCheck. The test results all matched very closely. |
| Quality Assurance Tool | Provide a secondary check for primary dose calculation. | Intended use as a quality assurance tool only, not as a treatment planning device. Provides secondary check for primary dose calculation, which is part of good treatment protocol and a recommendation by AAPM Task Group 43. |
| Accuracy (Inferred) | Match closely with primary treatment planning system or hand calculations. | "The test results all matched very closely." (This is the primary metric reported, implying accuracy relative to established methods). |
| Compliance with Regulations | Conformance to GMP regulations. | "Conformance to the GMP regulations helped to insure that the finished product is safe and effective to use." |
| Hazard Resolution | Hazards resolved as appropriate. | "A Hazard Analysis was completed and hazards were resolved as appropriate." |
| System Specifications | All system specifications met. | "All system specifications were met and testing performed to demonstrate substantial equivalence." |
| Substantial Equivalence (Overall) | Demonstrate substantial equivalence to predicate devices. | The FDA determined the device is "substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices." The document explicitly states: "Based upon the technological characteristics, intended use, and non-clinical tests, BrachyCheck is substantially equivalent to the predicate device. The documentation submitted for review supports this claim." |
2. Sample Size Used for the Test Set and Data Provenance
The document does not explicitly state a specific sample size for a "test set" in the context of typical clinical study reporting. The evaluation appears to be based on non-clinical tests comparing BrachyCheck's calculations to those of a brachytherapy treatment planning system or hand calculations.
- Test Set Size: Not explicitly stated as a number of cases or plans. It refers to "tests" that were "conducted using a brachytherapy treatment planning system or hand calculations and BrachyCheck."
- Data Provenance: The data used for these non-clinical tests would be internally generated or from established brachytherapy treatment planning systems and hand calculations. The origin is not specified as a country, and it is inherently retrospective in the sense that it evaluates the device against existing calculation methods rather than collecting new patient data.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The ground truth for the non-clinical tests appears to have been established by:
- "A brachytherapy treatment planning system": These are established systems used in clinical practice, which serve as a de facto "expert" in terms of established calculation methods.
- "Hand calculations": These would be performed by qualified medical physicists or dosimetrists. The document mentions "The staff at Oncology Data Systems includes a certified medical dosimetrist with over 34 years of clinical experience." While not explicitly stated that this individual established the ground truth for every test case, their expertise underpins the company's capabilities and the validity of hand calculations.
- Number of Experts: Not explicitly stated for specific test cases. It implies reliance on output from existing validated systems and potentially multiple dosimetrists/physicists for hand calculations.
- Qualifications: For hand calculations, the internal expertise includes a "certified medical dosimetrist with over 34 years of clinical experience."
4. Adjudication Method for the Test Set
The document does not describe a formal adjudication method (like 2+1 or 3+1). The "tests results all matched very closely" implies a direct comparison rather than an adjudicated consensus.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was Done
No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. The document explicitly states: "Summary of Clinical Testing: Clinical testing was not required to demonstrate substantial equivalence or safety and effectiveness." The evaluation was based on non-clinical tests. Therefore, no effect size of human readers improving with AI vs. without AI assistance is applicable or reported.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) was Done
Yes, a standalone (algorithm only) performance evaluation was done. The BrachyCheck module is a software tool that performs calculations and verifies doses. The non-clinical tests directly compare its computational output to existing systems or hand calculations. It functions as a "standalone mode independent of any brachytherapy treatment planning system" for its verification task. The results reported ("test results all matched very closely") refer to the algorithm's performance in this standalone capacity.
7. The Type of Ground Truth Used
The ground truth used for the non-clinical tests was based on:
- Established gold standard calculations: Output from a "primary radiation brachytherapy treatment planning system."
- Expert hand calculations: Performed by qualified personnel (e.g., certified medical dosimetrist).
This implicitly represents a form of expert consensus or established methodologies rather than pathology or outcomes data.
8. The Sample Size for the Training Set
The document does not mention a training set or machine learning components. BrachyCheck appears to be a deterministic software system that performs dose calculations based on established physics algorithms, not a machine learning model that requires training data.
9. How the Ground Truth for the Training Set Was Established
Since no training set is mentioned (as the device is not described as involving machine learning), this question is not applicable to the provided information.
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.