(90 days)
The intended use for the muCheck Monitor Unit Validation Program is the same as for the predicate device: to calculate a monitor unit or timer setting for the purpose of validating a monitor unit or timer setting previously calculated by a primary radiation treatment planning system or hand calculation. The intended use is as a quality assurance tool only and not as a treatment planning device.
The muCheck Monitor Unit Validation Program is a software program that is designed to operate on an IBM compatible personal computer in a Windows environment. It has been designed to operate on a stand alone mode independent of any radiation treatment planning system. It does not connect to or control any radiation hardware device.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
The provided document does not explicitly state numerical acceptance criteria in a structured table. However, the core acceptance criterion is substantial equivalence to a predicate device, specifically the "K & S Associates, PC Setup Program (510(k) K 914698)".
The "reported device performance" is the claim that muCheck's calculations "matched very closely" with those of the predicate device and the manual/treatment plan calculations.
| Acceptance Criteria (Stated or Implied) | Reported Device Performance |
|---|---|
| Substantial Equivalence to Predicate Device (K & S Associates, PC Setup Program) in calculating monitor unit/timer settings. | Non-clinical tests: "The test results all matched very closely." |
| Functionality as a Quality Assurance Tool | Beta Clinical Testing: "The results received and summarized from the beta testing conducted in St. John Hospital support the claim of substantial equivalence." |
| Safety and Effectiveness | "The combined expertise [of staff] as well as conformance to the GMP regulations helped to insure that the finished product is safe and effective to use." |
| Accuracy of MU Calculation | "The muCheck Monitor Unit Validation Program (the device under review) makes this process much faster and more accurate." |
Study Details
The document describes two types of testing to support the claim of substantial equivalence: Non-clinical tests and Beta Clinical Testing.
-
Sample Size Used for the Test Set and the Data Provenance:
- Non-clinical tests: The document doesn't specify a numerical sample size for the test set. It mentions "each test was further validated using a treatment plan which provide a monitor unit or hand calculation." This implies multiple test cases were used, but the exact number is not stated.
- Data Provenance: The data provenance is not explicitly stated. The tests seem to be internal comparisons of calculations, rather than using patient-derived data directly.
-
Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications:
- Non-clinical tests: The ground truth for these tests was established by comparison to the predicate device's calculations and "treatment plan which provide a monitor unit or hand calculation." The involvement of specific experts for establishing this ground truth for each test case is not detailed. However, the overall development involved:
- A certified medical dosimetrist with over 20 years of experience.
- A systems analyst with over 20 years of experience in design and development of systems.
- It can be inferred that these individuals, or similar qualified personnel, would have been involved in verifying the accuracy of the manual/treatment plan calculations.
- Non-clinical tests: The ground truth for these tests was established by comparison to the predicate device's calculations and "treatment plan which provide a monitor unit or hand calculation." The involvement of specific experts for establishing this ground truth for each test case is not detailed. However, the overall development involved:
-
Adjudication Method for the Test Set:
- The document does not describe a formal adjudication method (like 2+1 or 3+1). The "non-clinical tests" involved comparing
muCheck's results against both the predicate device and a "treatment plan which provide a monitor unit or hand calculation." The "Beta Clinical Testing" results were "received and summarized" from St. John Hospital. This suggests a direct comparison of calculated values rather than an adjudicated consensus process.
- The document does not describe a formal adjudication method (like 2+1 or 3+1). The "non-clinical tests" involved comparing
-
Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study was not conducted. The device is a "Monitor Unit Validation Program" and acts as a quality assurance tool, not a diagnostic imaging device where human reader performance with or without AI assistance would typically be evaluated.
-
Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study:
- Yes, the non-clinical tests and beta clinical testing were essentially standalone performance evaluations of the
muCheckalgorithm's ability to calculate monitor units. The comparison was to established calculation methods (predicate device, treatment plan, hand calculation). The device is designed to "operate on a stand alone mode independent of any radiation treatment planning system."
- Yes, the non-clinical tests and beta clinical testing were essentially standalone performance evaluations of the
-
Type of Ground Truth Used:
- For the non-clinical tests, the ground truth was established by:
- Calculations from the predicate device (K & S Associates, PC Setup Program).
- Calculations derived from a "treatment plan" or "hand calculation." This can be considered a form of expert-validated calculation/reference standard.
- For the beta clinical testing, the ground truth is implicitly the clinical setting's existing methods for monitor unit validation, against which
muCheck's results were compared to support "substantial equivalence."
- For the non-clinical tests, the ground truth was established by:
-
Sample Size for the Training Set:
- The document describes
muCheckas a calculation program that stores "beam data factors" in a database. It doesn't mention a "training set" in the context of machine learning, as this is not an AI/machine learning model in the modern sense. Instead, the system is populated with predefined, measured "beam data factors" as part of its pre-installation.
- The document describes
-
How the Ground Truth for the Training Set Was Established:
- Given that this is not an AI/ML device, there isn't a "training set" in the conventional sense. The "ground truth" for the system's operational data (beam data factors) is established through physical measurements by a medical physicist, unique to each machine. These factors are then "entered into the system one time and stored in a database as part of the pre-installation process." The accuracy of this data entry is verified by "plotting on a graph and printed along with tables of actual values entered."
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.