(60 days)
The Monaco system is used to make treatment plans with prescriptions for external beam radiation therapy. The system calculates dose for photon and electron treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.
The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:
- contouring
- image manipulation
- simulation
- image fusion
- plan optimization
- QA and plan review
Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), when Dynamic Conformal Arc planning was added (K110730), and most recently when the system's intended use was expanded to include electron treatment planning, among other changes (K132971). The Monaco system accepts patient diagnostic imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.
Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of beam modifiers between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes. The Monaco system supports 3D conformal planning, IMRT, and Dynamic Conformal. It supports inverse and forward planning workflows.
The provided text describes the Monaco RTP System, a medical device for radiation treatment planning. It includes information on its intended use, technological characteristics, and a comparison with predicate devices. However, the document does not contain specific acceptance criteria, a detailed study proving the device meets acceptance criteria, or information on ground truth establishment, sample sizes for training/test sets, expert qualifications, or adjudication methods in the manner typically expected for AI/ML device submissions.
The document states:
- Clinical trials were not performed.
- Validation testing involved simulated clinical workflows.
- Over 600 test procedures were executed (verification tests) to ensure the system works as designed, including new functionality, risk mitigations, and regression tests.
Therefore, most of the requested information cannot be extracted from this document as it pertains to a different type of device (a treatment planning system, not an AI/ML diagnostic aid) and an earlier regulatory submission context where such detailed performance studies for AI/ML were not standard.
Here's a breakdown of what can be extracted and what cannot:
1. A table of acceptance criteria and the reported device performance:
This document does not present quantitative acceptance criteria or corresponding reported device performance metrics like sensitivity, specificity, or AUC, which are common for AI/ML devices. Instead, it states that "Monaco passed testing and was deemed safe and effective for its intended use." The "performance" described is in terms of functionality and passing verification tests.
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| System works as designed | Passed over 600 test procedures, including new functionality, risk mitigations, and regression tests. Deemed safe and effective for its intended use. |
| Functionality (e.g., contouring, dose calculation, plan optimization, image manipulation & fusion, CT simulation, QA/Plan Review) | All listed functionalities are supported and passed verification tests. |
| Substantial equivalence to predicate device | Demonstrated via comparison table (Monaco is substantially equivalent to K132971 and AdvantageSim MD K132944 in intended use and safety/effectiveness). |
2. Sample size used for the test set and the data provenance:
- Sample Size for Test Set: Not specified. The document mentions "over 600 test procedures" but doesn't detail the number of cases or data points used within these procedures.
- Data Provenance: Not specified. The testing involved "simulated clinical workflows," but the origin (e.g., country of origin of data, retrospective or prospective) of the data used in these simulations is not mentioned.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
This information is not provided. The testing described is verification-based ("system is working as designed"), not ground truth establishment by experts for specific diagnostic or prognostic outcomes.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:
Not specified.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, and if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
No MRMC study was performed or mentioned. The device is a treatment planning system, not an AI-assisted diagnostic device, and clinical trials were explicitly stated as not being performed.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
The device's performance is described as standalone in the sense that its functional verification tests ensure it performs its calculations and operations correctly. However, it's explicitly stated that "Monaco does not directly control the linear accelerator that delivers the radiation. Once completed, plans are reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place." This implies that the system is always used with human oversight, but its core calculation and planning functionalities are "standalone" in their execution. The document does not describe specific "standalone performance" metrics in the context of an AI/ML algorithm.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
The concept of "ground truth" as typically applied to diagnostic AI/ML devices based on expert consensus or pathology is not present in this document. The "truth" for this device's performance would be the accuracy of its physical dose calculations and the correct execution of its planning algorithms against established physics models and pre-defined test cases, rather than clinical outcomes or diagnostic interpretations.
8. The sample size for the training set:
The document does not refer to a "training set" as this is not an AI/ML device in the modern sense that learns from data. It's a deterministic software system.
9. How the ground truth for the training set was established:
Not applicable, as there is no "training set."
Summary of what is available from the document:
The Monaco RTP System is a radiation treatment planning system. Its regulatory submission (K151233) describes its intended use, technological characteristics, and compares it to predicate devices (Monaco K132971, AdvantageSim MD K132944) to demonstrate substantial equivalence.
Device Performance and Testing:
- Type of Study: Verification testing and simulated clinical workflows.
- Number of Tests: Over 600 test procedures were executed.
- Purpose of Tests: To verify requirements for new product functionality, ensure risk mitigations function as intended, and regression tests to ensure continued safety and effectiveness of existing functionality.
- Outcome: "Monaco passed testing and was deemed safe and effective for its intended use."
- Clinical Trials: Explicitly stated as not performed because "Clinical testing on patients is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk."
- Human-in-the-Loop: The system's plans are always "reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place."
- "Level of Concern": Classified as "major level of concern" because "should a flaw in the treatment plan escape the notice of the qualified professionals using the Monaco system, serious injury or death could result."
§ 892.5050 Medical charged-particle radiation therapy system.
(a)
Identification. A medical charged-particle radiation therapy system is a device that produces by acceleration high energy charged particles (e.g., electrons and protons) intended for use in radiation therapy. This generic type of device may include signal analysis and display equipment, patient and equipment supports, treatment planning computer programs, component parts, and accessories.(b)
Classification. Class II. When intended for use as a quality control system, the film dosimetry system (film scanning system) included as an accessory to the device described in paragraph (a) of this section, is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 892.9.