Search Results

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon, electron, and proton treatment plans and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• plan optimization
• QA and plan review

The Monaco RTP System accepts patient diagnostic imaging data from CT and MR scans, and source dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation, on these diagnostic images. Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. Monaco RTP system then produces a display of radiation dose distribution within the patient, indicating not only doses to the target volume but to surrounding tissue and structures. The optimal plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

The parameters of the plan are output for later reference and for inclusion in the patient file. Monaco planning methods and modalities:

• Intensity Modulated Radiation Treatment (IMRT) planning
• Electron, photon and proton treatment planning
• Planning for dynamic delivery methods (e.g. dMLC, dynamic conformal, Volumetric Modulated Arc Therapy (VMAT))
• Stereotactic planning and support of cone-based stereotactic
• 3D conformal planning
• Adaptive planning (e.g. for the Elekta Unity MR-Linac)
• Monaco basic systems tools, characteristics, and functions:
• Plan review tools
• Manual and automated contouring tools
• DICOM connectivity
• Windows operating system
• Simulation
• Support for a variety of beam modifiers (e.g. MLCs, blocks, etc.)
• Standardized uptake value (SUV)
• Specialty Image Creation (MIP, MinIP, and Avg)
• Monaco dose and Monitor Unit (MU) calculation:
• Dose calculation algorithms for electron, photon, proton planning

Monaco is programmed using C and C++ computer programming languages. Monaco runs on Windows operating system and off-the-shelf computer server/hardware.

The provided text is a 510(k) summary for the Monaco RTP System, an updated version of a previously cleared device. It largely focuses on demonstrating substantial equivalence to the predicate device and does not contain detailed acceptance criteria or a study proving the device meets them in the format requested. The document explicitly states: "No animal or clinical tests were performed to establish substantial equivalence with the predicate device." Therefore, I cannot provide a table of acceptance criteria, reported device performance, or details about a clinical study.

However, based on the non-clinical performance testing described, here's what information can be extracted:

1. A table of acceptance criteria and the reported device performance:

Based on the provided text, specific numerical acceptance criteria and corresponding reported device performance values are not available. The document states that "Design verification and performance testing were carried out in accordance with design controls... against design and risk management requirements at sub-system, integration and system levels." It also mentions "Software verification testing was conducted and documented in accordance with FDA quidance 1 for devices that pose a major level of concern (Class C per IEC 62304)." However, the actual criteria for these verifications (e.g., specific error margins for dose calculation, response times for image manipulation) and the measured performance against those criteria are not detailed.

2. Sample size used for the test set and the data provenance:

• Sample size used for the test set: Not specified. The document mentions "sub-system, integration and system levels" testing, but no specific number of test cases or data sets are provided.
• Data provenance: Not specified. As no clinical studies were performed, the "data" would refer to test cases, models, or simulated data used in the non-clinical verification. The origin of this data is not mentioned.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• Not applicable/Not specified. Since no clinical studies or expert consensus activities are described for establishing ground truth on a test set, this information is not available.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set:

• Not applicable/None specified. No adjudication method is mentioned as there were no clinical studies.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No MRMC comparative effectiveness study was conducted. The device is a Radiation Treatment Planning (RTP) system, not an AI-assisted diagnostic tool that would typically involve human readers in this context. The document explicitly states "No animal or clinical tests were performed."

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• The document describes "Design verification and performance testing" and focuses on the system's ability to calculate dose, perform image manipulation, optimization, etc., which implies a standalone performance evaluation of the software components. However, specific performance metrics for the algorithm only without any human interaction involved in setting up the plan or interpreting the output are not quantified. The mention of "Software verification testing" suggests an evaluation of the algorithm's correctness.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

• For the non-clinical testing, the "ground truth" would likely be established through:
  • Reference data/models: For dose calculations, comparison against established physics models, phantom measurements, or other validated dose calculation systems.
  • Known input/output pairs: For software functionalities like image manipulation or contouring, where the expected output for a given input is pre-defined.
  • Compliance with standards: The document lists several ISO and IEC standards (e.g., ISO 62083 for radiotherapy treatment planning systems), implying that meeting the requirements of these standards serves as a form of "ground truth" for safety and performance.

8. The sample size for the training set:

• Not applicable/Not specified. The document does not describe the device as employing machine learning or AI that would require a "training set" in the conventional sense for a diagnostic algorithm. It describes a physics-based dose calculation and treatment planning system.

9. How the ground truth for the training set was established:

• Not applicable. As no training set is described, there's no information on how its ground truth would be established.

In summary, the provided document is a 510(k) premarket notification for an updated Radiation Treatment Planning (RTP) system. It focuses on demonstrating substantial equivalence to a predicate device through non-clinical verification and validation testing against design requirements and recognized standards, rather than providing details of a clinical study or specific quantitative acceptance criteria and performance data.

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon, electron, and proton treatment plans and displays, on-screen and in hard-copy, two- or threedimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• . plan optimization
• QA and plan review

The Monaco RTP System accepts patient diagnostic imaging data from CT and MR scans, and source dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation, on these diagnostic images. Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. Monaco RTP system then produces a display of radiation dose distribution within the patient, indicating not only doses to the target volume but to surrounding tissue and structures. The optimal plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

The parameters of the plan are output for later reference and for inclusion in the patient file.

Monaco planning methods and modalities:

• Intensity Modulated Radiation Treatment (IMRT) planning .
• . Electron, photon and proton treatment planning
• . Planning for dynamic delivery methods (e.g. dMLC, dynamic conformal, Volumetric Modulated Arc Therapy (VMAT))
• . Stereotactic planning and support of cone-based stereotactic
• . 3D conformal planning
• . Adaptive planning (e.g. for the Elekta Unity MR-Linac)

Monaco basic systems tools, characteristics, and functions:

• . Plan review tools
• . Manual and automated contouring tools
• DICOM connectivity .
• . Windows operating system
• . Simulation
• . Support for a variety of beam modifiers (e.g. MLCs, blocks, etc.)
• . Standardized uptake value (SUV)
• Specialty Image Creation (MIP, MinIP, and Avq) •
• . Monaco dose and Monitor Unit (MU) calculation:
• Dose calculation algorithms for electron, photon, proton planning .

Monaco is programmed using C and C++ computer programming languages. Monaco runs on Windows operating system and off-the-shelf computer server/hardware.

This document, K213787, is an FDA 510(k) premarket notification for the Elekta Monaco RTP System, Release 6.1. It asserts substantial equivalence to a predicate device, Monaco RTP System (K202789). The document focuses on the non-clinical performance testing of the device, particularly for enhancements in proton therapy functionality.

Here's an analysis of the acceptance criteria and the study that proves the device meets them, based on the provided text:

1. A table of acceptance criteria and the reported device performance

The document does not provide a specific table with numerical acceptance criteria and corresponding reported device performance values in a format like "Target Value (X%) vs. Observed Value (Y%)". Instead, it describes a more qualitative approach to verifying the enhancements.

However, based on the "SUMMARY OF PERFORMACE TESTING (NON-CLINICAL)" section, we can infer the following:

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

The document does not explicitly state a "sample size" in terms of number of patient cases for the non-clinical testing. The tests described are computational comparisons (Monaco vs. Geant4, Monaco vs. manual summation) for various energies, materials, spot arrangements, and beam arrangements. These are not patient-specific data sets but rather simulated or theoretical scenarios designed to evaluate the computational accuracy of the new features.

There is no mention of "country of origin" or whether it was "retrospective or prospective" as these terms typically apply to studies involving patient data, which this non-clinical testing does not appear to use.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

For the non-clinical tests described:

• LET calculation accuracy: The ground truth for monoenergetic spots was established by comparison to Geant4 (a Monte Carlo simulation toolkit) and "other published results". This implies a reliance on established physics models and potentially peer-reviewed literature rather than human expert interpretation of images.
• LET distribution accuracy: Ground truth for complex arrangements was established by "expected values as obtained through manual summation of individual spots according to the design equations." This suggests a mathematical derivation as the ground truth.
• Clinical workflows: "Formal validation... by competent and professionally qualified personnel." No specific number or detailed qualifications (e.g., "radiologist with 10 years of experience") are provided for these personnel.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

Given that the non-clinical tests involve comparisons to established algorithms (Geant4), published results, and mathematical derivations, there is no mention or indication of an adjudication method like "2+1" or "3+1", which are typically used for establishing consensus among human interpreters. The comparisons are to objective, established computational or theoretical benchmarks.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, a multi-reader, multi-case (MRMC) comparative effectiveness study was not done. The document explicitly states: "No animal or clinical tests were performed to establish substantial equivalence with the predicate device." Therefore, there is no information on how much human readers improve with or without AI assistance, as the changes are to the dose calculation algorithms themselves (new proton functionalities like robust optimization, robust evaluation, and LET calculation) rather than an AI-assisted diagnostic or contouring tool that directly impacts human reader performance.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, the described performance testing is primarily a standalone (algorithm only) evaluation. The comparisons are:

• Monaco vs. Geant4 (algorithm vs. algorithm/physics model)
• Monaco vs. manual summation based on design equations (algorithm vs. mathematical derivation)

The "clinical workflow validation" does involve "competent and professionally qualified personnel" interacting with the system, but the core performance evaluation of the new proton features (LET, robustness) is algorithmic.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used for the non-clinical testing appears to be a combination of:

• Established physics models/simulations: Geant4 for monoenergetic LET calculations.
• Mathematical derivations/design equations: For complex LET distributions.
• "Published results": For qualitative comparison of monoenergetic LET.

There is no mention of expert consensus (for image interpretation), pathology, or outcomes data being used as ground truth for these specific non-clinical tests.

8. The sample size for the training set

The document does not mention a "training set" or "training data". This is because the Monaco RTP System (as described in this submission) uses deterministic algorithms for dose calculation (e.g., Monte Carlo algorithm) rather than machine learning or AI models that require a separate training phase. The "development" mentioned refers to software engineering and algorithm implementation, not machine learning model training.

9. How the ground truth for the training set was established

Since no training set is mentioned or implied for the deterministic algorithms described, this question is not applicable based on the provided document.

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon, proton, and electron treatment plans and displays, on-screen and in hard-copy, two- or threedimensional radiation dose distributions inside patients for given treatment plan setups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• · contouring
• · image manipulation
• · simulation
• · image fusion
• · plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009. when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), again when Dynamic Conformal Arc planning was added (K110730), and electron planning, support for stereotactic cones, and SUV calculation were added (K132971). Specialty image creation was added in 2015 (K151233), and adaptive planning and dose calculation in the presence of a magnetic field (e.g., MR-Linac) was added in 2018 (K183037). A 510(k) was filed in 2017 for the addition of carbon ion planning. The 510(k) was withdrawn because there was no hardware cleared for the US market capable of delivering carbon ion plans. Monaco's carbon ion planning functionality remains licensed off and inaccessible to US users.

The Monaco system accepts patient imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the clinican prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

Monaco 6.00 supports Proton Pencil Beam Scanning (Proton PBS) planning for IBA Proteus®ONE and Proteus®PLUS delivery systems (Ion Beam Applications S.A.).

The provided text is a 510(k) summary for the Elekta Monaco RTP System. It describes the device, its intended use, and a comparison to predicate devices, but it explicitly states that "Clinical trials were not performed as part of the development of this product. Clinical testing on patients is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk. Validation testing involved simulated clinical workflows using actual patient data, such as patient images. Pre-defined pass/fail criteria were also equivalent to that of the previous version of Monaco."

Therefore, I cannot provide a detailed answer to your request regarding acceptance criteria and a study proving the device meets them in the way you've outlined, as there was no clinical study (MRMC, standalone, etc.) involving human readers or a traditional test set/ground truth establishment as would be done for an AI-based diagnostic device.

The study referenced is a non-clinical verification study that focuses on software functionality, safety, and effectiveness compared to an existing predicate device, primarily through regression testing and verification of new functionalities. The acceptance criteria are "pre-defined pass/fail criteria" that were equivalent to those used for the previous version of Monaco.

However, based on the information provided, I can infer and summarize what was done:

1. A table of acceptance criteria and the reported device performance:

The document broadly states that "Pre-defined pass/fail criteria were also equivalent to that of the previous version of Monaco." and "Conformity to the same pass/fail criteria as the predicate version of Monaco indicated that Monaco 6.00 was substantially equivalent in safety and effectiveness."

While specific numeric acceptance criteria and performance metrics are not detailed in this summary, the overall acceptance criterion was Substantial Equivalence to the predicate devices. The performance reported is that "Monaco 6.00 was deemed safe and effective for its intended use" based on the non-clinical testing.

2. Sample size used for the test set and the data provenance:

• Sample Size for Test Set: The document mentions "Over 600 test procedures were executed" and "Validation testing involved simulated clinical workflows using actual patient data, such as patient images." The exact number of patient datasets or specific test cases within those 600 procedures is not specified.
• Data Provenance: "actual patient data, such as patient images." No specific country of origin is mentioned, but "simulated clinical workflows" suggests internally generated or existing de-identified data. The testing was retrospective in the sense that it used pre-existing patient data for simulation, not prospective patient enrollment.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• This wasn't a ground truth establishment study for a diagnostic AI. The "ground truth" for this system's validation was primarily the expected output of the algorithms (e.g., dose calculations, plan optimizations) as per documented specifications and comparisons to known good results from the predicate device.
• The document states that "Once completed, plans are reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place." This implies that the system is used by "Dosimetrist or Medical Physicist" and reviewed by "qualified clinicians" in a clinical setting, but these roles were not part of a formal "ground truth" establishment for the validation study itself.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

• Not applicable. This was a software verification and validation against specified requirements and predicate performance, not a clinical adjudication of diagnostic findings.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No MRMC comparative effectiveness study was performed. The document explicitly states: "Clinical trials were not performed as part of the development of this product." and "Clinical testing on patients is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk."

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• The testing described is essentially a form of "standalone" algorithm verification in a simulated environment, focused on the software's functional correctness for tasks like dose calculation and plan optimization, rather than a diagnostic AI's performance. The product itself is a "Radiation Treatment Planning System," which is inherently a human-in-the-loop device, where the software outputs are reviewed and approved by clinicians before implementation. The verification ensured the software's outputs were correct according to its specifications and predicate performance.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

• The "ground truth" for this validation was primarily derived from:
  • Validated algorithms/physical models: For dose calculation, the "ground truth" is based on established physics principles and validated dose calculation algorithms (Monte Carlo, Collapsed Cone, Pencil Beam).
  • Predicate device performance: Functional equivalence and similar calculation results to the previously cleared Monaco RTP System (K190178) and RayStation 8.1 (K190387).
  • Pre-defined pass/fail criteria: These would be based on engineering specifications, clinical requirements for accuracy in dose distribution, and comparison to known good results for test cases.
  • "Simulated clinical workflows using actual patient data": This implies that for these simulated workflows, the expected correct outcome (e.g., the accurate dose distribution for a given patient anatomy and treatment plan) served as the reference.

8. The sample size for the training set:

• This device is not an AI/ML model that undergoes a "training" phase in the typical sense (i.e., learning from annotated data). It's a software system built on established algorithms for radiation treatment planning. Therefore, there is no "training set" as would be applicable to a deep learning model.

9. How the ground truth for the training set was established:

• Refer to point 8. Not applicable for this type of device.

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon and electron treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

· contouring

• · image manipulation
• · simulation
• · image fusion
• plan optimization
• · QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), again when Dynamic Conformal Arc planning was added (K110730), and electron planning, support for stereotactic cones, and SUV calculation were added (K132971). Specialty image creation was added in 2015 (K151233), and adaptive planning and dose calculation in the presence of a magnetic field (e.g., MR-Linac) was added in 2018 (K183037). A 510(k) was filed in 2017 for the addition of carbon ion planning. The 510(k) was withdrawn because there was no hardware cleared for the US market capable of delivering carbon ion plans. Monaco's carbon ion planning functionality remains licensed off and inaccessible to US users.

The Monaco system accepts patient imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the clinican prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

Here's a summary of the acceptance criteria and study information for the Monaco RTP System based on the provided text:

Acceptance Criteria and Reported Device Performance

Study Information:

1. Sample size used for the test set and the data provenance:

   • Test Set Sample Size: Not explicitly stated as a number of cases or patients. The validation testing involved "simulated clinical workflows using actual patient data, such as patient images."
   • Data Provenance: "Actual patient data, such as patient images." The country of origin is not specified, but the context of an FDA submission implies a focus on data relevant to the U.S. market, though not exclusively. The study was retrospective in the sense that it used pre-existing "actual patient data."

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • This information is not provided in the document. The adjudication of ground truth for the test set is not explicitly detailed.

3. Adjudication method for the test set:

   • The document states that plans are "reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place." However, for the specific test set used in validation, the adjudication method (e.g., 2+1, 3+1 consensus) is not explicitly described. The testing involved "pre-defined pass/fail criteria" that were "equivalent to that of the predicate, K183037."

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • A multi-reader multi-case (MRMC) comparative effectiveness study was not performed. The device is a treatment planning system, not an AI-assisted diagnostic tool for human readers in the traditional sense discussed in MRMC studies.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, the primary validation was effectively a standalone performance evaluation of the software. The document states: "Verification tests were written and executed to ensure that the system is working as designed. Over 600 test procedures were executed, including tests to verify requirements for new product functionality, tests to ensure that risk mitigations function as intended, and regression tests to ensure continued safety and effectiveness of existing functionality." This describes an algorithm-only evaluation against predefined criteria.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The "ground truth" for the test set verification was based on pre-defined pass/fail criteria and ensuring the system's calculations and functionality matched expectations established by the predicate device (K183037) and internal Elekta requirements. It also relied on "simulated clinical workflows using actual patient data" to ensure the system produced expected dose distributions and plan outputs. It is not framed as comparing to pathology or long-term outcomes data, but rather the accurate computation and display of dose distributions as per established physics and clinical planning principles.

7. The sample size for the training set:

   • The document does not specify a distinct "training set" for the Monaco RTP System. As a radiation treatment planning system, it relies on physics models and algorithms rather than machine learning models that typically require a training set in the AI sense. The development likely involved extensive testing and calibration against known physics principles and clinical data, which is distinct from a machine learning training set.

8. How the ground truth for the training set was established:

   • Since a distinct "training set" in the machine learning context is not mentioned, the concept of establishing ground truth for it is not applicable based on the provided text. The accuracy of the system is established through rigorous verification against physics models, calculations, and clinical expectations, rather than learning from a labeled training dataset.

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon and electron treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), again when Dynamic Conformal Arc planning was added (K110730), and electron planning, support for stereotactic cones, and SUV calculation were added (K132971). Finally, specialty image creation was added in 2015 (K15123). A 510(k) was filed in 2017 for the addition of carbon ion planning. The 510(k) was withdrawn because there was no hardware cleared for the US market capable of delivering carbon ion plans. Monaco's carbon ion planning functionality remains licensed off and inaccessible to US users.

The Monaco system accepts patient imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the clinican prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

The Monaco RTP System is a radiation treatment planning system. The provided text indicates that no clinical trials were performed. Instead, validation testing involved simulated clinical workflows using actual patient data, and algorithm testing verified the accuracy of the new dose calculation algorithm.

1. Table of acceptance criteria and reported device performance:

2. Sample size used for the test set and data provenance:

• Test Set Sample Size: The document mentions "actual patient data" was used for simulated clinical workflows and algorithm testing, but it does not specify the sample size of this patient data.
• Data Provenance: The document does not specify the country of origin. It indicates the data was "actual patient data," but does not state whether it was retrospective or prospective.

3. Number of experts used to establish the ground truth for the test set and their qualifications:

• The document states that "qualified clinicians" review and approve treatment plans and that the process "can be performed such that no human subjects are exposed to risk." This implies that the validation testing was based on expert consensus or established clinical standards rather than live patient outcomes. However, the number of experts and their specific qualifications are not specified.

4. Adjudication method for the test set:

• The document mentions that treatment plans are "reviewed and approved by qualified clinicians," and that a "flaw in the treatment plan [could] escape the notice of the qualified professionals." This suggests a review process by experts. However, a formal adjudication method (e.g., 2+1, 3+1) is not explicitly described.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:

• No, an MRMC comparative effectiveness study was not done. The document explicitly states: "Clinical trials were not performed as part of this product."

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

• Yes, a standalone component was done for algorithm testing. The document states: "Algorithm testing verified the accuracy of the new dose calculation algorithm in Monaco 5.40 using the same test methods as the predicate version of Monaco." This implies evaluation of the algorithm's performance independent of a human user.

7. The type of ground truth used:

• The ground truth appears to be based on established clinical standards and expert review/consensus rather than direct pathology or outcomes data from clinical trials. The document mentions "pre-defined pass/fail criteria" and validation against "simulated clinical workflows using actual patient data." The acceptance criteria focused on demonstrating "equivalent accuracy" to the predicate device, which would have its own established standards.

8. The sample size for the training set:

• The document does not specify a training set sample size. The focus of the provided text is on validation and verification testing of the already developed Monaco RTP system, which includes a new dose calculation algorithm. It does not describe the development or training phase of the algorithm itself.

9. How the ground truth for the training set was established:

• Since a training set is not explicitly mentioned and the document focuses on the validation of an existing system's new algorithm, the method for establishing ground truth for a hypothetical training set is not described.

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179). The Monaco system accepts patient diagnostic imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a multileaf collimator (MLC) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

The Monaco RTP System is a radiation treatment planning system. Here's a breakdown of its acceptance criteria and the supporting study:

1. Table of Acceptance Criteria and Reported Device Performance

The provided summary does not explicitly list distinct, quantifiable acceptance criteria with corresponding performance metrics in a readily extractable table format for dose calculation or planning accuracy. Instead, it states that verification tests were "written and executed to ensure that the system is working as designed" and that "Pass/fail requirements and results of this testing can be found in section 18 of this submission." However, Section 18 is not included in the provided text.

Based on the available information, the general performance criteria can be inferred as:

2. Sample Size Used for the Test Set and the Data Provenance

The summary states that "Clinical trials were not performed as part of the development of this product." Instead, "Algorithm testing was performed to compare calculated against measured doses," and "clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

Therefore:

• Test Set Sample Size: Not specified in terms of number of patient cases. The testing involved "algorithm testing" (comparing calculated vs. measured doses) and an unspecified number of "clinically oriented validation test cases."
• Data Provenance: Not explicitly stated regarding origin (e.g., country). However, the testing was "in-house" by the manufacturer (Computerized Medical Systems, Inc., USA). This implies the data used for the algorithm and validation tests would be internally generated or sourced. The context suggests it was not patient data from clinical settings.
• Retrospective/Prospective: The testing appears to be retrospective in the sense that it did not involve prospective human subjects but rather validation against pre-existing data (measured doses) or simulated/representative cases for the "clinically oriented validation test cases."

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

• Number of Experts: Not explicitly stated. The "clinically oriented validation test cases" were "written and executed in-house by CMS customer support personnel."
• Qualifications of Experts: The personnel were "CMS customer support personnel." While they handled "clinically oriented" test cases, their specific clinical qualifications (e.g., medical physicist, dosimetrist, or specific years of experience) are not provided. The summary also notes that plans are "reviewed and approved by qualified clinicians" in a clinical setting, but this refers to post-approval clinical use, not the ground truth establishment for the premarket testing.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for establishing ground truth for the test set. Since the testing involved "algorithm testing" comparing calculated against measured doses, and "clinically oriented validation test cases" executed in-house, it is unlikely a multi-expert adjudication method was employed in the traditional sense. The "ground truth" for algorithmic accuracy would be established by the physical measurements, and for validation cases, by adherence to predefined clinical expectations or specifications.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If so, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance?

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. The device is a radiation treatment planning system, not an AI-assisted diagnostic tool for human readers. Its primary function is to calculate dose and aid in plan creation, not to improve human reader performance in interpreting images or making diagnoses.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, a form of standalone performance assessment was done. "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy." This directly evaluates the algorithm's output (calculated dose) against an objective standard (measured dose) without a human-in-the-loop decision-making process. The "clinically oriented validation test cases" also assessed the system's ability to produce acceptable plans based on defined criteria.

7. The Type of Ground Truth Used

• For Algorithm Testing: The ground truth was measured doses. The summary states "Algorithm testing was performed to compare calculated against measured doses." This implies physical measurements were used as the gold standard.
• For "Clinically Oriented Validation Test Cases": The ground truth was based on predefined clinical expectations/specifications or internal standards established by the CMS customer support personnel who wrote and executed these cases.

8. The Sample Size for the Training Set

The document does not specify a separate "training set" sample size. The Monaco system is a radiation treatment planning system that calculates dose and optimizes plans based on established physics models and algorithms. It does not appear to be a machine learning model that requires a distinct "training set" in the common understanding of AI devices. Its development would involve calibration, verification, and validation, rather than a training process on a large dataset of patient images or outcomes.

9. How the Ground Truth for the Training Set Was Established

Since a "training set" in the context of machine learning is not mentioned or implied for this device, the method for establishing its ground truth is not applicable/not provided. The system's foundational accuracy would stem from its underlying physical models and their calibration, which would involve experimental data and established physics principles, rather than a labeled training dataset.

Ask a specific question about this device

The Monaco system is used to create treatment plans for any cancer patient for whom external beam intensity modulated radiation therapy (IMRT) has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or threedimensional radiation dose distributions within a patient for a given treatment plan setup.

The Monaco product line is intended for use in radiation treatment planning using generally accepted methods for contouring, image manipulation, simulation, image fusion, plan optimization and QA and plan review.

Monaco uses local biological measures for optimization to create intensity modulated radiation therapy (IMRT) plans using Multileaf Collimators. With the addition of VMAT planning capability, Monaco also allows users to creatment plans in which the devices that aim and shape the beam are in motion while the beam is on.

The provided text describes the Monaco RTP System - VMAT Option, a software used for radiation treatment planning. It states that "Clinical trials were not performed as part of the development of this product." Instead, "Clinically oriented validation test cases were written and executed in-house by Customer Support personnel in a simulated clinical environment. Algorithm testing was also performed by qualified Medical Physicists using measured data from clinical facilities."

However, the document does not provide:

• A table of acceptance criteria and reported device performance.
• Sample sizes used for the test set or data provenance.
• Number or qualifications of experts used to establish ground truth.
• Adjudication methods.
• Information on multi-reader multi-case (MRMC) comparative effectiveness studies.
• Specific details on standalone algorithm performance.
• Type of ground truth used (beyond "measured data").
• Sample size for the training set.
• How ground truth for the training set was established.

Therefore,Based on the provided text, a comprehensive description of acceptance criteria and the study proving the device meets them cannot be fully constructed for many of the requested categories. The document explicitly states that "Clinical trials were not performed."

Here's what can be extracted and what is missing:

Acceptance Criteria and Study Details:

Reported Device Performance: The text states, "Monaco with the VMAT option successfully passed both testing efforts and was deemed fit for clinical use." No quantitative performance metrics (e.g., accuracy, precision, dose deviation thresholds) are provided in this summary. |
| 2. Sample size for test set and data provenance | Sample Size: Not specified. The document mentions "Clinically oriented validation test cases" and "measured data from clinical facilities" but does not give a number of cases or patients.

Data Provenance: "measured data from clinical facilities." Specific countries or retrospective/prospective nature are not mentioned. |
| 3. Number and qualifications of experts for ground truth | Number of Experts: Not specified.

Qualifications: "qualified Medical Physicists" performed algorithm testing. For clinically oriented validation, "Customer Support personnel in a simulated clinical environment" executed tests. No specific experience or board certifications are provided for either group. |
| 4. Adjudication method for the test set | Not documented. |
| 5. MRMC comparative effectiveness study | No. The document explicitly states: "Clinical trials were not performed as part of the development of this product." |
| 6. Standalone (algorithm only) performance study | Yes, to some extent. "Algorithm testing was also performed by qualified Medical Physicists using measured data from clinical facilities." This suggests an assessment of the algorithm's output against known measured data, which is a form of standalone performance evaluation. However, detailed results or metrics are not provided in this summary. |
| 7. Type of ground truth used | "Measured data from clinical facilities." This implies physical measurements of radiation dose distributions that the system's calculations were compared against. |
| 8. Sample size for the training set | Not applicable/Not specified. This is a radiation treatment planning system that calculates dose distributions, not a machine learning model that would typically have a "training set" in the conventional sense. The "algorithm" was developed, and then tested. |
| 9. How ground truth for the training set was established | Not applicable/Not specified (see point 8). |

Summary of Study (as described in the document):

The Monaco RTP System - VMAT Option underwent two main types of testing:

1. Clinically oriented validation test cases: These were executed in-house by Customer Support personnel within a simulated clinical environment. The specific nature of these "test cases" (e.g., number of plans, complexity) is not detailed.
2. Algorithm testing: This was performed by qualified Medical Physicists. The testing involved comparing the algorithm's outputs against "measured data from clinical facilities." The report states that "Test reports are included in section 20 of this submission," but these details are not provided in the summary.

Both testing efforts were successfully passed, leading the device to be deemed "fit for clinical use." Verification tests, with associated pass/fail requirements, were also conducted to ensure the system functioned as designed, and these were also successfully passed.

Crucially, no clinical trials involving human subjects were performed or deemed necessary for this premarket notification, as the manufacturer argued that "testing can be performed such that no human subjects are exposed to risk" and that clinical testing was "not advantageous in demonstrating substantial equivalence or safety and effectiveness." The device's safety and effectiveness were established through comparison to predicate devices and the non-clinical testing described above.

Ask a specific question about this device

The Monaco system is used to create treatment plans for any cancer patient for whom external beam intensity modulated radiation therapy (IMRT) has been prescribed. The system will calculate and display, both on-screen and in hard-copy, either two- or three-dimensional radiation dose distributions within a patient for a given treatment plan set-up.

The Monaco product line is intended for use in radiation treatment planning using generally accepted methods for contouring, image manipulation, simulation, image fusion, plan optimization and QA and plan review.

Monaco uses local biological measures for optimization to create intensity modulated radiation therapy (IMRT) plans using Multileaf Collimators.

The provided text describes the Monaco RTP System, a radiation treatment planning system, and its non-clinical testing for substantial equivalence. It does not contain information about clinical trials or the specific acceptance criteria and performance metrics typically found in studies involving AI performance for diagnosis or image interpretation.

Based on the provided text, here is an analysis of the acceptance criteria and study information:

1. Table of Acceptance Criteria and Reported Device Performance:

The document does not provide a table with specific quantitative acceptance criteria or detailed performance metrics. Instead, it states:

2. Sample Sizes Used for Test Set and Data Provenance:

The document explicitly states that clinical trials were not performed. Therefore, there isn't a "test set" in the traditional sense involving patient data for an effectiveness study.

• Test Set Sample Size: Not applicable, as no clinical test set was used for effectiveness.
• Data Provenance: Not applicable for effectiveness testing. However, for "On-site validation testing," it mentions "a small group of customers, using actual patient data." The origin of this patient data (country, retrospective/prospective) is not specified. This validation was not a formal clinical trial for effectiveness.

3. Number of Experts and Qualifications for Ground Truth:

• Number of Experts: Not specified.
• Qualifications of Experts: For the "on-site validation testing," it mentions "qualified clinicians" who review and approve plans and "a small group of customers." Specific qualifications (e.g., radiologist with X years of experience) are not provided. The document highlights that "qualified clinicians" review plans and may subject them to quality assurance before treatment, implying their role in validating the plans generated by the system.

4. Adjudication Method for Test Set:

Not applicable, as no formal clinical test set or adjudication process for a diagnostic outcome was performed. The document mentions "qualified clinicians" review and approve plans, but this isn't an adjudication method for a test set in the context of an effectiveness study.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

No. The document explicitly states: "Clinical trials were not performed as part of the development of this feature." Therefore, an MRMC comparative effectiveness study was not conducted.

6. Standalone (Algorithm Only) Performance Study:

Yes, in a non-clinical context. The document refers to "Verification tests, including algorithm test cases," and "algorithm accuracy requirements." It states that "Monaco successfully passed both verification and algorithm testing," suggesting a standalone evaluation of the algorithms.

7. Type of Ground Truth Used:

For the non-clinical "verification tests" and "algorithm test cases," the ground truth would likely be established through:

• Pre-defined Pass/Fail Criteria: These would be based on engineering specifications and expected output from the algorithms.
• Comparison to Known Analytical Solutions/Benchmarks: For specific algorithm calculations (e.g., dose distribution), "ground truth" would be established by comparing the software's output to mathematically derived solutions or outputs from trusted reference calculators.
• Outputs from Predicate Devices: The text mentions "the same algorithm accuracy requirements as those used to evaluate the XiO RTP System and the Focal Workstation," implying that the performance of predicate devices might serve as a benchmark or a form of 'ground truth' for comparison during testing.

For the "on-site validation testing using actual patient data," the "ground truth" for clinical fitness would likely involve:

• Clinical Acceptability: Expert opinion from the "small group of customers" (clinicians) on whether the generated plans were clinically appropriate and safe.
• Quality Assurance (QA) Practices: Comparison against established QA standards for radiation oncology treatment plans.

8. Sample Size for the Training Set:

Not specified. The document does not provide details on the training methodology or the datasets used to develop the algorithms within the Monaco RTP System.

9. How Ground Truth for Training Set Was Established:

Not specified. Since details about the training set are not provided, how its ground truth was established is also not mentioned.

Ask a specific question about this device