Search Results

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon, proton, and electron treatment plans and displays, on-screen and in hard-copy, two- or threedimensional radiation dose distributions inside patients for given treatment plan setups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• · contouring
• · image manipulation
• · simulation
• · image fusion
• · plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009. when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), again when Dynamic Conformal Arc planning was added (K110730), and electron planning, support for stereotactic cones, and SUV calculation were added (K132971). Specialty image creation was added in 2015 (K151233), and adaptive planning and dose calculation in the presence of a magnetic field (e.g., MR-Linac) was added in 2018 (K183037). A 510(k) was filed in 2017 for the addition of carbon ion planning. The 510(k) was withdrawn because there was no hardware cleared for the US market capable of delivering carbon ion plans. Monaco's carbon ion planning functionality remains licensed off and inaccessible to US users.

The Monaco system accepts patient imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the clinican prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

Monaco 6.00 supports Proton Pencil Beam Scanning (Proton PBS) planning for IBA Proteus®ONE and Proteus®PLUS delivery systems (Ion Beam Applications S.A.).

The provided text is a 510(k) summary for the Elekta Monaco RTP System. It describes the device, its intended use, and a comparison to predicate devices, but it explicitly states that "Clinical trials were not performed as part of the development of this product. Clinical testing on patients is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk. Validation testing involved simulated clinical workflows using actual patient data, such as patient images. Pre-defined pass/fail criteria were also equivalent to that of the previous version of Monaco."

Therefore, I cannot provide a detailed answer to your request regarding acceptance criteria and a study proving the device meets them in the way you've outlined, as there was no clinical study (MRMC, standalone, etc.) involving human readers or a traditional test set/ground truth establishment as would be done for an AI-based diagnostic device.

The study referenced is a non-clinical verification study that focuses on software functionality, safety, and effectiveness compared to an existing predicate device, primarily through regression testing and verification of new functionalities. The acceptance criteria are "pre-defined pass/fail criteria" that were equivalent to those used for the previous version of Monaco.

However, based on the information provided, I can infer and summarize what was done:

1. A table of acceptance criteria and the reported device performance:

The document broadly states that "Pre-defined pass/fail criteria were also equivalent to that of the previous version of Monaco." and "Conformity to the same pass/fail criteria as the predicate version of Monaco indicated that Monaco 6.00 was substantially equivalent in safety and effectiveness."

While specific numeric acceptance criteria and performance metrics are not detailed in this summary, the overall acceptance criterion was Substantial Equivalence to the predicate devices. The performance reported is that "Monaco 6.00 was deemed safe and effective for its intended use" based on the non-clinical testing.

2. Sample size used for the test set and the data provenance:

• Sample Size for Test Set: The document mentions "Over 600 test procedures were executed" and "Validation testing involved simulated clinical workflows using actual patient data, such as patient images." The exact number of patient datasets or specific test cases within those 600 procedures is not specified.
• Data Provenance: "actual patient data, such as patient images." No specific country of origin is mentioned, but "simulated clinical workflows" suggests internally generated or existing de-identified data. The testing was retrospective in the sense that it used pre-existing patient data for simulation, not prospective patient enrollment.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• This wasn't a ground truth establishment study for a diagnostic AI. The "ground truth" for this system's validation was primarily the expected output of the algorithms (e.g., dose calculations, plan optimizations) as per documented specifications and comparisons to known good results from the predicate device.
• The document states that "Once completed, plans are reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place." This implies that the system is used by "Dosimetrist or Medical Physicist" and reviewed by "qualified clinicians" in a clinical setting, but these roles were not part of a formal "ground truth" establishment for the validation study itself.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

• Not applicable. This was a software verification and validation against specified requirements and predicate performance, not a clinical adjudication of diagnostic findings.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No MRMC comparative effectiveness study was performed. The document explicitly states: "Clinical trials were not performed as part of the development of this product." and "Clinical testing on patients is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk."

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• The testing described is essentially a form of "standalone" algorithm verification in a simulated environment, focused on the software's functional correctness for tasks like dose calculation and plan optimization, rather than a diagnostic AI's performance. The product itself is a "Radiation Treatment Planning System," which is inherently a human-in-the-loop device, where the software outputs are reviewed and approved by clinicians before implementation. The verification ensured the software's outputs were correct according to its specifications and predicate performance.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

• The "ground truth" for this validation was primarily derived from:
  • Validated algorithms/physical models: For dose calculation, the "ground truth" is based on established physics principles and validated dose calculation algorithms (Monte Carlo, Collapsed Cone, Pencil Beam).
  • Predicate device performance: Functional equivalence and similar calculation results to the previously cleared Monaco RTP System (K190178) and RayStation 8.1 (K190387).
  • Pre-defined pass/fail criteria: These would be based on engineering specifications, clinical requirements for accuracy in dose distribution, and comparison to known good results for test cases.
  • "Simulated clinical workflows using actual patient data": This implies that for these simulated workflows, the expected correct outcome (e.g., the accurate dose distribution for a given patient anatomy and treatment plan) served as the reference.

8. The sample size for the training set:

• This device is not an AI/ML model that undergoes a "training" phase in the typical sense (i.e., learning from annotated data). It's a software system built on established algorithms for radiation treatment planning. Therefore, there is no "training set" as would be applicable to a deep learning model.

9. How the ground truth for the training set was established:

• Refer to point 8. Not applicable for this type of device.

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon and electron treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

· contouring

• · image manipulation
• · simulation
• · image fusion
• plan optimization
• · QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), again when Dynamic Conformal Arc planning was added (K110730), and electron planning, support for stereotactic cones, and SUV calculation were added (K132971). Specialty image creation was added in 2015 (K151233), and adaptive planning and dose calculation in the presence of a magnetic field (e.g., MR-Linac) was added in 2018 (K183037). A 510(k) was filed in 2017 for the addition of carbon ion planning. The 510(k) was withdrawn because there was no hardware cleared for the US market capable of delivering carbon ion plans. Monaco's carbon ion planning functionality remains licensed off and inaccessible to US users.

The Monaco system accepts patient imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a beam modifier (MLC, block, etc.) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the clinican prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

Here's a summary of the acceptance criteria and study information for the Monaco RTP System based on the provided text:

Acceptance Criteria and Reported Device Performance

Study Information:

1. Sample size used for the test set and the data provenance:

   • Test Set Sample Size: Not explicitly stated as a number of cases or patients. The validation testing involved "simulated clinical workflows using actual patient data, such as patient images."
   • Data Provenance: "Actual patient data, such as patient images." The country of origin is not specified, but the context of an FDA submission implies a focus on data relevant to the U.S. market, though not exclusively. The study was retrospective in the sense that it used pre-existing "actual patient data."

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • This information is not provided in the document. The adjudication of ground truth for the test set is not explicitly detailed.

3. Adjudication method for the test set:

   • The document states that plans are "reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place." However, for the specific test set used in validation, the adjudication method (e.g., 2+1, 3+1 consensus) is not explicitly described. The testing involved "pre-defined pass/fail criteria" that were "equivalent to that of the predicate, K183037."

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • A multi-reader multi-case (MRMC) comparative effectiveness study was not performed. The device is a treatment planning system, not an AI-assisted diagnostic tool for human readers in the traditional sense discussed in MRMC studies.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, the primary validation was effectively a standalone performance evaluation of the software. The document states: "Verification tests were written and executed to ensure that the system is working as designed. Over 600 test procedures were executed, including tests to verify requirements for new product functionality, tests to ensure that risk mitigations function as intended, and regression tests to ensure continued safety and effectiveness of existing functionality." This describes an algorithm-only evaluation against predefined criteria.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The "ground truth" for the test set verification was based on pre-defined pass/fail criteria and ensuring the system's calculations and functionality matched expectations established by the predicate device (K183037) and internal Elekta requirements. It also relied on "simulated clinical workflows using actual patient data" to ensure the system produced expected dose distributions and plan outputs. It is not framed as comparing to pathology or long-term outcomes data, but rather the accurate computation and display of dose distributions as per established physics and clinical planning principles.

7. The sample size for the training set:

   • The document does not specify a distinct "training set" for the Monaco RTP System. As a radiation treatment planning system, it relies on physics models and algorithms rather than machine learning models that typically require a training set in the AI sense. The development likely involved extensive testing and calibration against known physics principles and clinical data, which is distinct from a machine learning training set.

8. How the ground truth for the training set was established:

   • Since a distinct "training set" in the machine learning context is not mentioned, the concept of establishing ground truth for it is not applicable based on the provided text. The accuracy of the system is established through rigorous verification against physics models, calculations, and clinical expectations, rather than learning from a labeled training dataset.

Ask a specific question about this device

The Monaco system is used to make treatment plans with prescriptions for external beam radiation therapy. The system calculates dose for photon and electron treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179), when Dynamic Conformal Arc planning was added (K110730), and most recently when the system's intended use was expanded to include electron treatment planning, among other changes (K132971). The Monaco system accepts patient diagnostic imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of beam modifiers between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes. The Monaco system supports 3D conformal planning, IMRT, and Dynamic Conformal. It supports inverse and forward planning workflows.

The provided text describes the Monaco RTP System, a medical device for radiation treatment planning. It includes information on its intended use, technological characteristics, and a comparison with predicate devices. However, the document does not contain specific acceptance criteria, a detailed study proving the device meets acceptance criteria, or information on ground truth establishment, sample sizes for training/test sets, expert qualifications, or adjudication methods in the manner typically expected for AI/ML device submissions.

The document states:

• Clinical trials were not performed.
• Validation testing involved simulated clinical workflows.
• Over 600 test procedures were executed (verification tests) to ensure the system works as designed, including new functionality, risk mitigations, and regression tests.

Therefore, most of the requested information cannot be extracted from this document as it pertains to a different type of device (a treatment planning system, not an AI/ML diagnostic aid) and an earlier regulatory submission context where such detailed performance studies for AI/ML were not standard.

Here's a breakdown of what can be extracted and what cannot:

1. A table of acceptance criteria and the reported device performance:

This document does not present quantitative acceptance criteria or corresponding reported device performance metrics like sensitivity, specificity, or AUC, which are common for AI/ML devices. Instead, it states that "Monaco passed testing and was deemed safe and effective for its intended use." The "performance" described is in terms of functionality and passing verification tests.

2. Sample size used for the test set and the data provenance:

• Sample Size for Test Set: Not specified. The document mentions "over 600 test procedures" but doesn't detail the number of cases or data points used within these procedures.
• Data Provenance: Not specified. The testing involved "simulated clinical workflows," but the origin (e.g., country of origin of data, retrospective or prospective) of the data used in these simulations is not mentioned.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

This information is not provided. The testing described is verification-based ("system is working as designed"), not ground truth establishment by experts for specific diagnostic or prognostic outcomes.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

Not specified.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, and if so, what was the effect size of how much human readers improve with AI vs without AI assistance:

No MRMC study was performed or mentioned. The device is a treatment planning system, not an AI-assisted diagnostic device, and clinical trials were explicitly stated as not being performed.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

The device's performance is described as standalone in the sense that its functional verification tests ensure it performs its calculations and operations correctly. However, it's explicitly stated that "Monaco does not directly control the linear accelerator that delivers the radiation. Once completed, plans are reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place." This implies that the system is always used with human oversight, but its core calculation and planning functionalities are "standalone" in their execution. The document does not describe specific "standalone performance" metrics in the context of an AI/ML algorithm.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

The concept of "ground truth" as typically applied to diagnostic AI/ML devices based on expert consensus or pathology is not present in this document. The "truth" for this device's performance would be the accuracy of its physical dose calculations and the correct execution of its planning algorithms against established physics models and pre-defined test cases, rather than clinical outcomes or diagnostic interpretations.

8. The sample size for the training set:

The document does not refer to a "training set" as this is not an AI/ML device in the modern sense that learns from data. It's a deterministic software system.

9. How the ground truth for the training set was established:

Not applicable, as there is no "training set."

Summary of what is available from the document:

The Monaco RTP System is a radiation treatment planning system. Its regulatory submission (K151233) describes its intended use, technological characteristics, and compares it to predicate devices (Monaco K132971, AdvantageSim MD K132944) to demonstrate substantial equivalence.

Device Performance and Testing:

• Type of Study: Verification testing and simulated clinical workflows.
• Number of Tests: Over 600 test procedures were executed.
• Purpose of Tests: To verify requirements for new product functionality, ensure risk mitigations function as intended, and regression tests to ensure continued safety and effectiveness of existing functionality.
• Outcome: "Monaco passed testing and was deemed safe and effective for its intended use."
• Clinical Trials: Explicitly stated as not performed because "Clinical testing on patients is not advantageous in demonstrating substantial equivalence or safety and effectiveness of the device since testing can be performed such that no human subjects are exposed to risk."
• Human-in-the-Loop: The system's plans are always "reviewed and approved by qualified clinicians and may be subject to quality assurance practices before treatment actually takes place."
• "Level of Concern": Classified as "major level of concern" because "should a flaw in the treatment plan escape the notice of the qualified professionals using the Monaco system, serious injury or death could result."

Ask a specific question about this device

The Monaco system is used to make treatment plans for patients with prescriptions for external beam radiation therapy. The system calculates dose for photon treatment plans and displays, on-screen and in hard-copy, two- or three-dimensional radiation dose distributions inside patients for given treatment plan set-ups.

The Monaco product line is intended for use in radiation treatment planning. It uses generally accepted methods for:

• contouring
• image manipulation
• simulation
• image fusion
• plan optimization
• QA and plan review

Monaco is a radiation treatment planning system that first received FDA clearance in 2007 (K071938). The modified system received clearance in 2009, when Volumetric Modulated Arc Therapy (VMAT) planning capability was added (K091179). The Monaco system accepts patient diagnostic imaging data and "source" dosimetry data from a linear accelerator. The system then permits the user to display and define (contour) the target volume to be treated and critical structures which must not receive above a certain level of radiation on these diagnostic images.

Based on the prescribed dose, the user, a Dosimetrist or Medical Physicist, can create multiple treatment scenarios involving the number, position(s) and energy of radiation beams and the use of a multileaf collimator (MLC) between the source of radiation and the patient to shape the beam. The Monaco system then produces a display of radiation dose distribution within the patient, indicating doses to the target volume and surrounding structures. The "best" plan satisfying the prescription is then selected, one that maximizes dose to the target volume while minimizing dose to surrounding healthy volumes.

The Monaco RTP System is a radiation treatment planning system. Here's a breakdown of its acceptance criteria and the supporting study:

1. Table of Acceptance Criteria and Reported Device Performance

The provided summary does not explicitly list distinct, quantifiable acceptance criteria with corresponding performance metrics in a readily extractable table format for dose calculation or planning accuracy. Instead, it states that verification tests were "written and executed to ensure that the system is working as designed" and that "Pass/fail requirements and results of this testing can be found in section 18 of this submission." However, Section 18 is not included in the provided text.

Based on the available information, the general performance criteria can be inferred as:

2. Sample Size Used for the Test Set and the Data Provenance

The summary states that "Clinical trials were not performed as part of the development of this product." Instead, "Algorithm testing was performed to compare calculated against measured doses," and "clinically oriented validation test cases were written and executed in-house by CMS customer support personnel."

Therefore:

• Test Set Sample Size: Not specified in terms of number of patient cases. The testing involved "algorithm testing" (comparing calculated vs. measured doses) and an unspecified number of "clinically oriented validation test cases."
• Data Provenance: Not explicitly stated regarding origin (e.g., country). However, the testing was "in-house" by the manufacturer (Computerized Medical Systems, Inc., USA). This implies the data used for the algorithm and validation tests would be internally generated or sourced. The context suggests it was not patient data from clinical settings.
• Retrospective/Prospective: The testing appears to be retrospective in the sense that it did not involve prospective human subjects but rather validation against pre-existing data (measured doses) or simulated/representative cases for the "clinically oriented validation test cases."

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts

• Number of Experts: Not explicitly stated. The "clinically oriented validation test cases" were "written and executed in-house by CMS customer support personnel."
• Qualifications of Experts: The personnel were "CMS customer support personnel." While they handled "clinically oriented" test cases, their specific clinical qualifications (e.g., medical physicist, dosimetrist, or specific years of experience) are not provided. The summary also notes that plans are "reviewed and approved by qualified clinicians" in a clinical setting, but this refers to post-approval clinical use, not the ground truth establishment for the premarket testing.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for establishing ground truth for the test set. Since the testing involved "algorithm testing" comparing calculated against measured doses, and "clinically oriented validation test cases" executed in-house, it is unlikely a multi-expert adjudication method was employed in the traditional sense. The "ground truth" for algorithmic accuracy would be established by the physical measurements, and for validation cases, by adherence to predefined clinical expectations or specifications.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If so, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance?

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. The device is a radiation treatment planning system, not an AI-assisted diagnostic tool for human readers. Its primary function is to calculate dose and aid in plan creation, not to improve human reader performance in interpreting images or making diagnoses.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, a form of standalone performance assessment was done. "Algorithm testing was performed to compare calculated against measured doses to ensure dose calculation accuracy." This directly evaluates the algorithm's output (calculated dose) against an objective standard (measured dose) without a human-in-the-loop decision-making process. The "clinically oriented validation test cases" also assessed the system's ability to produce acceptable plans based on defined criteria.

7. The Type of Ground Truth Used

• For Algorithm Testing: The ground truth was measured doses. The summary states "Algorithm testing was performed to compare calculated against measured doses." This implies physical measurements were used as the gold standard.
• For "Clinically Oriented Validation Test Cases": The ground truth was based on predefined clinical expectations/specifications or internal standards established by the CMS customer support personnel who wrote and executed these cases.

8. The Sample Size for the Training Set

The document does not specify a separate "training set" sample size. The Monaco system is a radiation treatment planning system that calculates dose and optimizes plans based on established physics models and algorithms. It does not appear to be a machine learning model that requires a distinct "training set" in the common understanding of AI devices. Its development would involve calibration, verification, and validation, rather than a training process on a large dataset of patient images or outcomes.

9. How the Ground Truth for the Training Set Was Established

Since a "training set" in the context of machine learning is not mentioned or implied for this device, the method for establishing its ground truth is not applicable/not provided. The system's foundational accuracy would stem from its underlying physical models and their calibration, which would involve experimental data and established physics principles, rather than a labeled training dataset.

Ask a specific question about this device