Search Results

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation consists of multiple applications:

• The main RayStation application is used for treatment planning.
• The RayPhysics application is used for commissioning of treatment machines to make them available for treatment planning and used for commissioning of imaging systems.

The devices to be marketed, RayStation/RayPlan 2024A SP3, 2024A and 2023B, contain modified features compared to last cleared version RayStation 12A including:

• Improved sliding window VMAT (Volumetric Modulated Arc Therapy) sequencing
• Higher dose grid resolution for proton PBS (Pencil Beam Scanning)
• Automated field in field planning
• LET optimization (Linear Energy Transfer)

These applications are built on a software platform, containing the radiotherapy domain model and providing GUI, optimization, dose calculation and storage services. The platform uses three Microsoft SQL databases for persistent storage of the patient, machine and clinic settings data.

As a treatment planning system, RayStation aims to be an extensive software toolbox for generating and evaluating various types of radiotherapy treatment plans. RayStation supports a wide variety of radiotherapy treatment techniques and features an extensive range of tools for manual or semi-automatic treatment planning.

The RayStation application is divided in modules, which are activated through licensing. A simplified license configuration of RayStation is marketed as RayPlan.

The provided document is a 510(k) clearance letter for the RayStation/RayPlan 2024A SP3, 2024A, and 2023B devices. It discusses the substantial equivalence of these devices to a predicate device (RayStation 12A).

However, the document does not contain specific acceptance criteria tables nor detailed study results for a single, comprehensive study proving the device meets acceptance criteria in the format typically requested (e.g., a specific clinical validation study with explicitly defined acceptance metrics like sensitivity, specificity, or AUC, and corresponding reported performance values).

Instead, the document describes a broad software verification and validation process, stating that the software underwent:

• Unit Testing
• Integration Testing
• System Level Testing
• Cybersecurity Testing
• Usability Testing (Validation in a clinical environment)
• Regression Testing

For several "Added/updated functions," the document provides a description of the verification and validation data used to demonstrate substantial equivalence and simply states "Yes" under the "Substantially Equivalent?" column if the validation was "successful." The acceptance criteria for these tests are described narratively within the text, not in a consolidated table format with numerical performance outcomes.

Therefore, I cannot generate the requested table of "acceptance criteria and the reported device performance" as a single, consolidated table with numerical results for the entire device's performance against specific, pre-defined acceptance criteria for a single study. The document describes a process of demonstrating substantial equivalence through various verification and validation activities rather than a single, large-scale study with quantitative acceptance criteria for the overall device performance.

However, I can extract the information related to the validation activities for specific features and the general approach to proving substantial equivalence.

Here's a breakdown of the requested information based on the provided document, addressing each point to the best of my ability given the available details:

Acceptance Criteria and Device Performance (Based on provided verification and validation descriptions)

As noted, a single, consolidated table of quantitative acceptance criteria and overall device performance is not provided. Instead, the document describes various verification and validation activities with implicit or explicit pass criteria for individual features or system aspects to demonstrate substantial equivalence to the predicate device.

Below are examples of how some "acceptance criteria" (pass criteria) and "reported performance" are described for specific features. These are not aggregated performance metrics for the entire device but rather success criteria for sub-components or changes.

2. Calculated values are numerically equal to values obtained from calling the method (regression test).
3. Calculated values are exported correctly from RayStation to DICOM (equality between calculated and exported point, only for Helical Tomo Synchrony plans, only in correct DICOM item).
4. Calculated values are converted correctly from DICOM to Accuray's system format (equality of point coordinates, only for relevant plan types). | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies all pass criteria were met). |
   | Point-dose optimization in brachy plans | 1. Position from the correct image set is used for point-dose objectives/constraints.
5. Possible to add optimization objective/constraint to a point, referring to the correct point.
6. When adding objective/constraint, selection of function type and dose level is possible and reflected in description.
7. Saving and loading an optimization function template containing point objectives/constraints works correctly (loaded functions are same as saved).
8. Results from single/multiple point optimization are as expected (dose in point(s) should be equal to specified dose in objective(s)). | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies all pass criteria were met). |
   | Electron Monte Carlo dose engine improvements | Comparing calculated doses with:

• Measured doses obtained from clinics,
• Doses computed in independent, well-established TPS,
• Doses computed with earlier versions of RayStation,
• Doses computed in BEAMnrc/egs++
  using Gamma evaluation criteria. | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies adequate agreement based on Gamma criteria). |
  | Evaluation on converted CBCT images for protons | For proton MC/PB dose computation:
• Gamma 2%/2mm pass rate above 90%
• Gamma 3%/3mm pass rate above 95% | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies specified Gamma pass rates were achieved). |
  | Overall Device (Software Verification/Validation) | Software specifications conform to user needs and intended uses, and particular requirements implemented through software can be consistently fulfilled. Conformance to applicable requirements and specifications. Successful outcome of unit, integration, system, cybersecurity, usability, and regression testing. Safety and effectiveness validated. | "RayStation/RayPlan 2024A SP3, 2024A and 2023B have met specifications and are as safe, as effective and perform as well as the legally marketed predicate devices." All general software tests (unit, integration, system, cybersecurity, usability, regression) were acceptable/successful. |

Study Details (Based on the document)

Given the nature of the 510(k) submission for a treatment planning system, the "study" is primarily a comprehensive software verification and validation effort to demonstrate substantial equivalence, rather than a single, standalone clinical trial or diagnostic accuracy study.

1. Sample sizes used for the test set and the data provenance:

   • Test Set Sample Sizes: Not explicitly stated as a single numerical value for a global "test set." Testing was conducted at multiple levels (unit, integration, system, usability, regression) across various features.
     • For "Evaluation on converted CBCT images for protons," it mentions "Test cases consist of CBCTs from the MedPhoton imaging ring on a Mevion S250i system, as well as the on-board CBCT systems on a Varian ProBeam and an IBA P1," implying a set of patient or phantom imaging data, but the exact number of cases/patients is not specified.
     • For other features, it refers to "tests," "validation data," or "computed doses" but doesn't quantify the number of distinct data points/cases used.
   • Data Provenance:
     • Country of Origin: Not specified in the document. Likely internal RaySearch data and potentially data from collaboration with clinical sites, but no specific countries are mentioned.
     • Retrospective or Prospective: Not explicitly stated. The verification and validation activities appear to be primarily retrospective (using existing data, phantom measurements, or simulated scenarios) as part of the software development lifecycle, rather than prospective clinical data collection for a specific study.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • This information is not provided in the document. The document refers to "measured doses obtained from clinics" and "doses computed in independent, well-established TPS" as part of the validation for dose engine improvements, suggesting some form of external or expert-derived ground truth, but the number and qualifications of experts involved are not detailed. For "Evaluation on converted CBCT images for protons," it states "For each case, a ground truth CT image has been prepared to serve as ground truth," implying expert or established reference standard, but again, no details on experts.

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • This information is not provided. The document focuses on computational and functional verification rather than multi-reader clinical assessment.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not explicitly done or reported in this document. The device, RayStation/RayPlan, is a treatment planning system that assists users in creating treatment plans, not primarily an AI-driven image interpretation or diagnostic aid where human reader performance improvement is typically measured. The AI-related feature mentioned is "deep learning segmentation," but the document states, "(The model training is performed offline on clinical CT and structure data.)" It does not detail an MRMC study related to its performance or impact on human readers.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, standalone (algorithm-only) performance was central to the validation. The document describes extensive "Unit Testing," "Integration Testing," "System Level Testing," and "Dose engine validation" which are all a form of standalone algorithmic evaluation. For example, the Gamma evaluation criteria for dose calculations or the numerical equality checks for dose compensation points are purely algorithmic performance assessments.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • The ground truth varied depending on the feature being validated:
     • "Measured doses" from clinics / Independent TPS computations / BEAMnrc/egs++ calculations: For dose engine validation. This represents a highly accurate, often physical measurement or well-established computational standard.
     • "Ground truth CT image": For evaluation of converted CBCT images for protons. This implies a high-quality reference image.
     • Internal "expected results" and "specifications": For functional and system-level tests (e.g., for point-dose optimization, the expected result was that the dose in the point should equal the dose specified in the objective).
     • "Clinical objectives": Used for plan comparisons (e.g., in segment weight optimization validation), likely representing desired dose distributions defined by clinical experts.

7. The sample size for the training set:

   • The document mentions "deep learning segmentation" and states that "The model training is performed offline on clinical CT and structure data." However, the sample size for this training set is not provided.

8. How the ground truth for the training set was established:

   • For "deep learning segmentation," the ground truth for training would implicitly be the "clinical CT and structure data" mentioned. This typically means expert-delineated structures (ROIs) on clinical CT images, but the exact method (e.g., single expert, consensus, specific software tools) is not detailed.

Ask a specific question about this device

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation is a treatment planning system for planning, analysis and administration of radiation therapy and medical oncology treatment plans. The device lets the user import patient images and data, identify treatment targets and organs at risk, create an optimal treatment plan taking into account patient anatomy, prescribe treatment dose and organ at risk sensitivity, review and approve the plan and then administer the treatment. A scientific basis for the device is the implementation of peer reviewed algorithms of plan parameter optimization and photon and particle dose calculation.

RayStation consists of multiple applications:

• The main RayStation application is used for treatment planning.
• The RayPhysics application is used for commissioning of treatment machines to make them available for treatment planning and used for commissioning of imaging systems.
• The RayTreat application is used for sending plans to treatment delivery devices for treatment and receiving records of performed treatments.

These applications are built on a software platform, containing the radiotherapy domain model and providing GUI, optimization, dose calculation and storage services. The platform uses three Microsoft SQL databases for persistent storage of the patient, machine and clinic settings data.

The RayStation application is divided in modules, which are activated through licensing. A simplified license configuration of RayStation is marketed as RayPlan has a limited set of modules, indicated in the following table.

Here's a breakdown of the acceptance criteria and study information for RayStation 12A, based on the provided text:

Acceptance Criteria and Device Performance

Study Information:

2. Sample sizes used for the test set and the data provenance

• Eye Planning with Wedges: Specific sample size for test cases is not provided, but they cover "line doses in homogeneous phantoms using a square aperture and a wedge mounted with varying opening angles and positions." The origin is internal testing/validation.
• Automatic Field in Field Planning: Specific sample size for test cases is not provided. The origin is internal testing/validation.
• Brachy Therapy Support for Elekta Flexitron® afterloaders: Specific sample size for test cases is not provided. Reference doses consist of "point doses, line doses, as well as 2D and 3D doses." Origin includes published consensus data, measured doses, doses computed in two major competing TPS, and doses computed with an independent Monte Carlo software.
• Electron Monte Carlo Dose Engine Update: Specific sample size for test cases is not provided. Reference doses include measured doses, doses computed in a well-established competing TPS, doses computed with earlier versions of RayStation, and doses computed in BEAMnrc/egs++. Data provenance is thus mixed, including internal comparisons, external commercial TPS, and external academic/research software.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The document does not specify the number or qualifications of experts used to establish ground truth for any of the test sets. The ground truth for dose calculation is primarily based on:

• Published consensus data
• Measured doses
• Comparison to other established commercial Treatment Planning Systems (TPS)
• Comparison to independent Monte Carlo software

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

The document does not describe any adjudication method involving human experts for resolving discrepancies in the test sets. The comparisons are quantitative against established reference data or other computational models.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

A multi-reader multi-case (MRMC) comparative effectiveness study was not mentioned or performed. This device is a treatment planning system, not an AI-assisted diagnostic tool that would typically involve human "readers" in that context. The improvements are in the accuracy and functionality of the dose calculation and planning algorithms, not in improving human interpretation of an AI output.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, the studies described are standalone performance evaluations of the algorithm's accuracy in specific calculation tasks (dose computation for various modalities and planning features). The "device performance" described in the table above refers to the algorithm's performance against reference standards.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used is primarily based on:

• Physical Measurements: "Measured doses," "depth-dose profiles along the central axis were acquired with a plane-parallel chamber in a water tank."
• Published Consensus Data: For brachytherapy, "Published consensus data."
• Established Computational Models: Doses computed in well-established competing TPS, independent Monte Carlo software (BEAMnrc/egs++), and comparison to previous versions of RayStation.

8. The sample size for the training set

The document does not provide any information regarding the sample size for a training set. The descriptions focus on the validation of specific algorithms and features, which often rely on physics models and deterministic calculations rather than machine learning training sets. While "deep learning segmentation" is mentioned as a module (available in RayStation 11B and 12A), the provided validation details for the new features in 12A do not describe training data for this specific version's changes.

9. How the ground truth for the training set was established

As no training set information is provided for the features specifically validated in K222312, there is no information on how its ground truth would have been established. For the general mention of "deep learning segmentation," the text indicates "The model training is performed offline on clinical CT and structure data," but no further details on ground truth establishment for this training.

Ask a specific question about this device

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation is a treatment planning system for planning, analysis and administration of radiation therapy and medical oncology treatment plans. It has a modern user interface and is equipped with fast and accurate dose and optimization engines.

RayStation consists of multiple applications:

• . The main RayStation application is used for treatment planning.
• . The RayPhysics application is used for commissioning of treatment machines to make them available for treatment planning and used for commissioning of imaging systems.
• The RayTreat application is used for sending plans to treatment delivery devices for treatment and receiving records of performed treatments.

The device to be marketed, RayStation 11B, contains modified features compared to RayStation 11.0 as indicated below:

A simplified license configuration of RayStation is marketed as RayPlan. RayPlan has a limited set of purchasable licenses and some modules will not be accessible. RayPlan is marketed as RayPlan 11B.

EQD2 dose computation (new) - From photon and/or braction doses, it is possible to compute, deform and accumulate the two Gray equivalent (EQD2) dose. The computation of the EQD2 dose uses the biological linear quadratic model, which is also the basis for the already released biological optimization and evaluation functionality.

Generation of synthetic CT from CBCT (new) - Two new methods (algorithms) for synthetic CT generation will be included. The synthetic CT images are created by combining information in the CBCT image and a CT image for the specific patient to allow for dose computation using the HU values in the image, as for regular CT images. In RayStation 11.0 it is possible to compute dose on CBCT images for photons using bulk density assignments. The added functionality will improve the photon dose calculation accuracy on CBCT images. Handling of LET and other RBE components (new) - This functionality enables possibility to compute and evaluate the dose weighted LET (Linear Energy Transfer) for proton and light ion plans. LET is an additional dosimetric measure that can be used to assess the radiobiological effect of the proton and light ion radiation. Radiobiological equivalent (RBE) dose is a derived quantity with dependence on both the physical dose and the LET. In RayStation 11.0, it is possible to compute and evaluate RBE doses.

The provided text describes a 510(k) submission for RayStation 11B, a radiation therapy treatment planning system. However, the document does not contain the detailed information required to answer many of the questions about acceptance criteria and the study that proves the device meets them, especially regarding the performance of a machine learning component if one exists and its specific acceptance criteria.

The document states: "Related to machine learning, there is no change compared to the predicate device." This implies that while the RayStation product family might have machine learning components, RayStation 11B does not introduce new machine learning features that would necessitate specific performance studies for acceptance as part of this 510(k) submission. The focus of the provided text is on the validation of new dose computation methods and synthetic CT generation.

Therefore, many sections below will indicate "Information Not Provided in Text" or state that the question is not applicable given the document's content.

Acceptance Criteria and Reported Device Performance

Given the information provided, the acceptance criteria are largely related to the successful verification and validation of the new features (EQD2 dose computation, synthetic CT generation from CBCT, and handling of LET and other RBE components) demonstrating that the dose computations are "adequate for clinical use" and that the system "has met specifications and is as safe, as effective and performs as well as or better than the legally marketed predicate device."

Without specific numerical thresholds or performance metrics in the provided text, a formal table of acceptance criteria and reported performance cannot be fully constructed for these new features. The document suggests that the previous predicate device's performance benchmarks likely served as the implicit standard for "as well as or better than."

Implicit Acceptance Criteria and Reported Performance (based on text):

Study Information

1. A table of acceptance criteria and the reported device performance
   See table above. Specific numerical thresholds are not provided in the text.

2. Sample sizes used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

   • Sample Sizes: Not explicitly stated for any of the validations (e.g., Clinical Evaluation for Brachy, CBCT Conversion validation, Proton PBS Monte Carlo validation). The text mentions "User Site Validation," which implies real-world data, but details on the number of cases or patients are absent.
   • Data Provenance: Not specified (e.g., country of origin).
   • Retrospective or Prospective: Not specified. "User Site Validation" might imply prospective or real-world use data, but this is not confirmed.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

   • Number of Experts: Not specified.
   • Qualifications of Experts: Not specified. Mention of "User Site Validation in cooperation with cancer clinics" suggests involvement of clinical professionals, but their specific roles, number, or qualifications are not detailed.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

   • Adjudication Method: Not specified.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

   • An MRMC study is not mentioned for the new features. The document explicitly states: "Related to machine learning, there is no change compared to the predicate device." This indicates that no new AI-assisted workflows requiring human reader improvement studies were part of this submission. The validation efforts focus on the accuracy of the new computational functions themselves, not human-AI interaction.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

   • Yes, the validation activities described (e.g., "EQD2 dose computation (new) – Validation," "Generation of synthetic CT from CBCT (new) – Validation," "Handling of LET and other RBE components (new) - Validation") are, by their nature, evaluations of the algorithm's performance in generating accurate dose calculations or synthetic CTs. These are standalone evaluations of the new features. The "User Site Validation" is likely a real-world system test, but the core 'algorithm-only' performance is implied by the specific validation names.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

   • The term "ground truth" is not explicitly used, but the validation implies comparisons against established or expected physical and radiobiological models and potentially clinical standards of care.
     • For dose computation, validation would typically involve comparing computed doses against known physics models, phantom measurements, or existing clinical systems deemed accurate.
     • For synthetic CT, the "ground truth" would likely be the actual CT data that the synthetic CT is intended to replicate or approximate for dose calculation purposes.
   • The text doesn't specify if clinical outcomes were used as ground truth for these specific validations.

8. The sample size for the training set

   • Information Not Provided in Text. The document states "Related to machine learning, there is no change compared to the predicate device," and the new features are described as "algorithms" or "functionality," not explicitly machine learning models that would require a distinct training set for this submission. If the original RayStation product had ML components, their training set information is not part of this document.

9. How the ground truth for the training set was established

   • Information Not Provided in Text. (See point 8).

In summary, the provided FDA 510(k) summary focuses on demonstrating the substantial equivalence of RayStation 11B to its predicate device by verifying and validating new computational features rather than new or modified AI/ML features requiring human-in-the-loop or specific training/test set performance metrics commonly associated with AI/ML device clearances. The acceptance criteria are implicitly tied to the successful completion of these verification and validation activities, proving the new functions are "adequate for clinical use" and the system performs "as well as or better than" the predicate.

Ask a specific question about this device

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

RayStation is a treatment planning system for planning, analysis and administration of radiation therapy and medical oncology treatment plans. It has a modern user interface and is equipped with fast and accurate dose and optimization engines.

RayStation consists of multiple applications:

• . The main RayStation application is used for treatment planning.
• . The RayPhysics application is used for commissioning of treatment machines to make them available for treatment planning and used for commissioning of imaging systems.
• . The RayTreat application is used for sending plans to treatment delivery devices for treatment and receiving records of performed treatments.

The device to be marketed, RayStation 11.0, marketing name "RayStation 11A", contains modified features compared to version 10.1. The main change is planning for the CyberKnife linear accelerator treatment unit.

The device to be marketed supports planning and dose calculation for M6/S7 CyberKnife treatment machines. Older CyberKnife versions are not supported. The CyberKnife has one delivery technique, and different node sets. The node set defines the possible robot positions for different collimations (cones or MLC) and patient geometry (head or body). All node sets available for patient treatments are included in the validation.

The document provided does not contain specific acceptance criteria for a device in the traditional sense of a medical diagnostic or AI-powered device. Instead, it describes a software system, RayStation 11.0, which is a radiation therapy treatment planning system. The document focuses on demonstrating substantial equivalence to a predicate device (RayStation 10.1) and validating its functionality for a new feature: planning for CyberKnife linear accelerator treatment units.

Therefore, the "acceptance criteria" discussed are more about the software's performance meeting its specifications and being suitable for clinical use, rather than quantitative performance metrics for a diagnostic output.

Based on the provided text, here's an attempt to answer your questions by interpreting "acceptance criteria" as the measures taken to validate the software's functionality and safety:

Table of Acceptance Criteria and Reported Device Performance

Given that RayStation is a treatment planning system and not a diagnostic device with specific output metrics like sensitivity/specificity, the "acceptance criteria" are related to the accuracy and safety of its dose calculations and overall system functionality for treatment planning. The document broadly states the "validation shows that the dose computation is suited for clinical use" and "adequate for clinical use."

Study that proves the device meets the acceptance criteria:

The provided document describes the verification and validation activities performed for RayStation 11.0. This is not a single "study" in the sense of a clinical trial with a defined sample size of patients with a specific condition. Instead, it's a comprehensive software development and quality assurance process.

1. Sample sizes used for the test set and the data provenance:

• Test Set Sample Size: The document does not specify a numerical "sample size" for a typical test dataset as one would expect in an AI/diagnostic device study. Instead, it refers to various testing activities:
  • CyberKnife Validation: "All node sets available for patient treatments are included in the validation." This implies comprehensive testing across different configurations of the CyberKnife treatment unit. It doesn't quantify the number of patient plans or variations tested.
  • System Tests, Unit/Subsystem Tests, Risk Analysis-based Tests: These are internal software testing activities, and their "sample size" would be defined by the number of test cases executed, which is not detailed.
  • User Validation: No specific number of cases or users is given.
• Data Provenance: Not explicitly stated for specific test cases. Given it's a software for radiation therapy planning, the "data" would consist of simulated patient anatomies, treatment plans, and test configurations. The "user validation in cooperation with cancer clinics" implies real-world clinical context, but details are not provided. It's implicitly retrospective as it's part of a software release validation. No country of origin is specified for the data used in testing.

2. Number of experts used to establish the ground truth for the test set and their qualifications:

• CyberKnife Validation: The validation was performed internally by RaySearch Laboratories, likely by their physicists and engineers. The document does not specify the number of external experts or their qualifications for establishing a "ground truth" for the dose calculations. The "ground truth" for dose calculations would be established through established physics models and comparisons to known phantoms or reference data, not necessarily expert consensus in the typical sense of image interpretation.
• User Validation: Conducted "in cooperation with cancer clinics." This indicates involvement of clinical experts (e.g., radiation oncologists, medical physicists) who would evaluate the usability and clinical appropriateness of the plans generated. The number and qualifications of these experts are not specified.
• General: The document does not mention the use of experts to establish 'ground truth' in the context of diagnostic interpretation, which is common for AI-powered image analysis tools. For a treatment planning system, 'ground truth' relates to the accuracy of physics calculations and the clinical utility of the generated plans.

3. Adjudication method for the test set:

• No explicit adjudication method (e.g., "2+1" or "3+1") is described, as the validation is not focused on subjective interpretation or a diagnostic outcome requiring such consensus. For dose calculations, discrepancies would be resolved by identifying and correcting calculation errors. For user validation, feedback would be gathered and addressed.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, and its effect size:

• No MRMC study was performed or described. The study type is focused on validating the software's internal performance and its capability to generate treatment plans, not on how human readers (e.g., radiologists) improved their performance using the software. RayStation is a planning tool, not an AI-assisted diagnostic tool for human readers.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

• Yes, standalone performance was assessed for core functionalities. The "Dose engine validation" for the CyberKnife is a standalone assessment of the algorithm's accuracy in calculating dose. "System Tests," "Unit and subsystem testing," and "Risk analysis-based tests" also represent assessments of the algorithm's performance without direct human intervention during the test execution. The "user validation" component does involve humans in the loop to assess usability and clinical utility.

6. The type of ground truth used:

• For Dose Calculations: The ground truth for dose engine validation would be based on physical principles, established dose measurement techniques (e.g., phantom measurements), and validated physics models. It's about the objective accuracy of numerical calculations.
• For System Functionality: The ground truth for system tests would be the expected output or behavior based on the software's design specifications and requirements.
• For Clinical Appropriateness/Usability (User Validation): This would be based on expert clinical judgment from the cancer clinics involved, but it's not a "ground truth" for a diagnostic outcome, rather an assessment of the tool's utility.

7. The sample size for the training set:

• Not applicable / Not provided. RayStation is a physics-based treatment planning system, not typically an AI/machine learning model in the sense of being "trained" on a large dataset of patient images or outcomes to learn patterns. While it might contain machine learning features (the document mentions "Related to machine learning, there is no change compared to the predicate device," implying some existing ML, but not as the primary function or new feature for the 11.0 update), the core CyberKnife planning feature validated here is based on physics algorithms, not a trained model. Therefore, there's no "training set" in the context of supervised machine learning.

8. How the ground truth for the training set was established:

• Not applicable / Not provided. As explained above, the core functionality validated (CyberKnife planning) is algorithm-driven, not learned from a training set. If there are existing ML components, the document doesn't detail their training or ground truth establishment.

Ask a specific question about this device

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation is a treatment planning system for planning, analysis and administration of radiation therapy and medical oncology treatment plans. It has a modern user interface and is equipped with fast and accurate dose and optimization engines.

RayStation consists of multiple applications:

• The main RayStation application is used for treatment planning.
• . The RayPhysics application is used for commissioning of treatment machines to make them available for treatment planning and used for commissioning of imaging systems.
• . The RayTreat application is used for sending plans to treatment delivery devices for treatment and receiving records of performed treatments.
• o The RayCommand application is used for treatment session management including treatment preparation and sending instructions to the treatment delivery devices.

The provided text details the 510(k) summary for RayStation 10.1, a software system for radiation therapy and medical oncology. The document indicates that the determination of substantial equivalence to the primary predicate device (RayStation 9.1) is not based on an assessment of non-clinical performance data. Instead, it relies on the entire system verification and validation specifications and reports.

However, the document does describe the performance data for several new features and explicitly states that these features have been "successfully validated for accuracy in clinically relevant settings according to specification" or "successfully validated according to specification." While these statements imply acceptance criteria were met, the specific numerical acceptance criteria and the reported device performance values are not explicitly provided in a comparative table format within the given text.

Therefore, the following response will extract the implied acceptance criteria and reported performance from the descriptions provided, and note where specific numerical values are absent.

Acceptance Criteria and Device Performance Study for RayStation 10.1

The provided document, K210645 for RayStation 10.1, indicates that the determination of substantial equivalence to the primary predicate device (RayStation 9.1) is not based on non-clinical performance data directly comparing the existing features of 10.1 to 9.1. Instead, it relies on the comprehensive system verification and validation reports.

However, for the new features introduced in RayStation 10.1, the document states that these features have undergone validation and met their respective specifications. While specific, quantifiable acceptance criteria and reported performance values are not presented in a direct comparative table within the provided text, the descriptions imply the following:

1. A table of acceptance criteria and the reported device performance

Note: The provided text does not contain specific numerical acceptance criteria (e.g., "accuracy within X%") or quantitative reported performance data for any of these features. The reported performance essentially states that the criteria were "met" or "validated."

2. Sample size used for the test set and the data provenance

The document indicates that RayStation 10.1's test specification is a further developed version of RayStation 9.1's, supported by requirements specification. The verification activities included "User validation in cooperation with cancer clinics." However, no specific sample sizes for test sets (e.g., number of patient cases) or data provenance (e.g., country of origin, retrospective/prospective nature) are provided in the given text for any of the validations.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The text mentions "User validation in cooperation with cancer clinics" but does not specify the number of experts, their qualifications, or how ground truth was established for the "test set" (if a distinct clinical test set was used for ground truth establishment).

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

No information on adjudication methods is provided in the supplied text.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

The text does not mention any multi-reader multi-case (MRMC) comparative effectiveness study, nor does it discuss human readers improving with or without AI assistance. The device is a treatment planning system, and while it states it "proposes treatment plans" based on user input, it does not describe AI-assisted diagnostic or interpretation tasks. It explicitly states, "Related to machine learning, there is no change compared to the primary predicate device." suggesting limited or no direct machine learning components in the new features where such a study would typically be relevant.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

The validations described for brachytherapy, medical oncology, proton ocular treatment, and robust planning of organ motion appear to be standalone algorithm performance assessments against defined specifications. These validations verify the accuracy and appropriateness of the software's calculations and functionalities independently, assuming "intended qualified user" interaction for medical oncology, but not as part of a human-in-the-loop performance study.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

For the dose calculation features (brachytherapy, proton ocular treatment), the "ground truth" implicitly refers to theoretical models and established physical principles (e.g., accurate modeling of TG43 formalism, proton dose calculations) as compared against the output of the software. For medical oncology functions, the "ground truth" for validation appears to be whether the functions are "appropriate" for planning workflows, likely assessed against clinical guidelines or expert workflows. For robust planning of organ motion, the ground truth relates to the correct generation and application of deformed image sets according to specifications. The document does not explicitly state that ground truth was established through pathology or outcomes data.

8. The sample size for the training set

The document refers to the system as "built on the same software platform" and "developed under the same quality system, by the same development teams." It mentions that "related to machine learning, there is no change compared to the primary predicate device." Given this, and the nature of treatment planning software, the concept of a "training set" in the context of machine learning (e.g., for image classification or prediction models) is not directly applicable or discussed for the validations mentioned. The system's development would involve software engineering and clinical validation rather than machine learning training sets for the described functionalities.

9. How the ground truth for the training set was established

As there is no mention of a training set or machine learning components for the new features, information on how its ground truth was established is not provided.

Ask a specific question about this device

RayStation is a software system designed for treatment planning and analysis of radiation therapy. The treatment plans provide treatment unit set-up parameters and estimates of dose distributions expected during the proposed treatment, and may be used to administer treatments after review and approval by the intended user.

The system functionality can be configured based on user needs.

The intended users of RayStation shall be clinically qualified radiation therapy staff trained in using the system.

RayStation 9.1/RayStation 9.2 are radiation therapy treatment planning systems, i.e. a software program for planning and analysis of radiation therapy. The functionality includes fusion capabilities (CT, PET and MRI), contouring, collapsed cone convolution dose computation, 4D data compatibility and treatment console interfacing, as well as unique features such as machine learning planning and segmentation, multi-criteria optimization, dose tracking, treatment adaptation and deformable registration, all available in one platform.

The provided text primarily focuses on the FDA 510(k) clearance for RayStation 9.1/9.2, comparing it to a predicate device (RayStation 8.1). While it mentions "test specifications," "successful verification and validation," and "non-clinical tests," it does not contain the detailed information required to answer the questions about acceptance criteria, study design, sample sizes, expert involvement, or specific performance metrics related to a device's performance against acceptance criteria.

The document states:

• "The test specifications of RayStation 9.2 are further developed versions of the test specifications of RayStation 8.1."
• "The successful verification and validation of RayStation 9.2 therefore support the substantial equivalence of the above RayStation versions."
• "The summary of the performed non-clinical tests shows that RayStation 9.1/RayStation 9.2 are as safe and effective and performs as well as the predicate device."

These statements indicate that testing was performed to demonstrate substantial equivalence, but they do not provide the details of those tests. The document is about regulatory clearance based on substantial equivalence, not a clinical performance study with specific acceptance criteria and detailed results.

Therefore, I cannot extract the requested information from the provided text. To answer your questions, I would need a description of a study specifically designed to prove the device meets acceptance criteria, including quantitative performance data.

Ask a specific question about this device

RayStation is a software system designed for treatment planning and analysis of radiation therapy. The treatment plans provide treatment unit set-up parameters and estimates of dose distributions expected during the proposed treatment, and may be used to administer treatments after review and approval by the intended user.

The system functionality can be configured based on user needs.

The intended users of RayStation shall be clinically qualified radiation therapy staff trained in using the system.

RayStation 8.1 is a radiation therapy treatment planning system, i.e. a software program for planning, management and analysis of radiation therapy treatment. The functionality includes fusion capabilities (CT, PET and MRI), contouring, collapsed cone convolution dose computation, 4D data compatibility and treatment console interfacing, as well as unique features such as machine learning and segmentation, multi-criteria optimization, dose tracking, treatment adaptation and deformable registration, all available in one platform.

The provided text is a 510(k) Summary for RayStation 8.1. It details the device's intended use, technological characteristics, and a comparison to its predicate device, RayStation 7.0. However, it does not contain acceptance criteria or a detailed study description with performance metrics, sample sizes, ground truth establishment, or expert qualifications for the new functionality mentioned (machine learning and segmentation).

The document mainly focuses on establishing substantial equivalence to the predicate device, RayStation 7.0, based on shared underlying software platform and quality system. It states that "The test specification of RayStation 8.1 is a further developed version of the test specification of RayStation 7.0. This is supported by the requirements specification, for which the same is true. The successful verification and validation of RayStation 8.1 therefore support the substantial equivalence of the above RayStation versions."

Therefore, based only on the provided text, the specific details requested regarding acceptance criteria and a study proving their fulfillment are not available. The document asserts that RayStation 8.1 performs "as well as the predicate device" based on verification and validation against an updated version of the previous device's test and requirements specifications, but it does not provide the specifics of these tests or their results in a measurable format.

To answer your request, I will indicate that the information is not present in the provided text for each point.

1. A table of acceptance criteria and the reported device performance

• Acceptance Criteria: Not provided in the document.
• Reported Device Performance: Not provided in a quantifiable manner for specific acceptance criteria. The document states "The summary of the performed non-clinical tests shows that RayStation 8.1 is as safe and effective and performs as well as the predicate device," but no specific metrics are given.

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Not provided in the document.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

• Not provided in the document.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Not provided in the document.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• Not provided in the document. The document primarily focuses on demonstrating substantial equivalence to a predicate device, not on MRMC studies for AI assistance. The mention of "machine learning and segmentation" is under new features, but no specific performance study for these is detailed.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• Not provided in the document.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• Not provided in the document.

8. The sample size for the training set

• Not provided in the document.

9. How the ground truth for the training set was established

• Not provided in the document.

Ask a specific question about this device

RayStation is a software system designed for treatment planning and analysis of radiation therapy. The treatment plans provide treatment unit set-up parameters and estimates of dose distributions expected during the proposed treatment, and may be used to administer treatments after review and approval by the intended user.

The system functionality can be configured based on user needs.

The intended users of RayStation shall be clinically qualified radiation therapy staff trained in using the system.

RayStation 7.0 is a radiation therapy treatment planning system, i.e. a software program for planning, management and analysis of radiation therapy treatment. The functionality includes fusion capabilities (CT, PET and MRI), contouring, collapsed cone convolution dose computation, 4D data compatibility and treatment console interfacing, as well as unique features such as multi-criteria optimization, dose tracking, treatment adaptation and deformable registration, all available in one platform.

This document is a 510(k) premarket notification for RayStation 7.0, a radiation therapy treatment planning system. It demonstrates the device's substantial equivalence to a legally marketed predicate device (RayStation 6.1). However, the document does not contain specific acceptance criteria, performance data, or details about a study proving the device meets these criteria in the manner typically associated with an AI/machine learning device submission.

The provided text focuses on:

• Regulatory information: FDA approval, classification, general controls, and compliance requirements.
• Device description: RayStation 7.0 as a software for radiation therapy planning, its workflow, and functionalities.
• Comparison to predicate device: Stating that the technological characteristics are the same, with improvements in usability, computational speed, and connectivity.
• Non-clinical performance data assessment: A general statement that the test specification of RayStation 7.0 is a "further developed version" of RayStation 6.1's, and successful verification and validation support substantial equivalence.

Therefore, I cannot provide the requested information about acceptance criteria and a study proving device performance because the provided document does not contain this level of detail for RayStation 7.0.

The document implies that the device's performance is demonstrated through its successful verification and validation against an updated test specification derived from the predicate device. However, it does not detail:

• Specific quantitative acceptance criteria (e.g., accuracy, precision metrics).
• Reported device performance against those criteria.
• Sample sizes for test sets, data provenance, expert qualifications, or ground truth establishment methods as you would expect for typical AI/ML medical device submissions.
• MRMC studies or standalone AI performance.

It's important to note that RayStation 7.0, classified as a "Medical charged-particle radiation therapy system," is primarily a treatment planning software and not explicitly presented as an AI/ML diagnostic or predictive device in this document. The "improvements" mentioned are general software enhancements rather than specific algorithmic improvements subject to the rigorous validation studies you've outlined in your request.

Ask a specific question about this device

RayStation is a software system designed for treatment planning and analysis of radiation therapy. The treatment plans provide treatment unit set-up parameters and estimates of dose distributions expected during the proposed treatment, and may be used to administer treatments after review and approval by the intended user.

The system functionality can be configured based on user needs.

The intended users of RayStation shall be clinically qualified radiation therapy staff trained in using the system.

RayStation 6.1 is a radiation therapy treatment planning system, i.e. a software program for planning and analysis of radiation therapy treatment plans. Functionality includes fusion capabilities (CT, PET and MRI), contouring, collapsed cone convolution dose computation and 4D data compatibility, as well as unique features such as multi-criteria optimization, dose tracking, treatment adaptation and deformable registration, all available on one platform.

I'm sorry, but the provided text does not contain any information about acceptance criteria, device performance, sample sizes, data provenance, expert qualifications, ground truth establishment, MRMC studies, or standalone performance testing for a medical device.

The document is a 510(k) premarket notification for a software system called "RayStation 6.1," which is a radiation therapy treatment planning system. It discusses the regulatory approval process, classification, device description, intended use, and a summary of non-clinical performance data, stating that "The summary of the performed non-clinical tests shows that RayStation 6.1 is as safe and effective, and performs as well as the predicate device." However, it does not provide the specific details of these tests or the data results that would be needed to answer your questions.

Therefore, I cannot provide the requested information based on the given input.

Ask a specific question about this device

RayStation is a software system designed for treatment planning and analysis of radiation therapy. The treatment plans provide treatment unit set-up parameters and estimates of dose distributions expected during the proposed treatment, and may be used to administer treatments after review and approval by the intended user.

The system functionality can be configured based on user needs.

The intended users of RayStation shall be clinically qualified radiation therapy staff trained in using the system.

RayStation 6.0 is a radiation therapy treatment planning system, i.e. a software program for planning and analysis of radiation therapy treatment plans. Functionality includes fusion capabilities (CT, PET and MRI), contouring, collapsed cone convolution dose computation and 4D data compatibility, as well as unique features such as multi-criteria optimization, dose tracking, treatment adaptation and deformable registration, all available on one platform.

This document is describing a 510(k) premarket notification for a medical device (RayStation 6.0), which aims to show substantial equivalence to a predicate device (RayStation 5.0). The provided text describes the software's functionality and its intended use but does not contain details about a study with acceptance criteria and device performance data in the context of an AI/human reader comparative study or standalone algorithm evaluation.

The sections "5.11 Assessment of non-clinical performance data" and "5.12 Test conclusion" briefly state that "The test specification of RayStation 6.0 is a further developed version of the test specification of RayStation 5.0" and that "The successful verification and validation of RayStation 6.0 therefore support the substantial equivalence of the above RayStation versions." However, it does not provide any specific acceptance criteria or performance metrics.

Therefore, I cannot fulfill your request for the detailed table and study information based solely on the provided text. The document focuses on regulatory submission aspects (e.g., 510(k) summary, intended use, technological characteristics comparison to a predicate device) rather than clinical performance data with specific acceptance criteria.

To answer your request, I would need a document that presents:

• Specific quantitative acceptance criteria (e.g., sensitivity, specificity, AUC thresholds).
• The actual reported performance of the device against these criteria.
• Details about the study design (sample size, data provenance, ground truth establishment, reader studies if applicable).

Ask a specific question about this device