Search Results

The EmpNia eMotus system is used to measure and record the patient's respiratory waveform to aid with respiratory-synchronized image acquisition or reconstruction during CT diagnostic imaging or radiation treatment planning procedures, where there is a risk of respiratory motion compromising the resulting image.

The EmpNia eMotus system is used to derive and communicate a Gate signal to aid with organ position verification for radiation therapy treatment using CT or Xray imaging by monitoring the patient's respiratory waveform during the image acquisition, where there is a risk of respiratory motion compromising the resulting image.

The EmpNia eMotus system is used to derive and communicate a Gate signal to aid with radiation therapy treatment, where there is a risk of respiratory motion compromising the resulting treatment accuracy.

The eMotus Respiratory Motion Management System ("eMotus system") is designed to monitor patient respiratory motion and to provide information about this respiratory motion to an external medical device system, such as a radiation therapy delivery device (TDD) or a diagnostic imaging device (DX). The main components of the eMotus system include:

• Sensor pad with optical fiber sensors,
• Optical fiber cables,
• Optical transceiver,
• Data acquisition computer with eMotus software application,
• Communication modules for compatible external systems, and
• Cables to allow data transmission between the components.

The sensor pad is a single-use, disposable component with an adhesive backing that is placed directly on the patient's thorax or abdomen. The sensor pad is attached to optical fiber cables that connect to the optical transceiver, which collects optical signal data based on deflection of the sensors in response to respiratory motion. The transceiver digitizes the data and transmits it to the eMotus computer, which visualizes the data as a waveform that can be highlighted when the waveform amplitude reaches a user-specified threshold or the patient's respiratory cycle reaches a user-specified phase. The user can utilize the respiratory threshold and phase information to manually control an external TDD or DX system.

When connected to an external TDD or DX, the eMotus system supports the following functions (as applicable given the functions of the external system):

• Threshold-gated therapy delivery: Automatic gating (turning on / off) of the radiation treatment beam based on user-set parameters for the amplitude of the respiratory waveform.
• Phase-gated therapy delivery: Automatic gating (turning on / off) of the radiation treatment beam based on user-set parameters for the phase of the respiratory waveform cycle.
• Retrospective four-dimensional planning scan: Delivery of the respiratory waveform to an imaging device to synchronize the waveform data with the scan data, enabling retrospective four-dimensional reconstruction of the imaging session for use in treatment planning.
• Prospective four-dimensional planning scan: Automatic patient's respiratory waveform are within preset limits, which is used to disable the radiation beam automatically.

The eMotus device is an ancillary device and does not provide stand-alone therapy or diagnostic information.

Unfortunately, the provided text does not contain the detailed study information required to answer many of your questions. The 510(k) summary focuses on demonstrating "substantial equivalence" to a predicate device, and while it mentions "bench performance," it lacks the specific methodology, sample sizes, and expert involvement that would typically be present in a comprehensive clinical or standalone performance study report.

Here's a breakdown of what can and cannot be answered based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document mentions "Bench performance" testing but does not explicitly state formal acceptance criteria in a quantitative sense, nor does it provide specific numerical performance metrics beyond "nearly identical signals" and "stable dynamics."

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Sample Size for Test Set: Not specified. The text mentions "infant and adult phantoms" and "multiple sensors" but does not give specific numbers.
• Data Provenance: The study described as "bench performance" clearly implies a laboratory/simulated environment rather than clinical data from human patients. Therefore, information about country of origin, retrospective or prospective data, is not applicable or provided.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

Not specified. Given it was "bench performance" with phantoms and a comparison to a predicate device, it's unlikely human experts were establishing ground truth in the traditional sense. The "ground truth" was likely derived from the known simulated respiratory patterns and the output of the predicate device.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

Not applicable/Not specified. Adjudication methods are typically used when human reviewers are involved in assessing complex outputs. This was a bench performance study comparing waveforms and functionality.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study was not explicitly done or described. The device is a "Respiratory Motion Management System," which aids in synchronizing image acquisition or radiation treatment – it's not an AI diagnostic tool that human readers would directly interpret to improve diagnostic accuracy in the way an MRMC study typically assesses. Therefore, the effect size for human reader improvement is not applicable to the information provided.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, the "bench performance" described primarily represents a standalone evaluation of the eMotus system's technical capabilities in a controlled environment, comparing its output directly to known inputs and the predicate device's output. The "human factors" testing mentioned separately focuses on usability, but the core performance data is standalone.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

The "ground truth" for the bench performance testing appears to be based on:

• Known simulated respiratory patterns (for assessing stable dynamics, peak frequency, irregular breathing alerts).
• Output of the predicate device (for comparing respiratory waveforms).

8. The sample size for the training set

The document does not mention any training set size, which suggests that the device, being a physiological signal monitoring and gating system, likely does not involve machine learning or AI that requires a labeled training set in the conventional sense for its core functionality. Its "software functions" are verified and validated, indicating traditional software engineering practices.

9. How the ground truth for the training set was established

Not applicable/Not specified, as no training set is mentioned in the provided text.

In summary, the provided FDA 510(k) clearance letter and summary are designed to demonstrate substantial equivalence, not to provide a detailed clinical or standalone performance study report with the specific metrics you've requested beyond what's inferable from the "bench performance" section.

Ask a specific question about this device

Cranial 4Pi is intended for patient immobilization in radiotherapy and radiosurgery procedures.

Cranial 4Pi is indicated for any medical condition in which the use of radiotherapy or radiosurgery may be appropriate for cranial and head & neck treatments.

Cranial 4Pi is an assembly of the following medical device/ accessory groups:

• CRANIAL 4PI OVERLAYS (CRANIAL 4PI CT OVERLAY, CRANIAL 4PI TREATMENT OVERLAY)
• CRANIAL 4PI HEADRESTS (CRANIAL 4PI HEADREST STANDARD, CRANIAL 4PI HEADREST LOW-NECK, CRANIAL 4PI HEADREST PLATFORM)
• CRANIAL 4PI HEADREST INLAYS (CRANIAL 4PI HEADREST INLAY STANDARD, CRANIAL 4PI HEADREST INLAY OPEN FACE, CRANIAL 4PI HEADREST INLAY H&N, CRANIAL 4PI HEAD SUPPORT STANDARD, CRANIAL 4PI HEAD SUPPORT WIDE)
• CRANIAL 4PI MASKS (CRANIAL 4PI BASIC MASK, CRANIAL 4PI OPEN FACE MASK, CRANIAL 4PI EXTENDED MASK, CRANIAL 4PI STEREOTACTIC MASK, CRANIAL 4PI STEREOTACTIC MASK 3.2MM)
• CRANIAL 4PI WEDGES AND SPACERS (CRANIAL 4PI WEDGE 5 DEG., CRANIAL 4PI WEDGE 10 DEG., CRANIAL 4PI SPACER 20MM, CRANIAL 4PI INDEXING PLATE)

The Cranial 4Pi Overlays are medical devices used for fixation of the patient in a CT- resp. linear accelerator - environment.

The Cranial 4Pi Headrests and the Cranial 4Pi Headrest Inlays are accessories to the Cranial 4Pi Overlays to allow an indication specific positioning of the patient's head and neck. The Cranial 4Pi Wedges and Spacers are accessories to the Cranial 4Pi Headrest Platform to adapt the inclination of the head support to the patients necks.

The Cranial 4Pi Masks are accessories to the Cranial 4Pi Overlays used for producing individual custom-made masks for patient immobilization to the Cranial 4Pi Overlay.

The provided text is a 510(k) Clearance Letter and 510(k) Summary for a medical device called "Cranial 4Pi Immobilization." This document focuses on demonstrating substantial equivalence to a predicate device, as required for FDA 510(k) clearance.

However, the provided text does not contain the detailed information typically found in a clinical study report or a pre-market approval (PMA) submission regarding acceptance criteria, study methodologies, or specific performance metrics with numerical results (like sensitivity, specificity, or AUC) that would be used to "prove the device meets acceptance criteria" for an AI/ML-driven device. The document primarily describes the device's components, indications for use, and a comparison to a predicate device to establish substantial equivalence.

The "Performance Data" section primarily addresses biocompatibility, mechanical verification, dosimetry, compatibility with another system, and mask stability. It does not describe a study to prove AI model performance against clinical acceptance criteria. The "Usability Evaluation" section describes a formative usability study, which is different from a performance study demonstrating clinical effectiveness or accuracy.

Therefore, many of the requested elements (especially those related to AI/ML model performance, ground truth establishment, expert adjudication, MRMC studies, or standalone algorithm performance) cannot be extracted from the provided text. The Cranial 4Pi Immobilization device appears to be a physical immobilization system, not an AI/ML diagnostic or prognostic tool.

Given the nature of the document (510(k) for an immobilization device), the concept of "acceptance criteria for an AI model" and "study that proves the device meets the acceptance criteria" in the traditional sense of an AI/ML clinical study does not apply here.

I will answer the questions based on the closest relevant information available in the provided text, and explicitly state where the information is not available or not applicable to the type of device described.

Preamble: Nature of the Device and Submission

The Cranial 4Pi Immobilization device is a physical medical device designed for patient immobilization during radiotherapy and radiosurgery. The 510(k) premarket notification for this device seeks to demonstrate substantial equivalence to an existing predicate device (K202050 - Cranial 4Pi Immobilization). This type of submission typically focuses on comparable intended use, technological characteristics, and safety/performance aspects relevant to the physical device's function (e.g., biocompatibility, mechanical stability, dosimetry interaction).

The provided documentation does not describe an AI/ML-driven component that would require acceptance criteria related to AI model performance (e.g., accuracy, sensitivity, specificity, AUC) or a study to prove such performance. Therefore, many of the questions asking about AI-specific validation (like ground truth, expert adjudication, MRMC studies, training/test sets for AI) are not applicable to this type of device and submission.

1. A table of acceptance criteria and the reported device performance

Based on the provided document, specific numerical "acceptance criteria" and "reported device performance" in the context of an AI/ML model are not available and not applicable. The document focuses on demonstrating substantial equivalence of a physical immobilization device.

However, the "Performance Data" section lists several tests and their outcomes, which serve as evidence that the device performs as intended for its physical function. These are not acceptance criteria for an AI model.

• Saline Extraction: No sensitization reactions observed.
• Cottonseed Oil Extraction: No sensitization reactions observed.
  Test article did not elicit sensitization reactions (guinea pigs). Positive controls validated sensitivity.
  Irritation Testing (ISO 10993-23):
• No irritation observed (rabbits) compared to control based on erythema and edema scores for saline and cottonseed oil extracts.
  Test article met requirements for Intracutaneous (Intradermal) Reactivity Test. Positive controls validated sensitivity. |
  | Mechanical Tests | Relevant for fulfillment of IEC 60601-1 requirements. | All mechanical tests relevant for fulfillment of IEC 60601-1 requirements were carried out successfully. |
  | Dosimetry Tests | Verify that dose attenuation is acceptable. | Tests to verify that dose attenuation is acceptable with the hardware components were carried out successfully. |
  | Compatibility Tests| Compatibility with ExacTrac Dynamic 2.0. | Compatibility with ExacTrac Dynamic 2.0 was tested successfully. |
  | Mask Stability | Cranial 4Pi SRS mask 3.2 mm (vs. 2mm predicate) to have higher stability against head movement. | Technical validation test to prove that the Cranial 4Pi SRS mask 3.2 mm... having a 3.2 mm top mask sheet instead of 2mm has a higher stability against head movement was carried out successfully. |
  | Usability Evaluation| Evaluate the usability of the subject devices. | Formative usability evaluation performed in three different clinics with seven participants to evaluate the usability of the subject devices. (Specific findings not detailed, but the study was performed). |

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Sample Size for Test Set: Not applicable/not stated in the context of an AI/ML test set. The usability evaluation involved "seven participants" in "three different clinics." For biocompatibility, animal studies were performed (guinea pigs for sensitization, rabbits for irritation; specific number of animals not stated but implied to be sufficient for ISO standards).
• Data Provenance: Not applicable for an AI/ML test set. The usability evaluation involved "three different clinics" but the country of origin is not specified.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

• Not applicable. This device is a physical immobilization system, not an AI/ML diagnostic or prognostic tool that requires expert-established ground truth on medical images.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Not applicable. This information is relevant to validating AI/ML diagnostic performance against ground truth, which is not described for this device.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• Not applicable. This is an AI/ML-specific study design. The device is a physical immobilization system, not an AI assistance tool for human readers.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

• Not applicable. This is an AI/ML-specific validation. There is no AI algorithm component described for this physical device.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

• Not applicable. No ground truth for diagnostic or prognostic purposes is established for this physical device. The "performance data" relies on standards compliance (e.g., ISO, IEC), physical measurements, and usability feedback.

8. The sample size for the training set

• Not applicable. There is no AI model described that would require a training set.

9. How the ground truth for the training set was established

• Not applicable. There is no AI model described.

Ask a specific question about this device

The device is intended for radiation treatment planning for use in stereotactic, conformal, computer planned, Linac based radiation treatment and indicated for cranial, head and neck and extracranial lesions.

RT Elements are computed-based software applications for radiation therapy treatment planning and dose optimization for linac-based conformal radiation treatments, i.e. stereotactic radiosurgery (SRS), fractionated stereotactic radiotherapy (SRT) or stereotactic ablative radiotherapy (SABR), also known as stereotactic body radiation therapy (SBRT) for use in stereotactic, conformal, computer planned, Linac based radiation treatment of cranial, head and neck, and extracranial lesions.

The device consists of the following software modules: Multiple Brain Mets SRS 4.5, Cranial SRS 4.5, Spine SRS 4.5, Cranial SRS w/ Cones 4.5, RT Contouring 4.5, RT QA 4.5, Dose Review 4.5, Brain Mets Retreatment Review 4.5, and Physics Administration 7.5.

Here's the breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) clearance letter for RT Elements 4.5, specifically focusing on the AI Tumor Segmentation feature:

Acceptance Criteria and Reported Device Performance

Note: For "Gliomas and glio-/neuronal tumors," the reported lower bound 95% CI for Recall (0.74) is slightly below the stated acceptance criteria of 0.8. Additional clarification from the submission would be needed to understand how this was reconciled for clearance. However, for all other categories and overall, the reported performance meets or exceeds the acceptance criteria.

Study Details for AI Tumor Segmentation

2. Sample size used for the test set and the data provenance:

• Sample Size: 412 patients (595 scans, 1878 annotations)
• Data Provenance: De-identified 3D CE-T1 MR images from multiple clinical sites in the US and Europe. Data was acquired from adult patients with one or multiple contrast-enhancing tumors. ¼ of the test pool corresponded to data from three independent sites in the USA.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

• Number of Experts: Not explicitly stated as a number, but referred to as an "external/independent annotator team."
• Qualifications of Experts: US radiologists and non-US radiologists. No further details on years of experience or specialization are provided in this document.

4. Adjudication method for the test set:

• The document mentions "a well-defined data curation process" followed by the annotator team, but it does not explicitly describe a specific adjudication method (e.g., 2+1, 3+1) for resolving disagreements among annotators.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

• No, a multi-reader multi-case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance was not reported for the AI tumor segmentation. The study focused on standalone algorithm performance against ground truth.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

• Yes, a standalone performance study was done. The validation was conducted quantitatively by comparing the algorithm's automatically-created segmentations with the manual ground-truth segmentations.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

• Expert Consensus Segmentations: The ground truth was established through "manual ground-truth segmentations, the so-called annotations," performed by the external/independent annotator team of radiologists.

8. The sample size for the training set:

• The sample size for the training set is not explicitly stated in this document. The document mentions that "The algorithm was trained on MRI image data with contrast-enhancing tumors from multiple clinical sites, including a wide variety of scanner models and patient characteristics."

9. How the ground truth for the training set was established:

• How the ground truth for the training set was established is not explicitly stated in this document. It can be inferred that it followed a similar process to the test set, involving expert annotations, but the details are not provided.

Ask a specific question about this device

The Visualase V2 ™ MRI-Guided Laser Ablation System is a neurosurgical tool and is indicated for use to ablate, necrotize, or coagulate intracranial soft tissue including brain structures (for example, brain tumor, radiation necrosis, and epileptic foci as identified by non-invasive and invasive neurodiagnostic testing, including imaging) through interstitial irradiation or thermal therapy in pediatrics and adults with 980 nm lasers. The intended patients are adults and pediatric patients from the age of 2 years and older.

The Visualase MRI-Guided Laser Ablation System comprises hardware and software components used in combination with three MR-compatible (conditional), sterile, single-use, saline-cooled laser applicators with proprietary diffusing tips that deliver controlled energy to the tissue of interest. The system consists of:

• a diode laser (energy source)
• a coolant pump to circulate saline through the laser application
• Visualase workstation which interfaces with MRI scanner's host computer
• Visualase software which provides the system's ability to visualize and monitor relative changes in tissue temperature during ablation procedures, set temperature limits and control the laser output; one monitors to display all system imaging and laser ablation via a graphical user interface and peripherals for interconnections

The provided FDA 510(k) clearance letter for the Visualase V2 MRI-Guided Laser Ablation System does not contain the detailed information necessary to fully address all aspects of the request. Specifically, the document focuses on regulatory compliance, substantial equivalence to predicate devices, and general testing summaries (software V&V, system V&V, electrical safety). It does not include specific acceptance criteria with performance metrics, details of a clinical study (like sample sizes, ground truth establishment, expert qualifications, or MRMC studies), or direct data proving the device met specific performance criteria.

The letter explicitly states: "A clinical trial was not required to establish substantial equivalence. Clinical evidence provided in a literature summary format supports the safe use of the Visualase V2 System in the intended patient population." This indicates that a prospective clinical performance study, often associated with detailed acceptance criteria and reported performance, was not the primary method for demonstrating substantial equivalence for this particular submission.

Therefore, many sections of your request cannot be fulfilled based on the provided document. I will fill in the information that is present and explicitly state where information is not available.

Acceptance Criteria and Device Performance for Visualase V2 MRI-Guided Laser Ablation System

Based on the provided FDA 510(k) clearance letter (K250307), the device's acceptance criteria and proven performance are primarily demonstrated through verification and validation activities for its software and system, and compliance with electrical safety standards. A formal clinical trial with quantitative performance metrics against specific acceptance criteria (e.g., sensitivity, specificity, accuracy) was not required for this submission to establish substantial equivalence, but rather clinical evidence was provided via a literature summary.

1. Table of Acceptance Criteria and Reported Device Performance

Given the nature of this 510(k) for the Visualase V2 System as described in the document, performance acceptance criteria are focused on safety, functionality, and equivalence to predicate devices, rather than clinical efficacy metrics typically found in AI/diagnostic device submissions.

2. Sample Size for the Test Set and Data Provenance

The document explicitly states: "A clinical trial was not required to establish substantial equivalence. Clinical evidence provided in a literature summary format supports the safe use of the Visualase V2 System in the intended patient population."

Therefore, no specific "test set" sample size for a clinical performance study is reported in this document. The "testing summary" refers to internal verification and validation against design controls and standards, not a clinical data set for performance evaluation of an AI algorithm.

Data Provenance: Not applicable for a clinical test set in this context, as a clinical performance study was not the basis for substantial equivalence for this upgrade. The clinical evidence was a literature summary.

3. Number of Experts Used to Establish Ground Truth and Qualifications

Not applicable, as a specific clinical test set for performance evaluation (e.g., for an AI algorithm's diagnostic accuracy which would require ground truth labeling) was not conducted as part of this 510(k) as described. The "ground truth" for the device's functionality and safety was established via engineering verification, validation, and regulatory compliance.

4. Adjudication Method for the Test Set

Not applicable, as a clinical test set requiring adjudication was not reported as part of this 510(k) as described.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

No, a MRMC comparative effectiveness study was not reported in this 510(k) clearance letter. The submission focused on establishing substantial equivalence through other means (software/system V&V, safety testing, literature review) rather than demonstrating AI assistance performance improvement.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study was Done

The Visualase V2 System is a medical device system that includes hardware and software for MRI-guided laser ablation, with software providing monitoring and control capabilities related to temperature and thermal damage estimation. It is not an AI diagnostic algorithm for which a standalone performance evaluation (e.g., AUC, sensitivity/specificity) would typically be required or reported in this format. The software's performance is intrinsically linked to the system's function and user interaction.

Therefore, a "standalone algorithm only" performance study in the sense of a diagnostic AI product is not applicable and not reported.

7. The Type of Ground Truth Used

For the system's functional and safety validation, the "ground truth" would be engineering specifications, design requirements, and established medical and electrical safety standards (e.g., IEC standards, 21 CFR 820.30 Design Controls).

For any inferred clinical claims from the "literature summary," the ground truth would originate from the clinical data reported in the summarized peer-reviewed literature, which could involve histological confirmation, long-term patient outcomes, or expert clinical diagnosis, but these details are not provided in the 510(k) letter itself.

8. The Sample Size for the Training Set

Not applicable. The document describes a medical device system update, not an AI algorithm developed using a specific training dataset in the machine learning sense. The "training" for the system's software would derive from its design and programming, verified through the V&V processes.

9. How the Ground Truth for the Training Set Was Established

Not applicable, as this is not an AI algorithm developed through data-driven training in the machine learning sense. The "ground truth" for the device's design and engineering would be based on scientific and engineering principles, preclinical testing, and existing medical knowledge, as per design control procedures.

Ask a specific question about this device

MapRT is indicated for assisting with planning of radiation therapy by:

• Assessing which combinations of gantry/couch angle and isocentre may result in a collision and which are available to potentially enhance the dose distribution; and
• Predicting when a treatment plan might result in a collision between the treatment machine and the patient or support structures

MapRT is used by radiotherapy professionals during the CT simulation and treatment planning stages of radiotherapy for collision avoidance and facilitating dose optimisation.

MapRT uses two lateral wide-field cameras in simulation to deliver a full 3D model of patients and accessories. This model is then used to calculate a clearance map for every couch (x-axis) and gantry (y-axis) angles. Radiotherapy treatment plans can then be imported automatically to check beams, arcs, and the transition clearance.

The provided document is a 510(k) clearance letter for a software device called MapRT, which assists in radiation therapy planning by predicting collisions. However, the document explicitly states: "As with the predicate device, no clinical investigations were performed for MapRT. Verification tests were performed to ensure that the module works as intended and pass/fail criteria were used to verify requirements. Validation testing was performed using summative evaluation techniques per 62366-1:2015/A1:2020. Verification and validation testing passed in all test cases."

This means the submission did not include a study design or performance data in the typical sense of a clinical trial or a multi-reader multi-case (MRMC) study to prove the device meets acceptance criteria related to clinical performance metrics like sensitivity, specificity, accuracy, or reader improvement. Instead, the clearance relies on:

1. Substantial Equivalence: The primary argument for clearance is that MapRT v1.2 is substantially equivalent to its predicate device (MapRT v1.0, K231185). The document highlights that the indications for use, functionality, technological characteristics, and intended users are the same as the predicate.
2. Verification and Validation (V&V) Testing: The document mentions that "Verification and validation testing passed in all test cases," indicating that the software meets its design specifications and functions as intended, primarily in terms of software functionality and accuracy of collision prediction within its defined operational parameters.

Given this information, it's not possible to fill out all aspects of your requested response, particularly those related to clinical studies, ground truth establishment, expert consensus, and MRMC studies, as they were explicitly not performed.

Here's an attempt to answer based on the provided document, noting where information is not available:

Device Acceptance Criteria and Study Performance for MapRT

The FDA 510(k) clearance for MapRT v1.2 is primarily based on demonstrating substantial equivalence to a legally marketed predicate device (MapRT v1.0, K231185) and successful completion of software verification and validation activities. The submission explicitly states that "no clinical investigations were performed for MapRT." Therefore, the acceptance criteria and performance proof are framed in the context of software verification and validation, and functional accuracy rather than clinical efficacy studies.

1. Acceptance Criteria and Reported Device Performance

The core functional acceptance criterion is the accuracy of collision prediction.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size for Test Set: The document does not specify a "test set" in the context of patient data or clinical cases. The performance data mentioned refer to software verification and validation tests, which would involve a set of test cases designed to cover various scenarios and functional requirements. The specific number or nature of these test cases is not detailed.
• Data Provenance: Not applicable for a clinical test set, as no clinical investigations were performed. The V&V testing would likely involve simulated data, synthetic models, or potentially anonymized patient models used for testing collision detection scenarios. The provenance (country of origin, retrospective/prospective) of such test data is not provided.

3. Number of Experts Used to Establish Ground Truth for Test Set and Their Qualifications

Not applicable. Since no clinical investigations were performed, there was no clinical "ground truth" established by experts in the context of patient outcomes or image interpretation. The ground truth for functional testing of collision prediction would be derived from precise engineering specifications and physical measurements, likely validated internally by the manufacturer's engineering team.

4. Adjudication Method for the Test Set

Not applicable, as no clinical test set requiring expert adjudication was used.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

No. The document explicitly states: "As with the predicate device, no clinical investigations were performed for MapRT." Therefore, no MRMC study was conducted to compare human reader performance with or without AI assistance.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study Was Done

Yes, in essence. The stated "accuracy of ± 2cm" for gantry clearance calculation and the passing of "all test cases" in verification and validation testing refer to the isolated performance of the MapRT algorithm in predicting collisions and calculating clearance. This implies an evaluation of the algorithm's functional accuracy independent of human interaction beyond inputting treatment plans. However, the details of how this accuracy was measured (e.g., against a gold standard derived from physical models or high-precision simulations) are not provided in this summary.

7. The Type of Ground Truth Used

For the accuracy of gantry clearance calculation (± 2cm), the ground truth would typically be established through:

• Precise engineering specifications and measurements of physical models of the treatment machine, patient, and support structures.
• High-fidelity simulation data where collision events and clearances can be precisely calculated geometrically.

It is not based on expert consensus, pathology, or outcomes data, as these are typically associated with clinical diagnostic or prognostic devices.

8. Sample Size for the Training Set

Not applicable. MapRT is a software device that simulates radiation treatment plans and predicts collisions based on geometric models and calculations. There is no indication that it is an AI/Machine Learning model that requires a "training set" of data in the conventional sense (e.g., for image classification or pattern recognition). Its "knowledge" of collision mechanics and geometries comes from programmed rules and pre-loaded models (e.g., LiDAR scans or 3D CAD models of equipment), not from learning from a dataset.

9. How the Ground Truth for the Training Set Was Established

Not applicable, as there is no "training set" for an AI/ML model for MapRT based on the provided information.

Ask a specific question about this device

Mobius3D software is used for quality assurance, treatment plan verification, and patient alignment and anatomy analysis in radiation therapy. It calculates radiation dose three dimensionally in a representation of a patient or a phantom. The calculation is based on read-in treatment plans that are initially calculated by a treatment planning system, and may additionally be based on external measurements of radiation fields from other sources such as linac delivery log data. Patient alignment and anatomy analysis is based on read-in treatment planning images (such as computed tomography) and read-in daily treatment images (such as registered cone beam computed tomography).

Mobius3D is not a treatment planning system. It is to be used only by trained radiation oncology personnel as a quality assurance tool.

Mobius3D is a software product used within a radiation therapy clinic for quality assurance and treatment plan verification. It is important to note that while Mobius3D operates in the field of radiation therapy, it is neither a radiation delivery device (e.g. a linear accelerator), nor is it a Treatment Planning System (TPS). Mobius3D cannot design or transmit instructions to a delivery device, nor does it control any other medical device. Mobius3D is an analysis tool meant solely for quality assurance (QA) purposes when used by trained medical professionals. Being a software only QA tool, Mobius3D never comes into contact with patients.

It appears there's a misunderstanding based on the provided document. The request asks for acceptance criteria and a study that proves the device meets those criteria, including specifics like sample sizes, expert qualifications, and ground truth establishment.

However, the provided FDA 510(k) clearance letter for Mobius3D (4.1) does not contain the detailed performance study results that would prove the device meets specific acceptance criteria.

The 510(k) summary (pages 5-7) primarily discusses:

• Device Description and Intended Use: What Mobius3D is and what it's used for (QA, treatment plan verification, patient alignment).
• Comparison to Predicate Device: How Mobius3D 4.1 differs from 4.0.
• Summary of Performance Testing (Non-Clinical):
  • Mentions software verification and validation, including unit, integration, and end-to-end testing.
  • Highlights MLC Modelling Accuracy testing comparing different Mobius3D versions, measurements, and a Treatment Planning System (Eclipse TPS 16.1).
  • States that "studies and reviews have been performed to assess the accuracy of newly introduced features and modifications" for Rapid Arc Dynamic Support and MLC Tongue and Groove Modelling.
  • Notes conformance to cybersecurity and interoperability requirements.
  • Crucially, it explicitly states: "No animal studies or clinical tests have been included in this pre-market submission." This means there isn't a human-in-the-loop study or a study directly demonstrating clinical performance against ground truth in a clinical setting.
• Use of Consensus Standards: A list of standards the device's design and evaluation conform to.
• Determination of Substantial Equivalence: Varian's conclusion that the device is substantially equivalent to the predicate.

Therefore, many of the specific details requested (Table of acceptance criteria, sample sizes for test sets, number/qualifications of experts for ground truth, adjudication methods, MRMC study, standalone performance, type of ground truth, training set sample size/ground truth establishment) are NOT present in this 510(k) clearance letter.

The letter focuses on the regulatory submission process and the FDA's determination of substantial equivalence based on the provided non-clinical testing and comparison to a predicate device. It doesn't typically include the full, detailed study reports with precise performance metrics and ground truth methodologies. Such details would typically be found in the more extensive technical documentation submitted by the manufacturer to the FDA, but they are summarized at a high level in the public 510(k) summary.

In summary, based only on the provided text, I cannot provide the detailed information requested regarding the specific acceptance criteria and the study that proves the device meets those criteria in the format you've requested. The document indicates non-clinical software testing and accuracy assessments were performed but does not provide the specific metrics or study design details for clinical performance proof.

Ask a specific question about this device

The PPTS is a medical device designed to produce and deliver a proton beam for the treatment of patients with localized tumors and other condition susceptible to treatment by radiation.

When the patient is in the seated position using the chair, the System is indicated for treatment of patients with localized tumors and other conditions susceptible to treatment by radiation in the head, neck and thorax.

The P-CURE Proton Therapy System (PPTS) is comprised of four main subsystems that function in tandem to generate the desired dose level and distribution at the target site:

• Beam production system (Synchrotron based accelerator)

  • Injector – produces and delivers protons to the synchrotron
  • Synchrotron ring – accelerates the proton beam in circular orbit (within the ring) to the desired energy level
  • Extraction system - extracts the beam from the ring to the beam delivery subsystem

• Beam delivery system for a single fixed beam treatment room. Steers and monitors the extracted proton pencil beam from the synchrotron to the desired treatment location (Nozzle).

• Patient Positioning System (P-ARTIS). Mechanically orients the patient (seated or on supine); provides independent means of patient registration using CT (3D) and X-ray (2D)

  • CT system (P-ARTIS CT)
  • Robotic arm and chair/couch (6 Degree of freedom Couch) (P-ARTIS PPS)
  • X-ray system (P-ARTIS XR)
  • Positioning Software (P-ART)

• Control and Safety Systems

  • Control Subsystem (TSM). Synchronizes the various subsystem actions and connects with hospital oncology information systems and PACS.
  • Safety Subsystem. Includes hardware and software means to ensure safe system operation for patient and personnel. It includes subsystem interlocks, treatment beam parameters monitoring, and others.

The provided FDA 510(k) clearance letter for the P-Cure Proton Therapy System (PPTS) does not contain specific acceptance criteria or a detailed study description with performance metrics that would allow for a comprehensive table and answer to all the questions. This document is a clearance letter, which summarizes the outcome of a review, rather than providing the full technical details of the submission.

However, based on the information provided, here's what can be extracted and inferred:

1. Table of Acceptance Criteria and Reported Device Performance

The clearance letter does not list specific numerical acceptance criteria (e.g., minimum accuracy percentages, maximum error values) or direct quantitative performance results in a table format. It states that:

• "In all instances, the PTTS functioned as intended and met its specifications."
• "Testing demonstrated substantial equivalence in terms of performance and safety to the predicate."

To construct a table, we would need the actual specifications and the measured performance against those specifications, which are typically found in the full 510(k) submission, not the clearance letter.

Inferred Performance Claims (from "Performance Data" section):

2. Sample Size Used for the Test Set and Data Provenance

The document does not provide details on the "sample size" in terms of patient data or case numbers for the performance testing. The testing described is primarily technical and engineering verification and validation of the system's components and functions (mechanical, beam, safety, software integration, repeatability). It does not mention clinical studies with patient data.

• Sample Size for Test Set: Not specified for clinical cases. The testing appears to be system-level verification and validation, not patient-based clinical performance data.
• Data Provenance: Not applicable as no patient data (e.g., country of origin, retrospective/prospective) is referenced for the performance testing cited. The submitter, P-Cure, Ltd., is located in Israel, but this pertains to the company, not necessarily the origin of any clinical data.

3. Number of Experts Used to Establish Ground Truth and Qualifications

The document does not mention any "ground truth" established by experts in the context of clinical performance, as it focuses on the technical verification and validation of the device's physical and software functions. Therefore, this question is not applicable based on the provided text.

4. Adjudication Method for the Test Set

As no expert review or clinical case evaluation is mentioned, there is no adjudication method described. This question is not applicable based on the provided text.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

No MRMC study is mentioned. The clearance letter details technical and engineering performance testing, not studies comparing human reader performance with or without AI assistance. This question is not applicable based on the provided text.

6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done

The device described is a physical medical device (Proton Therapy System), not an AI algorithm to be used standalone or with human-in-the-loop for diagnostic or prognostic purposes. While the system has software, its "performance" refers to the entire system's ability to produce and deliver a proton beam accurately and safely. This question is not applicable in the typical sense of AI standalone performance.

7. The Type of Ground Truth Used

The "ground truth" for the performance testing appears to be based on:

• Known engineering specifications and physical laws: For beam performance, mechanical movements, dose delivery accuracy, etc.
• Safety standards: Compliance with IEC and AAPM standards.
• Software requirements: Validation against specified software functions.

There is no mention of expert consensus, pathology, or outcomes data as "ground truth" for the reported performance testing.

8. The Sample Size for the Training Set

The concept of a "training set" is usually applicable to machine learning algorithms. While the system involves software, the document describes traditional software validation and verification for system control and safety functions, not the development of a machine learning model that would require a distinct training set. Therefore, this question is not applicable based on the provided text.

9. How the Ground Truth for the Training Set Was Established

As no "training set" (in the context of machine learning) is mentioned, the method for establishing its ground truth is also not applicable.

Ask a specific question about this device

RayCare is an oncology information system intended to provide information which is used to take decisions for diagnosis, treatment management, treatment planning, scheduling, treatment and follow-up of radiation therapy, medical oncology and surgical oncology.

For these disciplines, as applicable, RayCare enables the user to define the clinical treatment intent, prescribe treatment, specify the detailed course of treatment delivery, manage the treatment course and monitor the treatment course.

In the context of radiation therapy, the RayCare image viewer can be used for viewing images, annotating images, performing and saving image registrations as well as image fusion to enable offline image review of patient positioning during treatment delivery.

RayCare is not intended for use in diagnostic activities.

RayCare is an oncology information system intended to provide information which is used to take decisions for diagnosis, treatment management, treatment planning, scheduling, treatment and follow-up of radiation therapy, medical oncology and surgical oncology.

For these disciplines, as applicable, RayCare enables the user to define the clinical treatment intent, prescribe treatment, specify the detailed course of treatment delivery, manage the treatment course and monitor the treatment course.

In the context of radiation therapy, the RayCare image viewer can be used for viewing images, annotating images, performing and saving image registrations as well as image fusion to enable offline image review of patient positioning during treatment delivery.

As an oncology information system, RayCare supports healthcare professionals in managing cancer care treatments. The system provides functionalities as described briefly in the sections below. These functionalities are not provided separately in different applications and have a joint purpose for the treatment of the patient.

RayCare is a software-as a Medical Device with a client part that allows the user to interact with the system and a server part that performs the necessary processing and storage functions. Selected aspects of RayCare are configurable, such as adapting workflow templates to the specific needs of the clinic.

This document describes the premarket notification for RayCare (2024A SP1), an oncology information system. The relevant sections for acceptance criteria and study details are primarily found under "VII. Non-Clinical and/or Clinical Tests Summary" and the tables within it.

Based on the provided text, RayCare (2024A SP1) is not an AI/ML device in the sense of making autonomous diagnostic decisions or image-based classifications. It is an Oncology Information System that supports clinical workflows for radiation therapy and other oncology disciplines. The "acceptance criteria" and "study that proves the device meets the acceptance criteria" in this context refer to the software verification and validation (V&V) activities. Therefore, the information provided focuses on demonstrating the software's functional correctness, safety, and effectiveness compared to a predicate device, rather than performance metrics specifically for an AI model (e.g., sensitivity, specificity, AUC).

Here's a breakdown of the requested information based on the provided document:

Acceptance Criteria and Device Performance (Software V&V)

The acceptance criteria for RayCare (2024A SP1) are implicitly defined by the successful completion of various software verification and validation activities designed to demonstrate that the device performs as intended and is as safe and effective as its predicate. These are primarily functional and system-level criteria.

Table of Acceptance Criteria and Reported Device Performance:

Since this is a software verification and validation summary for an oncology information system, the "performance" is demonstrated through successful compliance with system specifications and validated functionality. The "acceptance criteria" are the "Pass criteria" of the specific tests.

Overall Conclusion:
"From the successful verification and validation activities, the conclusion can be drawn that RayCare 2024A SP1 has met specifications and is as safe, as effective and performs as well as or better than the legally marketed predicate device."

1. Sample sizes used for the test set and the data provenance:

   • Test Set Sample Size: The document does not specify a numerical sample size for "test sets" in the traditional sense of patient cases or images for evaluating an AI model. Instead, it refers to software verification and validation ("V&V") activities including unit testing, integration testing, system-level testing, cybersecurity testing, usability testing, and regression testing. These involve testing against requirements and specifications, often using simulated data, test cases, or specific user scenarios, rather than a fixed "dataset" of patient images.
   • Data Provenance: The document does not explicitly state the country of origin of testing data or if it was retrospective or prospective. Given it's software V&V for an oncology information system, the "data" would primarily be test inputs and expected outputs generated internally during the development process (e.g., test scripts, simulated patient data to exercise specific functionalities). It's not a study on real patient data for diagnostic performance.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • This concept is not applicable as this is a software verification and validation summary for an oncology information system, not a study evaluating an AI model's diagnostic or prognostic performance against expert-determined ground truth. The "ground truth" for V&V activities is the system's specified behavior and functional requirements. Software engineers and QA professionals establish whether the software meets these pre-defined requirements.

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • Not applicable. Adjudication methods are typically used in clinical studies involving human readers to resolve discrepancies in annotations or diagnoses, especially when establishing ground truth for AI model evaluation. This document describes software V&V.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not done. The document explicitly states: "No Clinical trials were required to demonstrate substantial equivalence."
   • This type of study is relevant for AI-assisted diagnostic devices. RayCare is described as an oncology information system, not an AI diagnostic tool.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • While the system has automated functions, the concept of "standalone performance" as it relates to an AI algorithm making a clinical decision (e.g., classifying a lesion) is not directly applicable here. The V&V described focuses on the system's ability to correctly manage and process information, integrate with other systems, and support user workflows, which are inherent to its "standalone" operation as an information system. The "performance" is demonstrated through successful execution of its intended software functions as per its specifications.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The "ground truth" for the software verification and validation described here is the functional specifications and requirements of the RayCare system. Successful "verification" means the design output meets the requirements, and "validation" means the software conforms to user needs and intended uses. This is established through internal testing against defined expected behaviors.

7. The sample size for the training set:

   • Not applicable. RayCare (2024A SP1) is an oncology information system, and the document does not indicate that it incorporates a machine learning model that was "trained" on a dataset in the way an AI diagnostic or predictive algorithm would be. The device's "development" involved standard software engineering practices.

8. How the ground truth for the training set was established:

   • Not applicable. As there is no mention of an AI/ML training set, the concept of establishing ground truth for it does not apply.

In summary, this FDA review document pertains to the clearance of an Oncology Information System (OIS) through the 510(k) pathway, demonstrating substantial equivalence to a predicate device. The "acceptance criteria" and "proof" come from a robust set of software verification and validation activities (unit, integration, system, cybersecurity, usability, regression testing) rather than clinical studies or the evaluation of an AI model's diagnostic performance against a clinical ground truth. The device is not presented as an AI-driven diagnostic tool.

Ask a specific question about this device

Oncospace is used to configure and review radiotherapy treatment plans for a patient with malignant or benign disease in the head and neck, thoracic, abdominal, and pelvic regions. It allows for set up of radiotherapy treatment protocols, association of a potential treatment plan with the protocol(s), submission of a dose prescription and achievable dosimetric goals to a treatment planning system, and review of the treatment plan. It is intended for use by qualified, trained radiation therapy professionals (such as medical physicists, oncologists, and dosimetrists). This device is for prescription use by order of a physician.

The Oncospace software supports radiation oncologists and medical dosimetrists during radiotherapy treatment planning. The software includes locked machine learning algorithms. During treatment planning, the Oncospace software works in conjunction with, and does not replace, a treatment planning system (TPS).

The Oncospace software is intended to augment the treatment planning process by:

• allowing the radiation oncologist to select and customize a treatment planning protocol that includes dose prescription (number of fractions, dose per fraction, dose normalization), a delivery method (beam type and geometry), and protocol-based dosimetric goals/objectives for treatment targets, and organs at risk (OAR);
• predicting dosimetric goals/objectives for OARs based on patient-specific anatomical geometry;
• automating the initiation of plan optimization on a TPS by supplying the dose prescription, delivery method, protocol-based target objectives, and predicted OAR objectives;
• providing a user interface for plan evaluation against protocol-based and predicted goals.

Diagnosis and treatment decisions occur prior to treatment planning and do not involve Oncospace. Decisions involving Oncospace are restricted to setting of dosimetric goals for use during plan optimization and plan evaluation. Human judgement continues to be applied in accepting these goals and updating them as necessary during the iterative beam optimization process. Human judgement is also still applied as in standard practice during plan quality assessment; the protocol-based OAR goals are used as the primary means of plan assessment, with the role of the predicted goals being to provide additional information as to whether dose to an OAR may be able to be further lowered.

When Oncospace is used in conjunction with a TPS, the user retains full control of the TPS, including finalization of the treatment plan created for the patient. Oncospace also does not interface with the treatment machines. The risk to patient safety is lower than a TPS since it only informs the treatment plan, does not allow region of interest editing, does not make treatment decisions, and does not interface directly with the treatment machine or any record and verify system.

Oncospace's OAR dose prediction approach, and the use of predictions in end-to-end treatment planning workflow, has been tested for use with a variety of cancer treatment plans. These included a wide range of target and OAR geometries, prescriptions and boost strategies (sequential and simultaneous delivery). Validity has thus been demonstrated for the range of prediction model input features encountered in the test cases. This range is representative of the diversity of the same feature types (describing target-OAR proximity, target and OAR shapes, sizes, etc.) encountered across all cancer sites. Given that the same feature types will be used in OAR dose prediction models trained for all sites, the modeling approach validated here is not cancer site specific, but rather is designed to predict OAR DVHs based on impactful features common to all sites. The software is designed to be used in the context of all forms of intensity-modulated photon beam radiotherapy. The planning objectives themselves are intended to be TPS-independent: these are instead dependent on the degree of organ sparing possible given the beam modality and range of delivery techniques for plans in the database. To facilitate streamlined transmission of DICOM files and plan parameters Oncospace includes scripts using the treatment planning system's scripting language (for example, Pinnacle).

The Oncospace software includes an algorithm for transforming non-standardized OAR names used by treatment planners to standardized names defined by AAPM Task Group 263. This matching process primarily uses a table of synonyms that is updated as matches are made during use of the product, as well as a Natural Language Processing (NLP) model that attempts to match plan names not already in the synonym table. The NLP model selects the most likely match, which may be a correct match to a standard OAR name, an incorrect match, or no match (when the model considers this to be most likely, such as for names resembling a target). The user can also manually match names using a drop-down menu of all TG-263 OAR names. The user is instructed to check each automated match and make corrections using the drop-down menu as needed.

Based on the provided 510(k) Clearance Letter, here's a detailed description of the acceptance criteria and the study proving the device meets them:

1. Table of Acceptance Criteria and Reported Device Performance

The document describes two main types of performance testing: clinical performance testing and model performance testing. The acceptance criteria are implicitly defined by the reported performance achieving non-inferiority or being within acceptable error margins.

• Thoracic: 2.2 Gy
• Abdominal: 1 Gy
• Pelvis (Gynecological): 1.9 Gy | Achieved Non-Inferiority:
• Mean OAR dose was statistically significantly lower for 5 OARs for abdominal and 4 OARs for pelvis (gynecological).
• No statistically significant differences in mean dose for remaining 11 OARs for thoracic, 3 OARs for abdominal, and 2 OARs for pelvis (gynecological).
• Non-inferiority demonstrated to 2.2 Gy for thoracic, 1 Gy for abdominal, and 1.9 Gy for pelvis (gynecological). |
  | Clinical Performance (Secondary Outcome) | Target Coverage Maintenance: No statistically significant difference in target coverage compared to clinical plans without Oncospace. | Achieved: No statistically significant difference in target coverage between clinical plans and plans created with use of the Oncospace system. |
  | Clinical Performance (Effort Reduction) | No increased optimization cycles when using Oncospace vs. traditional workflow. (Implicit acceptance criteria) | Achieved: Out of all the plans tested, no plan required more optimization cycles using Oncospace versus using traditional radiation treatment planning clinical workflow. |
  | Model Performance (H&N External Validation) | Mean Absolute Error (MAE) in OAR DVH dose values:
• Institution 2: Within 5% of prescription dose for all OARs.
• Institution 3: Within 5% of prescription dose for all OARs. | Achieved (with some exceptions):
• Institution 2: MAE within 5% for 9/12 OARs; does not exceed 9% for any OARs.
• Institution 3: MAE within 5% for 10/12 OARs; does not exceed 8% for any OARs. |
  | Model Performance (Prostate External Validation) | Mean Absolute Error (MAE) in OAR DVH dose values: Within 5% of prescription dose for all OARs. | Achieved (with some exceptions):
• Institution 3: MAE within 5% for 4/6 OARs; 5.1% for one OAR; 15.9% for one OAR. |
  | NLP Model Performance (Cross-Validation) | Validation macro-averaged F1 score above 0.92 and accuracy above 96% for classifying previously unseen terms. | Achieved: All models achieved a validation macro-averaged F1 score above 0.92 and accuracy above 96%. |
  | NLP Model Performance (External Validation) | Correctly match a high percentage of unique and total structure names. (Implicit acceptance criteria) | Achieved: Correctly matched 207/221 (94.1%) of all structure names, or 131/145 (91.0%) unique structure names. |
  | General Verification Tests | All system requirements and acceptance criteria met (clinical, standard UI, cybersecurity). | Achieved: Met all system requirements and acceptance criteria. |

2. Sample Sizes and Data Provenance

The document provides detailed sample sizes for training/tuning, and external performance/clinical validation datasets.

• Test Set Sample Sizes:

  • Clinical Validation Dataset:
    • Head and Neck: 18 patients (previously validated)
    • Thoracic: 20 patients (14 lung, 6 esophagus)
    • Abdominal: 17 patients (11 pancreas, 6 liver)
    • Pelvis: 17 patients (12 prostate, 5 gynecological) (prostate previously validated)
  • External Performance Test Dataset(s) (Model Performance):
    • Head and Neck: Dataset A: 265 patients (Institution_2); Dataset B: 27 patients (Institution_3)
    • Prostate: 40 patients (Institution_3)
  • NLP Model External Testing Dataset: 221 structures with 145 unique original names.

• Data Provenance (Country of Origin, Retrospective/Prospective):

  • Training/Tuning/Internal Testing Datasets: Acquired from Johns Hopkins University (JHU) between 2008-2019. Johns Hopkins University is located in the United States. This data is retrospective.
  • External Performance Test Datasets: Acquired from Institution_2 and Institution_3. Locations of these institutions are not specified but are implied to be distinct from JHU. This data is retrospective.
  • Clinical Validation Datasets: Acquired from Johns Hopkins University (JHU) between 2021-2024 (for Thoracic, Abdominal, Pelvis) and 2021-2022 (for H&N, previously validated). This data is retrospective.
  • NLP Model Training/External Validation: Trained and validated using "known name matches in the prostate, gynecological, head and neck, thoracic, and pancreas cancer datasets licensed to Oncospace by Johns Hopkins University." This indicates retrospective data from the United States.

3. Number of Experts and Qualifications for Ground Truth

The document does not explicitly state the number of experts or their specific qualifications (e.g., years of experience, types of radiologists) used to establish the ground truth for the test sets.

Instead, it refers to "heterogenous sets of traditionally-planned clinical treatment plans" and "curated, gold-standard treatment plans" (for the predicate device comparison table, implying similar for the subject). This suggests that the ground truth for the OAR dose values reflects actual clinical outcomes from existing treatment plans.

For the NLP model, the ground truth was "known name matches" in the acquired datasets, implying consensus or established naming conventions from the institutions, rather than real-time expert adjudication for the study.

4. Adjudication Method for the Test Set

The document does not describe an explicit adjudication method (e.g., 2+1, 3+1 reader adjudication) for establishing the ground truth dose values or treatment plan quality for the test sets. The "ground truth" seems to be defined by:

• Clinical Performance Testing: "heterogenous sets of traditionally-planned clinical treatment plans," implying the actual clinical plans serve as the comparative ground truth.
• Model Performance Testing: "comparison of predicted dose values to ground truth values." These ground truth values appear to be the actual recorded DVH dose values from the clinical plans in the external test datasets.
• NLP Model: "known name matches" from the licensed datasets, suggesting pre-defined or institutional standards.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, an MRMC comparative effectiveness study involving human readers improving with AI vs. without AI assistance was not described in this document.

The study focused on:

• Comparing plans generated with Oncospace to traditionally-planned clinical treatment plans (effectively comparing AI-assisted plan generation to human-only plan generation, but without a specific MRMC design to measure human reader improvement).
• Assessing the ability of Oncospace to maintain or improve OAR sparing and target coverage.
• Evaluating the accuracy of the model's dose predictions and the NLP module.

The study design described is a non-inferiority trial for clinical performance, and model performance accuracy assessments, not an MRMC study quantifying human reader performance change.

6. Standalone (Algorithm Only) Performance

Yes, standalone performance was done for:

• Model Performance Testing: This involved comparing the model's predicted OAR DVH dose values directly against "ground truth values" (actual recorded dose values from clinical plans) in the external test datasets. This is an algorithm-only (standalone) assessment of the dose prediction accuracy, independent of the overall human-in-the-loop clinical workflow.
• NLP Model Performance: The NLP model's accuracy in mapping non-standardized OAR names to TG-263 names was evaluated in a standalone manner using cross-validation and an external test dataset.

The "clinical performance testing," while ultimately comparing plans with Oncospace assistance to traditional plans, is also evaluating the algorithm's influence on the final plan quality. However, the explicit "model performance testing" sections clearly describe standalone algorithm evaluation.

7. Type of Ground Truth Used

The ground truth used in this study primarily relied on:

• Existing Clinical Treatment Plans/Outcomes Data: For clinical performance testing, "heterogenous sets of traditionally-planned clinical treatment plans" served as the comparative baseline. The OAR doses and target coverage from these real-world clinical plans constituted the "ground truth" for comparison. This can be categorized as outcomes data in terms of actual treatment parameters delivered in a clinical setting.
• Recorded Dosimetric Data: For model performance testing, the "ground truth values" for predicted DVH doses were the actual, recorded DVH dose values from the clinical plans in the external datasets. This data is derived from expert consensus in practice (as these were actual clinical plans deemed acceptable by clinicians) and outcomes data (the resulting dose distributions).
• Established Reference/Consensus (NLP): For the NLP model, the ground truth was based on "known name matches" or "standardized names defined by AAPM Task Group 263," which represents expert consensus or authoritative standards.

8. Sample Size for the Training Set

The document refers to the "Development (Training/Tuning) and Internal Performance Testing Dataset (randomly split 80/20)" for each anatomical location. Assuming the 80% split is for training:

• Head and Neck: 1145 patients (80% for training) = approx. 916 patients
• Thoracic: 1623 patients (80% for training) = approx. 1298 patients
• Abdominal: 712 patients (80% for training) = approx. 569 patients
• Pelvis: 1785 patients (80% for training) = approx. 1428 patients

9. How the Ground Truth for the Training Set Was Established

The ground truth for the training set for the dose prediction models was established based on retrospective clinical data from Johns Hopkins University between 2008-2019. These were actual treatment plans for patients who received radiation therapy, meaning their dosimetric parameters (like OAR doses and target coverage from DVHs) and anatomical geometries (from imaging) were used as the input features and target outputs for the machine learning models.

The plans were selected based on certain criteria (e.g., "required to exhibit 90% target coverage" for H&N, "92% target coverage" for Thoracic, etc.), implying these were clinically acceptable plans. This essentially makes the ground truth for training derived from expert consensus in practice (as these were plans approved and delivered by medical professionals at a major institution) and historical outcomes data (the actual treatment parameters achieved).

For the NLP model, the training ground truth was "known name matches" in these same prostate, gynecological, head and neck, thoracic, and pancreas cancer datasets, meaning established mappings between unstandardized and standardized OAR names were used. This again points to expert consensus/standardization.

Ask a specific question about this device

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation is a software system for radiation therapy and medical oncology. Based on user input, RayStation proposes treatment plans. After a proposed treatment plan is reviewed and approved by authorized intended users, RayStation may also be used to administer treatments.

The system functionality can be configured based on user needs.

RayStation consists of multiple applications:

• The main RayStation application is used for treatment planning.
• The RayPhysics application is used for commissioning of treatment machines to make them available for treatment planning and used for commissioning of imaging systems.

The devices to be marketed, RayStation/RayPlan 2024A SP3, 2024A and 2023B, contain modified features compared to last cleared version RayStation 12A including:

• Improved sliding window VMAT (Volumetric Modulated Arc Therapy) sequencing
• Higher dose grid resolution for proton PBS (Pencil Beam Scanning)
• Automated field in field planning
• LET optimization (Linear Energy Transfer)

These applications are built on a software platform, containing the radiotherapy domain model and providing GUI, optimization, dose calculation and storage services. The platform uses three Microsoft SQL databases for persistent storage of the patient, machine and clinic settings data.

As a treatment planning system, RayStation aims to be an extensive software toolbox for generating and evaluating various types of radiotherapy treatment plans. RayStation supports a wide variety of radiotherapy treatment techniques and features an extensive range of tools for manual or semi-automatic treatment planning.

The RayStation application is divided in modules, which are activated through licensing. A simplified license configuration of RayStation is marketed as RayPlan.

The provided document is a 510(k) clearance letter for the RayStation/RayPlan 2024A SP3, 2024A, and 2023B devices. It discusses the substantial equivalence of these devices to a predicate device (RayStation 12A).

However, the document does not contain specific acceptance criteria tables nor detailed study results for a single, comprehensive study proving the device meets acceptance criteria in the format typically requested (e.g., a specific clinical validation study with explicitly defined acceptance metrics like sensitivity, specificity, or AUC, and corresponding reported performance values).

Instead, the document describes a broad software verification and validation process, stating that the software underwent:

• Unit Testing
• Integration Testing
• System Level Testing
• Cybersecurity Testing
• Usability Testing (Validation in a clinical environment)
• Regression Testing

For several "Added/updated functions," the document provides a description of the verification and validation data used to demonstrate substantial equivalence and simply states "Yes" under the "Substantially Equivalent?" column if the validation was "successful." The acceptance criteria for these tests are described narratively within the text, not in a consolidated table format with numerical performance outcomes.

Therefore, I cannot generate the requested table of "acceptance criteria and the reported device performance" as a single, consolidated table with numerical results for the entire device's performance against specific, pre-defined acceptance criteria for a single study. The document describes a process of demonstrating substantial equivalence through various verification and validation activities rather than a single, large-scale study with quantitative acceptance criteria for the overall device performance.

However, I can extract the information related to the validation activities for specific features and the general approach to proving substantial equivalence.

Here's a breakdown of the requested information based on the provided document, addressing each point to the best of my ability given the available details:

Acceptance Criteria and Device Performance (Based on provided verification and validation descriptions)

As noted, a single, consolidated table of quantitative acceptance criteria and overall device performance is not provided. Instead, the document describes various verification and validation activities with implicit or explicit pass criteria for individual features or system aspects to demonstrate substantial equivalence to the predicate device.

Below are examples of how some "acceptance criteria" (pass criteria) and "reported performance" are described for specific features. These are not aggregated performance metrics for the entire device but rather success criteria for sub-components or changes.

2. Calculated values are numerically equal to values obtained from calling the method (regression test).
3. Calculated values are exported correctly from RayStation to DICOM (equality between calculated and exported point, only for Helical Tomo Synchrony plans, only in correct DICOM item).
4. Calculated values are converted correctly from DICOM to Accuray's system format (equality of point coordinates, only for relevant plan types). | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies all pass criteria were met). |
   | Point-dose optimization in brachy plans | 1. Position from the correct image set is used for point-dose objectives/constraints.
5. Possible to add optimization objective/constraint to a point, referring to the correct point.
6. When adding objective/constraint, selection of function type and dose level is possible and reflected in description.
7. Saving and loading an optimization function template containing point objectives/constraints works correctly (loaded functions are same as saved).
8. Results from single/multiple point optimization are as expected (dose in point(s) should be equal to specified dose in objective(s)). | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies all pass criteria were met). |
   | Electron Monte Carlo dose engine improvements | Comparing calculated doses with:

• Measured doses obtained from clinics,
• Doses computed in independent, well-established TPS,
• Doses computed with earlier versions of RayStation,
• Doses computed in BEAMnrc/egs++
  using Gamma evaluation criteria. | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies adequate agreement based on Gamma criteria). |
  | Evaluation on converted CBCT images for protons | For proton MC/PB dose computation:
• Gamma 2%/2mm pass rate above 90%
• Gamma 3%/3mm pass rate above 95% | "The successful validation of this feature demonstrates that the device is as safe and effective as the predicate device." (Implies specified Gamma pass rates were achieved). |
  | Overall Device (Software Verification/Validation) | Software specifications conform to user needs and intended uses, and particular requirements implemented through software can be consistently fulfilled. Conformance to applicable requirements and specifications. Successful outcome of unit, integration, system, cybersecurity, usability, and regression testing. Safety and effectiveness validated. | "RayStation/RayPlan 2024A SP3, 2024A and 2023B have met specifications and are as safe, as effective and perform as well as the legally marketed predicate devices." All general software tests (unit, integration, system, cybersecurity, usability, regression) were acceptable/successful. |

Study Details (Based on the document)

Given the nature of the 510(k) submission for a treatment planning system, the "study" is primarily a comprehensive software verification and validation effort to demonstrate substantial equivalence, rather than a single, standalone clinical trial or diagnostic accuracy study.

1. Sample sizes used for the test set and the data provenance:

   • Test Set Sample Sizes: Not explicitly stated as a single numerical value for a global "test set." Testing was conducted at multiple levels (unit, integration, system, usability, regression) across various features.
     • For "Evaluation on converted CBCT images for protons," it mentions "Test cases consist of CBCTs from the MedPhoton imaging ring on a Mevion S250i system, as well as the on-board CBCT systems on a Varian ProBeam and an IBA P1," implying a set of patient or phantom imaging data, but the exact number of cases/patients is not specified.
     • For other features, it refers to "tests," "validation data," or "computed doses" but doesn't quantify the number of distinct data points/cases used.
   • Data Provenance:
     • Country of Origin: Not specified in the document. Likely internal RaySearch data and potentially data from collaboration with clinical sites, but no specific countries are mentioned.
     • Retrospective or Prospective: Not explicitly stated. The verification and validation activities appear to be primarily retrospective (using existing data, phantom measurements, or simulated scenarios) as part of the software development lifecycle, rather than prospective clinical data collection for a specific study.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • This information is not provided in the document. The document refers to "measured doses obtained from clinics" and "doses computed in independent, well-established TPS" as part of the validation for dose engine improvements, suggesting some form of external or expert-derived ground truth, but the number and qualifications of experts involved are not detailed. For "Evaluation on converted CBCT images for protons," it states "For each case, a ground truth CT image has been prepared to serve as ground truth," implying expert or established reference standard, but again, no details on experts.

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • This information is not provided. The document focuses on computational and functional verification rather than multi-reader clinical assessment.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not explicitly done or reported in this document. The device, RayStation/RayPlan, is a treatment planning system that assists users in creating treatment plans, not primarily an AI-driven image interpretation or diagnostic aid where human reader performance improvement is typically measured. The AI-related feature mentioned is "deep learning segmentation," but the document states, "(The model training is performed offline on clinical CT and structure data.)" It does not detail an MRMC study related to its performance or impact on human readers.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, standalone (algorithm-only) performance was central to the validation. The document describes extensive "Unit Testing," "Integration Testing," "System Level Testing," and "Dose engine validation" which are all a form of standalone algorithmic evaluation. For example, the Gamma evaluation criteria for dose calculations or the numerical equality checks for dose compensation points are purely algorithmic performance assessments.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

   • The ground truth varied depending on the feature being validated:
     • "Measured doses" from clinics / Independent TPS computations / BEAMnrc/egs++ calculations: For dose engine validation. This represents a highly accurate, often physical measurement or well-established computational standard.
     • "Ground truth CT image": For evaluation of converted CBCT images for protons. This implies a high-quality reference image.
     • Internal "expected results" and "specifications": For functional and system-level tests (e.g., for point-dose optimization, the expected result was that the dose in the point should equal the dose specified in the objective).
     • "Clinical objectives": Used for plan comparisons (e.g., in segment weight optimization validation), likely representing desired dose distributions defined by clinical experts.

7. The sample size for the training set:

   • The document mentions "deep learning segmentation" and states that "The model training is performed offline on clinical CT and structure data." However, the sample size for this training set is not provided.

8. How the ground truth for the training set was established:

   • For "deep learning segmentation," the ground truth for training would implicitly be the "clinical CT and structure data" mentioned. This typically means expert-delineated structures (ROIs) on clinical CT images, but the exact method (e.g., single expert, consensus, specific software tools) is not detailed.

Ask a specific question about this device