Search Results

The BEAMSCAN MR system is used preferably in combined MRI-radiation therapy systems with static magnetic fields of up to 1.5T and is intended to collect beam data in water under the aspect of machine QA for the following purposes:

• acceptance testing and/or commissioning of a combined MRI-radiation therapy system
• measurements after repair or replacement of major treatment unit components of a combined MRI-Radiation therapy system
• beam data analysis according to international therapy dosimetry protocols
• acquisition, formatting and transfer of basic data to treatment planning systems
• periodic QA procedures, e.g. constancy check
• high precision data acquisition for scientific research (not a medical device indication)

The BEAMSCAN MR system is comprised of a PMMA tank with a moving mechanism and radiation detectors. Further main components are a carriage with built-in electrometer, control unit and control interface. The carriage includes a water reservoir. The whole system is controlled by software for data display and processing.

Here's a breakdown of the acceptance criteria and the study information for the BEAMSCAN MR device, based on the provided FDA 510(k) summary:

The document focuses on demonstrating substantial equivalence to a predicate device (BEAMSCAN water phantom system, K161807) and adapting the system for use in combined MRI-radiation therapy systems with static magnetic fields up to 1.5T. Therefore, the "acceptance criteria" largely revolve around meeting relevant safety and performance standards for this new environment, and the "study" involves verification and validation testing to confirm these aspects.

1. A table of acceptance criteria and the reported device performance

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

The document does not specify a distinct "test set" with a number of cases in the traditional sense of a clinical study. The performance evaluation is based on:

• Bench and Non-clinical Testing: These are laboratory-based tests. The "sample size" would relate to the number of measurements taken or the number of units tested. This information is not provided beyond stating that verification and validation were performed.
• Clinical Workflow Validation: This was conducted "with qualified medical physicists and experienced PTW staff." There is no specific number of "cases" or "patients" as the device is for QA and not used with patients. The provenance is internal to the manufacturer (PTW staff) and likely conducted at their facilities or collaborator sites.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

No "ground truth" established by external experts in the way typical for AI/diagnostic devices. The "validation of the clinical workflow" was performed by:

• "qualified medical physicists"
• "experienced PTW staff"

The specific number of these individuals or their detailed qualifications (e.g., years of experience) are not provided. Given the nature of a QA device, the "ground truth" for its performance accuracy would be derived from physical dosimetry standards and established protocols, rather than expert interpretation of patient data.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

No adjudication method is mentioned as there isn't a traditional "test set" requiring expert consensus on findings. The performance evaluation relies on objective measurements against engineering specifications and relevant industry standards.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study was not done. This device is a quality assurance tool for radiation therapy systems, not a diagnostic imaging device that assists human readers. Therefore, the concept of "human readers improving with AI assistance" is not applicable here.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

The BEAMSCAN MR is a hardware system with integrated software for data display and processing. It performs "standalone" measurements of beam characteristics. There isn't a human-in-the-loop diagnostic process for the device itself; rather, humans (medical physicists) operate the device to perform QA. The software itself contains "no new functions" and was previously cleared. The focus is on the hardware's performance in an MR environment.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

The "ground truth" for the device's performance is based on:

• Physical Dosimetry Standards: Compliance with standards like IEC 60731:2016 for radiotherapy dose measurements.
• Engineering Specifications: The device's ability to accurately measure parameters like detector positioning, reproducibility, and mechanical alignment against its design inputs.
• Regulatory Standards: Adherence to electrical safety (IEC 61010-1:2010), EMC (IEC 60601-1-2:2014), and MR safety (ASTM F2052-15) standards.

There is no "expert consensus," "pathology," or "outcomes data" ground truth in the context of this device because it is a measurement tool, not a diagnostic or prognostic one.

8. The sample size for the training set

The document does not explicitly mention a "training set" in the context of machine learning, as this device primarily relies on physics principles and engineering. The "software contains no new functions" and was previously cleared. Therefore, any development data for the original software would have been part of the predicate device's clearance.

9. How the ground truth for the training set was established

Not applicable as there is no mention of a "training set" for machine learning purposes for this device. The software's ground truth (i.e., its correct functionality) would have been established through traditional software verification and validation, ensuring it correctly implements the algorithms for data acquisition, processing, and display as specified by physics principles and dosimetry protocols.

Ask a specific question about this device

The BEAMSCAN system is intended to collect beam data in water under the aspect of machine QA for the following purposes:

• acceptance testing and/or commissioning of a radiation therapy system.
• measurements after repair or replacement of major treatment unit components.
• beam data analysis according to international therapy dosimetry protocols.
• acquisition, formatting and transfer of basic data to treatment planning systems.
• periodic QA procedures, e.g. constancy checks.

The BEAMSCAN system is comprised of a PMMA tank with a moving mechanism and a radiation detector. Further main components are a lifting carriage with built-in electrometer and control unit with separate control pendant. The lifting carriage either includes a water reservoir or a separate reservoir is used. A positioning device can be mounted on top of the lifting carriage and allows the water phantom to be aligned manually. The whole system is controlled with software for data display and processing.

This document describes the BEAMSCAN system, a water phantom for dosimetry measurements in radiation therapy. It is intended for machine quality assurance (QA), including acceptance testing, commissioning, measurements after repair, beam data analysis, data transfer to treatment planning systems, and periodic QA procedures. Its use is restricted to qualified medical physicists and must not be used while a patient is present.

Here's an analysis of the acceptance criteria and study information provided:

1. Table of Acceptance Criteria and Reported Device Performance

The provided document doesn't explicitly list a table of acceptance criteria with corresponding performance metrics in a pass/fail format. However, it states that "Verification and validation testing demonstrated that BEAMSCAN fulfils the design specification and its intended use." It also mentions non-clinical performance testing was conducted according to IEC 60731 A1:2016 and specific properties of water phantom systems.

Based on the "Bench and Non-clinical Testing" section, we can infer the aspects that were tested and confirmed:

2. Sample Size Used for the Test Set and Data Provenance

The document does not explicitly specify a "sample size for the test set" in the context of clinical images or patient data. The BEAMSCAN system is for quality assurance of radiation therapy machines, not for patient diagnosis or treatment. The testing described focuses on the device's measurement capabilities and operational performance.

The "Bench and Non-clinical Testing" section mentions "Verification and validation testing" and "BEAMSCAN validation testing." It states that this testing was performed with "qualified medical physicists and experienced PTW staff." There is no specific number of test cases or "samples" quantified in the typical sense of a clinical study.

Data Provenance: The testing was "non-clinical" and involved "qualified medical physicists and experienced PTW staff," suggesting it was conducted internally by the manufacturer (PTW-Freiburg, Germany) or by independent test laboratories (for electrical safety and EMC), focusing on the device's physical and software performance. Therefore, the data provenance is from laboratory/bench testing and internal validation, not retrospective or prospective clinical patient data.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document mentions that "Validation of the clinical workflow has been conducted in BEAMSCAN validation testing with qualified medical physicists and experienced PTW staff." It does not specify a precise number of individual experts beyond the general terms "qualified medical physicists" and "experienced PTW staff."

Qualifications of Experts:

• Qualified medical physicists: This is a professional designation, implying they possess the necessary education, training, and certification to work in medical physics, which includes radiation dosimetry and QA.
• Experienced PTW staff: This refers to personnel employed by the manufacturer (PTW-Freiburg) who have expertise related to the device and its intended use.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method in the sense of resolving discrepancies among multiple expert assessments for ground truth, as would be common in a clinical imaging study. Since the testing involved device performance against established standards (e.g., IEC 60731 A1:2016) and design specifications, the "ground truth" would be determined by these objective criteria and the observed measurements. Workflow validation would likely involve confirmation by the "qualified medical physicists" that the system operates as expected for its intended QA purposes.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done

No, an MRMC comparative effectiveness study was not done. The BEAMSCAN is a measurement device for machine QA, not a diagnostic or treatment planning tool that would involve human readers interpreting cases with and without AI assistance. The study focuses on the device's inherent measurement accuracy and performance.

6. If a Standalone (i.e. algorithm only without human-in-the loop performance) Was Done

The BEAMSCAN system is a physical device that performs measurements through mechanical movement and detector readings, controlled by software. The "performance data" section details "Software Verification and Validation Testing" and "Bench and Non-clinical Testing." These tests evaluate the device's performance, including the software's role in controlling measurements and data processing. While the human user is "in the loop" for setting up the QA procedure and interpreting the results, the core measurement functionality of the system itself, including its embedded algorithms for positioning and data acquisition, is tested in a standalone manner to ensure it meets specifications. The "automated levelling by calculating ideal coordinate axes with respect to the measured water surface" is an example of an algorithmic function that would be tested for standalone performance.

7. The Type of Ground Truth Used

The ground truth used for testing the BEAMSCAN system is based on:

• Design Specifications: The device's performance was verified against its design inputs.
• International Standards: Compliance with standards such as IEC 60731 A1:2016 for dosimetry and IEC electrical safety and EMC standards.
• Objective Physical Quantities: Measurement accuracy, reproducibility, and mechanical alignment are compared to known or expected physical values and acceptable tolerances for radiation therapy QA.
• Clinical Workflow Expectations: Validation by qualified medical physicists ensures the system functions correctly within a radiotherapy QA workflow.

This is a form of objective performance ground truth, not a subjective ground truth like expert consensus on medical images or pathology reports.

8. The Sample Size for the Training Set

The document does not mention a "training set" in the context of machine learning or AI model development. The BEAMSCAN is a physics-based measurement system, not a device that employs machine learning for its core functionality of beam data acquisition. Therefore, the concept of a "training set" as understood in AI/ML is not applicable here. Its performance is based on its physical design, calibration, and deterministic algorithms.

9. How the Ground Truth for the Training Set Was Established

As there is no mention of a "training set" or machine learning algorithms in the context of the BEAMSCAN system's primary function, this question is not applicable based on the provided information.

Ask a specific question about this device