Search Results

CollisionCheck is intended to assist radiation treatment planners in predicting when a treatment plan might result in a collision between the treatment machine and the patient or support structures.

The CollisionCheck device (model RADCO) is software intended to assist users to identify where collisions between the treatment machine and the patient or support structures may occur in a treatment plan. The treatment plans are obtained from the Eclipse Treatment Planning System (also referred to as Eclipse TPS) of Varian Medical Systems. CollisionCheck runs as a dynamic link library (DLL) plugin to Varian Eclipse. It is designed to run on the Windows Operating System. CollisionCheck performs calculations on the plan obtained from Eclipse TPS (Version 12 (K131891), Version 13.5 (K141283), and Version 13.7 (K152393) which is a software used by trained medical professionals to install and simulate radiation therapy treatments for malignant or benign diseases.

The provided text describes the regulatory clearance of CollisionCheck (K171350) and compares it to a predicate device, Mobius3D (K153014). However, it does not contain specific details about acceptance criteria, the study design (e.g., sample size, data provenance, ground truth establishment, expert qualifications, or adjudication methods), or MRMC study results. The document states that "no clinical trials were performed for CollisionCheck" and mentions "Verification tests were performed to ensure that the software works as intended and pass/fail criteria were used to verify requirements." This implies that the performance demonstration was likely limited to software verification and validation, rather than a clinical performance study with human-in-the-loop or standalone AI performance metrics.

Therefore, many of the requested details cannot be extracted from the provided text. I will provide what can be inferred or stated as absent based on the document.

Acceptance Criteria and Device Performance

The document does not explicitly list quantitative acceptance criteria with corresponding performance metrics like sensitivity, specificity, or F1-score for the CollisionCheck device. Instead, the performance demonstration focuses on software verification and validation to ensure the device works as intended and is as safe and effective as the predicate device.

Table of Acceptance Criteria and Reported Device Performance (Inferred/Based on Document Context):

Study Details:

Given the statement "no clinical trials were performed for CollisionCheck," and the focus on "Verification tests," most of the questions regarding a typical AI performance study (like those involving test sets, ground truth experts, MRMC studies) cannot be answered with specific data from this document. The performance demonstration appears to have been solely based on internal software verification and validation activities.

1. Sample sizes used for the test set and data provenance:

   • Test Set Sample Size: Not specified. The document only mentions "verification tests" and "pass/fail criteria."
   • Data Provenance: Not specified. It's likely synthetic or internal clinical data used for software testing, rather than a distinct, prospectively collected, or retrospectively curated clinical test set for performance evaluation in a regulatory sense.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Not applicable/Not specified. Given that "no clinical trials were performed," it's highly improbable that a formal expert-adjudicated ground truth was established for a test set in the context of an AI performance study. Ground truth in this context would likely be defined by the physics-based simulation of collisions within the software's design.

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • Not applicable/Not specified. No adjudication method is mentioned, consistent with the absence of a clinical performance study involving human readers.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not done. The document explicitly states, "no clinical trials were performed." Therefore, no effect size of human reader improvement with AI assistance is reported.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • While the "verification tests" would evaluate the algorithm's standalone functionality, the document does not provide specific performance metrics (e.g., sensitivity, specificity) for its standalone performance that would typically be seen in a standalone AI evaluation. The device assists a human user, so its "standalone" performance wouldn't be in isolation but rather its ability to correctly identify collisions as defined by its internal models.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • The document implies a physics-based or computational ground truth. The device performs calculations and simulations. The "ground truth" for its verification and validation would be whether its simulation correctly identifies collisions based on defined geometric and physical parameters. It's not based on expert consensus, pathology, or outcomes data, as it's a planning assistance tool, not a diagnostic one.

7. The sample size for the training set:

   • Not applicable/Not specified. The document describes CollisionCheck as software that performs calculations and simulations (modeling the linac as a cylinder, supporting applicators, etc.). It is not described as an AI or machine learning model that requires a "training set" in the conventional sense of supervised learning on a large dataset. Its functionality is likely rule-based or physics-informed, rather than learned from data.

8. How the ground truth for the training set was established:

   • Not applicable/Not specified. Since it's not described as an ML model with a training set, the concept of establishing ground truth for a training set does not apply here. The "ground truth" for its development would be the accurate mathematical and physical modeling of collision scenarios.

Ask a specific question about this device

ClearCheck is intended for quality assessment of radiotherapy treatment plans.

The ClearCheck device (model RADCC) is a software intended to present treatment plans obtained from Eclipse Treatment Planning System (also referred to as Eclipse TPS) of Varian Medical Systems in a user friendly way (numerical form of data) for user approval of the treatment plan. ClearCheck runs as a dynamic link library (dll) plugin to Varian Eclipse. It is designed to run on the Windows Operating System and generated reports can be viewed on Internet Explorer. ClearCheck performs calculations on the plan obtained from Eclipse TPS (Version 12 (K131891) and Version 13.5 (K141283)) which is a software used by trained medical professionals to design and simulate radiation therapy treatments for malignant or benign diseases. ClearCheck has two components: A standalone Windows Operating System executable application that is used for 1. administrative operations to set specified default settings and user settings. 2. A plan evaluation application that is a dynamic link library (dll) file that is a plugin to the Varian Medical Systems Eclipse TPS. The plugin is designed to evaluate the quality of an Eclipse treatment plan. Plan quality is based on user specified Dose Constraints and Plan Check Parameters.

Here's an analysis of the provided text regarding the acceptance criteria and study for the ClearCheck device, organized according to your request.

Please note: The provided document is a 510(k) summary, which focuses on demonstrating substantial equivalence to a predicate device. It explicitly states that "no clinical trials were performed for ClearCheck" (Section 5.7). Therefore, a substantial portion of your requested information (e.g., MRMC studies, specific performance metrics against ground truth, expert qualifications, adjudication methods, sample sizes for test/training sets with ground truth derivation methods) is not present in this type of regulatory submission. The verification tests mentioned are likely internal software validation rather than clinical performance studies.

1. Table of acceptance criteria and the reported device performance

The document does not provide a specific table of quantitative acceptance criteria for device performance based on clinical outcomes or accuracy metrics. Instead, "pass/fail criteria were used to verify requirements" during internal verification tests. These requirements are implicit in the comparison to the predicate device and the claim of substantial equivalence.

2. Sample size used for the test set and the data provenance

The document does not specify a "test set" in the context of clinical or performance data. It mentions "Verification tests" were performed for the software. These tests would involve internal generated data or existing clinical plans to validate the software's functionality, but no details on sample size, data provenance, or specific test cases are provided for external review.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

This information is not provided. Since no clinical trials or external performance evaluations of this nature were conducted (as stated in Section 5.7), the concept of "ground truth" as derived by experts for a test set is not applicable to the submitted performance data.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

This information is not provided for the same reasons as above.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No MRMC comparative effectiveness study was done. The document explicitly states "no clinical trials were performed for ClearCheck." The device is a "quality assessment" tool for radiotherapy plans, not an AI for image interpretation or diagnosis that would typically involve human reader performance studies.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

A standalone performance assessment in the sense of the algorithm's internal calculations and functionality was performed as part of "Verification tests." However, specific quantitative metrics common for standalone AI algorithms (e.g., sensitivity, specificity, AUC against a clinical ground truth) are not provided in this regulatory summary. The device's "performance" is primarily assessed by its functional correctness and consistency with the predicate device's overall purpose of quality assessment.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

This is not explicitly stated. For the internal "Verification tests," the "ground truth" would likely be the expected outputs or calculated values based on established physics principles and treatment planning guidelines, which the software is designed to implement and report. This is not the same as clinical "ground truth" derived from patient outcomes or expert consensus on a diagnosis.

8. The sample size for the training set

This information is not applicable and not provided. ClearCheck is described as a software tool that performs calculations and presents data based on user-specified dose constraints and plan check parameters from an existing Eclipse TPS plan. It is not an AI/ML algorithm that learns from a "training set" of data to make predictions or classifications.

9. How the ground truth for the training set was established

This information is not applicable and not provided, as the device does not employ a machine learning model that requires a training set with associated ground truth.

Ask a specific question about this device