Search Results

The intended use of FOCUS RTP System is to provide radiation treatment planning capability, for both external beam and brachytherapy sources, to satisfy the prescription of a Radiation Oncologist. The resultant treatment plan is to be evaluated, modified as necessary, approved and delivered by qualified medical personnel. Operation of the system is identical to FOCUS systems cleared under previous Premarket Notifications with the exception the user can now select a third type of external beam particle for therapy (protons) in addition to the earlier two particles (electrons and photons)..

The FOCUS Radiation Treatment Planning System accepts a) patient diagnostic imaging data from CT and MR scans, or from films, and b) "source" dosimetry data, typically from a linear accelerator. The system then permits the user to display and define (contour) a) the target volume to be treated and b) critical structures which must not receive above a certain level of radiation, on these diagnostic images. Based on the prescribed dose, the user, typically a Dosimetrist or Medical Physicist, can then create multiple treatment scenarios involving the type, number, position(s) and energy of radiation beams and the use of treatment aids between the source of radiation and the patient (wedges, blocks, ports, etc.). The FOCUS system then produces a display of radiation dose distribution within the patient, indicating not only doses to the target volume but to surrounding tissue and structures. The "best" plan satisfying the prescription is then selected, one which maximizes dose to the target volume while minimizing dose to surrounding healthy volume. The parameters of the plan are output in hard-copy format for later reference and for placement in the patient file. Previously, for situations where external beam therapy was to be used, either Electron and/or Photon radiation beams could be selected.. These were delivered by a linear accelerator whose output characteristics are input to the treatment planning system prior to beginning planning. This Premarket Notification addresses the addition of a third type of radiation beam -Proton. The algorithm for calculating dose was provided by the Massachusetts General Hospital (MGH), based on their years of experience at the Harvard Cyclotron Lab. In addition to providing the algorithm, MGH also worked with CMS in its implementation. Software developers at MGH were trained on the CMS software development process to permit them to create code directly for use in FOCUS. As the final step, MGH provided the verification testing to assure the algorithm had been implemented correctly, measuring calculated dose against measures. A FOCUS RTP System with proton planning capability is now in clinical use at the Northeast Proton Therapy Center.

The provided text describes the K020027 FOCUS Radiation Treatment Planning System with Proton Planning Capability. It details the device's function, its new proton planning feature, and the non-clinical testing performed.

Here's an analysis of the acceptance criteria and study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size: Not explicitly stated in terms of a specific number of cases or plans. The text refers to "Algorithm test cases" without detailing their quantity.
• Data Provenance: The testing was a "non-clinical" study. The algorithm was developed based on "years of experience at the Harvard Cyclotron Lab" at Massachusetts General Hospital (MGH). The verification testing involved measuring "calculated dose against measures," implying an experimental setup or phantom measurements rather than patient data. The study, "Validation of the CMS Proton Treatment Planning System for Treatments In Large Field Beam Line at the Harvard Cyclotron Laboratory (HCL)," suggests a laboratory-based, prospective evaluation of the algorithm's performance against physical measurements or highly controlled simulated scenarios.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: Not explicitly stated. However, the algorithm itself was provided by the Massachusetts General Hospital (MGH), implying the expertise of their staff at the Harvard Cyclotron Lab (HCL). MGH also performed the verification testing.
• Qualifications of Experts: The text states the algorithm was based on "years of experience at the Harvard Cyclotron Lab." This implies the involvement of highly experienced medical physicists and other specialists familiar with proton therapy dose calculation. "Skip Rosenthal of MGH" authored the validation document, indicating his expertise in this area.

4. Adjudication Method for the Test Set

• Adjudication Method: Not applicable or not described. The validation appears to be a direct comparison of calculated dose values to measured dose values, rather than an expert consensus process.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

• MRMC Study: No, an MRMC comparative effectiveness study was not done. The device is a radiation treatment planning system algorithm, not a diagnostic AI intended to assist human readers in interpreting images or making a diagnosis. Its function is to calculate dose distributions based on user input for treatment planning. The study focused on the accuracy of the dose calculation algorithm itself.

6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done

• Standalone Performance Study: Yes, in essence, the "non-clinical testing" described is a standalone performance study. The "Algorithm test cases were written and executed to assure the system is calculating dose correctly for proton treatment plans." This focused solely on the algorithm's output (calculated dose) compared to a reference (measured dose), without evaluating user interaction or clinical outcomes in humans. The current device is meant to be used by "Dosimetrist or Medical Physicist" where they would evaluate, modify and approve the plans. However, the specific testing described in the summary is of the algorithm's accuracy in isolation.

7. The Type of Ground Truth Used

• Type of Ground Truth: The ground truth used was measured dose values. The text explicitly states "measuring calculated dose against measures" in the verification testing. This indicates that the algorithm's computed dose distributions were compared against physical measurements obtained from a phantom or experimental setup at the Harvard Cyclotron Laboratory.

8. The Sample Size for the Training Set

• Sample Size for Training Set: Not explicitly stated. The algorithm itself was provided by MGH, based on their "years of experience at the Harvard Cyclotron Lab." This implies that it was developed and implicitly "trained" or refined over a long period from extensive data and knowledge accumulated at HCL, but there's no defined "training set" in the context of supervised machine learning with a specific sample size.

9. How the Ground Truth for the Training Set Was Established

• Ground Truth for Training Set: The ground truth for the underlying algorithm's development (or its "training" in a broader sense) was established through extensive experience and data collection at the Harvard Cyclotron Lab (HCL) over many years. This would likely involve:
  • Clinical experience and outcomes: Understanding how protons interact with tissue and the effects of different dose distributions.
  • Physics measurements: Extensive experimental data on proton beam characteristics, interactions, and dose deposition in phantoms and biological samples.
  • Computational modeling and simulations: Development and validation of physics models used in dose calculation.
    The algorithm provided by MGH represents the culmination of this accumulated knowledge and empirical validation.

Ask a specific question about this device

The FOCUS Pilot Contouring Workstation permit contouring of patient tumors and critical structures on images (CT slices) at a workstation remote from the FOCUS Radiation Treatment Planning System. Images are drawn from the FOCUS RTP System, contoured and then returned to the FOCUS RTP System for completion of the treatment planning activity.

The FOCUS Pilot Contouring Workstation ("the CWS") is designed to be used in conjunction with the FOCUS Radiation Treatment Planning System (K915691). The CWS (the "client") relocates the time-consuming task of image contouring from the RTP System (the "server") and permits it to be done remotely on a workstation designed solely for contouring. This frees the RTP System for performance of dose calculation and display, treatment plan comparison, source file maintenance and other tasks associated with radiation treatment planning.

The CWS will accept patient image data (CT slices) from the FOCUS RTP System and stores them on its hard drive prior to recalling them for contouring. The CWS provides the user tools with which to review the images and contour and label tumor volumes and critical structures. Patient demographic data, in addition to that entered at the RTP System, is also able to be entered at this workstation. After this contouring has been performed, the images are stored on the PC hard drive prior to being returned to the RTP system for subsequent use in treatment planning. The Contouring Workstation does no treatment planning, and does not accept plan information for display and/or modification. Screen capture printing is available from the CWS.

The provided text does not contain any information about acceptance criteria or a study proving the device meets those criteria.

The document is a 510(k) summary for the FOCUS Pilot Contouring Workstation, focusing on demonstrating substantial equivalence to predicate devices. It describes the device's function, intended use, and technological characteristics compared to similar systems. It also discusses the safety measures and quality checks in place after the contouring process, but these are not presented as acceptance criteria for the device itself.

Therefore, I cannot provide the requested table or answer the specific questions about the study design, sample sizes, expert qualifications, or ground truth.

Ask a specific question about this device

The Advanced Radiation Therapy Systems, Inc., "Advanced Treatment Planning System" is used to plan patient treatments for radiation therapy with external photons and electrons.

The Advanced Treatment Planning System (ATPS) from Advanced Radiation Therapy Systems, Inc. (ARTS) is a software product that runs on a Silicon Graphics, Inc. UNIX Workstation in conjunction with specified accessory hardware. The ATPS provides the user the tools to easily perform patient treatment planning for the application of electron and photon radiation therapy utilizing input of patient anatomy from Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) devices. Incorporation of 3D visualization software from Advanced Visualization Systems (AVS), combined with X windows and Motif graphics, results in a graphical user interfacc that is both flexible and casy to use. See the User's Manual in Tab 5 for a more detailed description and examples of graphics. The ATPS provides both two dimensional (2D) and three dimensional (3D) dose calculation algorithms for photons and electrons.

The provided text describes a Premarket Notification [510(k)] Summary for a device called "Advanced Treatment Planning System" (ATPS) by Advanced Radiation Therapy Systems, Inc. (ARTS). This document is a regulatory submission to the FDA, asserting substantial equivalence to predicate devices, and as such, it does not contain a detailed study proving the device meets specific acceptance criteria in the way a clinical trial detailed report would.

However, based on the information provided, we can infer some aspects related to acceptance criteria and how a "study" (in this context, the 510(k) submission itself and the comparison with predicates) demonstrates compliance.

Here's an attempt to answer your questions based on the provided text, with the understanding that typical detailed study data is not present in this type of summary document:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state "acceptance criteria" with numerical targets and then explicitly report "device performance" against those targets in a table format. Instead, the core of a 510(k) submission for a treatment planning system relies on substantial equivalence to predicate devices. This implies that the device's performance, particularly its dose calculation algorithms and user interface, must be comparable or not raise new questions of safety and effectiveness compared to the predicate devices.

The "acceptance criteria" are implicitly the functional and performance benchmarks set by the predicate devices. The "reported device performance" is demonstrated through the "Predicate Comparison Table" (mentioned but not included in the provided text for "Technological Characteristics"). This table would detail how the ATPS's features and performance parameters align with or are similar to the predicate devices.

Implicit Acceptance Criteria (Inferred from 510(k) process):

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

The document does not provide details on a specific "test set" sample size or data provenance. In a 510(k) for a treatment planning system, the "test" often involves demonstrating that the dose calculation algorithms produce results consistent with established physics principles and industry standards, and that the software functions as intended across various clinical scenarios. This might involve:

• Benchmarking against publicly available phantom data: Often used to validate dose calculation accuracy.
• Comparison to predicate device outputs: Running the same patient cases on both the new device and the predicate to show comparable treatment plans.
• Internal validation cases: Cases designed by the manufacturer to test specific features and edge cases.

However, the specific numbers of such cases, their origin, or whether they were retrospective or prospective are not mentioned in this summary.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

The document does not specify the number or qualifications of experts used to establish ground truth. For a device like a treatment planning system, "ground truth" for dose calculations is typically established through:

• Physics principles and accepted dosimetry measurements: E.g., measurements in phantom tanks with ion chambers or film.
• Comparison to other validated algorithms: Using highly accurate research algorithms or established commercial systems as a reference.
• Clinical expert review: Radiation oncologists and medical physicists review generated treatment plans for clinical acceptability and reasonableness.

Given this is a 510(k) summary, these details are usually found in the full submission, not the summary.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

The document does not mention any adjudication method. This is typically relevant for studies involving human interpretation (e.g., image reading) where disagreement among experts needs to be resolved. For a treatment planning system, the "adjudication" is more about adherence to physical laws and clinical standards, and consistency with predicate device performance.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

The document does not indicate that a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was performed. MRMC studies are primarily for devices that assist human readers in tasks like diagnosis (e.g., CAD systems for mammography). A Radiation Therapy Treatment Planning System is a tool for physicists and oncologists to create treatment plans, not primarily to "read" or interpret images in a diagnostic sense where AI offers an assistive reading benefit in an MRMC context. Therefore, this type of study would not typically be applicable or described for this device.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

The ATPS is inherently a "human-in-the-loop" device. It provides "the user the tools to easily perform patient treatment planning." While the dose calculation algorithms themselves operate "stand-alone" on computed data, the overall "performance" of the system is the combination of the algorithms and the user's ability to create an optimal plan.
Therefore, "standalone" algorithm performance would have been assessed internally (e.g., dose calculation accuracy validation), but the device itself is designed for human interaction. The 510(k) likely focused on validating the accuracy of the underlying algorithms and the usability of the interface, not a "standalone vs. human-in-the-loop" comparison in the same way as AI diagnostic tools.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The document does not explicitly state the type of ground truth used. For a treatment planning system, the "ground truth" for validation would primarily involve:

• Physics dosimetry measurements: Using phantoms and physical detectors to verify the accuracy of calculated dose distributions.
• Reference calculations: Comparing the ATPS's dose calculations to those from well-validated and accepted algorithms or benchmark software.
• Clinical acceptability by physicists and oncologists: Reviewing treatment plans to ensure they meet clinical standards and safely deliver the prescribed dose to the target while sparing healthy tissue.

The most concrete "ground truth" for the dose calculation algorithms would be based on physical measurements and established physics models, often vetted by expert medical physicists.

8. The sample size for the training set

The document does not mention a "training set" or its sample size. The ATPS is described as a software product providing dose calculation algorithms. Such algorithms, especially those developed in the late 1990s, would typically be based on analytical models of radiation transport (e.g., Clarkson, convolution/superposition for photons; various electron algorithms) derived from fundamental physics, rather than machine learning models that require labeled training data in the modern sense. Therefore, a "training set" as understood in AI/ML contexts would not be applicable here. The algorithms are "trained" by incorporating physics principles and parameters, not by being fed a dataset.

9. How the ground truth for the training set was established

As there is no mention of a "training set" in the context of machine learning, this question is not applicable based on the provided text. The "ground truth" for the underlying physics models would be derived from fundamental scientific principles, experimental measurements in physics labs, and Monte Carlo simulations, all validated by expert physicists over decades.

Ask a specific question about this device