Search Results

ART-Plan+'s indicated target population is cancer patients for whom radiotherapy treatment has been prescribed. In this population, any patient for whom relevant modality imaging data is available.

ART-Plan+'s includes several modules:

• SmartPlan which allows automatic generation of radiotherapy treatment plan that the users import into their own Treatment Planning System (TPS) for the dose calculation, review and approval. This module is available for supported prescriptions for prostate only.

• Annotate which allows automatic generation of contours for organs at risk, lymph nodes and tumors, based on medical practices, on medical images such as CT and MR images

ART-Plan+ is not intended to be used for patients less than 18 years of age.

The indicated users are trained medical professionals including, but not limited to, radiotherapists, radiation oncologists, medical physicists, dosimetrists and medical professionals involved in the radiation therapy process.

The indicated use environments include, but are not limited to, hospitals, clinics and any health facility offering radiation therapy.

ART-Plan+ is a software platform allowing contour regions of interest on 3D images and to provide an automatic treatment plan. It includes several modules:

-Home: tasks

-Annotate and TumorBox: contouring of regions of interest

-SmartPlan: creation of an automatic treatment plan based on a planning CT and a RTSS

-Administration and settings : preferences management, user account management, etc.

-Institute Management: institute information, including licenses, list of users, etc.

-About: information about the software and its use, as well as contact details.

Annotate, TumorBox and SmartPlan are partially based on a batch mode, which allows the user to launch the operations of autocontouring and autoplanning without having to use the interface or the viewers. In that way, the software is completely integrated into the radiotherapy workflow and offer to the user a maximum of flexibility.

ART-Plan+ offers deep-learning based automatic segmentation of OARs and LNs for the following localizations:

-Head and neck (on CT images)

-Thorax/breast (on CT images)

-Abdomen (on CT and male on MR images)

-Pelvis male (on CT and MR images)

-Pelvis female (on CT images)

-Brain (on CT images and MR images)

ART-Plan+ offers deep-learning based automatic segmentation of targets for the following localizations:

-Brain (on MR images)

Based on the provided text, here's a detailed breakdown of the acceptance criteria and the study that proves the device meets them:

1. Table of Acceptance Criteria and the Reported Device Performance:

The document describes five distinct types of evaluations with their respective acceptance criteria. While the exact "reported device performance" (i.e., the specific numerical results obtained for each metric) is not explicitly stated, the document uniformly concludes, "All validation tests were carried out using datasets representative of the worldwide population receiving radiotherapy treatments. Finally, all tests passed their respective acceptance criteria, thus showing ART-Plan + v3.0.0 clinical acceptability." This implies all reported device performances met or exceeded the criteria.

2. Sample Sizes Used for the Test Set and the Data Provenance:

• Non-regression Testing (Autosegmentation of ORs): Minimum sample size of 24 patients.
• Qualitative Evaluation (Autosegmentation of ORs): Minimum sample size of 18 patients.
• Quantitative Evaluation (Autosegmentation of ORs): Minimum sample size of 24 patients.
• Inter-expert Variability Evaluation (Autosegmentation of ORs): Minimum sample size of 13 patients.
• Quantitative Evaluation (Brain Metastasis, MR images): Minimum sample size of 51 patients.
• Quantitative Evaluation (Glioblastoma, MR images): Minimum sample size of 43 patients.
• Quantitative and Qualitative Evaluation (Automatic Treatment Plans): Minimum sample size of 20 patients.

Data Provenance: The document states, "All validation tests were carried out using datasets representative of the worldwide population receiving radiotherapy treatments." It does not specify the country of origin or whether the data was retrospective or prospective.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts:

The document refers to "medical experts" or "clinicians" for establishing ground truth and performing evaluations.

• For the non-regression testing of autosegmentation, "manual contours performed by medical experts" were used.
• For qualitative evaluation of autosegmentation, "medical experts" performed the qualitative evaluation.
• For inter-expert variability evaluation of autosegmentation, "two independent medical experts" were asked to contour the same images.
• For brain metastasis and glioblastoma segmentation, "contours provided by medical experts" were used for comparison.
• For the evaluation of automatic treatment plans, "medical experts" determined the clinical acceptability.

The specific number of experts beyond "two independent" for inter-expert variability is not consistently provided, nor are their exact qualifications (e.g., specific specialties like "radiation oncologist" or years of experience). However, the stated users of the device include "trained medical professionals including, but not limited to, radiotherapists, radiation oncologists, medical physicists, dosimetrists and medical professionals involved in the radiation therapy process," implying these are the types of professionals who would serve as experts.

4. Adjudication Method for the Test Set:

• For the inter-expert variability test, it involved comparing contours between two independent medical experts and with the software's contours. This implies a comparison rather than an explicit formal adjudication method (like 2+1 voting).
• For other segmentation evaluations, the ground truth was "manual contours performed by medical experts" or "contours provided by medical experts." It's not specified if these were consensus readings, or if an adjudication method was used if multiple experts contributed to a single ground truth contour for a case.
• For the automatic treatment plan qualitative evaluation, "expert review" is mentioned, but the number of reviewers or their adjudication process is not detailed.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

The document describes studies that evaluate the standalone performance of the AI for segmentation and treatment planning, and how its performance compares to expert-generated contours/plans or inter-expert variability. It does not explicitly describe an MRMC comparative effectiveness study designed to measure the improvement of human readers with AI assistance versus without AI assistance. The focus is on the AI's performance relative to expert-defined ground truths or benchmarks.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done:

Yes, the studies are largely focused on standalone algorithm performance.

• The "Non-regression testing," "Quantitative evaluation," and "Inter-expert variability evaluation" of autosegmentation explicitly compare the software's generated contours (algorithm only) against manual contours or inter-expert contours.
• The "Quantitative evaluation of autosegmentation of Brain metastasis" and "Glioblastoma" assess the algorithm's performance (sensitivity, precision, DSC, FP) against expert-provided contours.
• For "Automatic Treatment Plan Generations," the quantitative evaluation compares the algorithm's plans to manual plans, and the qualitative evaluation assesses the acceptance of the automatic plans by experts.

7. The Type of Ground Truth Used:

The primary ground truth relied upon in these studies is:

• Expert Consensus/Manual Contours: This is repeatedly stated as "manual contours performed by medical experts" or "contours provided by medical experts."
• Inter-expert Variability: For one specific study, the variability between two independent experts was used as a benchmark for comparison.
• Manual Treatment Plans: For the treatment plan evaluation, manual plans served as a benchmark for quantitative comparison.

No mention of pathology or outcomes data as ground truth is provided.

8. The Sample Size for the Training Set:

The document does not specify the sample size for the training set. It only mentions the training of the algorithm (e.g., "retraining or algorithm improvement").

9. How the Ground Truth for the Training Set Was Established:

The document does not explicitly describe how the ground truth for the training set was established. It only states that the device uses "deep-learning based automatic segmentation," implying that it would have been trained on curated data with established ground truth, likely also generated by medical experts, but the specifics are not detailed in this excerpt.

Ask a specific question about this device

ART-Plan's indicated target population is cancer patients for whom radiotherapy treatment has been prescribed. In this population, any patient for whom relevant modality imaging data is available.

ART-Plan is not intended to be used for patients less than 18 years of age.

The indicated users are trained medical professionals including, but not limited to, radiotherapists, radiation oncologists, medical physicists, dosimetrists and medical professionals involved in the radiation therapy process.

The indicated use environments are, but not limited to, hospitals, clinics and any health facility involved in radiation therapy.

ART-Plan is a software platform allowing users with an account to contour regions of interest on 3D images, perform multi-model registration of images, and help in the decision for the need for replanning based on contours and doses on daily images. It includes several modules:

• Home,
• Annotate,
• SmartFuse,
• AdaptBox,
• Administration,
• About

ART-Plan asks the user to work by project. It is necessary to create a "project" entity in the Patient page by associating it to a reference volume (preferably the positioning CT or the positioning MR) in order to create the contours on this volume in the Annotate module or to use this image as reference to compare it with CBCT images in the AdaptBox module. It is possible to create several projects for a given patient.

The Home module allows the user to search for a patient already present in the software's database or to import it from the structure's imaging servers or from another external source and to manage the different projects of this patient.

The Annotate module allows the user to contour regions of interest on the reference volume. It also allows generation of pseudo-CTs from MRI images. Users are able to visualize, evaluate and modify the HU values of the associated structures on the pseudo-CT, if needed. After validation, manual and automatic contours can be generated on the pseudo-CT images. Registration can also be performed using the pseudo-CT either as a target or source image. All results can be exported upon approval.

The SmartFuse module allows the user to fuse the primary volume of a project with secondary volumes.

The AdapBox module helps the user to decide if a replanning is necessary. For this purpose, the module allows the user to generate a pseudo-CT from a CBCT image, to auto-delineate regions of interest on the pseudo-CT, to compute the dose on both planning CT and pseudo-CT and then define if there is a need for replanning by comparing volume and dose metrics computed on both images and over the course of the treatment. Those metrics are defined by the user.

The Administration module allows users with specific rights to manage the platform's usage parameters. Some more restricted rights are also accessible in the drop-down menu linked to the user account through the Settings menu.

The About module allows the user to obtain information about the software and its use, as well as to contact TheraPanacea.

Here's a summary of the acceptance criteria and study details for ART-Plan (v.2.2.0), based on the provided text:

Acceptance Criteria and Reported Device Performance

Note: The provided text lists multiple acceptance criteria for different aspects of the device (auto-segmentation, synthetic-CT generation, dose engine validation), and not all reported performance metrics are explicitly linked one-for-one to a single acceptance criterion in a consolidated table within the document. The table below attempts to synthesize the acceptance criteria and the general statement of "all tests passed their respective acceptance criteria" for the reported performance.

Study Details

1. Sample size used for the test set and the data provenance:

   • Auto-segmentation (Quantitative & Non-regression): Minimum sample size of 17 patients per anatomical region (where applicable).
   • Auto-segmentation (Qualitative): Minimum sample size of 15 patients per anatomical region.
   • Auto-segmentation (US patient data performance comparison): Minimum sample size of 17 patients.
   • Contour Propagation: Minimum sample size of 15 patients.
   • Synthetic-CT Dosimetric Validation: 19 patients per supported anatomy.
   • Synthetic-CT Geometric and Anatomic Validation: 19 patients per supported anatomy.
   • Dose Engine Validation: 45 patients per supported anatomy.
   • Data Provenance: "datasets representative of the worldwide population receiving radiotherapy treatments." (No specific countries or retrospective/prospective information is given, beyond "US patient data" for one specific comparison.)

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The term "medical experts" is used generally. No specific number of experts or detailed qualifications (e.g., years of experience, subspecialty certification) are provided for the ground truth establishment.

3. Adjudication method for the test set:

   • Not explicitly stated. The ground truth seems to be established by "manual contours performed by medical experts," implying a single expert creation or a consensus if multiple experts were involved, but the method for consensus (e.g., 2+1, 3+1) is not detailed.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC comparative effectiveness study involving human readers improving with AI assistance is described. The studies focus on the performance of the AI algorithm (auto-segmentation, synthetic-CT generation, dose calculation) against expert-generated ground truth or established clinical benchmarks.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, the described studies are primarily standalone performance evaluations of the ART-Plan (v2.2.0) algorithms. This includes auto-segmentation, synthetic-CT generation, and dose engine validation, all assessed against a ground truth. While there are "qualitative evaluations" by clinicians, these assess the output of the auto-segmentation or contour propagation, not a measure of human performance with or without the device.

6. The type of ground truth used:

   • Expert Consensus: "manual contours performed by medical experts" is the primary ground truth for auto-segmentation and contour propagation assessments.
   • Clinical Dosimetric Criteria: For dose engine validation and synthetic-CT dosimetric validation, comparisons are made against established clinical dosimetric criteria and dose distributions from the planning CT.
   • Reference Imaging: For geometric/anatomic validation of synthetic-CT, comparisons seem to be made against the original CBCT using metrics like the Jacobian determinant.

7. The sample size for the training set:

   • Not specified in the provided text. The document focuses on the validation of the device (ART-Plan v2.2.0) and mentions "retraining or algorithm improvement" but does not give details about the training set size or composition.

8. How the ground truth for the training set was established:

   • Not specified in the provided text.

Ask a specific question about this device

ART-Plan is indicated for cancer patients for whom radiation treatment has been planned. It is intended to be used by trained medical professionals including, but not limited to, radiologists, radiation oncologists, dosimetrists, and medical physicists.

ART-Plan is a software application intended to display and visualize 3D multi-modal medical image data. The user may import, define, display, transform and store DICOM3.0 compliant datasets (including regions of interest structures). These images, contours and objects can subsequently be exported/distributed within the system, across computer networks and/or to radiation treatment planning systems. Supported modalities include CT, PET-CT, CBCT, 4D-CT and MR images.

ART-Plan supports Al-based contouring on CT and MR images and offers semi-automatic and manual tools for segmentation.

To help the user assess changes in image data and to obtain combined multi-modal image information, ART-Plan allows the registration of anatomical and functional images and display of fused and non-fused images to facilitate the comparison of patient image data by the user.

With ART-Plan, users are also able to generate, visualize, evaluate and modify pseudo-CT from MRI images.

The ART-Plan application consists of two kev modules: SmartFuse and Annotate, allowing the user to display and visualize 3D multi-modal medical image data. The user may process, render, review, store, display and distribute DICOM 3.0 compliant datasets within the system and/or across computer networks. Supported modalities cover static and gated CT (computerized tomography including CBCT and 4D-CT), PET (positron emission tomography) and MR (magnetic resonance).

The ART-Plan technical functionalities claimed by TheraPanacea are the following:

• Proposing automatic solutions to the user, such as an automatic delineation, automatic . multimodal image fusion, etc. towards improving standardization of processes/ performance / reducing user tedious / time consuming involvement.
• . Offering to the user a set of tools to assist semi-automatic delineation, semi-automatic reqistration towards modifying/editing manually automatically generated structures and adding/removing new/undesired structures or imposing user-provided correspondences constraints on the fusion of multimodal images.
• . Presenting to the user a set of visualization methods of the delineated structures, and registration fusion maps.
• . Saving the delineated structures / fusion results for use in the dosimetry process.
• Enabling rigid and deformable registration of patients images sets to combine information contained in different or same modalities.
• . Allowing the users to generate, visualize, evaluate and modify pseudo-CT from MRI images.

ART-Plan offers deep-learning based automatic segmentation for the following localizations:

• head and neck (on CT images) ●
• thorax/breast (for male/female and on CT images) ●
• abdomen (on CT images and MR images) ●
• pelvis male(on CT images and MR images) ●
• pelvis female (on CT images) ●
• brain (on CT images and MR images)

ART-Plan offers deep-learning based synthetic CT-generation from MR images for the following localizations:

• . pelvis male
• brain

The provided text describes the acceptance criteria and the study conducted to prove that the ART-Plan v1.10.1 device meets these criteria. Note that this submission is a Special 510(k) for modifications to an already cleared device (ART-Plan v1.10.0), focusing on the addition of 48 new structures to existing localizations and 8 bug fixes. The performance studies primarily validate these new structures.

Here's the detailed breakdown:

1. Table of Acceptance Criteria and Reported Device Performance

The device ART-Plan v1.10.1 is an AI-based contouring tool. The acceptance criteria and reported performance for the new structures are categorized into two main types: quantitative (using Dice Similarity Coefficient) and qualitative.

For Auto-segmentation Models (New Structures):

For Synthetic-CT Generation Tool (General, not specifically for new features in this submission):

2. Sample Size Used for the Test Set and Data Provenance

The document indicates that for the new structures, the sample sizes for the test set varied:

• For quantitative evaluations (Dice difference inter-expert):
  • Minimum sample size for evaluation method: 20
  • Reported sample size for most structures (e.g., Carina, Lad coronary, Left bronchia, Right cochlea): 33
  • Reported sample size for some Brain T1 (MR) structures (e.g., Anterior cerebellum, Left cochlea): 30
• For qualitative evaluations (A+B %):
  • Minimum sample size for evaluation method: 15
  • Reported sample size for most structures (e.g., Ascending aorta, Left atrium, Left main coronary artery): 20
  • Reported sample size for Left cervical lymph node IVB, Right cervical lymph node IVB: 15
  • Reported sample size for Sigmoid: 30

Data Provenance: The data used for training and testing are described as "real-world retrospective data which were initially used for treatment of cancer patients." The document mentions that the data originated from various centers, with a statistical analysis of imaging vendors in EU & USA to represent the market share. It also states that the data demographic distribution (gender, age) aligns with cancer incidence statistics in the US, UK, and globally.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

The document explicitly states that the "truthing" process "includes a mix of data created by different delineators (clinical experts) and assessment of intervariability, ground truth contours provided by the centers and validated by a second expert of the center, and qualitative evaluation and validation of the contours."

• Number of Experts: For the inter-expert variability comparison, at least two experts are implied (one for ground truth, and comparison to other delineators or a second expert validation). For qualitative evaluations, "experts" (plural) are mentioned.
• Qualifications of Experts: The document states "trained medical professionals including, but not limited to, radiation oncologists, dosimetrists, and medical physicists." The ground truth contours were "provided by the centers and validated by a second expert of the center," indicating a high level of clinical expertise.

4. Adjudication Method for the Test Set

The adjudication method is implied to be a form of expert consensus or validation. The "truthing process" includes:

• "data created by different delineators (clinical experts)"
• "assessment of intervariability"
• "ground truth contours provided by the centers and validated by a second expert of the center"
• "qualitative evaluation and validation of the contours"

This suggests that for creating the reference standard, multiple experts contributed, and a validation step often involving a second expert was performed. For comparing the AI model's performance to human experts, it was compared to "inter-expert variability" or validated qualitatively by "experts." This is not a strict "2+1" or "3+1" for every single case, but rather a process involving consensus, validation, and inter-variability analysis among clinical experts for establishing the ground truth and for evaluating the AI's performance against that truth and against other expert interpretations.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No explicit Multi-Reader Multi-Case (MRMC) comparative effectiveness study comparing human readers with AI assistance vs. without AI assistance is detailed in the provided text. The studies focus on the standalone performance of the AI model against the established ground truth and inter-expert variability.

6. Standalone Performance Study

Yes, a standalone performance study was done for the algorithm without human-in-the-loop. The tables and descriptions of acceptance criteria and results (Dice Similarity Coefficient, A+B% for qualitative evaluation) directly assess the performance of the AI-based contouring (Annotate module) in generating contours.

7. Type of Ground Truth Used

The ground truth used is primarily expert consensus/delineation. It is described as:

• "data created by different delineators (clinical experts)"
• "ground truth contours provided by the centers and validated by a second expert of the center"
• "qualitative evaluation and validation of the contours"

The contouring guidelines followed were confirmed with the data-providing centers, and the process aimed to be representative of delineation practice across centers and international guidelines.

8. Sample Size for the Training Set

• Training samples: 299,142
• Validation samples: 75,018
• Total samples: 374,160

Although the total number of samples is 374,160, the document clarifies that "The total number of patients used for training (8736) is lower than the number of samples (374160)." This indicates that one patient can contribute to multiple images and multiple structures, leading to a higher number of "samples" for training an AI model.

9. How the Ground Truth for the Training Set Was Established

The ground truth for the training set was established through "real-world retrospective data," where contours were generated by clinical experts. The process included:

• Contouring guidelines confirmed with data-providing centers.
• A mix of data created by different delineators (clinical experts).
• Ground truth contours provided by the centers and validated by a second expert of the center.
• Qualitative evaluation and validation of the contours to ensure representativeness of delineation practice and adherence to international guidelines.

This rigorous process aimed to account for expert annotation variability and ensure the training data was clinically relevant and accurate.

Ask a specific question about this device