(105 days)
ART-Plan is indicated for cancer patients for whom radiation treatment has been planned. It is intended to be used by trained medical professionals including, but not limited to, radiologists, radiation oncologists, dosimetrists, and medical physicists.
ART-Plan is a software application intended to display and visualize 3D multi-modal medical image data. The user may import, define, display, transform and store DICOM3.0 compliant datasets (including regions of interest structures). These images, contours and objects can subsequently be exported/distributed within the system, across computer networks and/or to radiation treatment planning systems. Supported modalities include CT, PET-CT, CBCT, 4D-CT and MR images.
ART-Plan supports Al-based contouring on CT and MR images and offers semi-automatic and manual tools for segmentation.
To help the user assess changes in image data and to obtain combined multi-modal image information, ART-Plan allows the registration of anatomical and functional images and display of fused and non-fused images to facilitate the comparison of patient image data by the user.
With ART-Plan, users are also able to generate, visualize, evaluate and modify pseudo-CT from MRI images.
The ART-Plan application consists of two kev modules: SmartFuse and Annotate, allowing the user to display and visualize 3D multi-modal medical image data. The user may process, render, review, store, display and distribute DICOM 3.0 compliant datasets within the system and/or across computer networks. Supported modalities cover static and gated CT (computerized tomography including CBCT and 4D-CT), PET (positron emission tomography) and MR (magnetic resonance).
The ART-Plan technical functionalities claimed by TheraPanacea are the following:
- Proposing automatic solutions to the user, such as an automatic delineation, automatic . multimodal image fusion, etc. towards improving standardization of processes/ performance / reducing user tedious / time consuming involvement.
- . Offering to the user a set of tools to assist semi-automatic delineation, semi-automatic reqistration towards modifying/editing manually automatically generated structures and adding/removing new/undesired structures or imposing user-provided correspondences constraints on the fusion of multimodal images.
- . Presenting to the user a set of visualization methods of the delineated structures, and registration fusion maps.
- . Saving the delineated structures / fusion results for use in the dosimetry process.
- Enabling rigid and deformable registration of patients images sets to combine information contained in different or same modalities.
- . Allowing the users to generate, visualize, evaluate and modify pseudo-CT from MRI images.
ART-Plan offers deep-learning based automatic segmentation for the following localizations:
- head and neck (on CT images) ●
- thorax/breast (for male/female and on CT images) ●
- abdomen (on CT images and MR images) ●
- pelvis male(on CT images and MR images) ●
- pelvis female (on CT images) ●
- brain (on CT images and MR images)
ART-Plan offers deep-learning based synthetic CT-generation from MR images for the following localizations:
- . pelvis male
- brain
The provided text describes the acceptance criteria and the study conducted to prove that the ART-Plan v1.10.1 device meets these criteria. Note that this submission is a Special 510(k) for modifications to an already cleared device (ART-Plan v1.10.0), focusing on the addition of 48 new structures to existing localizations and 8 bug fixes. The performance studies primarily validate these new structures.
Here's the detailed breakdown:
1. Table of Acceptance Criteria and Reported Device Performance
The device ART-Plan v1.10.1 is an AI-based contouring tool. The acceptance criteria and reported performance for the new structures are categorized into two main types: quantitative (using Dice Similarity Coefficient) and qualitative.
For Auto-segmentation Models (New Structures):
| Acceptance Criteria Type | Acceptance Criteria | Reported Device Performance (Examples from Table 4) | Pass/Fail |
|---|---|---|---|
| Quantitative | a) DSC (mean) ≥ 0.8 (AAPM standard) | (Not explicitly shown for new structures, but implied passed) | Pass |
| b) DSC (mean) ≥ 0.54 OR DSC (mean) ≥ mean (DSC inter-expert) + 5% | Carina: DICE diff inter-expert = 6.58% | Pass | |
| Lad coronary: DICE diff inter-expert = 15.56% | Pass | ||
| Left bronchia: DICE diff inter-expert = 14.75% | Pass | ||
| Right cochlea: DICE diff inter-expert = 29.22% | Pass | ||
| Qualitative | c) A+B % ≥ 85% (clinically acceptable without modifications or with minor corrections) | Ascending aorta: A+B = 100% | Pass |
| Left atrium: A+B = 100% | Pass | ||
| Left main coronary artery: A+B = 93% | Pass | ||
| Sigmoid: A+B = 100% | Pass |
For Synthetic-CT Generation Tool (General, not specifically for new features in this submission):
| Acceptance Criteria Type | Acceptance Criteria | Reported Device Performance | Pass/Fail |
|---|---|---|---|
| Quantitative | a) A median 2%/2mm gamma passing criteria of ≥95% | (Not explicitly shown in this document, but implied passed for prior clearance) | Pass |
| b) A median 3%/3mm gamma passing criteria of ≥99.0% | (Not explicitly shown in this document, but implied passed for prior clearance) | Pass | |
| c) A mean dose deviation (pseudo-CT compared to standard CT) of ≤2% in ≥88% of patients | (Not explicitly shown in this document, but implied passed for prior clearance) | Pass |
2. Sample Size Used for the Test Set and Data Provenance
The document indicates that for the new structures, the sample sizes for the test set varied:
- For quantitative evaluations (Dice difference inter-expert):
- Minimum sample size for evaluation method: 20
- Reported sample size for most structures (e.g., Carina, Lad coronary, Left bronchia, Right cochlea): 33
- Reported sample size for some Brain T1 (MR) structures (e.g., Anterior cerebellum, Left cochlea): 30
- For qualitative evaluations (A+B %):
- Minimum sample size for evaluation method: 15
- Reported sample size for most structures (e.g., Ascending aorta, Left atrium, Left main coronary artery): 20
- Reported sample size for Left cervical lymph node IVB, Right cervical lymph node IVB: 15
- Reported sample size for Sigmoid: 30
Data Provenance: The data used for training and testing are described as "real-world retrospective data which were initially used for treatment of cancer patients." The document mentions that the data originated from various centers, with a statistical analysis of imaging vendors in EU & USA to represent the market share. It also states that the data demographic distribution (gender, age) aligns with cancer incidence statistics in the US, UK, and globally.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document explicitly states that the "truthing" process "includes a mix of data created by different delineators (clinical experts) and assessment of intervariability, ground truth contours provided by the centers and validated by a second expert of the center, and qualitative evaluation and validation of the contours."
- Number of Experts: For the inter-expert variability comparison, at least two experts are implied (one for ground truth, and comparison to other delineators or a second expert validation). For qualitative evaluations, "experts" (plural) are mentioned.
- Qualifications of Experts: The document states "trained medical professionals including, but not limited to, radiation oncologists, dosimetrists, and medical physicists." The ground truth contours were "provided by the centers and validated by a second expert of the center," indicating a high level of clinical expertise.
4. Adjudication Method for the Test Set
The adjudication method is implied to be a form of expert consensus or validation. The "truthing process" includes:
- "data created by different delineators (clinical experts)"
- "assessment of intervariability"
- "ground truth contours provided by the centers and validated by a second expert of the center"
- "qualitative evaluation and validation of the contours"
This suggests that for creating the reference standard, multiple experts contributed, and a validation step often involving a second expert was performed. For comparing the AI model's performance to human experts, it was compared to "inter-expert variability" or validated qualitatively by "experts." This is not a strict "2+1" or "3+1" for every single case, but rather a process involving consensus, validation, and inter-variability analysis among clinical experts for establishing the ground truth and for evaluating the AI's performance against that truth and against other expert interpretations.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No explicit Multi-Reader Multi-Case (MRMC) comparative effectiveness study comparing human readers with AI assistance vs. without AI assistance is detailed in the provided text. The studies focus on the standalone performance of the AI model against the established ground truth and inter-expert variability.
6. Standalone Performance Study
Yes, a standalone performance study was done for the algorithm without human-in-the-loop. The tables and descriptions of acceptance criteria and results (Dice Similarity Coefficient, A+B% for qualitative evaluation) directly assess the performance of the AI-based contouring (Annotate module) in generating contours.
7. Type of Ground Truth Used
The ground truth used is primarily expert consensus/delineation. It is described as:
- "data created by different delineators (clinical experts)"
- "ground truth contours provided by the centers and validated by a second expert of the center"
- "qualitative evaluation and validation of the contours"
The contouring guidelines followed were confirmed with the data-providing centers, and the process aimed to be representative of delineation practice across centers and international guidelines.
8. Sample Size for the Training Set
- Training samples: 299,142
- Validation samples: 75,018
- Total samples: 374,160
Although the total number of samples is 374,160, the document clarifies that "The total number of patients used for training (8736) is lower than the number of samples (374160)." This indicates that one patient can contribute to multiple images and multiple structures, leading to a higher number of "samples" for training an AI model.
9. How the Ground Truth for the Training Set Was Established
The ground truth for the training set was established through "real-world retrospective data," where contours were generated by clinical experts. The process included:
- Contouring guidelines confirmed with data-providing centers.
- A mix of data created by different delineators (clinical experts).
- Ground truth contours provided by the centers and validated by a second expert of the center.
- Qualitative evaluation and validation of the contours to ensure representativeness of delineation practice and adherence to international guidelines.
This rigorous process aimed to account for expert annotation variability and ensure the training data was clinically relevant and accurate.
{0}------------------------------------------------
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health and Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the FDA logo is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
Therapanacea SAS % Catherine Martineau-Huynh COO 7 bis boulevard Bourdon Paris, 75004 FRANCE
April 19, 2023
Re: K230023/S001 Trade/Device Name: ART-Plan Regulation Number: 21 CFR 892.2050 Regulation Name: Medical Image Management And Processing System Regulatory Class: Class II Product Code: QKB, MUJ, LLZ Dated: December 28, 2022 Received: January 4, 2023
Dear Catherine Martineau-Huynh:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801 and Part 809); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see
{1}------------------------------------------------
https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Image /page/1/Picture/5 description: The image shows a digital signature. The signature is from Lora D. Weidner - S. The date of the signature is 2023.04.19. The time of the signature is 18:00:58-04'00'.
Lora D. Weidner -S 18:00:58 -04'00'
Lora D. Weidner, Ph.D. Assistant Director Radiation Therapy Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K230023
Device Name ART-Plan
Indications for Use (Describe)
ART-Plan is indicated for cancer patients for whom radiation treatment has been planned. It is intended to be used by trained medical professionals including, but not limited to, radiation oncologists, dosimetrists, and medical physicists.
ART-Plan is a software application intended to display and visualize 3D multi-modal medical image data. The user may mport, define, display, transform and store DICOM3.0 compliant datasets (including regions of interest structures). These images, contours and objects can subsequently be exported/distributed within the system, across computer networks and/or to radiation treatment planning systems. Supported modalities include CT, PET-CT, CBCT, 4D-CT and MR images.
ART-Plan supports AI-based contouring on CT and MR images and offers semi-automatic and manual tools for segmentation.
To help the user assess changes in image data and to obtain combined multi-modal image information, ART-Plan allows the registration of anatomical and functional images and display of fused and non-fused images to facilitate the comparison of patient image data by the user.
With ART-Plan, users are also able to generate, visualize, evaluate and modify pseudo-CT from MRI images.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| ☑ Prescription Use (Part 21 CFR 801 Subpart D) | |
| ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
510(k) Summary
This 510(k) Summary is submitted in accordance with 21 CFR Part 807, Section 807.92.
This summary of 510(k) safety and effectiveness information is being submitted in accordance with the requirement of 21 CFR 807.92
| Submitter Information: | TheraPanacea SAS |
|---|---|
| Name: | 7 bis boulevard Bourdon 75004 Paris |
| Address: | 3019834893 |
| Establishment Registration Number: | 10082087 |
| Owner/Operator Number: | +33 9 62 52 78 19 |
| Phone: | Catherine Martineau-Huynh |
| Contact: | c.huynh@therapanacea.eu |
| E-mail: | 28th of December 2022 |
| Date of Summary: |
Device Information:
Below summarises the Device Classification information regarding the ART-Plan v1.10.1.
| Device Proprietary Name | NA |
|---|---|
| Common Name: | ART-Plan |
| Trade Name: | ART-Plan |
| Product Code(s): | NA |
Primary Product Code
| RegulationNumber | Device | Device Class | ProductCode | ClassificationPanel |
|---|---|---|---|---|
| 892.2050 | Medical imagemanagement andprocessing system | Class II | QKB | Radiology |
Secondary Product Codes
| RegulationNumber | Device | Device Class | ProductCode | ClassificationPanel |
|---|---|---|---|---|
| 892.2050 | Medical imagemanagement andprocessing system | Class II | LLZ | Radiology |
| 892.5050 | Medicalcharged-particleradiation therapysystem | Class II | MUJ | Radiology |
{4}------------------------------------------------
Substantial Equivalence
| Manufacturer | Trade Name | ProductCode | Regulation | 510(k) Number |
|---|---|---|---|---|
| TheraPanaceaSAS | ART-Plan | QKB | 892.2050 | K220813 |
Submission Description
This Special 510(k) covers a modification to add 48 new structures to existing localizations and 8 bug fixes to ART-Plan v1.10.0, as cleared in 510(k) (K220813).
There are no significant changes presented to the other software components previously cleared in K220813 i.e., no change to the other modules such as Smartfuse, Home, Administration and to the other features such as generation of synthetic CT from MR.
Well-established methods described in the previously 510(k)-cleared ART-Plan v1.10.0, have been used to evaluate the change is provided in a summary in this submission.
This Special 510(k) presents the addition of 48 new structures to existing localizations (Annotate module). This modification extends the use of Annotate to other radiotherapy protocols, such as the SBRT for lung. It also includes 8 bug fixes.
Device Description
General Description .
The ART-Plan application consists of two kev modules: SmartFuse and Annotate, allowing the user to display and visualize 3D multi-modal medical image data. The user may process, render, review, store, display and distribute DICOM 3.0 compliant datasets within the system and/or across computer networks. Supported modalities cover static and gated CT (computerized tomography including CBCT and 4D-CT), PET (positron emission tomography) and MR (magnetic resonance).
The ART-Plan technical functionalities claimed by TheraPanacea are the following:
- Proposing automatic solutions to the user, such as an automatic delineation, automatic . multimodal image fusion, etc. towards improving standardization of processes/ performance / reducing user tedious / time consuming involvement.
- . Offering to the user a set of tools to assist semi-automatic delineation, semi-automatic reqistration towards modifying/editing manually automatically generated structures and adding/removing new/undesired structures or imposing user-provided correspondences constraints on the fusion of multimodal images.
- . Presenting to the user a set of visualization methods of the delineated structures, and registration fusion maps.
- . Saving the delineated structures / fusion results for use in the dosimetry process.
- Enabling rigid and deformable registration of patients images sets to combine information contained in different or same modalities.
- . Allowing the users to generate, visualize, evaluate and modify pseudo-CT from MRI images.
ART-Plan offers deep-learning based automatic segmentation for the following localizations:
{5}------------------------------------------------
- head and neck (on CT images) ●
- thorax/breast (for male/female and on CT images) ●
- abdomen (on CT images and MR images) ●
- pelvis male(on CT images and MR images) ●
- pelvis female (on CT images) ●
- brain (on CT images and MR images)
ART-Plan offers deep-learning based synthetic CT-generation from MR images for the following localizations:
- . pelvis male
- brain
Intended/ Indications for Use
Intended use :
ART-Plan is a software for multi-modal visualization, contouring and processing of 3D images of cancer patients for whom radiotherapy treatment has been prescribed.
It allows the user to view, create and modify contours for the regions of interest. It also allows to generate automatically, and based on medical practices, the contours for the organs at risk and healthy lymph nodes and to register combinations of anatomical and functional images. Contours and images require verifications, potential modifications, and subsequently the validation of a trained user with professional qualifications in anatomy and radiotherapy before their export to a Treatment Planning System.
ART-Plan offers the following visualization, contouring and manipulation tools to aid in the preparation of radiotherapy treatment:
- Multi-modal visualization and rigid- and deformable registration of anatomical and ● functional images such as CT, MR, PET-CT, 4D-CT and CBCT
- Display of fused and non-fused images to facilitate the comparison and delineation of ● image data by the user
- Manual modification and semi-automatic generation of contours for the regions of ● interest
- Automatic generation of contours for organs at risk and healthy lymph nodes, based . on medical practices, on medical images such as CT and MR images.
- Generation of pseudo-CT for supported anatomies
The device is intended to be used in a radiation therapy clinical setting, by trained professionals only.
Indications for use:
ART-Plan is indicated for cancer patients for whom radiation treatment has been planned. It is intended to be used by trained medical professionals including, but not limited to, radiologists, radiation oncologists, dosimetrists, and medical physicists.
ART-Plan is a software application intended to display and visualize 3D multi-modal medical image data. The user may import, define, display, transform and store DICOM3.0 compliant datasets (including regions of interest structures). These images, contours and objects can subsequently be exported/distributed within the system, across computer networks and/or to radiation treatment planning systems. Supported modalities include CT, PET-CT, CBCT, 4D-CT and MR images.
{6}------------------------------------------------
ART-Plan supports Al-based contouring on CT and MR images and offers semi-automatic and manual tools for segmentation.
To help the user assess changes in image data and to obtain combined multi-modal image information, ART-Plan allows the registration of anatomical and functional images and display of fused and non-fused images to facilitate the comparison of patient image data by the user.
With ART-Plan, users are also able to generate, visualize, evaluate and modify pseudo-CT from MRI images.
Comparison with the Predicate and Previously Cleared Device
The candidate device TheraPanacea SAS ART-Plan 1.10.1 is substantially equivalent to the predicate, K220813, the TheraPanacea SAS ART-Plan 1.10.0 and a comparison of the key characteristics is summarised in Table 1.
| Characteristic | ART-Plan v1.10.1 with Modification | ART-Plan v1.10.0 K220813(Predicate) | Equivalence |
|---|---|---|---|
| Device Name | ART-Plan v1.10.1 | ART-Plan v1.10.0 | Equivalent |
| Manufacturer | TheraPanacea SAS | TheraPanacea SAS | Equivalent |
| Device Classification | II | II | Equivalent |
| Primary Product Code | QKB | QKB | Equivalent |
| Secondary ProductCode | LLZ, MUJ | LLZ, MUJ | Equivalent |
| Indications for Use | ART-Plan is indicated for cancer patients for whom radiation treatment has been planned. It is intended to be used by trained medical professionals including, but not limited to, radiologists, radiation oncologists, dosimetrists, and medical physicists.ART-Plan is a software application intended to display and visualize 3D multi-modal medical image data. The user may import, define, display, transform and store DICOM 3.0 compliant datasets (including regions of interest structures). These images, contours and objects can subsequently be exported/distributed within the system, across computer networks and/or to radiation treatment planning systems. Supported modalities include CT, PET-CT, CBCT, 4D-CT and MR images.ART-Plan supports Al-based contouring on CT and MR images and offers semi-automatic and manual tools for segmentation.To help the user assess changes in image data and to obtain combined multi-modal image information, ART-Plan allows the registration of | ART-Plan is indicated for cancer patients for whom radiation treatment has been planned. It is intended to be used by trained medical professionals including, but not limited to, radiologists, radiation oncologists, dosimetrists, and medical physicists.ART-Plan is a software application intended to display and visualize 3D multi-modal medical image data. The user may import, define, display, transform and store DICOM 3.0 compliant datasets (including regions of interest structures). These images, contours and objects can subsequently be exported/distributed within the system, across computer networks and/or to radiation treatment planning systems. Supported modalities include CT, PET-CT, CBCT, 4D-CT and MR images.ART-Plan supports Al-based contouring on CT and MR images and offers semi-automatic and manual tools for segmentation.To help the user assess changes in image data and to obtain combined multi-modal image information, ART-Plan allows the registration of | Equivalent |
| Characteristic | ART-Plan v1.10.1 with Modification | ART-Plan v1.10.0 K220813(Predicate) | Equivalence |
| anatomical and functional images anddisplay of fused and non-fused imagesto facilitate the comparison of patientimage data by the user. | anatomical and functional images anddisplay of fused and non-fused imagesto facilitate the comparison of patientimage data by the user. | ||
| With ART-Plan, users are also able togenerate, visualize, evaluate andmodify pseudo-CT from MRI images | With ART-Plan, users are also able togenerate, visualize, evaluate andmodify pseudo-CT from MRI images | ||
| Intendeduser/Location | It is intended to be used by trainedmedical professionals including, but notlimited to, radiologists, radiationoncologists, dosimetrists, and medicalphysicists / Hospitals | It is intended to be used by trainedmedical professionals including, but notlimited to, radiologists, radiationoncologists, dosimetrists, and medicalphysicists / Hospitals | Equivalent |
| Segmentation features(Annotate module) | Automatically delineates OARs andhealthy lymph nodesDeep learning algorithm.Automatic segmentation includes thefollowing localizations:* head and neck (on CT images)* thorax/breast (for male/female and onCT images)* abdomen (on CT images and MRimages)* pelvis male(on CT images and MRimages)* pelvis female (on CT images)* brain (on CT images and MR images) | Automatically delineates OARs andhealthy lymph nodesDeep learning algorithm.Automatic segmentation includes thefollowing localizations:* head and neck (on CT images)* thorax/breast (for male/female and onCT images)* abdomen (on CT images and MRimages)* pelvis male(on CT images and MRimages)* pelvis female (on CT images)* brain (on CT images and MR images) | Equivalent -The candidatedevice andpredicate arecapable ofautomaticallycontouring theorgan-at-risk(OAR) andhealthy lymphnodes usingAl (deeplearning)algorithm.The candidatedeviceincludes 48additionalstructures tothe alreadyexistinglocalizations |
| Bugs | Correction of 8 bugs | NA | EquivalentThe bug fixesintroduced inthe candidatedevice do notaffect thesafety orperformanceof thepredicatedevice |
{7}------------------------------------------------
Table 1: Comparison of characteristics between the Modified System and the Predicate System.
{8}------------------------------------------------
In Table 2, structures included in ART-Plan v1.10.1 are presented.
| Head & Neck (CT) - 47 structures | ||||||
|---|---|---|---|---|---|---|
| Brainstem | Cerebellum | Chiasma | Encephalon | Esophagus | Hypophyse | Larynx |
| Left Brachialplexus | Left cervical lymphnode IB | Left cervicallymph node II | Left cervicallymph node III | Left cervicallymph node IVA | Left cervicallymph node IVB | Left cervicallymph node V |
| Left cervical lymphnode VIIA | Left cervical lymphnode VIIB | Left cochlea | Left eye | Left eye lens | Left opticalnerve | Left parotid |
| Left submandible | Lefttemporomandibularr joints | Lips | Mandible | Medullarcanal | Mouth | Right brachialplexus |
| Right cervicallymph node IB | Right cervicallymph node II | Rightcervicallymph nodeIII | Right cervicallymph nodeIVA | Right cervicallymph nodeIVB | Right cervicallymph node V | Right cervicallymph nodeVIIA |
| Right cervicallymph node VIIB | Right cochlea | Right eye | Right eye lens | Right opticalnerve | Right parotid | Rightsubmandible |
| Righttemporomandibularr joints | Spinal Cord | Thyroid | Trachea | Externalcontour | ||
| Thorax / Breast (CT) - 30 structures | ||||||
| Esophagus | Heart | Larynx | Left brachialplexus | Left breast | Left humeralhead | Left IMC(internalmammarychain) lymphnode |
| Left interpectorallymph node | Left lung | Left lymphnode L1 | Left lymphnode L2 | Left lymphnode L3 | Leftsupraclavicularr lymph nodes | Liver |
| Medullar canal | Right brachialplexus | Right breast | Right humeralhead | Right IMC(internalmammarychain) lymphnode | Rightinterpectorallymph node | Right lung |
| Right lymph nodeL1 | Right lymph nodeL2 | Right lymphnode L3 | Rightsupraclavicularr lymph nodes | Spinal cord | Thoracic aorta | Thyroid |
| Trachea | External Contour | |||||
| Pelvis Male (CT) - 19 structures | ||||||
| Anal canal | Bladder | Bowel bag | CTVnprostate | Left femoralhead | Left iliac | Left kidney |
| Liver | Medullar canal | Penile bulb | Prostate | Rectum | Rightfemoral head | Right iliac |
| Right kidney | Seminal vesicle | Sigmoid | Spinal cord | Externalcontour | ||
| Pelvis Female (CT) - 25 structures | ||||||
| Anal canal | Bladder | Bowel bag | Commoniliac gynecolymph node | CTVtgyneco | Left femoralhead | Left iliac |
| Left iliac gynecolymph node | Left inguinalgyneco lymphnode | Left kidney | Liver | Lomboaorticc lymphnode | Medullarcanal | Parametrium |
| Presacralgyneco lymphnode | Rectum | Rightfemoralhead | Right iliac | Rightinguinalgynecolymph node | Right kidney | |
| Sigmoid | Spinal cord | Vagina | Externalcontour | |||
| Heart substructures (part of thorax / breast) (CT) - 13 structures | ||||||
| Ascending aorta | Coronary sinus | Left atrium | Left maincoronaryartery | Leftventricle | Left ventricleanterior | Left ventricleapical |
| Left ventricleinferior | Left ventriclelateral | Leftventricleseptal | Right atrium | Rightventricle | Vena cavasuperior | |
| SBRT lung (part of thorax / breast) (CT) - 14 structures | ||||||
| Bronchial tree | Carina | Leftanteriordescendingaorta | Left bronchia | Leftbronchus | Left chestwall | Pericardium |
| Pulmonaryarteries | Right bronchia | Rightbronchus | Right chestwall | Spleen | Stomach | Vena cavainferior |
| Brain T1 (MR) - 28 structures | ||||||
| Anteriorcerebellum | Chiasma | Encephalon | Hypophyse | Left cochlea | Left cornea | Left eye lens |
| Lefthippocampus | Lefthypothalamus | Leftlacrimalgland | Left opticalnerve | Left retina | Leftvestibularsemicircularcanals(VSCC) | Medullaoblongata |
| Midbrain | Pons | Posteriorcerebellum | Rightcochlea | Rightcornea | Right eyelens | RighthippocampuS |
| Righthypothalamus | Right lacrimalgland | Rightopticalnerve | Right retina | Rightversibularsemicircularcanals(VSCC) | Spinal cord | Externalcontour |
| Pelvis T2 (male) (MR) - 12 structures | ||||||
| Anal canal | Bladder | Leftfemoralhead | Left pelvis | Penile bulb | Prostate | Rectum |
| Right femoralhead | Right pelvis | Sacrum | Seminalvesicle | Externalcontour | ||
| Pelvis TF (male) (MR) - 19 structures | ||||||
| Anal canal | Aorta | Bladder | Duodenum | Inferiorvena cava | Large bowel | Left femoralhead |
| Left kidney | Liver | Pancreas | Penile bulb | Prostate | Rectum | Rightfemoral head |
| Right kidney | Seminal vesicle | Sigmoid | Stomach | Externalcontour |
{9}------------------------------------------------
Table 2: Structures included in ART-Plan v1.10.1
{10}------------------------------------------------
Technological Characteristics
The proposed modification to the Annotate module on the TheraPanacea ART-Plan v1.10.1 has identical indications for use, operating principles, performance, and technical specification as the predicate device, the TheraPanacea SAS ART-Plan 1.10.0.
The proposed modification of the addition of 48 new structures to the existing localizations and the introduction of 8 bug fixes enables further help in the management of radiotherapy planning. Equivalence between both systems has been shown through the thorough performance testing performed.
Summary of Non-Clinical Tests (Performance data)
The TheraPanacea ART-Plan V1.10.1 was tested to ensure performance of the system, to verify and validate the product design and to characterise the performance and safety of TheraPanacea's ART-Plan v1.10.1.
The performance of the Annotate modification is identical to the predicate previously cleared device in terms of technical specification and safety. The primary difference between the predicate and the candidate devices is the addition of 48 new structures to existing localizations (Annotate module). This modification extends the use of Annotate to other radiotherapy protocols, such as the SBRT for lung,
All changes were verified and validated according to TheraPanacea SAS internal design control process and in accordance with special controls for software systems.
This is demonstrated through the extensive testing carried out on the system with the modification, which passes all performance and verification tests that follow the same protocol and acceptance criteria as the ones submitted to the FDA under the clearance of the predicate device (K220813). It also demonstrated that the proposed modification performed according to its specification and has met the technological and performance criteria which have not changed from the predicate device.
Information about our training dataset:
A method generalizes well if the observed performance on training and validation sets remains stable. In the case of strong presence of expert's annotation variability (that is not necessarily because of erroneous annotations but because image quality/organ visibility can be interpreted differently amonq experts), a method that can demonstrate similar performance with respect to a given metric on training, validation and later on testing is considered to generalize well.
In that process, both the loss function being optimization procedure (stochastic gradient descent) and the dice metric which is the main proxy of segmentation quality, are monitored over the train and validation sets. If the loss is non-increasing on the validation set and if the dice metrics follow similar in value trends in both the validation and training sets, it is considered that the model being trained does not overfit, and hence should generalize well, at least on input domains similar to ones in those sets.
{11}------------------------------------------------
On the contrary, overfitting can be detected whenever the training loss keeps decreasing while the validation loss after a while increases. This means that the model is focusing on features that are specific to the training data and not present in the validation data. This implies that the capacity of the model to generalize is poor. In that respect, the independence of the train/validation/test sets is fundamental.
We consider that a model is a good candidate for production when the following conditions are met: 1) the loss and dices have reached a plateau on the validation set, 2) there is no overfitting, i.e. training and validation curves are similar and 3) the level of the dice for the different organs are as good or above the clinical expectations according to well defined performance criteria.
The learning curves of organs may be different depending on the sizes and shapes (difficulties) of structures (organs). Thus, the range of testing scores, Dice Similarity Coefficient (DSC), may vary. It is important to remember that smaller orqans might have smaller DSC and yet be still clinically relevant and acceptable, as the DSC is a relative metric that is heavily dependent on the volume of the organ. This is due to the fact that the DSC scores are normalized from the union of organ volume between the two sets (ground truth, automatic annotations) and therefore lower DSC could correspond to clinically acceptable values for small organs, since the proposed contours might take just a few editions to make them usable for planning, whilst still saving time from the users, i.e. that these contours would be judged "clinically acceptable after minor corrections" in a qualitative evaluation.
Learning curves can have an average DSC and loss function for each epoch (which is an iteration of training where the whole training dataset has been passed to the network) over the training set and over the validation set. Our curves show that validation and training data are very close to each other, reaching convergence after some epochs (depending on the structure), demonstrating no overfitting of the training data. Once convergence is achieved, the model is considered ready to be tested and clinically validated on a different, yet representative data set, as described in the process that has already been submitted to and cleared by the FDA.
Some limitations have been identified that correspond either to the sex or the age of patients. For instance, for the auto-segmentation model following limitations are disclosed to the user in the Instruction For Use (User Manual) based on the sex of the patient:
- The Truefisp Pelvis MRI and T2 Elekta Pelvis MRI auto-contouring models only work on male anatomy.
- -The patient sex of the patient (dicom tag (0010, 0040)) is taken into account for the auto-segmentation:
- if the tag is "F" or "M", the sex specific organs (prostate, breast, etc.) are ' contoured according
to the tag
if the taq is empty or "O": -
- if batch: no contour is delineated except external contour '
- if auto seqmentation on Annotate: only common contours to the 2 sexes are delineated
- if the tag is incorrect, the generated contours may be inappropriate -
The automatic contouring (including external contour) function may generate inappropriate
{12}------------------------------------------------
contours in the following cases:
- When the volume used is an image taken of a child –
- -When the patient has a particular anatomy.
- When the considered volume is that of a patient not positioned on his back at the time of acquisition.
- -When the value entered in the Patient Position tag (0018, 5100) is erroneous.
- -When the DICOM-CT contains an unusually high number of slices.
- When the quality of the images used as input is not satisfying enough or the resolution is low such as CBCT. Therefore, the contours produced may have a low quality.
- -When the primary volume is an MRI whose acquisition sequence is not compatible with the selected auto-contouring model.
- -When the patient is unusually positioned on the image (image not centered on the patient, head rotated on the side ... )
Only some anatomies are covered by the automatic contouring:
- Automatic contouring on CT images covers all anatomies (head, Head & Neck, thorax, breast, abdominal region and pelvis (M/F)
- -Automatic contouring on MR images covers some sequences and anatomies: Brain T1, Abdo TF (TrueFisp), Pelvis T2, Pelvis TF.
- -In order to suggest the most relevant structures to the user, a CT that does not include a chiasma but does include a liver, is not considered as Head and Neck case. In that case, no Head and Neck structures will be automatically segmented.
All the above information on the limitations of some models is included in the Instruction For Use (User Manual) which is made available to all users of the software.
Summary test statistics or other test results including acceptance criteria or ● other information supporting the appropriateness of the characterized performance:
Acceptance criteria for performance of ART-Plan modules were established using performance ranges extracted from benchmark devices and alternative technologies in the literature. For an auto segmentation model to be judged acceptable, every organ included in the model must pass at least one acceptance criterion with success across the different testings it has been submitted to. These criteria are as follows:
a) The Dice Similarity Coefficient (DSC) is equal to or superior to the acceptance criteria set by the AAPM: DSC (mean)≥ 0.8.
Or
b) The Dice Similarity Coefficient (DSC) is equal to or superior to inter-expert variability: DSC (mean)≥ 0.54 or DSC (mean) ≥ mean (DSC inter-expert) + 5% .
Or
c) The clinicians' s qualitative evaluation of the auto-segmentation is considered acceptable for clinical use without modifications (A) or with minor modifications / corrections (B) with a A+B % above or equal to 85% considering the following scale:
A: the contour is acceptable for a clinical use without any modification
B: the contour would be acceptable for clinical use after minor modifications/corrections
C: the contour requires major modifications (e.g. it would be faster for the expert to manually delineate the structure)"
For the synthetic-CT generation tool, the acceptance criteria are as follows:
Page 10 of 22
{13}------------------------------------------------
a. A median 2%/2mm qamma passing criteria of ≥95%
b. A median 3%/3mm gamma passing criteria of ≥99.0%
c. A mean dose deviation (pseudo-CT compared to standard CT) of ≤2% in ≥88% of patients
Total number of individual patients images in the reported auto segmentation . tools and independence of test data and training data
Our training, validation and test cohorts are built from real-world retrospective data which were initially used for treatment of cancer patients. For the structures of a given anatomy for a given modality (MR or CT), two non-overlapping data sets were separated: the test patients (number selected based on thorough literature review and statistical power) and the train data. We make sure that those sets are non-overlapping and further split the train cases into train and validation sets and ensure enough train cases for the machine learning models to converge and achieve good performances of the validation set.
| Sample size | % | |
|---|---|---|
| Training | 299 142 | 0.8 |
| Validation | 75 018 | 0.2 |
| Total | 274 160 | 1 |
Table 3: Distribution of samples between training and validation data sets
. Total number of cases and samples images in the reported auto segmentation results
The total number of patients used for training (8736) is lower than the number of samples (374160). This is linked to the fact that one patient can be associated with more images (e.g. CT, MR) and that each image (anatomy) has the delineation of several structures (OARs and lymph nodes) which increases the number of samples used for training and validation.
. Demographic distribution including gender, age and ethnicity
All data used for training of the models have been pseudo-anonymised by the centers providing data before transfer. Around 80% of the data used for training contain information on gender and age of the patients. In terms of gender, around 44% and 56% of our data (that contains this information) are from female and male patients. respectively. In comparison, in 2020 according to the Global Cancer Observatory, 48% and 52% of the cancer patients were female and male, respectively.
In terms of age, our data follows the same trend observed and reported in the US (SEER NIH), UK (Cancer Research UK) and worldwide (Global Cancer Observatory) for cancer incidence according to age, with more than 95% of the data coming from patients between 20 and 85 years old. Our data has a slight overrepresentation (8% points) for the ages between 54 and 60 years old, at the cost of a slight underrepresentation of patients in the age range between
{14}------------------------------------------------
20-34 (1.5% points) and above 85 (6.5% points) years old. In addition, following the general global (incl US) trend, our data also depicts a steep rise in the incidence rate from in the age group of 55-64 years old, with a median age of 63 years old (as compared to 66 years old in the US).
Although this information is not exhaustive, this analysis shows that the demographic distribution in terms of age and gender of the data used for training and validation of the models are well aligned with the incidence cancer statistics found for instance in US, UK and globally. This comes from the fact that real clinical data provided by medical facilities without any selection criteria (i.e. no discrimination or selection has been applied to the cases retrieved), leading to the demographic distribution including gender and age across the data is representative of the distribution in the clinic and thus of the cancer patient population in general.
An exception is noted for following models that are gender-dependent:
- 100% of pelvis images for male pelvis model for automatic annotation are male patients
- 100% of pelvis images for female pelvis model for automatic annotation are female patients
- 100% of breast images for the breast automatic annotation are female patients
- 100 % of pelvis images for automatic synthetic-CT generation are male patients
The pseudo-anonymized data did not include any information on the ethnicity.
In addition, automatic delineation of the device demonstrated equivalent performances between non-US and US population.
● On the "truthing" and data collection process
The contouring guidelines followed to produce the contours were confirmed with the centers which provided the data. Our truthing process includes a mix of data created by different delineators (clinical experts) and assessment of intervariability, ground truth contours provided by the centers and validated by a second expert of the center, and qualitative evaluation and validation of the contours. This process ensures that the data used for training and testing can be considered representative of the delineation practice across centers and is following international guidelines.
. On clinical subgroups, confounders and equipment details
In general, confounding factors affecting health status present in the dataset could be related to patient clinical variables such as age, gender, ethnicity, economical and educational levels. As shown in "Demographic distribution including gender, age and ethnicity", our data is representative of the demographic cancer distribution in terms of gender and age. In addition, our models when appropriate (i.e. for gender independent anatomies) are shared across gender removing any further bias and augmenting substantially training cohorts.
Variables like ethnicity, economical and educational status that could be associated with obesity are further confounding factors that could impact global patient's anatomy and introduce bias in the performance of the obtained solution. To address this aspect, we have adopted a strategy that projects a patient's specific anatomy to common, multiple, different in size, full-body female and male patient templates, allowing a direct harmonization of data resulting in potential removal of bias of anatomical diversity across ethnic, economical and educational groups. Please note that this information (ethnic group, educational/economical level, etc.) is often not available in the pseudo-anonymised data and therefore performing
{15}------------------------------------------------
statistical tests and increasing the number of operations allowing to separate correlations from causality is often unattainable.
Regarding variables associated with treatment therapeutic and treatment implementation strategies; we can imaging devices and treatment devices being potential confounding factors as differences exist among CT and MR scanners manufacturers that could potentially introduce bias. We have addressed this concern through a statistical analysis of the different imaging vendors in EU & USA towards the creation of a data training, validation and testing cohort that globally appropriately represents the market share of the different vendors allowing generalization and removing hardware specific bias. In terms of treatment implementation, it should be noted that different quidelines exist and depending on the treatment device different therapeutic constraints and guidelines are applied. This is reflected in our database since different strategies and constraints are used depending on the choice of treatment (e.g external radiotherapy vs stereotactic treatment). Our solution, due to its concept of removing bias through projection to patient template anatomies as well as due to the component-based approach that is able to aggregate training data across imaging and treatment vendors, is able to address the maximum set of constraints. Therefore, we do not introduce any bias on the type of treatment that will be delivered (supporting any type of clinically conventionally adopted treatment from manufactures such as Varian, Elekta, Accuray, GE, Siemens, ViewRay), providing direct means for customization of the constraints to be met at the clinical expert level and offering a representative coverage of all vendors in radiation oncology world-wide.
An exception is noted for following models that are vendor-, machine- or sequence-dependent:
-
MR annotation tool for pelvis and abdominal regions were trained on data from a 0.35T MR machine provided by ViewRay
-
synthetic-CT generation tool for pelvis was trained on data from a 0.35T MR machine provided by ViewRay
-
synthetic-CT generation tool and annotation tool for MR pelvis was trained on data from 1.5T Philips (Elekta) for T2 sequences, and might not work on T1-weighted images
On generalizability of the models: ●
A method generalizes well also if the observed performance on training and validation sets remains stable. In the case of strong presence of expert's annotation variability (that is not necessarily because of erroneous annotations but because image quality/organ visibility can be interpreted differently among experts), a method that can demonstrate similar performance with respect to a given metric on training, validation and later on testing is considered to generalize well.
We consider that a model is a good candidate for production when the following conditions are met: 1) the loss and dices have reached a plateau on the validation set, 2) there is no overfitting, i.e. training and validation curves are similar and 3) the level of the dice for the different organs are as qood or above the clinical expectations.
Once convergence is achieved, the model is tested and clinically validated on a different, yet representative data set, following a well-established process of validation that has already been submitted to and cleared by the FDA.
{16}------------------------------------------------
Summary of Verification and Validation Activities
| Organ | Performance testmethod/Acceptance criterion | Summary of results | Any differences toprotocol? |
|---|---|---|---|
| 1.Carina | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=6.58%Passed | Sample size: 33which is above theminimum data samplesize |
| 2.Ladcoronary | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=15.56%Passed | Sample size: 33which is above theminimum data samplesize |
| 3.Leftbronchia | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=14.75%Passed | Sample size: 33which is above theminimum data samplesize |
| 4.Leftbronchus | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=6.17%Passed | Sample size: 33which is above theminimum data samplesize |
| 5.Left chestwall | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=0%Passed | Sample size: 33which is above theminimum data samplesize |
| 6.Pericardium | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=1.06%Passed | Sample size: 33which is above theminimum data samplesize |
| 7.pulmonary | Intervariability comparison to | DICE diff | Sample size: 33 |
| arteries | experts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | inter-expert=3.61%Passed | which is above theminimum data samplesize |
| 8. Rightbronchia | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=22.64%Passed | Sample size: 33which is above theminimum data samplesize |
| 9. Rightbronchus | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=7.41%Passed | Sample size: 33which is above theminimum data samplesize |
| 10. Rightchestwall | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=-1.10%Passed | Sample size: 33which is above theminimum data samplesize |
| 11. Spleen | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=1.08%Passed | Sample size: 33which is above theminimum data samplesize |
| 12. stomach | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=2.27%Passed | Sample size: 33which is above theminimum data samplesize |
| 13. Vena cavainf | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=9.59%Passed | Sample size: 33which is above theminimum data samplesize |
| 14. Bronchialtree | This structure corresponds to aboolean of other structures: carina+ leftbronchus + rightbronchus +leftbronchia + rightbronchia whichwhich have all passed theperformance tests | ||
| 15. Ascendingaorta | Qualitative evaluation by experts | A+B=100% | Sample size: 20 |
| Criterion D | Passed | which is above theminimum data sample | |
| Min sample size for evaluationmethod: 15 | size | ||
| 16. coronarysinus | Intervariability comparison toexperts(Criterion C of performancecriteria) | diff inter-expert=3.59% | Sample size: 20 |
| Min sample size for evaluationmethod: 20 | Passed | which is the minimumdata sample size | |
| 17. Left atrium | Qualitative evaluation by experts | A+B=100% | Sample size: 20 |
| Criterion D | Passed | which is above theminimum data sample | |
| Min sample size for evaluationmethod: 15 | size | ||
| 18. Left maincoronaryartery | Qualitative evaluation by experts | A+B=93% | Sample size: 20 |
| Criterion D | Passed | which is above theminimum data sample | |
| Min sample size for evaluationmethod: 15 | size | ||
| 19. Leftventricle | Qualitative evaluation by experts | A+B=100% | Sample size: 20 |
| Criterion D | Passed | which is above theminimum data sample | |
| Min sample size for evaluationmethod: 15 | size | ||
| 20. Leftventricleanterior | Qualitative evaluation by experts | A+B=100% | Sample size: 20 |
| Criterion D | Passed | which is above theminimum data sample | |
| Min sample size for evaluationmethod: 15 | size | ||
| 21. Leftventricleapical | Qualitative evaluation by experts | A+B=100% | Sample size: 20 |
| Criterion D | Passed | which is above theminimum data sample | |
| Min sample size for evaluationmethod: 15 | size | ||
| 22. Leftventricleinferior | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%Passed | Sample size: 20which is above theminimum data samplesize |
| 23. Leftventriclelateral | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%Passed | Sample size: 20which is above theminimum data samplesize |
| 24. Leftventricleseptal | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%Passed | Sample size: 20which is above theminimum data samplesize |
| 25. Right atrium | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%passed | Sample size: 20which is above theminimum data samplesize |
| 26. Rightventricle | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%Passed | Sample size: 20which is above theminimum data samplesize |
| 27. Vena cavasup | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%Passed | Sample size: 20which is above theminimum data samplesize |
| 28. Left cervicallymph nodeIVB | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B = 96.67%Passed | Sample size: 15which is above theminimum data samplesize |
| 29. Rightcervicallymph nodeIVB | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B = 96.67%Passed | Sample size: 15which is above theminimum data samplesize |
| 30. Anteriorcerebellum | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=6.47%Passed | Sample size: 30which is above theminimum data samplesize |
| 31. Left cochlea | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=19.96%Passed | Sample size: 30which is above theminimum data samplesize |
| 32. Left cornea | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=7.93%Passed | Sample size: 30which is above theminimum data samplesize |
| 33. Lefthypothalamus | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=4.19%Passed | Sample size: 30which is above theminimum data samplesize |
| 34. Left lacrimalgland | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=4.76%Passed | Sample size: 30which is above theminimum data samplesize |
| 35. Left retina | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=12.26%Passed | Sample size: 30which is above theminimum data samplesize |
| 36. Left vscc | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=-1.20%Passed | Sample size: 30which is above theminimum data samplesize |
| 37. Medullaoblangata | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=3.25%Passed | Sample size: 30which is above theminimum data samplesize |
| 38. Midbrain | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=5.78Passed | Sample size: 30which is above theminimum data samplesize |
| 39. Pons | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=3.39%Passed | Sample size: 30which is above theminimum data samplesize |
| 40. Posteriorcerebellum | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=2.07%Passed | Sample size: 30which is above theminimum data samplesize |
| 41. Rightcochlea | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=29.22%Passed | Sample size: 30which is above theminimum data samplesize |
| 42. Rightcornea | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=4.66%Passed | Sample size: 30which is above theminimum data samplesize |
| 43. Righthypothalamus | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=3.32%Passed | Sample size: 30which is above theminimum data samplesize |
| 44. Rightlacrimalgland | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=4.23%Passed | Sample size: 30which is above theminimum data samplesize |
| 45. Right retina | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=10.03%Passed | Sample size: 30which is above theminimum data samplesize |
| 46. Right vscc | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=3.08%Passed | Sample size: 30which is above theminimum data samplesize |
| 47. Spinal Cord | Intervariability comparison toexperts(Criterion C of performancecriteria)Min sample size for evaluationmethod: 20 | DICE diffinter-expert=13.01%Passed | Sample size: 30which is above theminimum data samplesize |
| 48. Sigmoid | Qualitative evaluation by expertsCriterion DMin sample size for evaluationmethod: 15 | A+B=100%Passed | Sample size: 30which is above theminimum data samplesize |
{17}------------------------------------------------
{18}------------------------------------------------
{19}------------------------------------------------
{20}------------------------------------------------
{21}------------------------------------------------
{22}------------------------------------------------
Table 4: Summary of Performance Test Results for the Annotate Module of ART-Plan
{23}------------------------------------------------
Summary of Non-Clinical Tests (Performance data)
The TheraPanacea ART-Plan V1.10.1 was tested to ensure performance of the system, to verify and validate the product design and to characterise the performance and safety of the TheraPanacea ART-Plan v1.10.1.
The performance of the Annotate modification is identical to the predicate previously cleared device in terms of technical specification and safety. The primary difference between the predicate and the candidate devices is the addition of 48 new structures to existing localizations (Annotate module). This modification extends the use of Annotate to other radiotherapy protocols, such as the SBRT for lung.
All changes were verified and validated according to TheraPanacea SAS internal design control process and in accordance with special controls for software systems.
This is demonstrated through the extensive testing carried out on the system with the modification, which passes all performance and verification tests that follow the same protocol and acceptance criteria as the ones submitted to the FDA under the clearance of the predicate device (K220813). It also demonstrated that the proposed modification performed according to its specification and has met the technological and performance criteria which has not changed from the predicate device.
As part of a "standard" lifecycle of a software, bugs were fixed (8 bug fixes). System verification and validation testing were performed to verify the software of the TheraPanacea ART-Plan v1.10.1 after the bug fixes using the same verification tests and acceptance criteria as the ones submitted to the FDA under the clearance of the predicate device (K220813). Related documents are available on request.
Table 5 summarises the non-clinical tests (performance tests) completed by TheraPanacea to validate the organs added in v1.10.1.
| Test Name | Test Description | Results |
|---|---|---|
| Study Protocol andReportAnnotatePerformancesSummary (v1.10.1) | The purpose of this document is to describe the testingprotocols and testing results for validating the performance ofthe Annotate module. The performance study of the ART-Planmodule, Annotate, evaluates the precision of the contoursdone by the software either i) against the one done by humanexperts through a direct comparison or ii) by a qualitativevalidation done by human experts.The objective of the tests is to demonstrate that theauto-segmentation algorithms (CT and MR) of the moduleAnnotate pass at least one acceptance criterion. Thisdocument includes test procedures, documentation,references, specifications, and acceptance criteria. Thisdocument is updated to take into account modifications madein ART-Plan v1.10.1 with the addition of heart substructuresand SBRT in the CT automatic segmentation. | Passed |
| Study Protocol andReportquantitativevalidation of Annotate | This test demonstrates that the Annotate provides clinicallyacceptable (compared to inter-expert variability) for SBRTstructures. All organs that have passed the acceptance | Passed |
{24}------------------------------------------------
| Test Name | Test Description | Results |
|---|---|---|
| in ART-Plan v1.10.1for SBRT CT. | criterion of reaching a percentage of a DSC(mean)≥ 0.8 or DSC(mean)≥0.54) or DSC(mean)≥mean(DSC inter-expert)+5% relative error (quantitative evaluation) have been released in v.1.10.1. | |
| Study Protocol and Report Qualitative Validation of Annotate in ART-Plan V1.10.1 for Heart substructures CT | This test demonstrates that the module Annotate provides acceptable contours for the organs evaluated on CT images of patients. All organs that have passed the acceptance criterion of reaching a percentage of at least 85% of A or B (qualitative evaluation) have been released in v.1.10.1. | Passed |
| Table 5: Summary of non-clinical performance tests performed for ART-Plan v1.10.1 |
|---|
| ----------------------------------------------------------------------------------- |
Summary of Animal & Clinical Studies
No animal studies were conducted as part of submission to prove substantial equivalence.
No clinical studies were conducted as part of submission to prove substantial equivalence.
Safety and Effectiveness/ Conclusion
Based on the information presented in this Special 510(k) submission, the TheraPanacea SAS ART-Plan v.1.10.1 is considered substantially equivalent. The TheraPanacea ART-Plan is as safe and effective as the currently marketed predicate device, TheraPanacea SAS ART-Plan v1.10.0 previously 510(k) cleared (K220813) .
Based on testing and comparison with the predicate device TheraPanacea SAS ART-Plan v1.10.0 previously 510(k) cleared (K220813), TheraPanacea SAS ART-Plan v1.10.1 indicated no adverse indications or results. It is our determination that the TheraPanacea SAS ART-Plan v.1.10.1 performs within its design specifications and is substantially equivalent to the predicate device, TheraPanacea SAS ART-Plan v1.10.0 previously 510(k) cleared (K220813).
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).