(27 days)
Trained medical professionals use Contour ProtégéAl as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAl supports the following indications:
· Creation of contours using machine-learning algorithms for applications including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management.
· Segmenting anatomical structures across a variety of CT anatomical locations.
· And segmenting the prostate, the seminal vesicles, and the urethra within T2-weighted MR images.
Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.
Contour ProtégéAl+ is an accessory to MIM software that automatically creates contours on medical images through the use of machine-learning algorithms. It is designed for use in the processing of medical images and operates on Windows, Mac, and Linux computer systems. Contour ProtégéAl+ is deployed on a remote server using the MIMcloud service for data management and transfer; or locally on the workstation or server running MIM software.
Here's a breakdown of Contour ProtégéAI+'s acceptance criteria and study information, based on the provided text:
Acceptance Criteria and Device Performance
The acceptance criteria for each structure's inclusion in the final models were a combination of statistical tests and user evaluation:
| Acceptance Criteria | Reported Device Performance (Contour ProtégéAI+) |
|---|---|
| Statistical non-inferiority of the Dice score compared with the reference predicate (MIM Atlas). | For most structures, the Contour ProtégéAI+ Dice score mean and 95th percentile confidence bound were equivalent to or better than the MIM Atlas. Equivalence was defined as the lower 95th percentile confidence bound of Contour ProtégéAI+ being greater than 0.1 Dice lower than the mean MIM Atlas performance. Results are shown in Table 2, with '*' indicating demonstrated equivalence. |
| Statistical non-inferiority of the Mean Distance Accuracy (MDA) score compared with the reference predicate (MIM Atlas). | For most structures, the Contour ProtégéAI+ MDA score mean and 95th percentile confidence bound were equivalent to or better than the MIM Atlas. Equivalence was defined as the lower 95th percentile confidence bound of Contour ProtégéAI+ being greater than 0.1 Dice lower than the mean MIM Atlas performance. Results are shown in Table 2, with '*' indicating demonstrated equivalence. |
| Average user evaluation of 2 or higher (on a three-point scale: 1=negligible, 2=moderate, 3=significant time savings). | The "External Evaluation Score" (Table 2) consistently shows scores of 2 or higher across all listed structures, indicating moderate to significant time savings. |
| (For models as a whole) Statistically non-inferior cumulative Added Path Loss (APL) compared to the reference predicate. | For all 4.2.0 CT models (Thorax, Abdomen, Female Pelvis, SurePlan MRT), equivalence in cumulative APL was demonstrated (Table 3), with Contour ProtégéAI+ showing lower mean APL values than MIM Atlas. |
| (For localization accuracy) No specific passing criterion, but results are included. | Localization accuracy results (Table 4) are provided as percentages of images successfully localized for both "Relevant FOV" and "Whole Body CT," ranging from 77% to 100% depending on the structure and model. |
Note: Cells highlighted in orange in the original document indicate non-demonstrated equivalence (not reproducible in markdown), and cells marked with '**' indicate that equivalence was not demonstrated because the minimum sample size was not met for that contour.
Study Details
-
Sample size used for the test set and the data provenance:
- Test Set Sample Size: The Contour ProtégéAI+ subject device was evaluated on a pool of 770 images.
- Data Provenance: The images were gathered from 32 institutions. The verification data used for testing is from a set of institutions that are totally disjoint from the datasets used to train each model. Patient demographics for the testing data are: 53.4% female, 31.3% male, 15.3% unknown; 0.3% ages 0-20, 4.7% ages 20-40, 20.9% ages 40-60, 50.0% ages 60+, 24.1% unknown; varying scanner manufacturers (GE, Siemens, Phillips, Toshiba, unknown). The data is retrospective, originating from clinical treatment plans according to the training set description.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The document implies that the ground truth for the test set was validated against "original ground-truth contours" when measuring Dice and MDA against MIM Maestro. However, the expert qualifications are explicitly stated for the training set ground truth, which often implies a similar standard for the test set.
- Ground truth (for training/re-segmentation) was established by:
- Consultants (physicians and dosimetrists) specifically for this purpose, outside of clinical practice.
- Initial segmentations were reviewed and corrected by radiation oncologists.
- Final review and correction by qualified staff at MIM Software (MD or licensed dosimetrists).
- All segmenters and reviewers were instructed to ensure the highest quality training data according to relevant published contouring guidelines.
-
Adjudication method for the test set:
- The document doesn't explicitly describe a specific adjudication method like "2+1" or "3+1" for the test set ground truth. However, it does state that "Detailed instructions derived from relevant published contouring guidelines were prepared for the dosimetrists. The initial segmentations were then reviewed and corrected by radiation oncologists against the same standards and guidelines. Qualified staff at MIM Software (MD or licensed dosimetrists) then performed a final review and correction." This process implies a multi-expert review and correction process to establish the ground truth used for both training and evaluation, ensuring a high standard of accuracy.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- A direct MRMC comparative effectiveness study measuring human readers' improvement with AI versus without AI assistance (i.e., human-in-the-loop performance) is not explicitly described in terms of effect size.
- Instead, the study evaluates the standalone performance of the AI device (Contour ProtégéAI+) against a reference device (MIM Maestro atlas segmentation) and user evaluation of time savings.
- The "Average user evaluation of 2 or higher" on a three-point scale (1=negligible, 2=moderate, 3=significant time savings) provides qualitative evidence of perceived improvement in workflow rather than a quantitative measure of diagnostic accuracy improvement due to AI assistance. "Preliminary user evaluation conducted as part of testing demonstrated that Contour ProtégéAI+ yields comparable time-saving functionality when creating contours as other commercially available automatic segmentation products."
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance evaluation was conducted. The primary comparisons for Dice score, MDA, and cumulative APL are between the Contour ProtégéAI+ algorithm's output and the ground truth, benchmarked against the predicate device's (MIM Maestro atlas segmentation) standalone performance. The results in Table 2 and Table 3 directly show the algorithm's performance.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Expert Consensus Contour (and review): The ground truth was established by expert re-segmentation of images (by consultants, physicians, and dosimetrists) specifically for this purpose, reviewed and corrected by radiation oncologists, and then subjected to a final review and correction by qualified MIM Software staff (MD or licensed dosimetrists). This indicates a robust expert consensus process based on established clinical guidelines.
-
The sample size for the training set:
- The document states that the CT images for the "training set were obtained from clinical treatment plans for patients prescribed external beam or molecular radiotherapy". However, it does not provide a specific numerical sample size for the training set, only for the test set (770 images). It only mentions being "re-segmented by consultants... specifically for this purpose".
-
How the ground truth for the training set was established:
- The ground truth for the training set was established through a multi-step expert process:
- CT images from clinical treatment plans were re-segmented by consultants (physicians and dosimetrists), explicitly for the purpose of creating training data, outside of clinical practice.
- Detailed instructions from relevant published contouring guidelines were provided to the dosimetrists.
- Initial segmentations were reviewed and corrected by radiation oncologists against the same standards and guidelines.
- A final review and correction was performed by qualified staff at MIM Software (MD or licensed dosimetrists).
- All experts were instructed to spend additional time to ensure the highest quality training data, contouring all specified OAR structures on all images according to referenced standards.
- The ground truth for the training set was established through a multi-step expert process:
{0}------------------------------------------------
February 3, 2025
Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: on the left, there is a seal of the Department of Health & Human Services - USA, which features an abstract image of an eagle. To the right of the seal, there is the text "FDA U.S. FOOD & DRUG ADMINISTRATION" in blue color. The word "FDA" is in a square box.
MIM Software Inc. Sydney Lindner Regulatory Affairs Engineer 25800 Science Park Drive, Suite 180 Cleveland, Ohio 44122
Re: K250035
Trade/Device Name: Contour ProtégéAI+ Regulation Number: 21 CFR 892.2050 Regulation Name: Medical Image Management And Processing System Regulatory Class: Class II Product Code: QKB, Dated: January 7, 2025 Received: January 7, 2025
Dear Sydney Lindner:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device"
{1}------------------------------------------------
(https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (OS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatory
{2}------------------------------------------------
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Locon Weidner
Lora D. Weidner, Ph.D. Assistant Director Radiation Therapy Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{3}------------------------------------------------
Indications for Use
Submission Number (if known)
Device Name
Contour ProtégéAl+
Indications for Use (Describe)
Trained medical professionals use Contour ProtégéAI as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAl supports the following indications:
· Creation of contours using machine-learning algorithms for applications including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management.
· Segmenting anatomical structures across a variety of CT anatomical locations.
· And segmenting the prostate, the seminal vesicles, and the urethra within T2-weighted MR images.
Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.
Type of Use (Select one or both, as applicable)
Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW!
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number.
{4}------------------------------------------------
Image /page/4/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, with a simple color palette.
510(k) Summary (The following information is in conformance with 21 CFR 807.92)
Submitter
MIM Software Inc. 25800 Science Park Drive - Suite 180 Cleveland, OH 44122
| Phone:Fax: | 216-455-0600216-455-0601 |
|---|---|
| Contact Person: | Sydney Lindner |
| Date Summary Prepared: | January 7, 2025 |
Device Name
Trade Name:
Common Name:
Regulation Number / Product Code:
Classification Name:
Contour ProtégéAl+
Medical Imaging Software
21 CFR 892.2050 Product Code QKB
System, Imaging Processing, Radiological
Predicate and Reference Devices
| Predicate: | K231765 | Contour ProtégéAl | MIM Software Inc. |
|---|---|---|---|
| Reference: | K071964 | MIM 4.1 (SEASTAR)[i.e., MIM Maestro] | MIMvista Corp. |
{5}------------------------------------------------
Image /page/5/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, to the left of the text "mim SOFTWARE." The text is in a sans-serif font, with "mim" in a larger size than "SOFTWARE."
Intended Use
Contour ProtégéAl is an accessory to MIM software used for the contouring of anatomical structures in imaging data using machine-learning-based algorithms automatically.
Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.
Contour ProtégéAl is not intended to detect or contour lesions.
Indications for Use
Trained medical professionals use Contour ProtégéAl as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAl supports the following indications:
- Creation of contours using machine-learning algorithms for applications ● including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management.
- · Segmenting anatomical structures across a variety of CT anatomical locations.
- And segmenting the prostate, the seminal vesicles, and the urethra within ● T2-weighted MR images.
Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAl.
Device Description
Contour ProtégéAl+ is an accessory to MIM software that automatically creates contours on medical images through the use of machine-learning algorithms. It is designed for use in the processing of medical images and operates on Windows, Mac, and Linux computer systems. Contour ProtégéAl+ is deployed on a remote server using the MIMcloud service for data management and transfer; or locally on the workstation or server running MIM software.
{6}------------------------------------------------
Image /page/6/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle cut out of the red square where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it.
Indications for use have not been modified. The intended use is the same as the last 510(k) clearance for Contour ProtégéAl (K231765).
In the upcoming 1.4.0 release, one new neural network model is being added and three existing neural network models are being updated by retraining with additional or updated contours which prompted this premarket submission of the subject device.
| ITEM | Subject Device:Contour ProtégéAl+(K250035) | Predicate Device:Contour ProtégéAl(K231765) | Substantial EquivalenceDiscussion |
|---|---|---|---|
| ClearanceDate | TBD | November 8, 2023 | N/A |
| OperatingPlatform | Server-based applicationsupporting:• Linux-based OS• Local deployment onWindows or Mac | Server-based applicationsupporting:• Linux-based OS• Local deployment onWindows or Mac | No change |
| Modalities | CT and MR | CT and MR | No change |
| Atlas-basedSegmentation | No | No | No change |
| Automaticallycontourimaging datausingmachine-learning | Yes | Yes | No change |
| Cloud-basedDeployment | Yes | Yes | No change |
| LocalDeploymentor Installation | Yes | Yes | No change |
| Neural | (1.0.0 models) | (1.0.0 models) | The subject device contains |
| Subject Device: | Predicate Device: | ||
| ITEM | Contour ProtégéAl+(K250035) | Contour ProtégéAl(K231765) | Substantial EquivalenceDiscussion |
| NetworkModelsIncluded | • Head and Neck CT• Prostate CT• Thorax CT• Liver CT• Prostate MR(1.1.0 model)• Prostate MR(2.0.0 models)• Head and Neck CT• Prostate CT• Thorax CT• Abdomen CT• Lungs and Liver CT(3.1.0 models)*• Head and Neck CT• Prostate CT• Thorax CT• Abdomen CT• Lungs and Liver CT• MRT Additional StructuresCT(4.0.0 models)• Head and Neck CT | • Head and Neck CT• Prostate CT• Thorax CT• Liver CT• Prostate MR(1.1.0 model)• Prostate MR(2.0.0 models)• Head and Neck CT• Prostate CT• Thorax CT• Abdomen CT• Lungs and Liver CT(3.1.0 models)*• Head and Neck CT• Prostate CT• Thorax CT• Abdomen CT• Lungs and Liver CT• MRT Additional StructuresCT(4.0.0 models)• Head and Neck CT | one new neural networkmodel and three updatedneural network modelscompared to the predicate.These 4 models were alltrained on the samearchitecture as thepredicate without anychanges. These 4 modelscover all the existingstructures for contouringand additional, newstructures, which aredenoted in bold font.These 4 models were alltested according to thesame procedures andacceptance criteria as thepredicate. No unexpectedresults were observed. Thechanges do not raise newquestions for safety andeffectiveness. |
| • Thorax CT• Abdomen CT• Pelvis CT• SurePlan MRT CT | • Thorax CT• Abdomen CT• Pelvis CT• SurePlan MRT CT | ||
| (4.1.0 models)• Head and Neck CT• Thorax CT• Whole Body -Physiological• Uptake Organs CT | (4.1.0 models)• Head and Neck CT• Thorax CT• Whole Body -Physiological• Uptake Organs CT | ||
| (4.2.0 models) | |||
| Subject Device: | Predicate Device: | ||
| ITEM | Contour ProtégéAl+(K250035) | Contour ProtégéAl(K231765) | Substantial EquivalenceDiscussion |
| • Thorax CT- LN_IMN_L- LN_IMN_R- LN_Sclav_L- LN_Sclav_R- LN_Ax_L1_L- LN_Ax_L1_R- LN_Ax_L2_L- LN_Ax_L2_R- LN_Ax_L3_L- LN_Ax_L3_R- BrachialPlex_L***- BrachialPlex_R***- Breast_L***- Breast_R***- Breast_L_RTOG**- Breast_R_RTOG**- Bronchus- Carina- Cricoid- Esophagus- Glnd_Thyroid- GreatVes***- Heart***- Humerus_Head_L- Humerus_Head_R- Kidney_L- Kidney_R- Larynx***- Liver- Lung_L- Lung_R- Musc_Constrict- Pancreas- SpinalCord- Stomach***- Trachea- Ribs**- Chestwall_L**- Chestwall_R**- A_Asc_Aorta**- A_LAD**• Abdomen CT- Bladder- Bowel*** | |||
| Subject Device: | Predicate Device: | Substantial Equivalence | |
| ITEM | Contour ProtégéAl+(K250035) | Contour ProtégéAl(K231765) | Discussion |
| - Bowel Bag***- Cauda Equina- Left Kidney- Right Kidney- Liver- Spinal Cord- Stomach***- Female Pelvis CT**- Bowel Bag**- Bowel**- LN Pelvics**- Bladder**- Uterocervix**- Colon Sigmoid**- Cauda Equina**- Sacral Plexus**- Sacrum**- Rectum**- Femur Head L**- Femur Head R**- SurePlan MRT CT- Bone***- Glnd_Lacrimal_L- Glnd_Lacrimal_R- Glnd_Submand_L- Glnd_Submand_R- Glnd_Thyroid- Kidney_L (Kidney_L w/oRenal Pelvis)- Kidney_R (Kidney_Rw/o Renal Pelvis)- Liver- Lung_L- Lung_R- Parotid_L- Parotid_R- Spleen |
Table 1 - Comparison to Predicate Device.
{7}------------------------------------------------
Image /page/7/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the word "SOFTWARE" in a smaller, sans-serif font.
{8}------------------------------------------------
Image /page/8/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, and the colors are eye-catching.
{9}------------------------------------------------
Image /page/9/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle in the red square. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the text "SOFTWARE" in a smaller, sans-serif font.
- The 3.1.0 models share the same training images and architecture as the 3.0.0 models. Some errors and style inconsistencies in the training segmentations were corrected before re-training, resulting in the 3.1.0 models. ** Indicates new contour/model.
*** Indicates updated contour.
{10}------------------------------------------------
Image /page/10/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, with a focus on simplicity and clarity.
Testing and Performance Data
Changes within this submission compared to the predicate device, Contour ProtégéAl (K231765), include one new 4.2.0 neural network model and three updated 4.2.0 neural network models with additional or updated contours. Non-inferiority testing was used to compare the Contour ProtégéAl+ subject device to Atlases created from the MIM Maestro (K071964) reference device.
Verification and validation tests were performed for each of the core features of Contour ProtégéAl+:
- Contour creation time savings ●
- Clinical quality contour generation ●
The testing methods include both internal verification by MIM Software's own qualified testers and external validation by trained medical professionals.
The Contour ProtégéAl+ subject device was evaluated on the test subjects from a pool of 770 images gathered from 32 institutions.
The CT images for this training set were obtained from clinical treatment plans for patients prescribed external beam or molecular radiotherapy, but the original segmentations were not used. Instead, the images were re-segmented by consultants (physicians and dosimetrists) specifically for this purpose, outside of clinical practice. Detailed instructions derived from relevant published contouring guidelines were prepared for the dosimetrists. The initial seqmentations were then reviewed and corrected by radiation oncologists against the same standards and guidelines. Qualified staff at MIM Software (MD or licensed dosimetrists) then performed a final review and correction. All segmenters and reviewers were instructed to spend additional time to ensure the highest quality training data. In particular, the consultants were asked to contour all specified OAR structures on all images according to referenced standards, whether or not they were proximal to the treatment field. All patients were imaged on an indexed couch in the treatment position ("simulation CT"). Series that were no-axial, had slices thinner than 0.5mm, or had non-Fan Beam or MV acquisitions were excluded.
The verification data used for testing is from a set of institutions that are totally disjoint from the datasets used to train each model in the Contour ProtégéAl+ device. The MIM Maestro (K071964) reference device was configured with Atlases built from the same training data used to train the models. We tested Contour ProtégéAl+ against the reference device, and the goal of this testing is to show that the subject device is equivalent or superior to the reference device. The performance of both segmentation devices was measured by calculating both the Dice score and MDA of the novel segmentations with the original ground-truth contours. User beta testing was also used to evaluate the performance of Contour ProtégéAl+ in the context of time savings
{11}------------------------------------------------
Image /page/11/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, on the left side. To the right of the squares is the text "mim" in a stylized, lowercase font, with the word "SOFTWARE" underneath in a smaller font.
compared to contouring from scratch. This user evaluation was made on a three-point scale for each contour, with one indicating negligible time savings, two indicating moderate, and three indicating significant time savings. Our acceptance criteria combine the statistical tests and the user evaluation - only structures that pass two or more of the following three tests could be included in the final models:
- Statistical non-inferiority of the Dice score compared with the reference predicate.
- Statistical non-inferiority of the MDA score compared with the reference predicate.
- Average user evaluation of 2 or higher, when measured on a three-point scale. ●
Further clinical validation was also conducted in-house to qualitatively evaluate contours compared to detailed criteria based on established clinical guidelines.
In addition, each model as a whole was also evaluated. In order to be included in the released product, the cumulative Added Path (APL) of the contours in the model was evaluated relative to the ground-truth. Cumulative APL has been found to correlate well with the total time spent in editing and correcting auto-segmented contours (Vaassen et al, 2020)1. Each model was required to have statistically non-inferior cumulative APL compared to the reference predicate.
Finally, the localization accuracy of each structure in all models was measured. We did not impose a passing criterion on localization accuracy; the results however are included in this summary document as well as in our User Guide and White Paper to allow the user to better understand the performance of the device.
Across the testing data images, 53.4% were female, 31.3% were male, and 15.3% were unknown for sex. The manufacturer was GE for 43.2%, Siemens for 21.7%, Phillips for 15.5%, Toshiba for 0.4%, and unknown or other scanner vendors for 19.2%. 0.3% were between the ages of 0-20, 4.7% were between the aqes of 20-40, 20.9% were between 40-60, and 50.0% were over the age of 60. 24.1% were unknown for age.
The mean and standard deviation Dice coefficients and MDA scores, along with the lower 95th percentile confidence bound, were calculated for both the proposed Contour ProtégéAl device and the MIM Maestro atlas segmentation reference device for each structure of each neural network model. Contour ProtégéAl+ results were equivalent or had better performance than the MIM Maestro atlas segmentation reference device. Equivalence is defined such that the lower 95th percentile confidence bound of the Contour ProtégéAl+ segmentation is greater than 0.1 Dice lower than the mean MIM atlas seqmentation reference device performance.
Results over the validation set compared to the reference device are presented below.
{12}------------------------------------------------
Image /page/12/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a lowercase, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, with a clean design.
| 4.2.0 CTModel | Structure | Dice MIMAtlas | DiceContourProtégéAl | MDAMIMAtlas | MDAContourProtégéAl | ExternalEvaluationScore |
|---|---|---|---|---|---|---|
| Thorax | BrachialPlex_L | 0.30 ± 0.14 | 0.41 ± 0.15(0.28) * | 3.01 ±1.19 | 2.89 ± 1.07(3.94) * | 2.43 |
| BrachialPlex_R | 0.31 ± 0.12 | 0.39 ± 0.15(0.26) * | 3.08 ±1.54 | 3.11 ± 1.38(4.55) * | 2.43 | |
| Breast_L | 0.74 ± 0.11 | 0.79 ± 0.07(0.73) * | 5.73 ±3.10 | 5.00 ± 2.53(6.98) * | 2.38 | |
| Breast_R | 0.76 ± 0.12 | 0.80 ± 0.11(0.71) * | 5.03 ±2.72 | 4.49 ± 2.02(6.24) * | 2.57 | |
| Breast_L_RTO_G | 0.74 ± 0.11 | 0.77 ± 0.11(0.69) * | 5.73 ±3.10 | 5.47 ± 3.72(7.87) | 2.5 | |
| Breast_R_RTO_G | 0.76 ± 0.12 | 0.77 ± 0.15(0.67) * | 5.03 ±2.72 | 5.10 ± 3.17(7.24) | 2.5 | |
| Bronchus | 0.57 ± 0.19 | 0.62 ± 0.14(0.50) ** | 2.69 ±2.22 | 1.86 ± 0.90(3.19) * | 2.63 | |
| Carina | 0.37 ± 0.18 | 0.50 ± 0.12(0.41) * | 2.67 ±2.65 | 1.93 ± 0.83(3.07) * | 2.43 | |
| Cricoid | 0.02 ± 0.04 | 0.06 ± 0.05(0.02) * | 4.77 ±1.53 | 5.01 ± 1.40(6.27) * | 2.86 | |
| Esophagus | 0.47 ± 0.17 | 0.69 ± 0.16(0.63) * | 2.73 ±2.19 | 1.08 ± 1.53(1.71) * | 2.5 | |
| Glnd_Thyroid | 0.46 ± 0.18 | 0.66 ± 0.18(0.53) * | 2.84 ±1.90 | 1.68 ± 1.44(2.86) * | 2.86 | |
| GreatVes | 0.64 ± 0.17 | 0.70 ± 0.16(0.53) ** | 4.59 ±1.99 | 3.44 ± 1.99(5.46) * | 2.63 | |
| Heart | 0.88 ± 0.08 | 0.90 ± 0.07(0.87) * | 3.05 ±2.03 | 2.49 ± 1.84(3.08) * | 2.63 | |
| Humerus_Head_L | 0.60 ± 0.24 | 0.62 ± 0.24(0.42) ** | 0.38 ±0.23 | 0.33 ± 0.22(0.52) * | 3 | |
| Humerus_Head_R | 0.57 ± 0.22 | 0.60 ± 0.22(0.41) ** | 0.78 ±2.00 | 0.33 ± 0.35(1.60) * | 3 | |
| 4.2.0 CTModel | Structure | Dice MIMAtlas | DiceContourProtégéAl | MDAMIMAtlas | MDAContourProtégéAl | ExternalEvaluationScore |
| Kidney_L | 0.73 ± 0.19 | 0.90 ± 0.09(0.83) * | 3.75 ±2.66 | 1.32 ± 1.13(2.27) * | 2.75 | |
| Kidney_R | 0.73 ± 0.19 | 0.89 ± 0.09(0.83) * | 3.97 ±2.60 | 1.41 ± 1.06(2.34) * | 2.75 | |
| Larynx | 0.47 ± 0.19 | 0.59 ± 0.14(0.51) * | 3.42 ±1.32 | 2.87 ± 1.34(3.51) * | 2.63 | |
| Liver | 0.84 ± 0.12 | 0.90 ± 0.13(0.85) * | 5.06 ±4.18 | 3.56 ± 9.12(6.39) * | 2.71 | |
| Lung_L | 0.95 ± 0.02 | 0.96 ± 0.02(0.96) * | 1.12 ±0.46 | 0.79 ± 0.40(0.94) * | 2.75 | |
| Lung_R | 0.95 ± 0.03 | 0.97 ± 0.03(0.96) * | 1.36 ±0.67 | 0.85 ± 0.48(1.04) * | 2.75 | |
| Musc_Constrict | 0.40 ± 0.17 | 0.50 ± 0.17(0.39) * | 2.00 ±1.79 | 1.67 ± 1.55(2.71) * | 3 | |
| Pancreas | 0.17 ± 0.17 | 0.47 ± 0.21(0.36) * | 16.43 ±16.42 | 6.80 ± 8.89(14.66) ** | 2.17 | |
| SpinalCord | 0.66 ± 0.16 | 0.64 ± 0.17(0.59) * | 1.29 ±0.91 | 1.28 ± 0.72(1.50) * | 2.5 | |
| Stomach | 0.46 ± 0.22 | 0.73 ± 0.21(0.64) * | 12.56 ±13.48 | 6.89 ± 20.57(14.36) ** | 2.13 | |
| Trachea | 0.68 ± 0.15 | 0.74 ± 0.17(0.66) * | 1.43 ±0.65 | 1.12 ± 0.64(1.46) * | 2.63 | |
| A LAD | 0.07 ± 0.09 | 0.32 ± 0.12(0.22) * | 8.77 ±10.69 | 3.43 ± 4.46(11.41) ** | 2.67 | |
| A_Aorta_Asc | 0.72 ± 0.16 | 0.83 ± 0.17(0.68) ** | 2.90 ±2.16 | 1.11 ± 0.60(2.64) * | 2.14 | |
| Rib | 0.25 ± 0.09 | 0.28 ± 0.11(0.21) * | 35.64 ±12.65 | 39.04 ± 14.62(49.68) ** | 2.63 | |
| Chestwall_L | 0.40 ± 0.16 | 0.39 ± 0.17(0.19) ** | 4.35 ±1.12 | 4.44 ± 1.24(5.87) * | 2.43 | |
| Chestwall_R | 0.45 ± 0.18 | 0.42 ± 0.18(0.22) ** | 4.66 ±1.65 | 4.69 ± 1.56(6.51) * | 2.43 | |
| LN_Ax_L1_L | 0.59 ± 0.09 | 0.63 ± 0.10(0.75) * | 2.76 ±0.89 | 2.25 ± 0.80(3.32) * | 2.67 | |
| 4.2.0 CTModel | Structure | Dice MIMAtlas | DiceContourProtégéAl | MDAMIMAtlas | MDAContourProtégéAl | ExternalEvaluationScore |
| LN_Ax_L1_R | 0.52 ± 0.12 | 0.58 ± 0.13(0.69) * | 3.46 ±1.04 | 2.90 ± 1.52(4.09) * | 2.67 | |
| LN_Ax_L2_L | 0.57 ± 0.16 | 0.64 ± 0.13(0.77) ** | 2.57 ±1.25 | 1.93 ± 0.88(2.88) * | 2.67 | |
| LN_Ax_L2_R | 0.53 ± 0.20 | 0.60 ± 0.18(0.73) ** | 2.91 ±1.29 | 2.44 ± 1.26(3.31) * | 2.67 | |
| LN_Ax_L3_L | 0.55 ± 0.17 | 0.62 ± 0.19(0.76) ** | 2.21 ±1.11 | 1.90 ± 1.26(2.87) * | 2.67 | |
| LN_Ax_L3_R | 0.46 ± 0.16 | 0.52 ± 0.17(0.65) * | 3.30 ±1.95 | 2.74 ± 1.73(4.24) * | 2.67 | |
| LN_IMN_L | 0.17 ± 0.11 | 0.41 ± 0.17(0.59) * | 4.75 ±3.26 | 1.82 ± 0.85(5.05) ** | 3 | |
| LN_IMN_R | 0.23 ± 0.16 | 0.48 ± 0.20(0.63) * | 3.36 ±1.95 | 1.69 ± 1.38(3.09) * | 3 | |
| LN_Sclav_L | 0.58 ± 0.15 | 0.66 ± 0.13(0.79) * | 2.84 ±1.94 | 2.49 ± 1.70(4.18) * | 2 | |
| LN_Sclav_R | 0.48 ± 0.09 | 0.55 ± 0.09(0.64) * | 2.97 ±0.75 | 2.67 ± 0.90(3.56) * | 2.33 | |
| Abdomen | Bladder | 0.72 ± 0.23 | 0.92 ± 0.16(0.81) * | 3.97 ±3.00 | 0.78 ± 0.73(1.93) * | 2.6 |
| Bowel | 0.34 ± 0.14 | 0.52 ± 0.19(0.37) * | 9.26 ±3.49 | 5.05 ± 3.17(8.13) ** | 2.5 | |
| BowelBag | 0.30 ± 0.09 | 0.36 ± 0.11(0.29) * | 14.13 ±3.62 | 10.08 ± 3.36(12.50) * | 2.89 | |
| CaudaEquina | 0.62 ± 0.15 | 0.69 ± 0.13(0.59) * | 1.17 ±0.51 | 0.95 ± 0.53(1.33) * | 2.6 | |
| Kidney_L | 0.74 ± 0.17 | 0.94 ± 0.03(0.85) * | 3.86 ±2.48 | 0.82 ± 0.41(2.12) * | 2.9 | |
| Kidney_R | 0.75 ± 0.18 | 0.92 ± 0.07(0.83) * | 3.97 ±3.35 | 0.96 ± 0.62(2.74) * | 2.9 | |
| Liver | 0.84 ± 0.12 | 0.93 ± 0.08(0.86) * | 5.23 ±3.65 | 2.01 ± 2.14(3.85) * | 2.8 | |
| SpinalCord | 0.60 ± 0.16 | 0.65 ± 0.14(0.56) * | 1.13 ±0.60 | 0.83 ± 0.32(1.10) * | 2.9 | |
| 4.2.0 CTModel | Structure | Dice MIMAtlas | DiceContourProtégéAl | MDAMIMAtlas | MDAContourProtégéAl | ExternalEvaluationScore |
| Stomach | 0.49 ± 0.21 | 0.82 ± 0.11(0.72) * | 11.56 ±14.68 | 2.68 ± 2.02(9.37) ** | 2.5 | |
| FemalePelvis | Bladder | 0.61 ± 0.18 | 0.91 ± 0.06(0.83) * | 5.53 ±3.76 | 1.04 ± 0.94(2.66) * | 2.75 |
| Bag_Bowel | 0.45 ± 0.17 | 0.52 ± 0.20(0.36) ** | 8.89 ±2.79 | 6.89 ± 2.25(9.06) * | 2.63 | |
| Bowel | 0.34 ± 0.10 | 0.55 ± 0.19(0.42) * | 8.81 ±3.05 | 4.84 ± 3.03(7.44) * | 2.75 | |
| Colon_Sigmoid | 0.05 ± 0.05 | 0.47 ± 0.24(0.36) * | 20.55 ±12.23 | 14.72 ± 15.14(23.56) ** | 2.5 | |
| Femur_Head_L | 0.86 ± 0.07 | 0.91 ± 0.08(0.84) * | 1.43 ±0.78 | 0.95 ± 1.03(1.75) * | 2.75 | |
| Femur_Head_R | 0.87 ± 0.05 | 0.92 ± 0.02(0.89) * | 1.22 ±0.59 | 0.69 ± 0.28(1.09) * | 2.75 | |
| UteroCervix | 0.16 ± 0.16 | 0.65 ± 0.27(0.47) * | 10.65 ±5.73 | 4.39 ± 13.21(13.49) ** | 2 | |
| LN_Pelvics | 0.64 ± 0.05 | 0.74 ± 0.06(0.69) * | 4.37 ±1.11 | 4.12 ± 1.52(5.28) * | 2.75 | |
| Rectum | 0.38 ± 0.16 | 0.75 ± 0.12(0.67) * | 6.30 ±3.30 | 1.52 ± 1.66(3.04) * | 2.63 | |
| SacralPlex | 0.02 ± 0.01 | 0.03 ± 0.01(0.02) * | 13.03 ±1.73 | 12.81 ± 1.86(14.38) * | 2.5 | |
| Sacrum | 0.84 ± 0.02 | 0.89 ± 0.01(0.87) * | 1.58 ±0.32 | 1.09 ± 0.17(1.31) * | 3 | |
| CaudaEquina | 0.65 ± 0.11 | 0.66 ± 0.11(0.58) * | 1.19 ±0.58 | 0.94 ± 0.59(1.39) * | 2.57 | |
| SurePlanMRT | Bone | 0.76 ± 0.08 | 0.83 ± 0.05(0.71) * | 4.77 ±1.97 | 4.57 ± 3.35(9.50) ** | 3 |
| Glnd_Lacrimal_L | 0.23 ± 0.17 | 0.30 ± 0.21(0.16) * | 1.99 ±0.86 | 1.26 ± 0.57(1.96) * | 2.67 | |
| Glnd_Lacrimal_R | 0.23 ± 0.16 | 0.36 ± 0.23(0.22) * | 1.53 ±0.80 | 1.18 ± 0.87(1.97) * | 2.67 | |
| Glnd_Subman | 0.58 ± 0.12 | 0.67 ± 0.29 | 2.19 ± | 1.00 ± 0.35 | 3 | |
| 4.2.0 CTModel | Structure | Dice MIMAtlas | DiceContourProtégéAl | MDAMIMAtlas | MDAContourProtégéAl | ExternalEvaluationScore |
| d_L | (0.52) * | 0.76 | (1.45) * | |||
| GInd_Submand_R | 0.56 ± 0.16 | 0.66 ± 0.31(0.48) * | 2.41 ±1.14 | 0.97 ± 0.33(1.68) * | 3 | |
| Glnd_Thyroid | 0.47 ± 0.19 | 0.75 ± 0.12(0.63) ** | 2.98 ±1.95 | 1.29 ± 1.20(2.46) * | 2.67 | |
| Kidney_L | 0.72 ± 0.18 | 0.91 ± 0.04(0.82) * | 4.08 ±2.52 | 1.56 ± 0.61(2.89) * | 3 | |
| Kidney_R | 0.76 ± 0.17 | 0.91 ± 0.03(0.82) * | 3.87 ±3.14 | 1.45 ± 0.63(3.12) * | 3 | |
| Liver | 0.85 ± 0.12 | 0.93 ± 0.07(0.88) * | 4.82 ±3.51 | 1.79 ± 1.50(3.25) * | 2.67 | |
| Lung_L | 0.94 ± 0.03 | 0.96 ± 0.04(0.93) * | 1.42 ±0.51 | 0.90 ± 0.49(1.32) * | 3 | |
| Lung_R | 0.94 ± 0.04 | 0.96 ± 0.05(0.92) * | 1.66 ±0.75 | 1.07 ± 0.86(1.75) * | 3 | |
| Parotid L | 0.71 ± 0.09 | 0.81 ± 0.05(0.78) * | 2.17 ±0.77 | 1.42 ± 0.40(1.73) * | 3 | |
| Parotid_R | 0.71 ± 0.09 | 0.82 ± 0.05(0.78) * | 2.17 ±0.73 | 1.34 ± 0.48(1.67) * | 3 | |
| Spleen | 0.72 ± 0.10 | 0.95 ± 0.02(0.87) * | 4.38 ±1.87 | 0.62 ± 0.45(2.08) * | 2.67 |
Table 2 – Dice, MDA, and external evaluation results.
{13}------------------------------------------------
Image /page/13/Picture/0 description: The image shows the MIM Software logo. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a stylized font, with the word "SOFTWARE" below it.
{14}------------------------------------------------
Image /page/14/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a stylized font, with the word "SOFTWARE" underneath in smaller letters. The logo is simple and modern, with a focus on the company's name.
{15}------------------------------------------------
Image /page/15/Picture/0 description: The image shows the logo for MIM Software. The logo consists of a gray square overlapping a red square with a white circle in the middle. To the right of the squares is the text "mim" in a sans-serif font, with the word "SOFTWARE" below it in a smaller font.
{16}------------------------------------------------
Image /page/16/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a lowercase, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, with a clean design.
Mean ± Std MDA mm (upper 95th percentile confidence bound based on normal distribution in parentheses) * Equivalence demonstrated at p=0.05 significance level between Contour ProtégéAI and MIM Atlas (green) ** Equivalence was NOT demonstrated because the minimum sample size was not met for this contour (yellow) Cells highlighted in orange indicate that equivalence was NOT demonstrated.
Additionally, preliminary user evaluation conducted as part of testing demonstrated that Contour ProtégéAl+ yields comparable time-saving functionality when creating contours as other commercially available automatic segmentation products.
{17}------------------------------------------------
Image /page/17/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle in the intersection. To the right of the squares is the text "mim" in a sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, and the colors are eye-catching.
Table 3 – 4.2.0 CT Models cumulative APL.
| 4.2.0 CT Models | MIM Atlas | Contour ProtégéAl |
|---|---|---|
| Thorax | 220.33 ± 232.42 | 181.44 ± 219.43 (200.79) * |
| Abdomen | 354.53 ± 386.05 | 238.05 ± 309.40 (312.07) * |
| Female Pelvis | 433.24 ± 392.19 | 314.13 ± 349.77 (383.86) * |
| SurePlan MRT | 216.52 ± 207.87 | 133.01 ± 160.23 (167.60) * |
Mean ± Std APL mm (upper 95th percentile confidence bound based on normal distribution in parentheses). * Equivalence demonstrated at p=0.05 significance level between Contour ProtégéAl and MIM Atlas.
| 4.2.0 CT Models | Structure | Relevant FOV | Whole Body CT |
|---|---|---|---|
| Thorax | BrachialPlex_L | 100 | 100 |
| BrachialPlex_R | 100 | 100 | |
| Breast_L | 100 | 100 | |
| Breast_R | 100 | 100 | |
| Breast_L_RTOG | 100 | 100 | |
| Breast_R_RTOG | 100 | 100 | |
| Bronchus | 100 | 100 | |
| Carina | 99 | 100 | |
| Cricoid | 91 | 100 | |
| Esophagus | 99 | 100 | |
| Glnd_Thyroid | 100 | 77 | |
| GreatVes | 100 | 91 | |
| Heart | 100 | 100 | |
| Humerus_Head_L | 100 | 100 | |
| Humerus_Head_R | 100 | 100 | |
| 4.2.0 CT Models | Structure | Relevant FOV | Whole Body CT |
| Kidney_L | 100 | 95 | |
| Kidney_R | 100 | 100 | |
| Larynx | 100 | 100 | |
| Liver | 99 | 95 | |
| Lung_L | 100 | 100 | |
| Lung_R | 100 | 100 | |
| Musc_Constrict | 100 | 91 | |
| Pancreas | 96 | 95 | |
| SpinalCord | 100 | 100 | |
| Stomach | 97 | 100 | |
| Trachea | 99 | 100 | |
| A_LAD | 100 | 86 | |
| A_Aorta_Asc | 96 | 100 | |
| Rib | 100 | 86 | |
| Chestwall_L | 100 | 100 | |
| Chestwall_R | 100 | 100 | |
| LN_Ax_L1_L | 100 | 100 | |
| LN_Ax_L1_R | 100 | 100 | |
| LN_Ax_L2_L | 100 | 100 | |
| LN_Ax_L2_R | 100 | 100 | |
| LN_Ax_L3_L | 100 | 100 | |
| LN_Ax_L3_R | 100 | 100 | |
| LN_IMN_L | 100 | 100 | |
| LN_IMN_R | 100 | 100 | |
| LN_Sclav_L | 100 | 100 | |
| 4.2.0 CT Models | Structure | Relevant FOV | Whole Body CT |
| Abdomen | LN_Sclav_R | 100 | 100 |
| Bladder | 98 | 95 | |
| Bowel | 100 | 100 | |
| BowelBag | 100 | 100 | |
| CaudaEquina | 100 | 100 | |
| Kidney_L | 100 | 91 | |
| Kidney_R | 100 | 100 | |
| Liver | 100 | 100 | |
| SpinalCord | 100 | 100 | |
| Stomach | 100 | 100 | |
| Female Pelvis | Bladder | 100 | 100 |
| Bag_Bowel | 100 | 100 | |
| Bowel | 100 | 100 | |
| Colon_Sigmoid | 93 | 86 | |
| Femur_Head_L | 100 | 100 | |
| Femur_Head_R | 100 | 95 | |
| UteroCervix | 97 | 100 | |
| LN_Pelvics | 100 | 100 | |
| Rectum | 100 | 100 | |
| SacralPlex | 100 | 100 | |
| Sacrum | 100 | 100 | |
| CaudaEquina | 100 | 100 | |
| SurePlan MRT | Bone | * | 100 |
| Glnd_Lacrimal_L | * | 100 | |
| Glnd_Lacrimal_R | * | 95 | |
| 4.2.0 CT Models | Structure | Relevant FOV | Whole Body CT |
| Glnd_Submand_L | * | 95 | |
| Glnd_Submand_R | * | 100 | |
| Glnd_Thyroid | * | 82 | |
| Kidney_L | * | 91 | |
| Kidney_R | * | 95 | |
| Liver | * | 100 | |
| Lung_L | * | 100 | |
| Lung_R | * | 100 | |
| Parotid_L | * | 100 | |
| Parotid_R | * | 100 | |
| Spleen | * | 100 |
Table 4 – 4.2.0 CT Models localization accuracy.
{18}------------------------------------------------
Image /page/18/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim SOFTWARE" in black, with the word "SOFTWARE" in a smaller font size.
{19}------------------------------------------------
Image /page/19/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with the red square partially covering the gray square. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. There is a trademark symbol to the right of the word software.
{20}------------------------------------------------
Image /page/20/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim SOFTWARE" in black. The "mim" is in a larger, bolder font than the "SOFTWARE".
Percentage of images that were successfully localized by Contour ProtégéAl.
- The relevant FOV for the "SurePlan MRT" model is the Whole Body, so a separate "Relevant FOV" test was not performed.
Each individual feature met the acceptance criteria defined for the verification and validation tests, and the entire software product was determined to be safe and effective for clinical use.
Known Limitations and Biases
Contour ProtégéAl+ can produce incorrect or implausible segmentations when the underlying anatomy is in the field of view of the scan but not clearly discernable due to image quality, noise, or the generally low contrast of some structures. This can occur in any situation where the organ at risk is not clearly discernable in the image.
While both images with and without IV contrast are represented in the training set for Contour ProtégéAl+, the effect of iodinated contrast can vary due to patient weight, contrast dosage, and time since injection. In cases where this is unusually intense brain or kidney enhancement, the posterior boundary can be incorrectly segmented. The age breakdown of the training data, where patient age was available, was predominantly in the 40-60 and 60+ ranges. While this is appropriate to the intended patient populations,
{21}------------------------------------------------
Image /page/21/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the word "SOFTWARE" in a smaller, sans-serif font, followed by a trademark symbol.
only a small proportion of testing and training data was for patients under 40, so more intensive review may be warranted for younger patients.
Where patient sex was available, the ratio of male to female patients in the training set was roughly 2:1. The testing data was chosen with a more even ratio (about 0.82:1) to establish applicability of the model to both.
Conclusion
Contour ProtégéAl+ has been developed according to MIM Software Inc.'s established design control process and software development life cycle. This includes risk management alongside verification and validation testing that includes testing of risk mitigations. Therefore, from all evidence gathered, it is MIM Software Inc.'s belief that Contour ProtégéAl+ (K250035) provides a device substantially equivalent to the predicate device.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).