Trained medical professionals use Contour ProtégéAl as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAl supports the following indications:

· Creation of contours using maching algorithms for applications including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management.

· Segmenting normal structures across a variety of CT anatomical locations.

· And segmenting normal structures of the prostate, seminal vesicles, and urethra within T2-weighted MR images.

Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.

Device Description

Contour ProtégéAl is an accessory to MIM software that automatically creates contours on medical images through the use of machine-learning algorithms. It is designed for use in the processing of medical images and operates on Windows, Mac, and Linux computer systems. Contour ProtégéAl is deployed on a remote server using the MIMcloud service for data management and transfer; or locally on the workstation or server running MIM software.

AI/ML Overview

The provided text outlines the 510(k) summary for Contour ProtégéAI, but it primarily focuses on establishing substantial equivalence to predicate devices and does not detail specific acceptance criteria or a comprehensive study report with numerical performance metrics against those criteria. The information provided is more about the regulatory submission process and general claims of equivalence rather than a detailed breakdown of a validation study.

However, based on the limited information regarding "Testing and Performance Data" (page 9), I can infer some aspects and highlight what is missing.

Here's an attempt to describe the acceptance criteria and study proving the device meets them, based on the provided text, while also pointing out the lack of detailed numerical results for the acceptance criteria.

Acceptance Criteria and Device Performance Study for Contour ProtégéAI

The provided 510(k) summary for Contour ProtégéAI states that "Equivalence is defined such that the lower 95th percentile confidence bound of the Contour ProtégéAI segmentation is greater than 0.1 Dice lower than the mean MIM Maestro atlas segmentation reference device performance." This statement defines the non-inferiority acceptance criterion used to compare Contour ProtégéAI against a reference device (MIM Maestro) rather than setting absolute performance thresholds for the contours themselves.

1. Table of Acceptance Criteria and Reported Device Performance

Acceptance Criteria (Inferred from Text)	Reported Device Performance
For each structure of each neural network model, the lower 95th percentile confidence bound of the Contour ProtégéAI Dice coefficient must be greater than 0.1 Dice lower than the mean Dice coefficient of the MIM Maestro atlas segmentation reference device.	Stated Outcome: "Contour ProtégéAI results were equivalent or had better performance than the MIM atlas segmentation reference device."
Specific numerical performance for each structure (Dice Coefficient)	Not provided in the document. The document states a qualitative conclusion of "equivalent or better performance" without the actual mean Dice coefficients or 95th percentile bounds for either Contour ProtégéAI or MIM Maestro.

2. Sample Size Used for the Test Set and Data Provenance

Test Set Sample Size: The document implies that the "test subjects" were used for evaluation, but the specific number of cases or patients in the test set is not explicitly stated.
Data Provenance: The text mentions that neural network models were trained on data that "did not include any patients from the same institution as the test subjects." This implies that the test set data originated from institutions different from the training data, suggesting a form of independent validation. The countries of origin for the data are not specified. The text indicates the study was retrospective as it involved evaluating pre-existing patient data.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Those Experts

The document states that "multiple atlases were created over the test subjects" for the MIM Maestro reference device. It does not explicitly state how the ground truth for the test set was established for Contour ProtégéAI's evaluation results. Instead, it refers to the MIM Maestro's performance as a reference. There is no information provided on the number or qualifications of experts who established any ground truth used in this comparison.

4. Adjudication Method for the Test Set

The document does not describe any specific adjudication method (e.g., 2+1, 3+1) for establishing ground truth or evaluating the test set. It mentions the "leave-one-out analysis" for creating atlases for MIM Maestro, which is a method of data splitting/resampling, not an adjudication process.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

Was an MRMC study done? Based on the provided text, there is no indication that a multi-reader multi-case (MRMC) comparative effectiveness study was conducted to evaluate how much human readers improve with AI vs. without AI assistance. The study described focuses on the comparison of the algorithm's performance (Contour ProtégéAI) against an existing atlas-based segmentation method (MIM Maestro).

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

Was a standalone study done? Yes, the described study appears to be a standalone (algorithm-only) performance evaluation. The comparison is between the Contour ProtégéAI algorithm's output and the MIM Maestro atlas segmentation reference device, with Dice coefficients calculated directly from these automated segmentations. The "Indications for Use" explicitly state: "Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI," implying that human modification is expected in clinical use, but the reported study does not include this human-in-the-loop performance.

7. Type of Ground Truth Used

The "ground truth" for the comparison appears to be the segmentation contours generated by the MIM Maestro atlas segmentation reference device. The study aims to demonstrate non-inferiority to this existing, cleared technology rather than a human expert-defined anatomical ground truth or pathology/outcomes data.

8. Sample Size for the Training Set

The document mentions a "pool of training data" but the specific sample size for the training set is not provided.

9. How the Ground Truth for the Training Set Was Established

The document states that "neural network models were trained for each modality (CT and MR) on a pool of training data." However, it does not describe how the ground truth (i.e., the "correct" contours) for this training data was established. It refers to the models being trained "on a pool of training data" without detailing the annotation or ground truth generation process for this training data.

Summary

{0}------------------------------------------------

Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.

MIM Software Inc. % Lynn Hanigan Quality Assurance Director 25800 Science Park Drive - Suite 180 CLEVELAND OH 44122

Re: K210632

Trade/Device Name: Contour ProtégéAI Regulation Number: 21 CFR 892.2050 Regulation Name: Medical image management and processing system Regulatory Class: Class II Product Code: QKB Dated: September 17, 2021 Received: September 20, 2021

Dear Lynn Hanigan:

We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see

{1}------------------------------------------------

https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely,

Thalia T. Mills, Ph.D. Director Division of Radiological Health OHT7: Office of In Vitro Diagnostics and Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

{2}------------------------------------------------

Indications for Use

510(k) Number (if known) K210632

Device Name

Contour ProtégéAI

Indications for Use (Describe)

Trained medical professionals use Contour ProtegeAI as a tool to assist in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. In addition, Contour ProtégéAI supports the following indications:

· Segmenting normal structures across a variety of CT anatomical locations.

· And segmenting normal structures of the prostate, seminal vesicles, and urethra within T2-weighted MR images.

Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.

Type of Use (Select one or both, as applicable)

Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{3}------------------------------------------------

Image /page/3/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one red and one gray, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, with a simple color palette.

510(k) Summary

K210632

(The following information is in conformance with 21 CFR 807.92)

Submitter:

MIM Software Inc. 25800 Science Park Drive - Suite 180 Cleveland, OH 44122

Phone: 216-455-0600 Fax: 216-455-0601

Contact Person:

Lynn Hanigan

Date Summary Prepared:

October 12, 2021

Device Name

Trade Name:	Contour ProtégéAl
Common Name:	Medical Imaging Software
Regulation Number / Product Code:	21 CFR 892.2050 / Product Code QKB
Classification Name:	Medical image management and processing system

Predicate Device

K193252	Contour ProtégéAl	MIM Software Inc.
Reference Device
K071964	MIM 4.1 SEASTAR (tradename MIM Maestro)	MIMvista Corp.

Intended Use

Contour ProtégéAl is an accessory to MIM software. It includes processing components to allow the contouring of anatomical structures using machine-learning-based algorithms automatically.

Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.

Contour ProtégéAl is not intended to detect or contour lesions.

{4}------------------------------------------------

Image /page/4/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle where they overlap. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, and the colors are eye-catching.

Indications for Use

Creation of contours using machine-learning algorithms for applications including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management.
Segmenting anatomical structures across a variety of CT anatomical locations.
. And segmenting normal structures of the prostate, seminal vesicles, and urethra within T2-weighted MR images.

Appropriate image visualization software must be used to review and, if necessary, edit results automatically generated by Contour ProtégéAI.

Device Description

Substantial Equivalence

ITEM	Contour ProtégéAl	Contour ProtégéAl(K193252)	MIM 4.1 SEASTAR[i.e., MIM Maestro](K071964)
Clearance Dates	TBD	7/2/2020	9/26/2007
Intended Use	Contour ProtégéAl is anaccessory to MIM softwareused for the contouring ofanatomical structures inimaging data usingmachine-learning-basedalgorithms automatically.Appropriate imagevisualization software mustbe used to review and, if	Contour ProtégéAl is anaccessory to MIM softwareused for the contouring ofanatomical structures inimaging data usingmachine-learning-basedalgorithms automatically.Contour ProtégéAl mustbe used in conjunctionwith MIM software to	MIM 4.1 (SEASTAR)software is intended fortrained medicalprofessionals including,but not limited to,radiologists, oncologists,physicians, medicaltechnologists,dosimetrists, andphysicists.
ITEM	Contour ProtégéAl	Contour ProtégéAl(K193252)	MIM 4.1 SEASTAR[i.e., MIM Maestro](K071964)
	necessary, edit resultsautomatically generated byContour ProtégéAl.Contour ProtégéAl is notintended to detect orcontour lesions.	review and, if necessary,edit results automaticallygenerated by ContourProtégéAl.Contour ProtégéAl is notintended to detect orcontour lesionsautomatically.	MIM 4.1 (SEASTAR) isa medical image andinformationmanagement systemthat is intended toreceive, transmit, store,retrieve, display, printand process digitalmedical images, as wellas create, display andprint reports from thoseimages. The medicalmodalities of thesemedical imagingsystems include, but arenot limited to, CT, MRI,CR, DX, MG, US,SPECT, PET and XA assupported byACR/NEMA DICOM 3.0.MIM 4.1 (SEASTAR)provides tools to quicklycreate, transform, andmodify contours forapplications including,but not limited to,quantitative analysis,aiding adaptive therapy,transferring contours toradiation therapytreatment planningsystems and archivingcontours for patientfollow-up andmanagement.
ITEM	Contour ProtégéAl	Contour ProtégéAl(K193252)	MIM 4.1 SEASTAR[i.e., MIM Maestro](K071964)
Indications forUse	Trained medicalprofessionals use ContourProtégéAl as a tool to assistin the automatedprocessing of digitalmedical images ofmodalities CT and MR, assupported by ACR/NEMADICOM 3.0. In addition,Contour ProtégéAl supportsthe following indications:Creation of contoursusing machine-learningalgorithms forapplications including,but not limited to,quantitative analysis,aiding adaptive therapy,transferring contours toradiation therapytreatment planningsystems, and archivingcontours for patientfollow-up andmanagement. Segmenting normalstructures across avariety of CTanatomical locations. And segmenting normalstructures of theprostate, seminalvesicles, and urethrawithin T2-weighted MRimages. Appropriate imagevisualization software mustbe used to review and, ifnecessary, edit results	Contour ProtégéAl is usedby trained medicalprofessionals as a tool toaid in the automatedprocessing of digitalmedical images ofmodalities CT and MR, assupported by ACR/NEMADICOM 3.0. ContourProtégéAl assists in thefollowing indications:The creation ofcontours usingmachine-learningalgorithms forapplications including,but not limited to,quantitative analysis,aiding adaptivetherapy, transferringcontours to radiationtherapy treatmentplanning systems, andarchiving contours forpatient follow-up andmanagement. Segmenting normalstructures across avariety of CTanatomical locations. And segmentingnormal structures ofthe prostate, seminalvesicles, and urethrawithin T2-weighted MRimages. Contour ProtégéAl mustbe used in conjunctionwith MIM software toreview and, if necessary,edit contours that were	MIM 4.1 (SEASTAR)software is used bytrained medicalprofessionals as a toolto aid in evaluation andinformationmanagement of digitalmedical images. Themedical imagemodalities include, butare not limited to, CT,MRI, CR, DX, MG, US,SPECT, PET and XA assupported byACR/NEMA DICOM 3.0.MIM 4.1 (SEASTAR)assists in the followingindications:Receive, transmit,store, retrieve, display,print, and processmedical images andDICOM objects. Create, display andprint reports frommedical images. Registration, fusiondisplay, and review ofmedical images fordiagnosis, treatmentevaluation, andtreatment planning. Localization anddefinition of objectssuch as tumors andnormal tissues inmedical images. Creation,transformation, andmodification of contoursfor applications
ITEM	Contour ProtégéAl	Contour ProtégéAl(K193252)	MIM 4.1 SEASTAR[i.e., MIM Maestro](K071964)
	automatically generated byContour ProtégéAl.	automatically generatedby Contour ProtégéAl.	including, but not limitedto, quantitative analysis,aiding adaptive therapy,transferring contours toradiation therapytreatment planningsystems, and archivingcontours for patientfollow-up andmanagement.
Modalities	CT and MR	CT and MR	CT, MRI, CR, DX, MG,US, SPECT, PET andXA
Atlas-BasedSegmentation	No	No	Yes
AutomaticallyContour ImagingData UsingMachine-Learning	Yes	Yes	No
ITEM	Contour ProtégéAl	Contour ProtégéAl(K193252)	MIM 4.1 SEASTAR[i.e., MIM Maestro](K071964)
Neural NetworkModels included	(1.0.0 models)Head and Neck CTProstate CTThorax CTLiver CTProstate MR(1.1.0 model)Prostate MR(2.0.0 models)Head and Neck CTProstate CTThorax CTAbdomen CTLungs and Liver CT	(1.0.0 models)Head and Neck CTProstate CTThorax CTLiver CTProstate MR	None
OperatingPlatform	Server-based applicationsupportingLinux-based OS- and -Local deployment onWindows or Mac	Server-based applicationsupportingLinux-based OS	Windows, Mac
Cloud-baseddeployment	Yes	Yes	No

{5}------------------------------------------------

Image /page/5/Picture/0 description: The image is a logo for MIM Software. The logo consists of two overlapping squares, one red and one gray, with a white circle cut out of the red square. To the right of the squares is the text "mim" in black, bold letters. Below the text "mim" is the text "SOFTWARE" in smaller, black letters.

{6}------------------------------------------------

Image /page/6/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle in the red square. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the text "SOFTWARE" in a smaller, sans-serif font.

{7}------------------------------------------------

Image /page/7/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red, with a white circle cut out of the red square. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is simple and modern, and the colors are eye-catching.

{8}------------------------------------------------

Image /page/8/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one gray and one red. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" in a smaller font below it. The logo is clean and modern, with a focus on simplicity and readability.

{9}------------------------------------------------

Image /page/9/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping rounded squares, one red and one gray, with a white circle cut out of the red square. To the right of the squares is the text "mim" in a bold, sans-serif font, with the word "SOFTWARE" underneath in a smaller font. The logo is simple and modern, and the colors are eye-catching.

ITEM	Contour ProtégéAl	Contour ProtégéAl(K193252)	MIM 4.1 SEASTAR[i.e., MIM Maestro](K071964)
Locally deployed(or installed)	Yes	No	Yes

Discussion

Changes within this submission include a slightly modified Intended Use and Indications for Use, new and modified CT neural network models with additional contours, a modified Prostate MR neural network model, and added functionality to install locally on a MIM workstation or server. These changes differ when comparing to Contour ProtégéAl 510(k)193252. Non-inferiority testing was used to compare the proposed Contour ProtégéAl device to atlases created from the MIM Maestro reference device.

Testing and Performance Data

For the proposed Contour ProtégéAl device, neural network models were trained for each modality (CT and MR) on a pool of training data that did not include any patients from the same institution as the test subjects. The models were then evaluated on the test subjects, and a Dice coefficient was calculated for each structure. These Dice coefficients were then aggregated, overall patients.

With the MIM Maestro atlas segmentation reference device, multiple atlases were created over the test subjects. Each Atlas contained images of the same anatomical field of view from the same institution. Each structure appeared in one Atlas. For each patient in an Atlas was used to segment the structures in that patient. The test patient itself was excluded from this Atlas (leave-one-out analysis).

The mean and standard deviation Dice coefficients, along with the lower 95th percentile confidence bound, were calculated for both the proposed Contour ProtégéAl device and the MIM Maestro atlas segmentation reference device for each structure of each neural network model. Contour ProtégéAI results were equivalent or had better performance than the MIM atlas seqmentation reference device. Equivalence is defined such that the lower 95th percentile confidence bound of the Contour ProtégéAl segmentation is greater than 0.1 Dice lower than the mean MIM Maestro atlas segmentation reference device performance.

{10}------------------------------------------------

Image /page/10/Picture/0 description: The image shows the logo for MIM Software. The logo consists of two overlapping squares, one gray and one red, with a white circle cut out of the red square. To the right of the squares is the text "mim" in a bold, sans-serif font. Below the text "mim" is the text "SOFTWARE" in a smaller, sans-serif font.

Conclusion

Based on the Discussion and Testing and Performance Data above, the proposed device is determined to be as safe and effective as its predicate device, Contour ProtégéAl 510(k)193252. In addition, the proposed device performs as well as the reference device, MIM 4.1 SEASTAR (k)071964 [MIM Maestro].

Regulation Number and Section

§ 892.2050 Medical image management and processing system.

(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).