(220 days)
Contour ProtégéAl is used by trained medical professionals as a tool to aid in the automated processing of digital medical images of modalities CT and MR, as supported by ACR/NEMA DICOM 3.0. Contour ProtégéAl assists in the following indications:
The creation of contours using machine-learning algorithms for applications including, but not limited to, quantitative analysis, aiding adaptive therapy, transferring contours to radiation therapy treatment planning systems, and archiving contours for patient follow-up and management.
Segmenting normal structures across a variety of CT anatomical locations.
And segmenting normal structures of the prostate, seminal vesicles, and urethra within T2-weighted MR images.
Contour ProtégéAI must be used in conjunction with MIM software to review and, if necessary, edit contours that were automatically generated by Contour ProtégAI.
Contour ProtégéAl is an accessory to MIM software that automatically creates contours on medical images through the use of machine-learning algorithms. It is designed for use in the processing of medical images and operates on Windows, Mac, and Linux computer systems. Contour ProtégéAl is deployed on a remote server using the MIMcloud service for data management and transfer.
Here's a breakdown of the acceptance criteria and study details for MIM Software Inc.'s Contour ProtégéAI, based on the provided FDA 510(k) summary:
Acceptance Criteria and Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Non-inferiority to predicate device | Contour ProtégéAI was shown to be non-inferior to the predicate device (MIM) with regards to the mean Dice coefficient of automatically generated contours. Non-inferiority was established with a limit of 0.1 Dice, meaning the performance of Contour ProtégéAI was no more than 0.1 Dice worse than the predicate. |
| Clinically acceptable performance | The non-inferiority limit of 0.1 Dice was determined to be the largest clinically acceptable difference based on previous studies. |
| Automated segmentation of CT images | Demonstrated through the non-inferiority study on a test set of 286 CT images. |
| Automated segmentation of MR images | Demonstrated through the non-inferiority study on a test set of 72 MR images. |
Study Details
-
Sample sizes used for the test set and data provenance:
- CT Images: 286 images
- MR Images: 72 images
- Data Provenance: The test images were gathered from "a different and disjoint set of institutions from the training data." This indicates an independent, external validation set, likely retrospective in nature given that it's an existing dataset. The specific country of origin is not specified.
-
Number of experts used to establish the ground truth for the test set and qualifications of those experts:
- The document does not explicitly state the number of experts or their qualifications for establishing the ground truth of the test set. It mentions "associated segmentations" for the training data but not how test set ground truth was created or by whom.
-
Adjudication method for the test set:
- The document does not specify an adjudication method (e.g., 2+1, 3+1). It states that the neural network models were evaluated against "associated segmentations," implying a reference truth was available.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, and the effect size of how much human readers improve with AI vs without AI assistance:
- No MRMC study evaluating human reader improvement with AI assistance was performed or reported in this summary. The study focused on the standalone performance of the Contour ProtégéAI against a predicate device.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone study was done. The non-inferiority test directly compared the automatically generated contours of Contour ProtégéAI against those of the predicate device, both operating without human intervention for the contouring process itself. The instructions for Contour ProtégéAI do state that it "must be used in conjunction with MIM software to review and, if necessary, edit contours." However, the reported performance study focused on the initial automated segmentation output.
-
The type of ground truth used:
- The ground truth for both training and testing datasets consisted of "associated segmentations." While not explicitly stated, these are typically expert-generated contours, often from trained medical professionals (e.g., oncologists, radiation oncologists, dosimetrists) or highly experienced image analysts. The document does not specify if pathology or outcomes data were used as ground truth.
-
The sample size for the training set:
- The document states that the neural networks were "trained on datasets from several large institutions." It does not provide a specific number of images or cases used in the training set.
-
How the ground truth for the training set was established:
- The training datasets included "CT images and MR images and their associated segmentations." This implies that expert-generated contours were available alongside the images for training the machine-learning models. The specific process or number of experts involved in creating these training segmentations is not detailed in the provided text.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).