(147 days)
Contour+ (MVision Al Segmentation) is a software system for image analysis algorithms to be used in radiation therapy treatment planning workflows. The system includes processing tools for automatic contouring of CT and MR images using machine learning based algorithms. The produced segmentation templates for regions of interest must be transferred to appropriate image visualization systems as an initial template for a medical professional to visualize, review, modify and approve prior to further use in clinical workflows.
The system creates initial contours of pre-defined structures of common anatomical sites, i.e., Head and Neck, Brain, Breast, Lung and Abdomen, Male Pelvis, and Female Pelvis.
Contour+ (MVision Al Segmentation) is not intended to detect lesions or tumors. The device is not intended for use with real-time adaptive planning workflows.
Contour+ (MVision Al Segmentation) is a software-only medical device (software system) that can be used to accelerate region of interest (ROI) delineation in radiotherapy treatment planning by automatic contouring of predefined ROIs and the creation of segmentation templates on CT and MR images.
The Contour+ (MVision Al Segmentation) software system is integrated with a customer IT network and configured to receive DICOM CT and MR images, e.g., from a CT or MRI scanner or a treatment planning system (TPS). Automatic contouring of predefined ROIs is performed by pre-trained, locked, and static models that are based on machine learning using deep artificial neural networks. The models have been trained on several anatomical sites, including the brain, head and neck, bones, breast, lung and abdomen, male pelvis, and female pelvis using hundreds of scans from a diverse patient population. The user does not have to provide any contouring atlases. The resulting segmentation structure set is connected to the original DICOM images and can be transferred to an image visualization system (e.g., a TPS) as an initial template for a medical professional to visualize, modify and approve prior to further use in clinical workflows.
The provided text does not include a table of acceptance criteria and the reported device performance, nor does it specify the sample sizes used for the test set, the number of experts for ground truth, or details on comparative effectiveness studies (MRMC).
However, based on the available information, here is a description of the acceptance criteria and study details:
Acceptance Criteria and Study for Contour+ (MVision AI Segmentation)
The study evaluated the performance of automatic segmentation models by comparing them to ground truth segmentations using Dice Score (DSC) and Surface-Dice Score (S-DSC@2mm) as metrics. The acceptance criteria were based on a "set level of minimum agreement against ground truth segmentations determined through clinically relevant similarity metrics DSC and S-DSC@2mm." While specific numerical thresholds for these metrics are not provided, the submission states that the device fulfills "the same acceptance criteria" as the predicate device.
It's important to note that the provided document is an FDA 510(k) clearance letter and not the full study report. As such, it summarizes the findings and affirms the device's substantial equivalence without detailing every specific test result or acceptance threshold.
1. A table of acceptance criteria and the reported device performance
| Metric | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Dice Score (DSC) | Based on a "set level of minimum agreement against ground truth segmentations" (specific thresholds not provided). | "Performance verification and validation results for various subsets of the golden dataset show the generalizability and robustness of the device..." |
| Surface-Dice Score (S-DSC@2mm) | Based on a "set level of minimum agreement against ground truth segmentations" (specific thresholds not provided). | "...Contour+ (MVision AI Segmentation) fulfills the same acceptance criteria, provides the intended benefits, and it is as safe and as effective as the predicate software version." |
2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Sample Size for Test Set: The exact sample size for the test (golden) dataset is not specified, but it's referred to as "various subsets of the golden dataset" and chosen to "achieve high granularity in performance evaluation tests."
- Data Provenance: The datasets originate from "multiple EU and US clinical sites (with over 50% of data coming from US sites)." It is described as containing "hundreds of scans from a diverse patient population," ensuring representation of the "US population and medical practice." The text does not explicitly state if the data was retrospective or prospective, but the description of "hundreds of scans" from multiple sites suggests it is likely retrospective.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
The number of experts used to establish the ground truth for the test set is not specified in the provided text. The qualifications are vaguely mentioned as "radiotherapy experts" who performed "Performance validation of machine learning-based algorithms for automatic segmentation." No specific years of experience or board certifications are detailed.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The adjudication method for establishing ground truth on the test set is not specified in the provided text. The text only states that the auto-segmentations were compared to "ground truth segmentations."
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
A multi-reader multi-case (MRMC) comparative effectiveness study focusing on the improvement of human readers with AI assistance versus without AI assistance is not explicitly described in the provided text.
The text states: "Performance validation of machine learning-based algorithms for automatic segmentation was also carried out by radiotherapy experts. The results show that Contour+ (MVision AI Segmentation) assists in reducing the upfront effort and time required for contouring CT and MR images, which can instead be devoted by clinicians on refining and reviewing the software-generated contours." This indicates that experts reviewed the output and perceived a benefit in efficiency, but it does not detail a formal MRMC study comparing accuracy or time, with a specific effect size.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance evaluation of the algorithm was conducted. The primary performance metrics (DSC and S-DSC@2mm) were calculated by directly comparing the "produced auto-segmentations to ground truth segmentations," which is a standalone assessment of the algorithm's output. The statement "Performance verification and validation results for various subsets of the golden dataset show the generalizability and robustness of the device" further supports this.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
The ground truth used was expert consensus segmentations. The text repeatedly refers to comparing the device's output to "ground truth segmentations" established by "radiotherapy experts." There is no mention of pathology or outcomes data being used for ground truth.
8. The sample size for the training set
The exact sample size for the training set is not specified, but the models were "trained on several anatomical sites... using hundreds of scans from a diverse patient population."
9. How the ground truth for the training set was established
The text states that the machine learning models were "trained on several anatomical sites... using hundreds of scans from a diverse patient population." While it doesn't explicitly detail the process for establishing ground truth for the training set, it is implied to be through expert contouring/segmentation, as the validation uses "ground truth segmentations" which are established by "radiotherapy experts." Given the extensive training data required for machine learning, it's highly probable that these "hundreds of scans" also had expert-derived segmentations as their ground truth for training.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).