(232 days)
MVision AI Segmentation is a software system for image analysis algorithms to be used in radiation therapy treatment planning workflows. The system includes processing tools for automatic contouring of CT images using machine learning based algorithms. The produced segmentation templates for regions of interest must be transferred to appropriate image visualization systems as an initial template for a medical professional to visualize, review, modify and approve prior to further use in clinical workflows.
The system creates initial contours of pre-defined structures of common anatomical sites, i.e. Head and Neck, Brain, Breast, Lung and Abdomen, Male Pelvis, and Female Pelvis in adult patients.
MVision AI Segmentation is not intended to detect lesions or tumors. The device is not intended for use with real-time adaptive planning workflows.
MVision AI Segmentation is a software only medical device which can be used to accelerate region of interest (ROI) delineation in radiotherapy treatment planning by creating automatic segmentation templates on CT images for these ROIs.
The segmentations are produced by pre-trained, locked, and static models that are based on deep artificial neural networks. The produced structure is intended to be used as a template for medical professionals to visualize, modify and approve prior to further use in clinical workflows.
The system is integrated with the customer IT network to receive DICOM images. CT images from, for example, a scanner or a treatment planning system (TPS) are exported to the device. A structure set is created in the device, and the created segmentation results are connected to the original images. These data are sent to the destination DICOM import folder to import the data to, for example, a treatment planning system. The produced structures can then be used as a template for manual ROI editing, review and approval workflow. The segmentations are produced by pre-trained and locked models that are based on deep artificial neural networks. To take the device into use, the user does not have to provide any contouring atlases. The models have been trained with the order of hundreds of scans, depending on the ROI in question. The MVision AI Segmentation device creates initial contours of pre-defined structures of common anatomical sites, i.e. Head and Neck, Brain, Breast, Lung and Abdomen, Male Pelvis, and Female Pelvis.
Here's a breakdown of the acceptance criteria and study information for the MVision AI Segmentation device, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The provided text does not explicitly state specific numerical acceptance criteria for evaluation metrics (e.g., a minimum Dice Similarity Coefficient (DSC) or Hausdorff Distance (HD)). Instead, it generally states that the device's performance will "reflect the real clinical performance" and that it produces "usable contours" that "save clinicians' time."
Therefore, I will extract relevant performance statements and structure them as best as possible, acknowledging the lack of specific thresholds.
| Criterion Type | Acceptance Criteria (Conceptual from text) | Reported Device Performance (from text) |
|---|---|---|
| Clinical Performance | Segmentation performance should reflect real clinical performance in any radiotherapy clinic following consensus guidelines. | "Performance verification results for various subsets of the golden dataset show the generalizability and robustness of the device for the US patient population and US medical practice." "MVision AI Segmentation assists in reducing the upfront effort and time on typical contouring which can be spent on refining and reviewing the results." "Performance validation data further suggests that the subject device produces usable contours (ROIs) as a starting point that will save clinicians' time and it will lead to sooner proceeding to essential parts of radiotherapy treatment planning stages." |
| Generalizability | Models should be generalizable and robust across different patient populations and medical practices. | "Performance verification results for various subsets of the golden dataset show the generalizability and robustness of the device for the US patient population and US medical practice." |
| Clinical Utility | Device should provide usable contours that contribute to efficiency and reduce effort in the radiotherapy workflow. | "Performance validation data further suggests that the subject device produces usable contours (ROIs) as a starting point that will save clinicians' time and it will lead to sooner proceeding to essential parts of radiotherapy treatment planning stages." "MVision AI Segmentation assists in reducing the upfront effort and time on typical contouring which can be spent on refining and reviewing the results." |
| Safety and Effectivnes | The device should be non-inferior, safe, and effective compared to the predicate device. | "Software verification and validation and Performance evaluation tests for machine learning based algorithms establish that the subject medical device is non-inferior, performs safely and effectively as the listed predicate device." |
2. Sample Sizes Used for the Test Set and Data Provenance
- Test Set ("Golden Dataset") Sample Size: The exact number of cases in the test set is not explicitly stated. The document mentions "various subsets of the golden dataset."
- Data Provenance: The data originates from "multiple different sources" to ensure generalizability. It is collected to reflect "the US patient population and US medical practice." The text does not specify countries of origin beyond "US patient population." The data type is implied to be CT images for use in radiotherapy. The text does not specify if the data is retrospective or prospective.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- Number of Experts: The number of experts used to establish ground truth for the test set is not explicitly stated.
- Qualifications of Experts: The ground truth for the test set was established by "radiotherapy experts." No further specific qualifications (e.g., years of experience, specific subspecialty) are provided.
4. Adjudication Method for the Test Set
- The text does not describe a specific adjudication method (e.g., 2+1, 3+1). It only states that the ground truth was established by "radiotherapy experts."
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No MRMC study comparing human readers with and without AI assistance was reported. The document focuses on the device's performance and its ability to "assist in reducing the upfront effort and time" for clinicians, implying an improvement in efficiency, but not a formal MRMC study demonstrating a quantified effect size of human improvement with AI vs without.
6. Standalone Performance Study (Algorithm Only)
- Yes, a standalone performance evaluation was clearly done. The entire "Performance Evaluation Summary" Section (Pages 7-8) describes the evaluation of the "model performance" and "machine learning based algorithms" on "training and test sets (golden dataset)." The results refer to the device producing contours and assisting in reducing effort, indicating an algorithm-only evaluation.
7. Type of Ground Truth Used
- The ground truth used is expert consensus, established by "radiotherapy experts" following "segmentation consensus guidelines."
8. Sample Size for the Training Set
- The models were trained with "the order of hundreds of scans, depending on the ROI in question."
9. How the Ground Truth for the Training Set Was Established
- The ground truth for the training set was established following "segmentation consensus guidelines" as the models were "trained to comply with" these guidelines. This implies expert-derived ground truth, consistent with the test set's ground truth methodology.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).