(29 days)
Ceevra Reveal 3+ is intended as a medical imaging system that allows the processing, review, analysis, communication and media interchange of multi-dimensional digital images acquired from CT or MR imaging devices and that such processing may include the generation of preliminary seqmentations of normal anatomy using software that employs machine learning and other computer vision algorithms. It is also intended as software for preoperative surgical planning, and as software for the intraoperative display of the aforementioned multi-dimensional digital images. Ceevra Reveal 3+ is designed for use by health care professionals and is intended to assist the clinician who is responsible for making all final patient management decisions.
The machine learning algorithms in use by Ceevra Reveal 3+ are for use only for adult patients (22 and over). Three-dimensional images for patients under the age of 22 or of unknown age will be generated without the use of any machine learning algorithms
Ceevra Reveal 3+ ("Reveal 3+"), manufactured by Ceevra, Inc. (the "Company"), is a software as a medical device with two main functions: (1) it is used by Company personnel to generate three-dimensional (3D) images from existing patient CT and MR imaging, and (2) it is used by clinicians to view and interact with the 3D images during preoperative planning and intraoperatively.
Clinicians view 3D images via the Reveal 3+ Mobile Image Viewer software application which runs on compatible mobile devices, and the Reveal 3+ Desktop Image Viewer software application which runs on compatible computers. The 3D images may also be displayed on compatible external displays, or in virtual reality (VR) format with a compatible off-the-shelf VR headset.
Reveal 3+ includes features that enable clinicians to interact with the 3D images including rotating, zooming, panning, selectively showing or hiding individual anatomical structures, and viewing measurements of or between anatomical structures.
Here's a detailed breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria (Metric) | Reported Device Performance |
|---|---|
| Machine Learning Model Performance | |
| Prostate (MR prostate imaging) | 0.87 Sørensen-Dice coefficient (DSC) |
| Bladder (MR prostate imaging) | 0.90 Sørensen-Dice coefficient (DSC) |
| Neurovascular bundles (MR prostate imaging) | 7.8 mm Hausdorff distance metric at the 95th percentile (HD-95) |
| Kidney (CT abdomen imaging) | 0.89 Sørensen-Dice coefficient (DSC) |
| Kidney (MR abdomen imaging) | 0.87 Sørensen-Dice coefficient (DSC) |
| Artery (CT abdomen imaging) | 0.87 Sørensen-Dice coefficient (DSC) |
| Artery (MR abdomen imaging) | 0.83 Sørensen-Dice coefficient (DSC) |
| Vein (CT abdomen imaging) | 0.86 Sørensen-Dice coefficient (DSC) |
| Vein (MR abdomen imaging) | 0.81 Sørensen-Dice coefficient (DSC) |
| Artery (CT chest imaging) | 0.85 Sørensen-Dice coefficient (DSC) |
| Vein (CT chest imaging) | 0.81 Sørensen-Dice coefficient (DSC) |
| Measurement Features Accuracy | All three types of measurements (volumes of structures, diameter of structure, distance between two points) produced by Ceevra Reveal 3+ were verified to be accurate within a mean difference of +/- 10%. |
Study Details:
2. Sample size used for the test set and the data provenance:
- Sample Size: A total of 141 imaging studies were used to evaluate the device's machine learning models.
- Data Provenance: The studies were actual CT or MR imaging studies of patients. No dataset contained more than one imaging study from any particular patient. The data ensured diversity in patient population and scanner manufacturers. Subgroup analysis was performed for patient age, patient sex, and scanner manufacturers.
- Patient Demographics: For non-prostate related datasets, 40% female patients and 60% male patients. Across all datasets, 32% of patients were under 60 years old, 32% were 60 to 70 years old, 30% were over 70 years old, and 6% were of unknown age.
- Scanner Manufacturers: Included GE Medical Systems, Siemens, Toshiba, and Philips Medical Systems.
- Ethnicity: Reasonably correlated to the overall US population.
- Retrospective/Prospective: The text does not explicitly state whether the data was retrospective or prospective, but it refers to "existing patient CT and MR imaging" and "datasets of actual CT or MR imaging studies of patients," which typically implies retrospective use of previously acquired data.
- Country of Origin: Not specified.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: The text states "segmentations generated by medical professionals," but does not explicitly quantify the number of individual experts or medical professionals involved in creating the ground truth for the test set.
- Qualifications of Experts: The experts are broadly described as "medical professionals." No further specific qualifications (e.g., years of experience, subspecialty) are provided.
4. Adjudication method for the test set:
- Adjudication Method: The text does not specify an adjudication method like "2+1" or "3+1." It only states that performance was verified by comparing model-generated segmentations against segmentations generated by medical professionals. This implies a direct comparison rather than a specific multi-expert adjudication workflow.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:
- MRMC Study: No, an MRMC comparative effectiveness study involving human readers with and without AI assistance was not explicitly described or reported in the provided text. The study focused on the standalone performance of the machine learning models.
- Effect Size: Not applicable, as no MRMC study was described.
6. If a standalone (i.e., algorithm-only without human-in-the-loop performance) was done:
- Standalone Performance: Yes, the described study evaluates the standalone performance of the machine learning algorithms. The performance metrics (DSC, HD-95) directly assess how well the algorithms' segmentations compare to the ground truth established by medical professionals.
7. The type of ground truth used:
- Ground Truth Type: Expert consensus/segmentation. The ground truth was established by "segmentations generated by medical professionals from the same imaging study."
8. The sample size for the training set:
- Training Set Sample Size: The text states, "No imaging study used to verify performance was used for training; independence of training and testing data were enforced at the level of the scanning institution, namely, studies sourced from a specific institution were used for either training or testing but could not be used for both." However, the specific sample size of the training set is not provided. It is only implied that it was distinct from the 141-study test set.
9. How the ground truth for the training set was established:
- Training Set Ground Truth: The text does not explicitly detail how the ground truth for the training set was established. It only emphasizes the independence of training and testing data and that the test set's ground truth was created by "medical professionals." It is reasonable to infer that the training set ground truth was similarly established by medical professionals, consistent with standard machine learning practices for supervised learning in medical imaging, but this is not explicitly stated.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).