Search Results

SubtleHD is an image processing software that can be used for image enhancement of all body parts MRI images. It can be used for noise reduction and increasing image sharpness.

SubtleHD is Software as a Medical Device (SaMD) consisting of a software algorithm that enhances images taken by MRI scanners. As it only processes images for the device has no user interface. It is intended to be used by radiologists and technologists in an imaging center, clinic, or hospital. The SubtleHD software can be used with MR images acquired as part of standard of care and accelerated MRI exams as the input. The outputs are the corresponding images with enhanced image quality. Original DICOM images are passed onto the SubtleHD software as an input argument and the enhanced images are saved in the designated location prescribed when running the SubtleHD software. The functionality of SubtleHD (noise reduction and sharpness enhancement) is identified from the DICOM series description and/or through configuration is specified as configuration files and OS environment variables.

SubtleHD software implements an image enhancement algorithm using a convolutional network based filtering. Original images are enhanced by running through a cascade of filter banks, where thresholding and scaling operations are applied. A single neural network is trained for adaptive noise reduction and sharpness increase. The parameters within the neural network were obtained through an image-guided optimization process. Additional nonlocal mean based denoising and unsharp masking based sharpening filters are applied to the deep learning processed image.

The software operates on DICOM files, enhances the images, and sends the enhanced images to any desired destination with an AE Title (e.g., PACS, MR device, workstation, and more). Enhanced images coexist with the original images.

The provided text describes the acceptance criteria and study proving the device, SubtleHD (1.x), meets these criteria.

Here's the breakdown of the requested information:

1. Table of Acceptance Criteria and Reported Device Performance

The provided text includes two distinct sets of acceptance criteria and performance results: one for "Performance Validation" and one for a "Reader Study."

Performance Validation Summary:

Reader Study Summary:

2. Sample sizes used for the test set and the data provenance

Standalone Image Quality Metric Testing:

• Paired Test Set (Unaligned SOC as Reference): 97 samples
• Paired Test Set (Unaligned SubtleHD-enhanced SOC as the Reference): 97 samples
• Aligned Test Set: 471 samples

Data Provenance for Standalone Image Quality Metric Testing:

• Countries of Origin: The aligned test set included data from both the US and OUS (Outside US). The paired test set's provenance isn't explicitly stated beyond "More than 50% of data is from sites in the United States."
• Retrospective or Prospective: The text states "retrospective clinical data" for the performance validation and reader study, and describes the data selection for the standalone metrics as using "a subset of our performance validation set," implying a retrospective nature for these as well.

Performance Validation & Reader Study Dataset Characteristics:

• Number of Image Series: 410 image series (205 input and 205 SubtleHD enhanced) for the Reader Study. The Performance Validation used the "SubtleHD performance validation test dataset," which is implied to be distinct from, but shares characteristics with, the reader study dataset.
• Retrospective or Prospective: Retrospective clinical data.
• Countries of Origin: The majority of performance data comes from sources in the United States (65%).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

Performance Validation:

• Ground Truth Establishment: Three Regions of Interest (ROIs) were drawn on each image by a Subtle Medical employee with an MD and/or PhD in a clinically-relevant field.
• Quality Review: A board-certified radiologist reviewed the ROI for acceptability. The exact number of radiologists is not specified, only "a board-certified radiologist."

Reader Study:

• Readers: The study involved "board-certified radiologists." The exact number of radiologists involved in the reading is not specified. Readers were blind to the image processing method.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• Performance Validation: Not applicable in the traditional sense, as ROIs were drawn by one expert and reviewed for acceptability by another. It's not a consensus reading model for diagnostic outcomes.
• Reader Study: The text does not specify an adjudication method for the reader study. It states that "Both the input images and the SubtleHD enhanced images were ranked by board-certified radiologists," implying individual readings without explicit mention of multiple readers reaching consensus or a tie-breaking mechanism.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• MRMC Study: A "Reader Study" was conducted, which involved multiple readers (board-certified radiologists) and multiple cases (410 image series). This qualifies as a multi-reader multi-case study.
• Comparative Effectiveness / Effect Size: The study assessed if SubtleHD-enhanced images statistically significantly improved perceived SNR, Overall Image Quality / Diagnostic Confidence, and Small Structure Visibility, and did not introduce artifacts. The results indicate a "PASS" for all these criteria based on a p-value < 0.05 from a Wilcoxon signed rank test, meaning there was a statistically significant improvement or no statistically significant difference for the positive criteria, and no artifact introduction. While it indicates an improvement in perceived image quality, the document does not report specific effect sizes (e.g., AUC uplift, specific improvement percentage in diagnostic accuracy) for how much human readers improve with AI assistance versus without. It focuses on the qualitative assessment of improvement in image characteristics and diagnostic confidence.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance evaluation was done through:

• Standalone Image Quality Metric Testing: This section reports quantitative metrics (L1 loss, SSIM, PSNR) for the algorithm's output compared to reference images. This is purely algorithm-based performance.
• Performance Validation: This also appears to be a standalone measurement of the algorithm's performance on a dataset, using quantitative metrics like SNR improvement, slope of image intensity change, FWHM reduction, and gradient entropy, without human readers directly influencing the primary outcome measurements.

Both of these sections describe the algorithm's objective performance without a human in the loop to modify or interact with the output for the purpose of the reported metrics.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• Standalone Image Quality Metric Testing:
  • For the Paired Test Set (Unaligned SOC as Reference), the "standard-of-care (SOC) images" were used as reference (ground truth). Also, "SubtleHD-enhanced SOC" was used as another reference.
  • For the Aligned Test Set, "high-quality ground truth images were reconstructed using the vendor's commercially available deep learning pipeline and represent standard-of-care image quality."
• Performance Validation: The ground truth for quantitative metrics (SNR, slope, FWHM, gradient entropy) was derived from characteristics within the images themselves, with ROIs drawn by a clinically-qualified employee and reviewed by a board-certified radiologist for acceptability. This is a form of expert-defined quantitative ground truth.
• Reader Study: The ground truth for this study was the perception of board-certified radiologists using a Likert scale for SNR, Overall Image Quality / Diagnostic Confidence, Small Structure Visibility, and Imaging Artifacts. This can be considered a form of expert consensus/opinion-based ground truth or perceived improvement, although not explicitly stated as a consensus amongst multiple readers for a single case.

8. The sample size for the training set

The document explicitly states that the test sets for all evaluations (Standalone Image Quality Metric Testing, Performance Validation, and Reader Study) were "Not used for algorithm training" and that "data was selected from sources not included in the training dataset (36.50% of the dataset is from non-training sources)." However, the actual sample size of the training set is not provided in the given text.

9. How the ground truth for the training set was established

The document states that a "single neural network is trained for adaptive noise reduction and sharpness increase" and "The parameters within the neural network were obtained through an image-guided optimization process." It also implies "pre- and post-processing is applied to configure desired perceived image quality." However, the specific method for how ground truth was established for the training set is not detailed in the provided text. It is generally understood that for deep learning models like this, the "ground truth" for training purposes would be pairs of original (possibly noisy/blurry) images and corresponding "ideal" or "high-quality" images that the model learns to transform inputs into. The text indicates that some of the performance data came from training sources.

Ask a specific question about this device