Search Results

The uMR Ultra system is indicated for use as a magnetic resonance diagnostic device (MRDD) that produces sagittal, transverse, coronal, and oblique cross sectional images, and spectroscopic images, and that display internal anatomical structure and/or function of the head, body and extremities. These images and the physical parameters derived from the images when interpreted by a trained physician yield information that may assist the diagnosis. Contrast agents may be used depending on the region of interest of the scan.

uMR Ultra is a 3T superconducting magnetic resonance diagnostic device with a 70cm size patient bore and 2 channel RF transmit system. It consists of components such as magnet, RF power amplifier, RF coils, gradient power amplifier, gradient coils, patient table, spectrometer, computer, equipment cabinets, power distribution system, internal communication system, and vital signal module etc. uMR Ultra is designed to conform to NEMA and DICOM standards.

Here's a breakdown of the acceptance criteria and study details for the uMR Ultra device, based on the provided FDA 510(k) clearance letter.

1. Table of Acceptance Criteria and Reported Device Performance

Given the nature of the document, which focuses on device clearance, multiple features are discussed. I will present the acceptance criteria and results for the AI-powered features, as these are the most relevant to the "AI performance" aspect.

Acceptance Criteria and Device Performance for AI-Enabled Features

• ACS has higher SNR than CS.
• ACS has higher (standard deviation (SD) / mean value(S)) values than CS.
• Bland-Altman analysis of image intensities acquired using fully sampled and ACS shown with less than 1% bias and all sample points falls in the 95% confidence interval.
• Measurement differences on ACS and fully sampled images of same structures under 5% is acceptable.
• Radiologists rate all ACS images with equivalent or higher scores in terms of diagnosis quality. | - Pass
• Pass
• Pass
• Pass
• Pass
• Verified that ACS meets the requirements of clinical diagnosis. All ACS images were rated with equivalent or higher scores in terms of diagnosis quality. |
  | DeepRecon | - DeepRecon images achieve higher SNR compared to NADR images.
• Uniformity difference between DeepRecon images and NADR images under 5%.
• Intensity difference between DeepRecon images and NADR images under 5%.
• Measurements on NADR and DeepRecon images of same structures, measurement difference under 5%.
• Radiologists rate all DeepRecon images with equivalent or higher scores in terms of diagnosis quality. | - NADR: 343.63, DeepRecon: 496.15 (PASS)
• 0.07% (PASS)
• 0.2% (PASS)
• 0% (PASS)
• Verified that DeepRecon meets the requirements of clinical diagnosis. All DeepRecon images were rated with equivalent or higher scores in terms of diagnosis quality. |
  | EasyScan | No Fail cases and auto position success rate P1/(P1+P2+F) exceeds 80%.
  (P1: Pass with auto positioning; P2: Pass with user adjustment; F: Fail) | 99.6% |
  | t-ACS | - AI prediction (AI module output) much closer to reference compared to AI module input images.
• Better consistency between t-ACS and reference than between CS and reference.
• No large structural difference appeared between t-ACS and reference.
• Motion-time curves and Bland-Altman analysis consistency between t-ACS and reference. | - Pass
• Pass
• Pass
• Pass |
  | AiCo | - AiCo images exhibit improved PSNR and SSIM compared to the originals.
• No significant structural differences from the gold standard.
• Radiologists confirm image quality is diagnostically acceptable, fewer motion artifacts, and greater benefits for clinical diagnosis. | - Pass
• Pass
• Confirmed. |
  | SparkCo | - Average detection accuracy needs to be > 90%.
• Average PSNR of spark-corrected images needs to be higher than spark images.
• Spark artifacts need to be reduced or corrected after enabling SparkCo. | - 94%
• 1.6 higher
• Successfully corrected |
  | ImageGuard | Success rate P/(P+F) exceeds 90%.
  (P: Pass if prompt appears for motion / no prompt for no motion; F: Fail if prompt doesn't appear for motion / prompt appears for no motion) | 100% |
  | EasyCrop | No Fail cases and pass rate P1/(P1+P2+F) exceeds 90%.
  (P1: Other peripheral tissues cropped, meets user requirements; P2: Cropped images don't meet user requirements, but can be re-cropped; F: EasyCrop fails or original images not saved) | 100% |
  | EasyFACT | Satisfied and Acceptable ratio (S+A)/(S+A+F) exceeds 95%.
  (S: All ROIs placed correctly; A: Fewer than five ROIs placed correctly; F: ROIs positioned incorrectly or none placed) | 100% |
  | Auto TI Scout | Average frame difference between auto-calculated TI and gold standard is ≤ 1 frame, and maximum frame difference is ≤ 2 frames. | Average: 0.37-0.44 frames, Maximum: 1-2 frames (PASS) |
  | Inline MOCO | Average Dice coefficient of the left ventricular myocardium after motion correction is > 0.87. | Cardiac Perfusion Images: 0.92
  Cardiac Dark Blood Images: 0.96 |
  | Inline ED/ES Phases Recognition | The average error between the phase indices calculated by the algorithm for the ED and ES of test data and the gold standard phase indices does not exceed 1 frame. | 0.13 frames |
  | Inline ECV | No failure cases, satisfaction rate S/(S+A+F) > 95%.
  (S: Segmentation adheres to myocardial boundary, blood pool ROI correct; A: Small missing/redundant areas but blood pool ROI correct; F: Myocardial mask fails or blood pool ROI incorrect) | 100% |
  | EasyRegister (Height Estimation) | PH5 (Percentage of height error within 5%); PH15 (Percentage of height error within 15%); MEAN_H (Average error of height). (Specific numerical criteria not explicitly stated beyond these metrics) | PH5: 92.4%
  PH15: 100%
  MEAN_H: 31.53mm |
  | EasyRegister (Weight Estimation) | PW10 (Percentage of weight error within 10%); PW20 (Percentage of weight error within 20%); MEAN_W (Average error of weight). (Specific numerical criteria not explicitly stated beyond these metrics) | PW10: 68.64%
  PW20: 90.68%
  MEAN_W: 6.18kg |
  | EasyBolus | No Fail cases and success rate P1+P2/(P1+P2+F) exceeds 100%.
  (P1: Monitoring point positioning meets user requirements, frame difference ≤ 1 frame; P2: Monitoring point positioning meets user requirements, frame difference = 2 frames; F: Auto position fails or frame difference > 2 frames) | P1: 80%
  P2: 20%
  Total Failure Rate: 0%
  Pass: 100% |

For the rest of the questions, I will consolidate the information where possible, as some aspects apply across multiple AI features.

2. Sample Sizes Used for the Test Set and Data Provenance

• ACS: 749 samples from 25 volunteers. Diverse demographic distributions covering various genders, age groups, ethnicity (White, Asian, Black), and BMI (Underweight, Healthy, Overweight/Obesity). Data collected from various clinical sites during separated time periods.
• DeepRecon: 25 volunteers (nearly 2200 samples). Diverse demographic distributions covering various genders, age groups, ethnicity (White, Asian, Black), and BMI. Data collected from various clinical sites during separated time periods.
• EasyScan: 444 cases from 116 subjects. Diverse demographic distributions covering various genders, age groups, and ethnicities. Data acquired from UIH MRI equipment (1.5T and 3T). Data provenance not explicitly stated (e.g., country of origin), but given the company location (China) and "U.S. credentials" for evaluators, it likely includes data from both. The document states "The testing dataset was collected independently from the training dataset".
• t-ACS: 1173 cases from 60 volunteers. Diverse demographic distributions covering various genders, age groups, ethnicities (White, Black, Asian) and BMI. Data acquired by uMR Ultra scanners. Data provenance not explicitly stated, but implies global standard testing.
• AiCo: 218 samples from 24 healthy volunteers. Diverse demographic distributions covering various genders, age groups, BMI (Under/healthy weight, Overweight/Obesity), and ethnicity (White, Black, Asian). Data provenance not explicitly stated.
• SparkCo: 59 cases from 15 patients for real-world spark raw data testing. Diverse demographic distributions including gender, age, BMI (Underweight, Healthy, Overweight, Obesity), and ethnicity (Asian, "N.A." for White, implying not tested as irrelevant). Data acquired by uMR 1.5T and uMR 3T scanners.
• ImageGuard: 191 cases from 80 subjects. Diverse demographic distributions covering various genders, age groups, and ethnicities (White, Black, Asian). Data acquired from UIH MRI equipment (1.5T and 3T).
• EasyCrop: Not explicitly stated as "subjects" vs. "cases," but tested on 5 intended imaging body parts. Sample size (N=65) implies 65 cases/scans, potentially from 65 distinct subjects or fewer if subjects had multiple scans. Diverse demographic distributions covering various genders, age groups, ethnicity (Asian, Black, White). Data acquired from UIH MRI equipment (1.5T and 3T).
• EasyFACT: 25 cases from 25 volunteers. Diverse demographic distributions covering various genders, age groups, weight, and ethnicity (Asian, White, Black).
• Auto TI Scout: 27 patients. Diverse demographic distributions covering various genders, age groups, ethnicity (Asian, White), and BMI. Data acquired from 1.5T and 3T scanners.
• Inline MOCO: Cardiac Perfusion Images: 105 cases from 60 patients. Cardiac Dark Blood Images: 182 cases from 33 patients. Diverse demographic distributions covering age, gender, ethnicity (Asian, White, Black, Hispanic), BMI, field strength, and disease conditions (Positive, Negative, Unknown).
• Inline ED/ES Phases Recognition: 95 cases from 56 volunteers, covering various genders, age groups, field strength, disease conditions (NOR, MINF, DCM, HCM, ARV), and ethnicity (Asian, White, Black).
• Inline ECV: 90 images from 28 patients. Diverse demographic distributions covering gender, age, BMI, field strength, ethnicity (Asian, White), and health status (Negative, Positive, Unknown).
• EasyRegister (Height/Weight Estimation): 118 cases from 63 patients. Diverse ethnic groups (Chinese, US, France, Germany).
• EasyBolus: 20 subjects. Diverse demographic distributions covering gender, age, field strength, and ethnicity (Asia).

Data Provenance (Retrospective/Prospective and Country of Origin):
The document states "The testing dataset was collected independently from the training dataset, with separated subjects and during different time periods." This implies a prospective collection for validation that is distinct from the training data. For ACS and DeepRecon, it explicitly mentions "US subjects" for some evaluations, but for many features, the specific country of origin for the test set is not explicitly stated beyond "diverse ethnic groups" or "Asian" which could be China (where the company is based) or other Asian populations. The use of "U.S. board-certified radiologists" and "licensed MRI technologist with U.S. credentials" for evaluation suggests the data is intended to be representative of, or directly includes, data relevant to the U.S. clinical context.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

• ACS & DeepRecon: Evaluated by "American Board of Radiologists certificated physicians" (plural, implying multiple, at least 2). Not specified how many exactly, but strong qualifications.
• EasyScan, ImageGuard, EasyCrop, EasyBolus: Evaluated by "licensed MRI technologist with U.S. credentials." For EasyBolus, it specifies "certified professionals in the United States." Number not explicitly stated beyond "the" technologist/professionals, but implying multiple for robust evaluation.
• Inline MOCO & Inline ECV: Ground truth annotations done by a "well-trained annotator" and "finally, all ground truth was evaluated by three licensed physicians with U.S. credentials." This indicates a 3-expert consensus/adjudication.
• SparkCo: "One experienced evaluator" for subjective image quality improvement.
• For other features (t-ACS, EasyFACT, Auto TI Scout, Inline ED/ES Phases Recognition, EasyRegister), the ground truth seems to be based on physical measurements (for EasyRegister) or computational metrics (for t-ACS based on fully-sampled images, and for accuracy of ROI placement against defined standards), rather than human expert adjudication for ground truth.

4. Adjudication Method (e.g., 2+1, 3+1, none) for the Test Set

• Inline MOCO & Inline ECV: "Evaluated by three licensed physicians with U.S. credentials." This implies a 3-expert consensus method for ground truth establishment.
• ACS, DeepRecon, AiCo: "Evaluated by American Board of Radiologists certificated physicians" (plural). While not explicitly stated as 2+1 or 3+1, it suggests a multi-reader review, where consensus was likely reached for the reported diagnostic quality.
• SparkCo: "One experienced evaluator" was used for subjective evaluation, implying no formal multi-reader adjudication for this specific metric.
• For features like EasyScan, ImageGuard, EasyCrop, EasyBolus (evaluated by MRI technologists) and those relying on quantitative metrics against a reference (t-ACS, EasyFACT, Auto TI Scout, EasyRegister, Inline ED/ES Phases Recognition), the "ground truth" is either defined by the system's intended function (e.g., correct auto-positioning) or a mathematically derived reference, so a traditional human adjudication method is not applicable in the same way as for diagnostic image interpretation.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done

The document does not explicitly state that a formal MRMC comparative effectiveness study was performed to quantify the effect size of how much human readers improve with AI vs. without AI assistance.

Instead, the evaluations for ACS, DeepRecon, and AiCo involve "American Board of Radiologists certificated physicians" who "verified that [AI feature] meets the requirements of clinical diagnosis. All [AI feature] images were rated with equivalent or higher scores in terms of diagnosis quality." For AiCo, they confirmed images "exhibit fewer motion artifacts and offer greater benefits for clinical diagnosis." This is a qualitative assessment of diagnostic quality by experts, but not a comparative effectiveness study in the sense of measuring reader accuracy or confidence change with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done

Yes, for many of the AI-enabled features, a standalone performance evaluation was conducted:

• ACS: Performance was evaluated by comparing quantitative metrics (NRMSE, SNR, Resolution, Contrast, Uniformity, Structure Measurement) against fully-sampled images or CS. This is a standalone evaluation.
• DeepRecon: Quantitative metrics (SNR, uniformity, contrast, structure measurement) were compared between DeepRecon and NADR (without DeepRecon) images. This is a standalone evaluation.
• t-ACS: Quantitative tests (MAE, PSNR, SSIM, structural measurements, motion-time curves) were performed comparing t-ACS and CS results against a reference. This is a standalone evaluation.
• AiCo: PSNR and SSIM values were quantitatively compared, and structural dimensions were assessed, between AiCo processed images and original/motionless reference images. This is a standalone evaluation.
• SparkCo: Spark detection accuracy was calculated, and PSNR of spark-corrected images was compared to original spark images. This is a standalone evaluation.
• Inline MOCO: Evaluated using Dice coefficient, a quantitative metric for segmentation accuracy. This is a standalone evaluation.
• Inline ED/ES Phases Recognition: Evaluated by quantifying the error between algorithm output and gold standard phase indices. This is a standalone evaluation.
• Inline ECV: Evaluated by quantitative scoring for segmentation accuracy (S, A, F criteria). This is a standalone evaluation.
• EasyRegister (Height/Weight): Evaluated by quantitative error metrics (PH5, PH15, MEAN_H; PW10, PW20, MEAN_W) against physical measurements. This is a standalone evaluation.

Features like EasyScan, ImageGuard, EasyCrop, and EasyBolus involve automated workflow assistance where the direct "diagnostic" outcome isn't solely from the algorithm, but the automated function's performance is evaluated in a standalone manner against defined success criteria.

7. The Type of Ground Truth Used

The type of ground truth varies depending on the specific AI feature:

• Reference/Fully-Sampled Data:
  • ACS, DeepRecon, t-ACS, AiCo: Fully-sampled k-space data transformed to image space served as "ground-truth" for training and as a reference for quantitative performance metrics in testing. For AiCo, "motionless data" served as gold standard.
  • SparkCo: Simulated spark artifacts generated from "spark-free raw data" provided ground truth for spark point locations in training.
• Expert Consensus/Subjective Evaluation:
  • ACS, DeepRecon, AiCo: "American Board of Radiologists certificated physicians" provided qualitative assessment of diagnostic image quality ("equivalent or higher scores," "diagnostically acceptable," "fewer motion artifacts," "greater benefits for clinical diagnosis").
  • EasyScan, ImageGuard, EasyCrop, EasyBolus: "Licensed MRI technologist with U.S. credentials" or "certified professionals in the United States" performed subjective evaluation against predefined success criteria for the workflow functionality.
  • SparkCo: One experienced evaluator for subjective image quality improvement.
• Anatomical/Physiological Measurements / Defined Standards:
  • EasyFACT: Defined criteria for ROI placement within liver parenchyma, avoiding borders/vascular structures.
  • Auto TI Scout, Inline ED/ES Phases Recognition: Gold standard phase indices were presumably established by expert review or a reference method.
  • Inline MOCO & Inline ECV: Ground truth for cardiac left ventricular myocardium segmentation was established by a "well-trained annotator" and "evaluated by three licensed physicians with U.S. credentials" (expert consensus based on anatomical boundaries).
  • EasyRegister (Height/Weight Estimation): "Precisely measured height/weight value" using "physical examination standards."

8. The Sample Size for the Training Set

• ACS: 1,262,912 samples (collected from variety of anatomies, image contrasts, and acceleration factors, scanned by UIH MRI systems).
• DeepRecon: 165,837 samples (collected from 264 volunteers, scanned by UIH MRI systems for multiple body parts and clinical protocols).
• EasyScan: Training data collection not explicitly detailed in the same way as ACS/DeepRecon (refers to "collected independently from the training dataset").
• t-ACS: Datasets collected from 108 volunteers ("large number of samples").
• AiCo: 140,000 images collected from 114 volunteers across multiple body parts and clinical protocols.
• SparkCo: 24,866 spark slices generated from 61 cases collected from 10 volunteers.
• EasyFACT, Auto TI Scout, Inline MOCO, Inline ED/ES Phases Recognition, Inline ECV, EasyRegister, EasyBolus: The document states that training data was independent of testing data but does not provide specific sample sizes for the training datasets for these features.

9. How the Ground Truth for the Training Set was Established

• ACS, DeepRecon, t-ACS, AiCo: "Fully-sampled k-space data were collected and transformed to image space as the ground-truth." For DeepRecon specifically, "multiple-averaged images with high-resolution and high SNR were collected as the ground-truth images." For AiCo, "motionless data" served as gold standard. All training data were "manually quality controlled."
• SparkCo: "The training dataset... was generated by simulating spark artifacts from spark-free raw data... with the corresponding ground truth (i.e., the location of spark points)."
• Inline MOCO & Inline ECV: The document states "all ground truth was annotated by a well-trained annotator. The annotator used an interactive tool to observe the image, and then labeled the left ventricular myocardium in the image."
• For EasyScan, EasyFACT, Auto TI Scout, Inline ED/ES Phases Recognition, EasyRegister, and EasyBolus training ground truth establishment is not explicitly detailed, only that the testing data was independent of the training data. For EasyRegister, it implies physical measurements were the basis for ground truth.

Ask a specific question about this device