(29 days)
TumorSight Viz is intended to be used in the visualization and analysis of breast magnetic resonance imaging (MRI) studies for patients with biopsy proven early-stage or locally advanced breast cancer. TumorSight Viz supports evaluation of dynamic MR data acquired from breast studies during contrast administration. TumorSight Viz performs processing functions (such as image registration, subtractions, measurements, 3D renderings, and reformats).
TumorSight Viz also includes user-configurable features for visualizing and analyzing findings in breast MRI studies. Patient management decisions should not be made based solely on the results of TumorSight Viz.
TumorSight Viz is an image processing system designed to assist in the visualization and analysis of breast DCE-MRI studies.
TumorSight reads DICOM magnetic resonance images. TumorSight processes and displays the results on the TumorSight web application.
Available features support:
- Visualization (standard image viewing tools, MIPs, and reformats)
- Analysis (registration, subtractions, kinetic curves, parametric image maps, segmentation and 3D volume rendering)
The TumorSight system consists of proprietary software developed by SimBioSys, Inc. hosted on a cloud-based platform and accessed on an off-the-shelf computer.
Here's a breakdown of the acceptance criteria and the study details for the TumorSight Viz device, based on the provided FDA 510(k) clearance letter:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are implicitly defined by the reported performance metrics, where the device's performance is deemed "adequate" and "clinically acceptable" if the variability is similar to inter-radiologist variability or differences in error are clinically insignificant.
| Measurement Description | Units | Acceptance Criterion (Implicit) | Reported Device Performance (Mean Abs. Error ± Std. Dev.) |
|---|---|---|---|
| Tumor Volume (n=218) | cubic centimeters (cc) | Similar to inter-radiologist variability | 5.2 ± 12.5 |
| Tumor-to-breast volume ratio (n=218) | % | Clinically acceptable | 0.4 ± 1.2 |
| Tumor longest dimension (n=242) | centimeters (cm) | Similar to inter-radiologist variability (e.g., 1.02 cm ± 1.33 cm) | 1.32 ± 1.65 |
| Tumor-to-nipple distance (n=241) | centimeters (cm) | Similar to inter-radiologist variability (e.g., 0.88 cm ± 1.12 cm) | 1.17 ± 1.55 |
| Tumor-to-skin distance (n=242) | centimeters (cm) | Similar to inter-radiologist variability (e.g., 0.42 cm ± 0.45 cm) | 0.60 ± 0.52 |
| Tumor-to-chest distance (n=242) | centimeters (cm) | Similar to inter-radiologist variability (e.g., 0.79 cm ± 1.14 cm) | 0.86 ± 1.22 |
| Tumor center of mass (n=218) | centimeters (cm) | Clinically acceptable | 0.60 ± 1.47 |
| Segmentation Accuracy | |||
| Volumetric Dice (n=218) | High agreement with reference standard | 0.76 ± 0.26 | |
| Surface Dice (n=218) | High agreement with reference standard (particularly for 3D rendering) | 0.92 ± 0.21 |
The document states: "We found that all tests met the acceptance criteria, demonstrating adequate performance for our intended use." This indicates that the reported performance metrics were considered acceptable by the regulatory body. For measurements where inter-radiologist variability is provided (e.g., longest dimension, tumor-to-skin), the device's error is compared to this variability. For other metrics, the acceptance is based on demonstrating "adequate performance," implying that the reported values themselves were within a predefined acceptable range.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 266 patients (corresponding to 267 samples, accounting for bilateral disease).
- Data Provenance:
- Country of Origin: U.S.
- Retrospective/Prospective: The document does not explicitly state "retrospective" or "prospective." However, the description of "DCE-MRI were obtained from... patients" and establishment of ground truth by reviewing images suggests a retrospective acquisition of data for validation. The mention of "All patients had pathologically confirmed invasive, early stage or locally advanced breast cancer" further supports a retrospective gathering of existing patient data.
- Clinical Sites: More than eight (8) clinical sites in the U.S.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
- Number of Experts: Three (3) U.S. Board Certified radiologists.
- Qualifications: U.S. Board Certified radiologists. (No specific experience in years is mentioned, but Board Certification implies a high level of expertise.)
4. Adjudication Method for the Test Set
- Adjudication Method: 2+1 (as described in the document).
- For each case, two radiologists independently measured various characteristics and determined if the candidate segmentation was appropriate.
- In cases of disagreement between the first two radiologists ("did not agree on whether the segmentation was appropriate"), a third radiologist provided an additional opinion, and the ground truth was established by majority consensus.
5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done
The document does not describe an MRMC comparative effectiveness study where human readers' performance with and without AI assistance is directly measured and compared.
Instead, it compares the device's performance to:
- Ground Truth: Radiologist consensus measurements.
- Predicate Device: Its own previous version.
- Inter-radiologist Variability: The inherent variability between human expert readers.
Therefore, no effect size of how much human readers improve with AI vs. without AI assistance is provided, as this type of MRMC study was not detailed.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, a standalone performance study was done. The sections titled "Performance Tests" and the tables detailing "Validation Testing (Mean Abs. Error ± Std. Dev.)" describe the algorithm's performance in comparison to the established ground truth. This is a standalone evaluation, as it assesses the device's output intrinsically against expert-derived truth without measuring human interaction or improvement. The statement "The measurements generated from the device result directly from the segmentation methodology and are an inferred reflection of the performance of the deep learning algorithm" supports this.
7. The Type of Ground Truth Used
- Type of Ground Truth: Expert Consensus (specifically, pathologist-confirmed lesions measured and evaluated by a consensus of U.S. Board Certified radiologists). The initial diagnosis of early-stage or locally advanced breast cancer for patient selection was based on pathology ("biopsy proven"). However, the ground truth for measurements and segmentation appropriateness for the study was established by radiologists.
8. The Sample Size for the Training Set
- Sample Size for Training Set: One thousand one hundred fifty-six (1156) patients/samples.
9. How the Ground Truth for the Training Set Was Established
The document states: "DCE-MRI were obtained from one thousand one hundred fifty-six (1156) patients from more than fifteen (15) clinical sites in the U.S. for use in training and tuning the device."
However, the document does not explicitly detail how the ground truth for this training set was established. It describes the ground truth establishment method only for the validation dataset (by three U.S. Board Certified radiologists with 2+1 adjudication). For training data, it is common practice to use similar rigorous methods for labeling, but the specifics are not provided in this excerpt.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).