(140 days)
Saige-Density is a software application intended for use with compatible full-field digital mammography (FFDM) and digital breast tomosynthesis (DBT) systems. Saige-Density provides an ACR BI-RADS Atlas 5th Edition breast density category to aid interpreting physicians in the assessment of breast tissue composition. Saige-Density produces adjunctive information. It is not a diagnostic aid.
Saige-Density is Software as a Medical Device that processes screening and diagnostic digital mammograms using deep learning techniques and generates outputs that serve as an aid for interpreting radiologists in assessing breast density. The software takes as input a single x-ray mammogram study and processes all acceptable 2D image DICOM files (FFDM and/or 2D synthetics) and generates a single study-level breast density category. Two DICOM files are outputted as a result: 1) a structured report (SR) DICOM object containing the case-level breast density category and 2) a secondary capture (SC) DICOM object containing a summary report with the study-level density category. Both output files contain the same breast density category ranging from "A" through "D" following Breast Imaging Reporting and Data System (BI-RADS) 5th Edition reporting guidelines. The SC report and/or the SR file may be viewed on a mammography viewing workstation.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
The acceptance criteria are implied by the reported performance metrics of the Saige-Density device. The primary objective of the standalone performance testing was to quantify the accuracy of Saige-Density's density category outputs. The reported performance is the accuracy of the device in classifying breast density into four categories (A, B, C, or D) and two categories (nondense: A, B; dense: C, D) compared to a consensus ground truth.
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Accuracy (Four-class categorization: A, B, C, D) vs. Ground Truth | 81.28% (95% CI: 78.42, 83.84) |
| Accuracy (Two-class categorization: Nondense, Dense) vs. Ground Truth | Implicitly represented by the confusion matrix, not a single percentage explicitly stated for this metric. |
- Nondense correctly classified: 87.8%
- Dense correctly classified: 95.2% |
Study Details
-
Sample size used for the test set and the data provenance:
- Sample Size: A total of 796 mammogram cases (representing 6,170 images) were retrospectively collected for the standalone performance testing.
- Data Provenance: The data was collected from five breast imaging centers in the United States. The collection sites selected for the pivotal study did not overlap with those used previously to collect data for training or testing the Saige-Density AI algorithm.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Five expert radiologists were used to establish the ground truth.
- Qualifications of Experts: The text refers to them as "expert radiologists," implying they are qualified to interpret mammograms, but specific details about their experience (e.g., years of experience) are not provided.
-
Adjudication method for the test set:
- Adjudication Method: Ground truth for each case was established as the consensus of the five expert radiologists' breast density categories on the same set of cases, and calculated as the median of the reported categories for each case. This suggests a form of consensus-based adjudication, specifically using the median.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- The provided text does not indicate that an MRMC comparative effectiveness study was conducted to evaluate human readers' improvement with AI assistance. The performance testing described is "Standalone Performance Testing," focusing on the algorithm's performance only.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance study was explicitly conducted and detailed: "Standalone Performance Testing: A multi-site retrospective study was conducted to evaluate the standalone performance of Saige-Density on DBT and FFDM mammograms."
-
The type of ground truth used:
- The type of ground truth used was expert consensus of five expert radiologists, based on ACR BI-RADS 5th Edition guidelines.
-
The sample size for the training set:
- The exact sample size for the training set is not explicitly stated. However, the text mentions that the training data consisted of "four datasets across various geographic locations within the US."
-
How the ground truth for the training set was established:
- The text does not explicitly describe how the ground truth for the training set was established. It only states that the data used for training the algorithm was distinct from the test set and came from "four datasets across various geographic locations within the US, including racially diverse regions such as New York City and Los Angeles."
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).