(266 days)
MammoScreen® BD is a software application intended for use with compatible full-field digital mammography and digital breast tomosynthesis systems. MammoScreen BD evaluates the breast tissue composition to provide an ACR BI-RADS 5th Edition breast density category. The device is intended to be used in the population of asymptomatic women undergoing screening mammography who are at least 40 years old.
MammoScreen BD only produces adjunctive information to aid interpreting physicians in the assessment of breast tissue composition. It is not a diagnostic software.
Patient management decisions should not be made solely based on analysis by MammoScreen BD.
MammoScreen BD is a software-only device (SaMD) using artificial intelligence to assist radiologists in the interpretation of mammograms. The purpose of the MammoScreen BD software is to automatically process a mammogram to assess the density of the breasts.
MammoScreen BD processes the 2D-mammograms standard views (CC and/or MLO of FFDM and/or the 2DSM from the DBT) to assess breast density.
For each examination, MammoScreen BD outputs the breast density following the ACR BI-RADS 5th Edition breast density category.
MammoScreen BD outputs can be integrated with compatible third-party software such as MammoScreen Suite. Results may be displayed in a web UI, as a DICOM Structured Report, a DICOM Secondary Capture Image, or within patient worklists by the third-party software.
MammoScreen BD takes as input a folder with images in DICOM formats and outputs breast density assessment in a form of a JSON file.
Note that the MammoScreen BD outputs should be used as complementary information by radiologists while interpreting breast density. Patient management decisions should not be made solely on the basis of analysis by MammoScreen BD, the medical professional interpreting the mammogram remains the sole decision-maker.
Here's a breakdown of the acceptance criteria and the study that proves MammoScreen BD meets them, based on the provided FDA 510(k) clearance letter:
Acceptance Criteria and Device Performance Study
The study primarily focuses on the standalone performance of MammoScreen BD in assessing breast density against an expert consensus Ground Truth. The key metric for performance is the quadratically weighted Cohen's Kappa (${\kappa}$).
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Primary Objective: Superiority in standalone performance for density assignment of MammoScreen BD compared to a pre-determined reference value (${\kappa_{\text{reference}} = 0.85}$). | Hologic: ${\kappa_{\text{quadratic}} = 89.03}$ [95% CI: 87.43 – 90.56] |
| Acceptance Criteria (Statistical): The one-sided p-value for the test $H_0: \kappa \leq 0.85$ is less than the significance level ($\alpha=0.05$) AND the lower bound of the 95% confidence interval for Kappa $> 0.85$, indicating that the observed weighted Kappa is statistically significantly greater than 0.85. | Hologic Envision: ${\kappa_{\text{quadratic}} = 89.54}$ [95% CI: 86.88 – 91.69] |
| GE: ${\kappa_{\text{quadratic}} = 93.19}$ [95% CI: 90.50 – 94.92] |
All reported Kappa values exceed the reference value of 0.85, and their 95% confidence intervals' lower bounds are also above 0.85, satisfying the acceptance criteria.
2. Sample Size and Data Provenance
Test Set:
- Hologic (original dataset): 922 patients / 1,155 studies
- Hologic Envision (new system for subject device): 500 patients / 500 studies
- GE (new system for subject device): 376 patients / 490 studies
Data Provenance:
- Hologic (original dataset):
- USA: 658 studies (distributed as A:85, B:269, C:241, D:63)
- EU: 447 studies (distributed as A:28, B:169, C:214, D:86)
- Hologic Envision: USA: 500 studies (distributed as A:50, B:200, C:200, D:50)
- GE:
- USA: 359 studies (distributed as A:38, B:155, C:139, D:31)
- EU: 129 studies (distributed as A:4, B:45, C:61, D:19)
All data for the test sets appears to be retrospective, as it's stated that the "Data used for the standalone performance testing only belongs to the test group" and is distinct from the training data.
3. Number of Experts and Qualifications for Ground Truth
- Number of Experts: 5 breast radiologists
- Qualifications: At least 10 years of experience in breast imaging interpretation.
4. Adjudication Method for the Test Set
The ground truth was established by majority rule among the assessment of the 5 breast radiologists. This implies a 3-out-of-5 or more agreement for a given breast density category to be assigned as ground truth.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
There is no mention of an MRMC comparative effectiveness study being performed to assess how much human readers improve with AI vs. without AI assistance. The study focuses solely on the standalone performance of the AI algorithm. The device is described as "adjunctive information to aid interpreting physicians," but its effect on radiologist performance isn't quantified in this document.
6. Standalone Performance (Algorithm Only)
Yes, a standalone performance study was explicitly conducted. The results for the quadratically weighted Cohen's Kappa presented in the table above (89.03 for Hologic, 89.54 for Hologic Envision, and 93.19 for GE) are all for the algorithm's performance only ("MammoScreen BD against the radiologist consensus assessment").
7. Type of Ground Truth Used
The ground truth used was expert consensus based on the visual assessment of 5 breast radiologists.
8. Sample Size for the Training Set
- Total number of studies: 108,775
- Total number of patients: 32,368
9. How the Ground Truth for the Training Set was Established
The document states that the training modules are "trained with very large databases of annotated mammograms." While "annotated" implies ground truth was established, the specific method for establishing ground truth for the training set is not detailed in the provided text. It only specifies the ground truth establishment method for the test set (majority rule of 5 radiologists). It's common for training data to use various methods for annotation, which might differ from the rigorous expert consensus used for the test set.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).