(101 days)
WRDensity is a software application intended for use with compatible full field digital mammography and digital breast tomosynthesis systems. WRDensity provides an ACR BI-RADS 5th Edition breast density category to aid interpreting physicians in the assessment of breast tissue composition. WRDensity produces adjunctive information. It is not a diagnostic aid.
WRDensity is a standalone software application that automatically analyzes "for presentation" data from digital breast x-ray systems with a deep learning algorithm to assess breast tissue composition. WRDensity primarily generates two outputs for an exam, the Breast Density Level (BDL) and the Breast Density Level Probabilities (BDLP).
The Breast Density Level is a categorical breast density assessment in accordance with the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS®) Atlas 5th Edition breast density categories "A" through "D". The BDL is the primary output of WRDensity.
The Breast Density Level Probabilities are the probabilities calculated by WRDensity for each of the four density categories. The BDLP is a secondary output that provides more information about the breast density of an exam and the device's confidence level.
WRDensity takes in images via a Digital Imaging and Communications in Medicine (DICOM) transfer from the facility's mammography imaging system, Picture Archive and Communication Server (PACS), or DICOM router. After analysis, WRDensity sends outputs to be stored in the PACS and Radiology Information System (RIS). These outputs can then be reviewed by the radiologist on the mammography workstation as a DICOM Secondary Capture Image, a DICOM Structured Report, and in the RIS. These outputs can be configured to match user preferences.
Here's a breakdown of the acceptance criteria and study details for WRDensity by Whiterabbit.ai, based on the provided text:
- 1. A table of acceptance criteria and the reported device performance
| Metric / Category | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Four-Class Task (A,B,C,D) | High agreement with radiologist consensus using quadratically-weighted Cohen's kappa. | Quadratically-weighted Cohen's kappa: 0.90 (95% CI [0.88, 0.92]) |
| BI-RADS A | - | 82% agreement with consensus |
| BI-RADS B | - | 90% agreement with consensus |
| BI-RADS C | - | 85% agreement with consensus |
| BI-RADS D | - | 85% agreement with consensus |
| Binary Task (Dense vs. Non-dense) | High agreement with radiologist consensus using Cohen's kappa. | Cohen's kappa: 0.88 (95% CI [0.85, 0.91]) |
| Non-dense (A+B) | - | 93% agreement with consensus |
| Dense (C+D) | - | 95% agreement with consensus |
| Consistency | High agreement between MLO and CC views of the same breast. | Assessed (Specific value not provided, but stated as evaluated). |
| Reproducibility | Low root mean square error between initial and second processing runs on predicted probabilities. | Assessed (Specific value not provided, but stated as evaluated). |
Note: The document implies "high agreement" as the acceptance criteria for kappa scores and other metrics, as specific numerical thresholds for acceptance criteria are not explicitly stated in this summary portion.
-
2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Test Set Sample Size: 871 exams from unique patients.
- Data Provenance: Not explicitly stated (e.g., country of origin, retrospective/prospective).
-
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
- Number of Experts: Five expert radiologists.
- Qualifications of Experts: Not explicitly stated, only referred to as "expert radiologists."
-
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
- Adjudication Method: "radiologist consensus." This implies a method where multiple radiologists (in this case, five) review cases and come to an agreement on the ground truth. The specific mechanics of how "consensus" was reached (e.g., majority vote, discussion and agreement, or specific rule sets like 3+1) are not detailed.
-
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No MRMC comparative effectiveness study is described where human readers' performance with and without AI assistance is compared. The study focuses solely on the standalone performance of the AI device against a radiologist consensus as ground truth.
-
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
- Yes, a standalone performance study was done. The performance metrics (quadratically-weighted Cohen's kappa, Cohen's kappa for binary classification, and agreement percentages from confusion matrices) directly compare the WRDensity output against the radiologist consensus, without human readers interacting with the AI output.
-
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
- The ground truth used was expert consensus from five expert radiologists.
-
8. The sample size for the training set
- The sample size for the training set is not provided in the given text.
-
9. How the ground truth for the training set was established
- How the ground truth for the training set was established is not provided in the given text.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).