(291 days)
Yes
The summary explicitly states "Artificial intelligence algorithm" under the "Mentions AI, DNN, or ML" section.
No.
This device is a software application designed to assist physicians in analyzing breast ultrasound images for diagnostic purposes, not to provide therapy or treatment.
Yes
The device aids trained interpreting physicians in analyzing breast ultrasound images to identify and assess suspicious soft tissue lesions for breast cancer, providing region-based analysis of lesion malignancy, scores of lesion characteristics (SLC), and corresponding BI-RADS categories. This direct assistance in identifying characteristics of disease indicates a diagnostic purpose.
Yes
The device description explicitly states that BU-CAD is a "software system" and describes its components as software modules (Viewer, Lesion Identification Module, Lesion Analysis Module). It processes digital images and outputs analysis and reports, without mentioning any dedicated hardware components included with the device.
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs are used to examine specimens derived from the human body. The intended use and device description clearly state that BU-CAD analyzes images of the breast (ultrasound and mammography), not biological samples like blood, tissue, or urine.
- IVDs provide information about a patient's health status based on the analysis of these specimens. While BU-CAD assists physicians in analyzing images to identify suspicious lesions and assess malignancy, it does not directly analyze biological specimens to provide diagnostic information. It's a tool for image interpretation.
- The device description and intended use focus on image processing and analysis. The core function is to process and analyze medical images, not biological samples.
Therefore, BU-CAD falls under the category of a medical device that aids in image interpretation, rather than an In Vitro Diagnostic device.
No
The letter does not explicitly state that the FDA has reviewed and approved or cleared a PCCP for this specific device. The relevant section indicates "Not Found" for PCCP authorization.
Intended Use / Indications for Use
BU-CAD is a software application indicated to assist trained interpreting physicians in analyzing the breast ultrasound images of patients with soft tissue breast lesions suspicious for breast cancer who are being referred for further diagnostic ultrasound examination.
Output of the device includes regions of interest (ROIs) and lesion contours placed on breast ultrasound images assisting physicians to identify suspicious soft tissue lesions from up to two orthogonal views of a single lesion, and region-based analysis of lesion malignancy upon the physician's query. The region-based analysis indicates the score of lesion characteristics (SLC), and corresponding BI-RADS categories in user-selected ROIs or ROIs automatically identified by the software. In addition, BU-CAD also automatically classifies lesion shape, orientation, margin, echo pattern, and posterior features according to BI-RADS descriptors.
BU-CAD may also be used as an image viewer of multi-modality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and document images, and output into a structured report (SR).
Patient management decisions should not be made solely on the basis of analysis by BU-CAD.
Limitations: BU-CAD is not to be used on sites of post-surgical excision, or images with Doppler, elastography, or other overlays present in them. BU-CAD is not intended for the primary interpretation of digital mammography images. BU-CAD is not intended for use on mobile devices.
Product codes
QDQ, LLZ
Device Description
BU-CAD developed by TaiHao Medical Inc. is a software system designed to assist users in analyzing breast ultrasound images including identification of regions suspicious for breast cancer and assessment of their malignancy. The following figure shows the architecture chart of BU-CAD which consists of a Viewer, a Lesion Identification Module, and a Lesion Analysis Module.
The Viewer is able to load breast ultrasound and mammography images (FDA-cleared fullfield digital mammography only) from local storage or a picture archiving and communication system (PACS) for review. The Viewer also includes tools that allow users to measure lession size and adjust the image (such as window level and window width adjustment). Additionally, the report may be saved in local storage or uploaded to PACS. BU-CAD also supports exporting CAD results to third-party reporting software to facilitate the reporting process.
The Lesion Identification Module identifies regions of interest (automated ROIs) of a single suspicious soft tissue lesion in up to two orthogonal views of breast ultrasound images for assisting users in detecting soft tissue lesions. Additionally, the Lesion Identification Module generates an ROI and a lesion contour on each breast ultrasound image. The lesion contour on each image will be automatically delineated by the given ROI. The Lesion Analysis Module analyzes given ROIs of a breast lesion on ultrasound images, and generates a score of lesion characteristics (SLC) in terms of malignancy or benignity of a lesion, BI-RADS category, and BI-RADS descriptors (with limitations as described in the User Manual) for the concurrent read. The users are able to replace the automated ROIs with re-delineated rectangular ROIs for analysis by Lesion Analysis Module. Only the last analysis results will be displayed on the user interface and are modifiable by the user. Note that the SLC is analyzed based on the rectangular ROIs, unless the user re-delineates the ROIs, the SLC will not be changed.
In clinical practice, after opening multi-modality digital images including ultrasound and mammography on the Viewer, the users may identify and analyze lesions with the assistance of the Lesion Identification Module and Lesion Analysis Module on the breast ultrasound images. Finally, the user confirms the diagnostic results (output from Lesion Analysis Module or modified by the user) shown on the user interface and saves them to the report.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Yes
Input Imaging Modality
Breast ultrasound data, Mammography
Anatomical Site
Breast
Indicated Patient Age Range
Not Found
Intended User / Care Setting
Trained interpreting physicians
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
A total of 1139 cases (628 reader study cases plus 511 extended cases) collected from multiple institutions were used in the standalone study. The source of cases is listed below.
- North America: 531 cases
- Europe: 36 cases
- Taiwan: 572 cases
The BI-RADS category distribution included in this study were listed below: - BI-RADS 2: 31 cases
- BI-RADS 3: 223 cases
- BI-RADS 4A: 356 cases
- BI-RADS 4B: 218 cases
- BI-RADS 4C: 181 cases
- BI-RADS 5: 130 cases
The number of benign and malignant cases included in this study were listed below. - Benign cases
- Pathology proof benign: 465 cases
- Two-year follow-up benign: 177 cases
- Malignant cases
- Ductal carcinomas in situ (DCIS): 53 cases
- invasive ductal carcinoma (IDC): 361 cases
- Invasive lobular carcinoma (ILC): 51 cases
- Other cancer types: 32 cases
The imaging hardware distribution included in this study were listed below:
- GE: 634 cases
- Siemens: 188 cases
- Canon/Toshiba: 90 cases
- Philips: 111 cases
- Supersonic: 24 cases
- Others: 92
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Summary of the Reader Study
The performance of physicians without and with the aid of BU-CAD decision support in interpreting breast ultrasound images was compared by using a fully crossed multi-reader multi-case receiver operating characteristic (MRMC-ROC) retrospective study (also known as Obuchowski-Rockette Dorfman-Berbaum-Metz MRMC-ROC or OR-DBM MRMC-ROC).
The study consisted of 628 cases, of which 456 cases (189 malignant and 267 benign) were collected from the United States and 172 cases (65 malignant and 107 benign) were collected from Taiwan. Sixteen readers participated in the study. Each reader was asked to identify the lesion, provide a linear score of lesion characteristics (SLC), select a BI-RADS category and select BI-RADS descriptors for an ultrasound breast lesion with or without the aid of BU-CAD.
Primary Objective
The primary objective of this clinical study is to prove that the user's performance (AUC of location-specific ROC) aided by the BU-CAD software is superior to the unaided performance. The aided AUC of the location-specific ROC for BU-CAD was superior to that of the unaided scenario for the diagnosis of breast ultrasound images. The mean AUC of location-specific ROC shift of 0.0374.
| Reading Scenario | AUC_LROC | 95% CI | p-value |
|---|---|---|---|
| Unaided | 0.7786 | (0.7463, 0.8109) | |
| Aided | 0.8160 | (0.7862, 0.8458) | |
| Aided - Unaided | 0.0374 | (0.0190, 0.0557) | 0.0001 |
Secondary Objective
The secondary objective of this clinical study is to compare that the user's performance (sensitivity, specificity, PPV, and NPV) between the unaided and aided readings. Sensitivity, specificity, PPV, and NPV produced from the aided arm were higher than unaided. The specificity, unadjusted PPV, and unadjusted NPV differed sigmificantly from zero between the aided and unaided sessions.
BU-CAD software was found to significantly decrease readers' interpretation times (by ~40%) which was shown in analyses including and excluding outliers. Statistical analyses also indicated that BU-CAD improved readers' determination of BI-RADS descriptors (Shape, Orientation, Margin, Echo Pattern, and Posterior Features), where at least one or more subcategories for each descriptor demonstrated improved aided read performance, with limitations described in the User Manual.
Summary of the Standalone Study
A total of 1139 cases (628 reader study cases plus 511 extended cases) collected from multiple institutions were used in the standalone study.
Lesion Identification Module (CADe) Performance
A total of 59 benign cases (including 11 of the 20 missing cases) and 18 malignant cases (including 9 of the 20 missing cases) did not meet the objective performance criteria (automated ROI center must be within ground truth ROI with at least 50% overlap in ROI area). The accuracy of the lesion identification algorithm was 93.24% (1062/1139). For the LROC analysis, 18 malignant cases were penalized due to wrong location or undetected by BU-CAD.
Comparison between Standalone and Unaided Reading Performance
The standalone performance of BU-CAD was measured in AUC LROC on the 628 reader study cases and the standalone study cases (combined the 628 reader study cases and 511 extended cases), a total of 1,139 cases (497 malignant and 642 benign). Table below shows the standalone AUC LROCs in both datasets are higher than that of unaided reading performance.
| Reading Scenario | AUC LROC | 95% CI |
|---|---|---|
| BU-CAD Standalone (628 reader study cases) | 0.7987 | (0.7626, 0.8348) |
| BU-CAD Standalone (1,139 standalone study cases) | 0.8203 | (0.7947, 0.8458) |
| Unaided Reading (628 reader study cases) | 0.7786 | (0.7463, 0.8109) |
Robustness of the Lesion Analysis Module (CADx)
To evaluate the robustness of the CADx algorithm (Lesion Analysis Module) when different rectangular ROIs are drawn around the same lesion on a given single-view image or two-view images, two reproducibility experiments of the same lesion cropped by different rectangular ROIs were conducted. In the first reproducibility experiment, each corner point of an ROI was shifted by randomly changing the horizontal and vertical dimensions up to 20% respectively from the ground truth ROI defined by the expert panel. The experiment was repeated 20 times with all 1139 test cases (the original dataset was 628 cases and the extended dataset was 511 cases). The results show that randomly enlarging the width and height of the ROIs did not affect the performance of the BU-CAD CADx algorithm (Lesion Analysis Module). The AUC remained stable between 0.840 and 0.846.
In the second reproducibility experiment, each corner point of ground truth ROI was altered by systematically shrinking the horizontal and vertical dimensions respectively from 1% to 30%. The experiment was conducted with all 1139 cases. The new ROIs and their corresponding images were then processed by the BU-CAD CADx algorithm (Lesion Analysis Module) to produce analysis outputs. The results show that as long as the shrinking percentage of the width and height of the ROIs is within 16%, the AUC remained above 0.8.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Secondary Objective (Reader Study)
| Statistical Parameter | Unaided (95% CI) | Aided (95% CI) |
|---|---|---|
| Sensitivity | 0.9225 (0.8896, 0.9554) | 0.9353 (0.9050, 0.9655) |
| Specificity | 0.3165 (0.2694, 0.3636) | 0.3611 (0.3124, 0.4098) |
| NPV (unadjusted) | 0.8623 (0.8048, 0.9198) | 0.8945 (0.8456, 0.9434) |
| NPV_U.S. (adjusted) | 0.9982 (0.9902, 1.0000) | 0.9986 (0.9918, 1.0000) |
| NPV_Taiwan (adjusted) | 0.9969 (0.9767, 1.0000) | 0.9975 (0.9809, 1.0000) |
| PPV (unadjusted) | 0.4876 (0.4433, 0.5319) | 0.5056 (0.4607, 0.5505) |
| PPV_U.S. (adjusted) | 0.0108 (-0.0001, 0.0216) | 0.0113 (0.0000, 0.0225) |
| PPV_Taiwan (adjusted) | 0.0256 (-0.0002, 0.0514) | 0.0283 (0.0006, 0.0560) |
Standalone Study (Sensitivity, Specificity, PPV, NPV)
| Statistical Parameter | Standalone (Frequency) | 95% CI |
|---|---|---|
| Sensitivity (%) | 88.33 (439/497) | (0.8551, 0.9115) |
| Specificity (%) | 57.94 (372/642) | (0.5413, 0.6176) |
| PPV (%) [unadjusted] | 61.92 (439/709) | (0.5834, 0.6549) |
| PPV_US (%) | 1.28 | (0.0011, 0.0245)* |
| PPV_TW (%) | 4.74 | (0.0246, 0.0703)* |
| NPV (%) [unadjusted] | 86.51 (372/430) | (0.8328, 0.8974) |
| NPV_US (%) | 99.82 | (0.9921, 1.0000)* |
| NPV_TW (%) | 99.67 | (0.9895, 1.0000)* |
- The 95% Confidence Interval (CI) was estimated conditioning on the obtained prevalence rates of 0.72% and 1.94% in U.S. and Taiwan, respectively.
Predicate Device(s)
TransparaTM (K181704), QuantX (K170195)
Reference Device(s)
Koios DS for Breast (K190442)
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 892.2090 Radiological computer-assisted detection and diagnosis software.
(a)
Identification. A radiological computer-assisted detection and diagnostic software is an image processing device intended to aid in the detection, localization, and characterization of fracture, lesions, or other disease-specific findings on acquired medical images (e.g., radiography, magnetic resonance, computed tomography). The device detects, identifies, and characterizes findings based on features or information extracted from images, and provides information about the presence, location, and characteristics of the findings to the user. The analysis is intended to inform the primary diagnostic and patient management decisions that are made by the clinical user. The device is not intended as a replacement for a complete clinician's review or their clinical judgment that takes into account other relevant information from the image or patient history.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithm, including a description of the algorithm inputs and outputs, each major component or block, how the algorithm and output affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide improved assisted-read detection and diagnostic performance as intended in the indicated user population(s), and to characterize the standalone device performance for labeling. Performance testing includes standalone test(s), side-by-side comparison(s), and/or a reader study, as applicable.
(iii) Results from standalone performance testing used to characterize the independent performance of the device separate from aided user performance. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, positive and negative predictive values, and diagnostic likelihood ratio). Devices with localization output must include localization accuracy testing as a component of standalone testing. The test dataset must be representative of the typical patient population with enrichment made only to ensure that the test dataset contains a sufficient number of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant disease, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Results from performance testing that demonstrate that the device provides improved assisted-read detection and/or diagnostic performance as intended in the indicated user population(s) when used in accordance with the instructions for use. The reader population must be comprised of the intended user population in terms of clinical training, certification, and years of experience. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, positive and negative predictive values, and diagnostic likelihood ratio). Test datasets must meet the requirements described in paragraph (b)(1)(iii) of this section.(v) Appropriate software documentation, including device hazard analysis, software requirements specification document, software design specification document, traceability analysis, system level test protocol, pass/fail criteria, testing results, and cybersecurity measures.
(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the device instructions for use, including the intended reading protocol and how the user should interpret the device output.
(iii) A detailed description of the intended user, and any user training materials or programs that address appropriate reading protocols for the device, to ensure that the end user is fully aware of how to interpret and apply the device output.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) A detailed summary of the performance testing, including test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders, such as anatomical characteristics, patient demographics and medical history, user experience, and imaging equipment.
0
December 21, 2021
Image /page/0/Picture/1 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
TaiHao Medical Inc. % Hsin Hung President 6F .- 1, No.100, Sec. 2 Heping E. Rd., Da'an Dist. Taipei. 10663 TAIWAN
Re: K210670
Trade/Device Name: BU-CAD Regulation Number: 21 CFR 892.2090 Regulation Name: Radiological computer assisted detection and diagnosis software Regulatory Class: Class II Product Code: QDQ, LLZ Dated: November 22, 2021 Received: November 22, 2021
Dear Hsin Hung:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for
1
devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely.
For
Thalia T. Mills, Ph.D. Director Division of Radiological Health OHT7: Office of In Vitro Diagnostics and Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K210670
Device Name BU-CAD
Indications for Use (Describe)
BU-CAD is a software application indicated to assist trained interpreting physicians in analyzing the breast ultrasound images of patients with soft tissue breast lesions suspicious for breast cancer who are being referred for further diagnostic ultrasound examination.
Output of the device includes regions of interest (ROIs) and lesion contours placed on breast ultrasound images assisting physicians to identify suspicious soft tissue lesions from up to two orthogonal views of a single lesion, and region-based analysis of lesion malignancy upon the physician's query. The region-based analysis indicates the score of lesion characteristics (SLC), and corresponding BI-RADS categories in user-selected ROIs or ROIs automatically identified by the software. In addition, BU-CAD also automatically classifies lesion shape, orientation, margin, echo pattern, and posterior features according to BI-RADS descriptors.
BU-CAD may also be used as an image viewer of multi-modality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and document images, and output into a structured report (SR).
Patient management decisions should not be made solely on the basis of analysis by BU-CAD.
Limitations: BU-CAD is not to be used on sites of post-surgical excision, or images with Doppler, elastography, or other overlays present in them. BU-CAD is not intended for the primary interpretation of digital mammography images. BU-CAD is not intended for use on mobile devices.
| Type of Use (Select one or both, as applicable) |
|---|
| Prescription Use (Part 21 CFR 801 Subpart D) |
| Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
510(K) Summary
I. Identification of Submitter
K210670
| Submitter: | TaiHao Medical Inc. |
|---|---|
| Address: | 6F.-1, No.100, Sec. 2, Heping E. Rd., Da'an Dist., Taipei City 106, Taiwan |
| (R.O.C.) | |
| Phone: | 886-2-2736-5679 |
| Contact: | HSIN HUNG (Simon) LAI |
|---|---|
| Title: | President |
| Phone: | 886-2-2736-5679 |
| Email: | simonlai@taihaomed.com |
| Manufacturer: | TaiHao Medical Inc. |
| Additional
| Contact: | HONG HAO CHEN, Ph.D. |
|---|---|
| Title: | Regulatory Affairs Manager |
| Address: | 6F.-1, No.100, Sec. 2, Heping E. Rd., Da'an Dist., Taipei City 106, Taiwan |
| (R.O.C.) | |
| Phone: | 886-2-2736-5679 |
| Email: | honghowc@taihaomed.com |
| Date of Prepared | December 16, 2021 |
II. Identification of Product
| Device Name: | BU-CAD |
|---|---|
| Regulation Number: | 892.2090 |
| Device Classification: | Class II |
| Classification Product Code: QDQ | |
| Subsequent Product Code: LLZ | |
| Classification Name: | Radiological Computer Assisted Detection/Diagnosis Software For |
| Lesions Suspicious For Cancer | |
| Review Panel: | Radiology |
| Manufacturer: | TaiHao Medical Inc. |
III. Predicate Device
| Predicate Device: | TransparaTM (K181704) (primary), QuantX (K170195) |
|---|---|
| Reference Device: | Koios DS for Breast (K190442) |
4
Device Description IV.
BU-CAD developed by TaiHao Medical Inc. is a software system designed to assist users in analyzing breast ultrasound images including identification of regions suspicious for breast cancer and assessment of their malignancy. The following figure shows the architecture chart of BU-CAD which consists of a Viewer, a Lesion Identification Module, and a Lesion Analysis Module.
Image /page/4/Figure/2 description: The image shows a diagram of the BU-CAD system. The input side shows local storage and PACS/RIS, which feed DICOM images into the BU-CAD system. The BU-CAD system consists of a viewer, lesion identification module, and lesion analysis module. The output side shows local storage and PACS, which receive the last diagnostic results.
Architecture chart of BU-CAD
The Viewer is able to load breast ultrasound and mammography images (FDA-cleared fullfield digital mammography only) from local storage or a picture archiving and communication system (PACS) for review. The Viewer also includes tools that allow users to measure lession size and adjust the image (such as window level and window width adjustment). Additionally, the report may be saved in local storage or uploaded to PACS. BU-CAD also supports exporting CAD results to third-party reporting software to facilitate the reporting process.
The Lesion Identification Module identifies regions of interest (automated ROIs) of a single suspicious soft tissue lesion in up to two orthogonal views of breast ultrasound images for assisting users in detecting soft tissue lesions. Additionally, the Lesion Identification Module generates an ROI and a lesion contour on each breast ultrasound image. The lesion contour on each image will be automatically delineated by the given ROI. The Lesion Analysis Module analyzes given ROIs of a breast lesion on ultrasound images, and generates a score of lesion characteristics (SLC) in terms of malignancy or benignity of a lesion, BI-RADS category, and BI-RADS descriptors (with limitations as described in the User Manual) for the concurrent read. The users are able to replace the automated ROIs with re-delineated rectangular ROIs for
5
analysis by Lesion Analysis Module. Only the last analysis results will be displayed on the user interface and are modifiable by the user. Note that the SLC is analyzed based on the rectangular ROIs, unless the user re-delineates the ROIs, the SLC will not be changed.
In clinical practice, after opening multi-modality digital images including ultrasound and mammography on the Viewer, the users may identify and analyze lesions with the assistance of the Lesion Identification Module and Lesion Analysis Module on the breast ultrasound images. Finally, the user confirms the diagnostic results (output from Lesion Analysis Module or modified by the user) shown on the user interface and saves them to the report.
| Region-based Analysis Item | Range |
|---|---|
| Score of lesion characteristics (SLC) | [0,100] |
| The SLC ranging from 0 to 25 corresponds to BI-RADS 2, | |
| from 26 to 50 corresponds to BI-RADS 3, from 51 to 97 | |
| corresponds to BI-RADS 4, and from 98 to 100 corresponds | |
| to BI-RADS 5. | |
| BI-RADS category | 2 / 3 / 4a / 4b / 4c / 5 |
| BI-RADS descriptors (mass) | Shape, Orientation, Margin, Echo Pattern, Posterior Features |
| (with limitations specified in User Manual) |
Output of BU-CAD analysis
Indications for Use V.
BU-CAD is a software application indicated to assist trained interpreting physicians in analyzing the breast ultrasound images of patients with soft tissue breast lesions suspicious for breast cancer who are being referred for further diagnostic ultrasound examination.
Output of the device includes regions of interest (ROIs) and lesion contours placed on breast ultrasound images assisting physicians to identify suspicious soft tissue lesions from up to two orthogonal views of a single lesion, and region-based analysis of lesion malignancy upon the physician's query. The region-based analysis indicates the score of lesion characteristics (SLC), and corresponding BI-RADS categories in user-selected ROIs or ROIs automatically identified by the software. In addition, BU-CAD also automatically classifies lesion shape, orientation, margin, echo pattern, and posterior features according to BI-RADS descriptors.
BU-CAD may also be used as an image viewer of multi-modality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and document images, and output into a structured report (SR).
Patient management decisions should not be made solely on the basis of analysis by BU-CAD.
6
Limitations: BU-CAD is not to be used on sites of post-surgical excision, or images with Doppler, elastography, or other overlays present in them. BU-CAD is not intended for the primary interpretation of digital mammography images. BU-CAD is not intended for use on mobile devices.
| | BU-CAD | TransparaTM
(K181704)
Predicate Device | QuantX
(K170195)
Predicate Device |
|------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Manufacturer | TaiHao Medical Inc. | ScreenPoint Medical
BV | Quantitative Insights,
Inc. |
| Regulation
Section | 21 CFR 892.2090 | 21 CFR 892.2090 | 21 CFR 892.2050 |
| Product Code | QDQ, LLZ | QDQ | LLZ |
| Intended Use | Intended to be used by
clinicians interpreting
radiological images, to
help them with
localizing and
characterizing breast
abnormalities.
Intended to be used
concurrently with the
reading of images and
are not intended as a
replacement for the
review of a clinician
or their clinical
judgement. | Intended to be used by
clinicians interpreting
radiological images, to
help them with
localizing and
characterizing breast
abnormalities.
Intended to be used
concurrently with the
reading of images and
are not intended as a
replacement for the
review of a clinician
or their clinical
judgement. | QuantX is a
quantitative image
analysis software
device used to assist
radiologists in the
assessment and
characterization of
breast abnormalities
using MR image data. |
| Characteristics | CADe and CADx
software used to assist
in localizing
suspicious soft tissue
lesions and region-
based analyze of
malignancy using
ultrasound image data. | CADe and CADx
software used to assist
in localizing
suspicious soft tissue
lesions and suspicious
calcifications; region-
based analyze of
malignancy using
mammography image
data. | The software
automatically registers
images, and segments
and analyzes user-
selected regions of
interest (ROI). QuantX
extracts image data
from the ROI to
provide volumetric
and surface area
analysis. |
| Target
Population | Patients with soft
tissue breast lesions
who are being referred
for ultrasound
interpretation. | Patients with soft
tissue breast lesions
and suspicious
calcifications who are
being referred for | Patients who are being
referred for breast
MRI interpretation. |
| | BU-CAD | TransparaTM
(K181704)
Predicate Device | QuantX
(K170195)
Predicate Device |
| Anatomical
Location | Breast | Breast | Breast |
| Design | Software-only device | Software-only device | Software-only device |
| Modality Used
for Analysis | Breast ultrasound data | Mammography | Breast MRI |
| Input | Medical images
provided in a DICOM
format | Medical images
provided in a DICOM
format | Medical images
provided in a DICOM
format |
| Output | ROIs and lesion
contours placed on
suspicious soft tissue
lesion.
A region-based score
of lesion malignancy,
a BI-RADS category,
and BI-RADS
descriptors. | Marks placed on
suspicious soft tissue
lesion and suspicious
calcifications.
A region-based score
of lesion malignancy,
and an overall score of
the mammogram. | QuantX extracts image
data from the ROI to
provide volumetric
and surface area
analysis. |
| Physical
Characteristics | Software Package
Operates on off-the-
shelf hardware | Software Package
Operates on off-the-
shelf hardware | Software Package
Operates on off-the-
shelf hardware |
| Comparative
Performance
Testing
(MRMC) | Metric: AUC
Cases: 628
Readers: 16 | Metric: AUC
Cases: 240
Readers: 14 | N/A |
| Modality Used
for Viewing | Breast Ultrasound and
Mammography
(FFDM) | N/A | Breast MRI, breast
ultrasound, and
mammography |
| Primary
Interpretation
of Digital
Mammography
Images | BU-CAD is not
intended for the
primary interpretation
of digital
mammography
images | N/A | QuantX is not
intended for primary
interpretation of
digital mammography
images |
Comparison with Predicate Device/Reference Device VI.
7
Intended Use �
The intended use of BU-CAD is the same as that of the legally marketed predicate device, Transpara™. Both are intended to be used by clinicians interpreting radiological images, to help them with localizing and characterizing breast abnormalities. BU-CAD and the predicate device are both intended to be used concurrently with the reading of images and are not intended as a replacement for the review of a clinician or their clinical judgement.
8
� Indications for Use
Both BU-CAD and Transpara™ are intended to identify regions suspicious for breast cancer and provide computer analytics that are then synthesized by an artificial intelligence algorithm into a single value. Both BU-CAD and Transpara™ generate region-based scores indicating the malignancy. Both BU-CAD and Koios DS for Breast characterizes a lesion based on categorical output and auto-classifies BI-RADS descriptors.
Intended Use Population and Modality �
BU-CAD and Transpara™ differ in the type of medical images the devices process, however, they are both aligned to the generic FDA device type for radiological computer-assisted detection and diagnosis for lesions suspicious for cancer. Transpara™ is intended for aiding physicians interpreting screening mammograms, while BU-CAD is intended for aiding physicians interpreting diagnostic ultrasound examination.
BU-CAD shares the intended use population and modality requirements of Koios DS for Breast. Both BU-CAD and the Koios DS for Breast are intended to be used for assisting trained interpreting physicians in analyzing patients with soft tissue breast lesions which are being referred for further diagnostic ultrasound examination.
BU-CAD and QuantX may also be used as image viewers of multi-modality digital images, including ultrasound and mammography, and are not intended for the primary interpretation of digital mammography images.
� Input
According to the respective device descriptions of Transpara™ and BU-CAD, the input to each consists of medical images provided in a DICOM format. While there are modality differences that are addressed above, the technical implementation for ingesting images for processing occurs via the same DICOM based interface.
Both BU-CAD and Koios DS for Breast analyzes breast lesion from up to two orthogonal views of a single lesion.
� Output
The outputs between BU-CAD and Transpara™ are not exactly the same. Outputs of Transpara™ consist of highlighted locations of detected suspicious soft tissue lesions and suspicious calcifications, and region-based scores (Transpara™ Score). Output of BU-CAD
9
consists of highlighted locations (ROI(s) and lesion contour(s)) of detected suspicious soft tissue lesion, region-based score (SLC), BI-RADS category, and BI-RADS descriptors.
Both BU-CAD and Koios DS for Breast characterizes a lesion based on categorical output and auto-classifies BI-RADS descriptors.
� Interface
Transpara™ consists of a processing server and an optional viewer. Processing results of Transpara™ can be transmitted to external destinations that allows PACS workstations to implement the interface of Transpara™ in mammography reading applications. BU-CAD is intended to be used as an image viewer of multi-modality digital images which provides analysis of breast ultrasound abnormalities.
Both BU-CAD and Koios DS for Breast are intended to be used as image viewers of multimodality digital images including ultrasound and mammography. Both sets of software include tools to allow users to measure and document images, and output the findings in structured DICOM formats.
� Performance Testing
Both BU-CAD and Koios DS for Breast are intended to be used for assisting trained interpreting physicians in analyzing patients with soft tissue breast lesions which are being referred for further diagnostic ultrasound examination.
When comparing clinical validation between BU-CAD, Transpara™, and Koios DS for Breast, the devices were evaluated using similar endpoints in their clinical studies and the Area Under the Curve (AUC) shift was used when comparing the performance of users alone versus users with the aid of the software platform. The number of cases evaluated in the MRMC reader study of Koios DS for Breast was 750 (150 additional cases for intra-operator variability evaluation without switching the reading condition), the number of cases evaluated in the MRMC reader study of Transpara™ was 240, while the BU-CAD MRMC reader study evaluated a total of 628 cases. The number of readers utilized in the Koios DS for Breast MRMC reader study was 15 (11 radiologist, 2 breast surgeon, and 2 OB/GYN), the number of readers utilized in the Transpara™ MRMC reader study was 14 radiologists, while the BU-CAD MRMC reader study used a total of 16 readers (14 radiologists and 2 breast surgeons).
10
The AUC shift between users alone and users with the aid of the software platforms (Transpara™, Koios DS for Breast, and BU-CAD) was similar. The results of Koios DS for Breast MRMC reader study showed a mean AUC shift of +0.037, the results of Transpara™ MRMC reader study showed mean AUC shift of +0.02, the BU-CAD MRMC reader study showed a mean shift +0.0374.
The standalone performance of BU-CAD reported an AUC LROC of 0.8203 (AUC from 0.8 to 0.9) compared to the reference device Koios DS for Breast of 0.882. For the BI-RADS descriptors, Koios DS for Breast provided BI-RADS descriptors of Shape and Orientation which the level of agreement between readers is similar to the agreement between readers and system. The MRMC showed that BU-CAD improved readers' determination of BI-RADS descriptors (Shape, Orientation, Margin, Echo Pattern, and Posterior Features) for at least one or more subcategories for each descriptor. In conclusion, the BU-CAD MRMC reader study has demonstrated substantially equivalent performance to Transpara™ and Koios DS for Breast by showing a statistically significant aided read performance using similar success criteria compared to Transpara™ and Koios DS for Breast.
� Discussion of the Comparison to Support Substantial Equivalence (SE) Determination
BU-CAD has the same intended use as the legally marketed predicate device. Transpara™ They are intended to be used by clinicians interpreting radiological images, to help them with localizing and characterizing breast abnormalities.
The input of BU-CAD and the predicate devices is composed of medical images provided in a DICOM format. The output of BU-CAD is similar to the predicate devices by providing ROIs and lesion contours placed on suspicious soft tissue lesion and region-based score are similar to Transpara™, while providing BI-RADS category and BI-RADS descriptors are similar to Koios DS for Breast.
Although BU-CAD and Transpara™ differ in the type of medical images the devices process, they are both aligned to the generic FDA device type for radiological computer-assisted detection and diagnosis for lesions suspicious for cancer (product code: QDQ). BU-CAD has similar intended use compared to the predicate devices that aim to localize and characterize breast abnormalities. Artificial intelligence algorithm of each device may have different technological characteristics from the legally marketed predicate devices. Therefore, a fully crossed multiple reader multiple case (MRMC) reader study was conducted in the US.
11
Compared to Transpara™ and QuantX as the primary and secondary predicates, and in consideration of the technological characteristics and test methods used in the legally marketed Koios DS for Breast, BU-CAD does not raise different questions of safety and effectiveness.
VII. Clinical Performance Data
� Summary of the Reader Study
The performance of physicians without and with the aid of BU-CAD decision support in interpreting breast ultrasound images was compared by using a fully crossed multi-reader multi-case receiver operating characteristic (MRMC-ROC) retrospective study (also known as Obuchowski-Rockette Dorfman-Berbaum-Metz MRMC-ROC or OR-DBM MRMC-ROC).
The study consisted of 628 cases, of which 456 cases (189 malignant and 267 benign) were collected from the United States and 172 cases (65 malignant and 107 benign) were collected from Taiwan. Sixteen readers participated in the study. Each reader was asked to identify the lesion, provide a linear score of lesion characteristics (SLC), select a BI-RADS category and select BI-RADS descriptors for an ultrasound breast lesion with or without the aid of BU-CAD.
Dataset Demographic
A total of 628 cases collected from two institutions were used in the reader study. The source of cases is listed below.
- U.S.: 456 cases
- . Taiwan: 172 cases
The BI-RADS category distribution included in this study were listed below:
- . BI-RADS 2: 5 cases
- . BI-RADS 3: 123 cases
- . BI-RADS 4A: 204 cases
- . BI-RADS 4B: 111 cases
- . BI-RADS 4C: 105 cases
- . BI-RADS 5: 80 cases
The number of benign and malignant cases included in this study were listed below.
- . Benign cases
- o Pathology proof benign: 197 cases
- Two-year follow-up benign: 177 cases O
12
- Malignant cases .
- Ductal carcinomas in situ (DCIS): 17 cases O
- invasive ductal carcinoma (IDC): 193 cases O
- Invasive lobular carcinoma (ILC): 40 cases O
- Other cancer types: 4 cases o
The imaging hardware distribution included in this study were listed below:
- . GE: 451 cases
- Acuson: 5 cases ●
- Philips: 100 cases
- Canon/Toshiba: 72 cases ●
Reader Experience
| Study
Reader | Specialty | MQSA | Received Breast
Image
Fellowship | Year of experience as
a radiologist |
|-----------------|----------------|------|----------------------------------------|----------------------------------------|
| Dr. X01 | Radiologist | Yes | No | 24 |
| Dr. X02 | Radiologist | Yes | Yes | 3 |
| Dr. X03 | Radiologist | Yes | No | 13 |
| Dr. X04 | Radiologist | Yes | No | 14 |
| Dr. X05 | Radiologist | Yes | No | 8 |
| Dr. X06 | Radiologist | Yes | Yes | 5 |
| Dr. X07 | Radiologist | Yes | Yes | 2 |
| Dr. X08 | Radiologist | Yes | No | 10 |
| Dr. X09 | Radiologist | Yes | Yes | 12 |
| Dr. X10 | Radiologist | Yes | No | 11 |
| Dr. X11 | Breast Surgeon | No | No | > 30 (breast surgeon) |
| Dr. X12 | Breast Surgeon | No | No | > 30 (breast surgeon) |
| Dr. X13 | Radiologist | Yes | No | 21 |
| Dr. X14 | Radiologist | Yes | No | 1 |
| Dr. X15 | Radiologist | Yes | No | 13 |
| Dr. X16 | Radiologist | Yes | No | 5 |
Primary Objective
The primary objective of this clinical study is to prove that the user's performance (AUC of location-specific ROC) aided by the BU-CAD software is superior to the unaided performance. The aided AUC of the location-specific ROC for BU-CAD was superior to that of the unaided scenario for the diagnosis of breast ultrasound images. The mean AUC of location-specific ROC shift of 0.0374.
13
| Reading Scenario | AUC_LROC | 95% CI | p-value |
|---|---|---|---|
| Unaided | 0.7786 | (0.7463, 0.8109) | |
| Aided | 0.8160 | (0.7862, 0.8458) | |
| Aided - Unaided | 0.0374 | (0.0190, 0.0557) | 0.0001 |
Primary Results of the Pivotal Study
Subgroup Analysis
Subgroup of reader specialty (with and without MQSA certification), with and without breast image fellowship training, ultrasound systems (GE, Acuson, Philips, and Canon/Toshiba), benign types (pathology proof benign and two-year follow-up benign), cancer types (DCIS, IDC, ILC, and others), lesion sizes (less than 1 cm, between 1 cm and 2 cm, and larger than 2 cm), lesion locations (center and not in center), ages (≤ 50 years, > 50 years, ≤ 55 years, and >55 years), and source of cases (U.S. and Taiwan) were performed. Except for the subgroup of Acuson ultrasound system, where the sample size was relatively low, the readers aided by the BU-CAD achieved higher performance than unaided reading in the other subgroups.
Secondary Objective
The secondary objective of this clinical study is to compare that the user's performance (sensitivity, specificity, PPV, and NPV) between the unaided and aided readings. Sensitivity, specificity, PPV, and NPV produced from the aided arm were higher than unaided. The specificity, unadjusted PPV, and unadjusted NPV differed sigmificantly from zero between the aided and unaided sessions.
| Statistical Parameter | Unaided (95% CI) | Aided (95% CI) |
|---|---|---|
| Sensitivity | 0.9225 (0.8896, 0.9554) | 0.9353 (0.9050, 0.9655) |
| Specificity | 0.3165 (0.2694, 0.3636) | 0.3611 (0.3124, 0.4098) |
| NPV (unadjusted) | 0.8623 (0.8048, 0.9198) | 0.8945 (0.8456, 0.9434) |
| NPV_U.S. (adjusted) | 0.9982 (0.9902, 1.0000) | 0.9986 (0.9918, 1.0000) |
| NPV_Taiwan (adjusted) | 0.9969 (0.9767, 1.0000) | 0.9975 (0.9809, 1.0000) |
| PPV (unadjusted) | 0.4876 (0.4433, 0.5319) | 0.5056 (0.4607, 0.5505) |
| PPV_U.S. (adjusted) | 0.0108 (-0.0001, 0.0216) | 0.0113 (0.0000, 0.0225) |
| PPV_Taiwan (adjusted) | 0.0256 (-0.0002, 0.0514) | 0.0283 (0.0006, 0.0560) |
Sensitivity, Specificity, PPV, and NPV between Unaided and Aided Reading Scenarios
Although the specificity in the aided scenario is 36.11%, the following confusion table summarizes the event count from a false-positive (FP) unaided to a true-negative (TN) when aided by BU-CAD or a reverse for all 374 benign cases. A total of 790 FP events unaided were changed to TN events aided by BU-CAD for all 16 readers, and a total of 523 TN events
14
unaided were changed to FP events aided by BU-CAD for all 16 readers. The overall benefit was +267 events and shows that BU-CAD is able to assist the majority of readers in reducing false positives even for datasets where readers have a low specificity performance in the unaided scenario.
| All benign (374) | X01 | X02 | X03 | X04 | X05 | X06 | X07 | X08 | X09 | X10 | X11 | X12 | X13 | X14 | X15 | X16 | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FP (unaided) → | |||||||||||||||||
| TN (aided) | 83 | 24 | 93 | 23 | 86 | 33 | 44 | 73 | 34 | 46 | 30 | 46 | 24 | 41 | 69 | 41 | 790 |
| TN (unaided) → | |||||||||||||||||
| FP (aided) | 33 | 28 | 18 | 70 | 16 | 67 | 16 | 52 | 38 | 33 | 22 | 47 | 4 | 20 | 17 | 42 | 523 |
| Difference | 50 | -4 | 75 | -47 | 70 | -34 | 28 | 21 | -4 | 13 | 8 | -1 | 20 | 21 | 52 | -1 | 267 |
Confusion Table FP to TN Net Benefit for Benign Cases
In addition, BU-CAD software was found to significantly decrease readers' interpretation times (by ~40%) which was shown in analyses including and excluding outliers. Statistical analyses also indicated that BU-CAD improved readers' determination of BI-RADS descriptors (Shape, Orientation, Margin, Echo Pattern, and Posterior Features), where at least one or more subcategories for each descriptor demonstrated improved aided read performance, with limitations described in the User Manual.
Accuracy of BI-RADS Descriptors
| Reading Scenario | Shape | Orientation | Margin | Echo Pattern | Posterior Features |
|---|---|---|---|---|---|
| Unaided | 78.14% | 82.15% | 79.22% | 76.49% | 66.51% |
| Aided | 78.92% | 82.20% | 77.34% | 66.52% | 67.53% |
| BU-CAD Standalone | 71.91% | 75.24% | 73.57% | 66.73% | 58.03% |
� Summary of the Standalone Study
A total of 1139 cases (628 reader study cases plus 511 extended cases) collected from multiple institutions were used in the standalone study.
Dataset Demographic
The source of cases is listed below.
- . North America: 531 cases
- Europe: 36 cases .
- Taiwan: 572 cases ●
The BI-RADS category distribution included in this study were listed below:
- . BI-RADS 2: 31 cases
15
- BI-RADS 3: 223 cases ●
- BI-RADS 4A: 356 cases .
- BI-RADS 4B: 218 cases
- . BI-RADS 4C: 181 cases
- BI-RADS 5: 130 cases ●
The number of benign and malignant cases included in this study were listed below.
- Benign cases .
- Pathology proof benign: 465 cases o
- Two-year follow-up benign: 177 cases o
- Malignant cases .
- Ductal carcinomas in situ (DCIS): 53 cases O
- invasive ductal carcinoma (IDC): 361 cases O
- Invasive lobular carcinoma (ILC): 51 cases O
- Other cancer types: 32 cases o
The imaging hardware distribution included in this study were listed below:
- . GE: 634 cases
- . Siemens: 188 cases
- . Canon/Toshiba: 90 cases
- . Philips: 111 cases
- Supersonic: 24 cases .
- Others: 92 .
Lesion Identification Module (CADe) Performance
A total of 59 benign cases (including 11 of the 20 missing cases) and 18 malignant cases (including 9 of the 20 missing cases) did not meet the objective performance criteria (automated ROI center must be within ground truth ROI with at least 50% overlap in ROI area). The accuracy of the lesion identification algorithm was 93.24% (1062/1139). For the LROC analysis, 18 malignant cases were penalized due to wrong location or undetected by BU-CAD.
Comparison between Standalone and Unaided Reading Performance
The standalone performance of BU-CAD was measured in AUC LROC on the 628 reader study cases and the standalone study cases (combined the 628 reader study cases and 511
16
extended cases), a total of 1,139 cases (497 malignant and 642 benign). Table below shows the standalone AUC LROCs in both datasets are higher than that of unaided reading performance.
| Reading Scenario | AUC LROC | 95% CI |
|---|---|---|
| BU-CAD Standalone (628 reader study cases) | 0.7987 | (0.7626, 0.8348) |
| BU-CAD Standalone (1,139 standalone study cases) | 0.8203 | (0.7947, 0.8458) |
| Unaided Reading (628 reader study cases) | 0.7786 | (0.7463, 0.8109) |
Standalone and Unaided Reading Performances
Summary of Subgroup Analysis
Subgroup of the different ultrasound systems (GE, Siemens, Canon/Toshiba, Philips, Supersonic, and others), benign types (pathology proof benign and two-year follow-up benign), cancer types (DCIS, IDC, ILC, and others), lesion size (less than 1 cm, between 1 cm and 2 cm, and larger than 2cm), Lesion Locations (center and not in center), view type (two view vs. single view), ages (≤ 50 years, > 50 years, ≤ 55 years, and >55 years), and sources of cases (North America, Europe, and Taiwan) were performed. The performance of distinguishing between benign and malignant in Siemens ultrasound system, DCIS and ILC cancer type, cases where the lesion is not in the center, two-orthogonal views, and source of North America and Europe achieved acceptable discrimination (AUC LROC from 0.7 to 0.8). The remaining subgroups achieved excellent (AUC LROC from 0.8 to 0.9) or outstanding (AUC LROC > 0.9) discrimination.
Sensitivity, Specificity, PPV, and NPV
The standalone performances of sensitivity and specificity were assessed by using the 1,139 cases and summarized in Table 9. Results show the standalone sensitivity and specificity were 88.53% and 57.94%. In addition, the adjusted PPV of U.S. and Taiwan were 1.28% and 4.74% respectively, the adjusted NPV of U.S. and Taiwan were 99.83% and 99.67% respectively. Because both the prevalence rates of U.S. and Taiwan are relatively low, the adjusted PPVs were relatively low and the adjusted NPVs were relatively high. However, the standalone PPVs in U.S. and Taiwan were higher than those of unaided and aided scenarios.
17
| Statistical Parameter | Standalone (Frequency) | 95% CI |
|---|---|---|
| With Modification for Wrong-location Penalty) | ||
| Sensitivity (%) | 88.33 (439/497) | (0.8551, 0.9115) |
| Specificity (%) | 57.94 (372/642) | (0.5413, 0.6176) |
| PPV (%) [unadjusted] | 61.92 (439/709) | (0.5834, 0.6549) |
| PPV_US (%) | 1.28 | (0.0011, 0.0245)* |
| PPV_TW (%) | 4.74 | (0.0246, 0.0703)* |
| NPV (%) [unadjusted] | 86.51 (372/430) | (0.8328, 0.8974) |
| NPV_US (%) | 99.82 | (0.9921, 1.0000)* |
| NPV_TW (%) | 99.67 | (0.9895, 1.0000)* |
Standalone Sensitivity, Specificity, PPV, NPV
- The 95% Confidence Interval (CI) was estimated conditioning on the obtained prevalence rates of 0.72% and 1.94% in U.S. and Taiwan, respectively.
The following table showed the calculated sensitivity and specificity using each BI-RADS category as the threshold. Since the clinical decision threshold for cancer vs. non-cancer is BI-RADS 3 vs BIRADS 4a and the BI-RADS fifth edition concluded that patients with category > 4a lesions are recommended to undergo biopsy, the analysis of sensitivity and specificity are still based on BI-RADS 4a as the cutoff point (i.e., a BI-RADS category of 4a or higher defines a positive call for cancer diagnosis).
Standalone Sensitivity and Specificity by Using Different Cut-Off Points
| Statistical
| Parameter | 3 | 4A* | 4 B | 4C | 5 |
|---|---|---|---|---|---|
| Sensitivity | 0.9416 | ||||
| (0.9210, 0.9623) | 0.8833 | ||||
| (0.8551, 0.9115) | 0.8249 | ||||
| (0.7915, 0.8584) | 0.6962 | ||||
| (0.6557, 0.7366) | 0.4588 | ||||
| (0.4149, 0.5026) | |||||
| Specificity | 0.3302 | ||||
| (0.2938, 0.3666) | 0.5794 | ||||
| (0.5413, 0.6176) | 0.6994 | ||||
| (0.6639, 0.7348) | 0.8271 | ||||
| (0.7979, 0.8564) | 0.9252 | ||||
| (0.9049, 0.9456) |
- The cut-off value used in the standalone study.
Robustness of the Lesion Analysis Module (CADx)
To evaluate the robustness of the CADx algorithm (Lesion Analysis Module) when different rectangular ROIs are drawn around the same lesion on a given single-view image or two-view images, two reproducibility experiments of the same lesion cropped by different rectangular ROIs were conducted. In the first reproducibility experiment, each corner point of an ROI was shifted by randomly changing the horizontal and vertical dimensions up to 20% respectively from the ground truth ROI defined by the expert panel. The experiment was repeated 20 times with all 1139 test cases (the original dataset was 628 cases and the
18
extended dataset was 511 cases). The results show that randomly enlarging the width and height of the ROIs did not affect the performance of the BU-CAD CADx algorithm (Lesion Analysis Module). The AUC remained stable between 0.840 and 0.846.
In the second reproducibility experiment, each corner point of ground truth ROI was altered by systematically shrinking the horizontal and vertical dimensions respectively from 1% to 30%. The experiment was conducted with all 1139 cases. The new ROIs and their corresponding images were then processed by the BU-CAD CADx algorithm (Lesion Analysis Module) to produce analysis outputs. The results show that as long as the shrinking percentage of the width and height of the ROIs is within 16%, the AUC remained above 0.8.
VIII. Non-Clinical Performance Data
In the design and development of BU-CAD, TaiHao applied the following voluntary FDA recognized standards:
| Standard | Standard Title |
|---|---|
| ISO 14971:2007 | Medical Devices - Application Of Risk Management To Medical Devices |
| IEC 62304:2015 | Medical Device Software - Software Life Cycle Processes |
| DEN180005 | Evaluation of automatic class III designation for OsteoDetect – Decision summary with special controls |
The following guidance documents were used to support this submission:
| FDA Guidance | Issued Date |
|---|---|
| Guidance for Industry and FDA Staff - Guidance for the Content of | |
| Premarket Submissions for Software Contained in Medical Devices. | May 11, 2005 |
| Guidance for Industry and Food and Drug Administration Staff - | |
| Computer-Assisted Detection Devices Applied to Radiology Images | |
| and Radiology Device Data – Premarket Notification [510(k)] | |
| Submissions. | July 3, 2012 |
| Guidance for Industry and FDA Staff - Clinical Performance | |
| Assessment: Considerations for Computer-Assisted Detection | |
| Devices Applied to Radiology Images and Radiology Device Data | |
| in - Premarket Notification (510(k)) Submissions. | January 22, 2020 |
19
| FDA Guidance | Issued Date |
|---|---|
| Guidance for Industry and Food and Drug Administration Staff - | |
| The 510(k) Program: Evaluating Substantial Equivalence in | |
| Premarket Notifications [510(k)]. | July 28, 2014 |
| Draft Guidance for Industry and Food and Drug Administration | |
| Staff - Content of Premarket Submissions for Management of | |
| Cybersecurity in Medical Devices. | October 18, 2018 |
BU-CAD is a software-only device. The level of concern for BU-CAD is identified as Moderate Level of Concern. Developmental testing was conducted to verify requirements according to the BU-CAD specifications. The purpose of the verification test was to assure that the software application satisfied the software requirements. Validation testing consisted of determining standalone performance of the algorithms in BU-CAD using a multiple-vendor testing dataset of breast ultrasound images. The testing dataset was not used for training of BU-CAD algorithms.
Conclusions IX.
TaiHao has applied a risk management process in accordance with FDA recognized standards to identify, evaluate, and mitigate all known hazards related to BU-CAD. Non-clinical and clinical performance tests demonstrate that BU-CAD performs similarly to the legally marketed predicates, and thatall identified risks are effectively mitigated. Therefore, it can be concluded that BU-CAD is as safe and effective as the identified predicates, Transpara™ (K181704) and QuantX (K170195)