(329 days)
Not Found
Yes
The document explicitly states that BraveCX is a "Deep Learning Artificial Intelligence (AI) software" and uses an "artificial intelligence algorithm" and "deep learning" to analyze images.
No.
The device is described as a radiological computer-assisted triage and notification software that analyzes images for suspected critical findings to prioritize worklists, not intended for diagnosis or treatment.
No
BraveCX is described as "prioritization-only software" and its results "are not intended to be used on a stand-alone basis for clinical decision-making," indicating it aids workflow rather than directly diagnosing.
Yes
The device description explicitly states that BraveCX is a "Deep Learning Artificial Intelligence (AI) software" and describes its function as analyzing images and providing output to PACS/workstation. There is no mention of any accompanying hardware component that is part of the device itself.
Based on the provided information, BraveCX is not an In Vitro Diagnostic (IVD).
Here's why:
- Definition of IVD: In Vitro Diagnostics are devices intended for use in the collection, preparation, and examination of specimens taken from the human body (such as blood, urine, or tissue) to provide information for diagnosis, monitoring, or screening.
- BraveCX's Function: BraveCX analyzes radiological images (chest X-rays), which are generated by imaging equipment and are not specimens taken from the human body in the sense of an IVD. It processes these images to identify potential findings, but it does not interact with biological samples.
Therefore, BraveCX falls under the category of medical imaging software or a radiological computer-assisted triage device, not an In Vitro Diagnostic.
No
The letter does not mention that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device. The "Control Plan Authorized (PCCP) and relevant text" section explicitly states "Not Found."
Intended Use / Indications for Use
BraveCX is a radiological computer-assisted triage and notification software that analyzes adult (≥18 years old) chest Xray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). BraveCX uses an artificial intelligence algorithm to analyze images for features suggestive of critical findings and provides case-level output available in the PACS/workstation for worklist prioritization or triage. As a passive notification for prioritization-only software tool within standard of care workflow, BraveCX does not send a proactive alert directly to the appropriately trained medical specialists. BraveCX is not intended to direct attention to specific portions of an image or to anomalies other than pleural effusion and/or pneumothorax. Its results are not intended to be used on a stand-alone basis for clinical decision-making.
Product codes (comma separated list FDA assigned to the subject device)
QFM
Device Description
BraveCX is supplied as a licensed Application Programming Interface (API) that can be deployed either as a cloud-based service, directly on premises, or integrated with third-party systems. The system can be configured to work with multiple DICOM storage platforms, including Picture Archiving and Communications (PACS) or other persistent data storage systems. BraveCX is a Deep Learning Artificial Intelligence (AI) software that analyzes adult (≥18 years old) chest X-ray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). It uses deep learning to analyze each image to identify features suggestive of pleural effusion and/or pneumothorax. Upon image acquisition from other radiological imaging equipment (e.g. X-ray systems), Anteroposterior (AP) and Posteroanterior (PA) chest X-Rays are received and processed by BraveCX. Following receipt of an image, BraveCX de-identifies a copy of each DICOM file and analyses it for features suggestive of pleural effusion and/or pneumothorax. Based on the analysis result, the software notifies PACS/workstation for the presence of the critical findings, indicated by "flag" or "(blank)". This allows the appropriately trained medical specialists to group suspicious exams together with potential for prioritization. Chest radiographs without an identified anomaly are placed in the worklist for routine review, which is the current standard of care. The software output to the user is a label of "flag" or "(blank)" that relates to the likelihood of presence of pneumothorax and/or pleural effusion. BraveCX platform ingests prediction requests with either attached DICOM images or DICOM UIDs referencing images already uploaded to DICOM storage. The results will be made available via a newly generated DICOM that is stored in DICOM storage or as a JSON file. The DICOM storage component may be a Picture Archiving and Communications (PACS) system or some other local storage platform. BraveCX works in parallel to and in conjunction with the standard of care workflow to enable prioritized review by the appropriately trained medical specialists who are qualified to interpret chest radiographs. As a passive notification for prioritization-only software tool within standard of care workflow, BraveCX does not send a proactive alert directly to the appropriately trained medical specialists who are qualified to interpret chest radiographs. BraveCX is not intended to direct attention to specific portions or anomalies of an image and it should not be used on a standalone basis for clinical decision-making. BraveCX automatically runs after image acquisition. It prioritises and displays the analysis results through the worklist interface of PACS/workstation. An on-device, technologist notification is generated within 15 minutes after interpretation by the user, indicating which cases were prioritized by BraveCX in PACS. The technologist notification is contextual and does not provide any diagnostic information. The on-device, technologist notification is not intended to inform any clinical decision, prioritization, or action.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Yes
Input Imaging Modality
Chest X-Ray
Anatomical Site
Chest/Lung
Indicated Patient Age Range
adult (≥18 years old)
Intended User / Care Setting
health care professional such as radiologist or another appropriately trained clinician.
Description of the training set, sample size, data source, and annotation protocol
Model training, validation, and testing sets were generated by stratified random partitions of 80%, 10%, and 10% respectively. Each partition was stratified according to the frequency of abnormalities, gender, and View Position. To avoid data leakage, each stratified split contained non-overlapping patient identifiers. Images used in the training, validation, and testing of the subject device were all manually-curated ground truths provided by three board-certified Radiologists with at least 10 years in specialist radiology training.
Description of the test set, sample size, data source, and annotation protocol
The company conducted an external independent testing to assess the performance of BraveCX. The studies were conducted with MIMIC Chest X-ray (MIMIC-CXR) Database v2.0.020, NIH Chest X-Ray dataset (NIH-CXR), and CheXpert dataset (Stanford Hospital) that represent the US population. The datasets contained 867 cases for pleural effusion and 2,114 cases for pneumothorax obtained from Beth Israel Deaconess Medical Center in Boston, MA, NIH Clinical Center, and Stanford Hospital. In all cases, each image corresponded to a single patient. Patients with multiple studies were excluded from the performance validation process. All images were manually labelled by three board-certified Radiologists with at least 10 years of experience in specialty radiology training.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Non-clinical performance tests were conducted. The company conducted an external independent testing to assess the performance of BraveCX.
For pleural effusion, a total of n=2,509 images were included (n=867 Pleural Effusion cases). ROC AUC was 0.988 (95% CI: 0.9885-0.9887).
For pneumothorax, a total of n=3,245 images were included (n=2,114 Pneumothorax cases). ROC AUC was 0.972 (95% CI: 0.9727-0.9729).
Standalone performance was demonstrated. The lower bound of ROC AUC exceeds 0.95 and the lower bounds of both sensitivity and specificity are above 0.85 for both pleural effusion and pneumothorax.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Pleural Effusion:
Sensitivity 92.62% (95% CI: 90.67%-94.27%)
Specificity 98.11% (97.33%-98.71%)
Pneumothorax:
Sensitivity 93.38% (95% CI: 92.23%-94.40%)
Specificity 97.27% (96.49%-97.92%)
Time-to-notification of BraveCX was 4.8 seconds-10.4 seconds (95% CI: 4.2-10.41s) for simultaneous prediction of Pleural Effusion and Pneumothorax.
Predicate Device(s): If the device was cleared using the 510(k) pathway, identify the Predicate Device(s) K/DEN number used to claim substantial equivalence and list them here in a comma separated list exactly as they appear in the text. List the primary predicate first in the list.
Reference Device(s): Identify the Reference Device(s) K/DEN number and list them here in a comma separated list exactly as they appear in the text.
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information for the subject device only (e.g. presence / absence, what scope was granted / cleared under the PCCP, any restrictions, etc).
Not Found
§ 892.2080 Radiological computer aided triage and notification software.
(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.
0
Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, with the letters "FDA" in a blue square. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
Bering Ltd % Stephan Toupin - Official Correspondent Dawa Medical LLC 7320 NW 12th Street Suite 103 Miami, FL 33126
November 9, 2023
Re: K223754
Trade/Device Name: BraveCX Regulation Number: 21 CFR 892.2080 Regulation Name: Radiological Computer Aided Triage And Notification Software Regulatory Class: Class II Product Code: QFM Dated: October 11, 2023 Received: October 11, 2023
Dear Stephan Toupin:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrb/cfdocs/cfpmn/pmn.cfm identifies.combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download). Your device is also subject to. among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100,
1
Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (OS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely.
Jessica Lamb
Jessica Lamb Assistant Director DHT8B: Division of Radiologic Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Ouality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K223754
Device Name BraveCX
Indications for Use (Describe)
BraveCX is a radiological computer-assisted triage and notification software that analyzes adult (≥18 years old) chest Xray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). BraveCX uses an artificial intelligence algorithm to analyze images for features suggestive of critical findings and provides caselevel output available in the PACS/workstation for worklist prioritization or triage. As a passive notification for prioritization-only software tool within standard of care workflow, BraveCX does not send a proactive alert directly to the appropriately trained medical specialists. BraveCX is not intended to direct attention to specific portions of an image or to anomalies other than pleural effusion and/or pneumothorax. Its results are not intended to be used on a stand-alone basis for clinical decision-making.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| ------------------------------------------------- | -- |
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
510(k) Summary
BraveCX
K223754
1. Submission Sponsor
Bering Ltd
54 Portland Place, 2nd Floor
London
W1B 1DY
United Kingdom
Contact: DROZDOV Ignat Title: Managing Director
2. Submission Correspondent
Stéphan Toupin, Msc
Dawa Medical LLC
7320 NW 12th Street Suite 103 Miami,
Florida, 33126, USA
Email: stoupin@dawamedical.com
Cellphone number: (786) 731-1159
3. Date Prepared
March 14, 2023
4. Device Identification
Trade/Proprietary Name: BraveCX
Common/Usual Name: BraveCX
Classification Name: Radiological computer aided triage and notification software
4
| Regulation Number: | 892.2080 |
|---|---|
| Product Code: | QFM, Radiological computer aided triage and notification software |
| Device Class: | Class II |
| Classification Panel: | Radiology |
5. Legally Marketed Predicate Device(s)
Primary Predicate
| 510(k) Number: | K211733 |
|---|---|
| DEVICE NAME: | Lunit INSIGHT CXR Triage |
| MANUFACTURER: | Lunit Inc. |
6. Indication for Use Statement
BraveCX is a radiological computer-assisted triage and notification software that analyzes adult (≥18 years old) chest X-ray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). BraveCX uses an artificial intelligence algorithm to analyze images for features suggestive of critical findings and provides case-level output available in the PACS/workstation for worklist prioritization or triage. As a passive notification for prioritization-only software tool within standard of care workflow, BraveCX does not send a proactive alert directly to the appropriately trained medical specialists. BraveCX is not intended to direct attention to specific portions of an image or to anomalies other than pleural effusion and/or pneumothorax. Its results are not intended to be used on a stand-alone basis for clinical decision-making.
7. Device Description
The company conducted an internal independent testing set to assess the performance of BraveCX. The validation was completed to determine whether a distinction between target findings is properly completed by the subject device. The internal independent testing set contained n=1,209 cases for pleural effusion and n=1,387 cases of pneumothorax, obtained between June 2007 and August 2019 across 14 acute sites in NHS Greater Glasgow and Clyde. Each image corresponded to a single patient. Images were obtained by following industry standards using both mobile and departmental X-Ray equipment. Table 5A lists X-Ray equipment manufacturers included in the training and internal independent testing of the BraveCX device.
5
Images used in the training, validation, and testing of the subject device were all manually-curated ground truths provided by three board-certified Radiologists with at least 10 years in specialist radiology training. Model training, validation, and testing sets were generated by stratified random partitions of 80%, 10%, and 10% respectively. Each partition was stratified according to the frequency of abnormalities, gender, and View Position. To avoid data leakage, each stratified split contained non-overlapping patient identifiers.
Table 5A - Distribution of device manufacturers used for training and internal validation of the BraveCX device.
| Manufacturer | Proportion of DICOMs |
|---|---|
| GE Healthcare | 2% |
| AGFA | 1% |
| Fujifilm Corporation | 70% |
| KODAK | 8% |
| Phillips Medical Systems | 1% |
| Samsung Electronics | 18% |
Summary of results:
ROC AUC was 0.96 (95% CI: 0.95 - 0.97) with 82% sensitivity and 95% specificity for pleural effusion.
For pneumothorax, 0.98 ROC AUC (95% CI: 0.98-0.99), 89% sensitivity, and 97% specificity were reported.
By confirming that the lower bound of ROC AUC exceeds 0.95 and the sensitivity and specificity for both target radiologic findings are above 80%, the performance of the algorithm of BraveCX is validated to demonstrate that the prespecified target performance is satisfied.
Product deployment:
BraveCX is supplied as a licensed Application Programming Interface (API) that can be deployed either as a cloud-based service, directly on premises, or integrated with third-party systems. The system can be configured to work with multiple DICOM storage platforms, including Picture Archiving and Communications (PACS) or other persistent data storage systems.
BraveCX is a Deep Learning Artificial Intelligence (AI) software that analyzes adult (≥18 years old) chest X-ray images for the presence of pre-specified suspected critical findings (pleural
6
effusion and/or pneumothorax. It uses deep learning to analyze each image to identify features suggestive of pleural effusion and/or pneumothorax.
Upon image acquisition from other radiological imaging equipment (e.g. X-ray systems), Anteroposterior (AP) and Posteroanterior (PA) chest X-Rays are received and processed by BraveCX. Following receipt of an image, BraveCX de-identifies a copy of each DICOM file and analyses it for features suggestive of pleural effusion and/or pneumothorax. Based on the analysis result, the software notifies PACS/workstation for the presence of the critical findings, indicated by "flag" or "(blank)". This allows the appropriately trained medical specialists to group suspicious exams together with potential for prioritization. Chest radiographs without an identified anomaly are placed in the worklist for routine review, which is the current standard of care.
The intended user of the BraveCX software is a health care professional such as radiologist or another appropriately trained clinician. The software does not alter the order or remove cases from the reading queue.
The software output to the user is a label of "flag" or "(blank)" that relates to the likelihood of presence of pneumothorax and/or pleural effusion.
BraveCX platform ingests prediction requests with either attached DICOM images or DICOM UIDs referencing images already uploaded to DICOM storage. The results will be made available via a newly generated DICOM that is stored in DICOM storage or as a JSON file. The DICOM storage component may be a Picture Archiving and Communications (PACS) system or some other local storage platform.
BraveCX works in parallel to and in conjunction with the standard of care workflow to enable prioritized review by the appropriately trained medical specialists who are qualified to interpret chest radiographs. As a passive notification for prioritization-only software tool within standard of care workflow, BraveCX does not send a proactive alert directly to the appropriately trained medical specialists who are qualified to interpret chest radiographs. BraveCX is not intended to direct attention to specific portions or anomalies of an image and it should not be used on a standalone basis for clinical decision-making.
BraveCX automatically runs after image acquisition. It prioritises and displays the analysis results through the worklist interface of PACS/workstation. An on-device, technologist notification is generated within 15 minutes after interpretation by the user, indicating which cases were prioritized by BraveCX in PACS. The technologist notification is contextual and does not provide
7
any diagnostic information. The on-device, technologist notification is not intended to inform any clinical decision, prioritization, or action.
8. Substantial Equivalence Discussion
The following table compares BraveCX to the predicate device with respect to indications for use, principles of operation, technological characteristics, and performance testing. The comparison of the devices provides more detailed information regarding the basis for the determination of substantial equivalence. The subject device does not raise any new issues of safety or effectiveness based on the similarities to the predicate device.
| Manufacturer | Bering Ltd | Lunit Inc. |
|---|---|---|
| Trade Name | BraveCX | Lunit INSIGHT CXR Triage |
| 510(k) Number | NA | K211733 |
| Product Code | QFM | QFM |
| Regulation | ||
| Number | 892.2080 | 892.2080 |
| Regulation | ||
| Name | Radiology | Radiology |
| Indications for | ||
| Use | BraveCX is a radiological | |
| computer-assisted triage and | ||
| notification software that analyzes | ||
| adult (≥18 years old) chest X-ray | ||
| images for the presence of pre- | ||
| specified suspected critical findings | ||
| (pleural effusion and/or | ||
| pneumothorax). BraveCX uses an | ||
| artificial intelligence algorithm to | ||
| analyze images for features | ||
| suggestive of critical findings and | ||
| provides case-level output available | ||
| in the PACS/workstation for | ||
| worklist prioritization or triage. As | ||
| a passive notification for | ||
| prioritization-only software tool | ||
| within standard of care workflow, | ||
| BraveCX does not send a proactive | Lunit INSIGHT CXR Triage is | |
| a radiological computer- | ||
| assisted triage and notification | ||
| software that analyzes adult | ||
| chest X-ray images for the | ||
| presence of pre-specified | ||
| suspected critical findings | ||
| (pleural effusion and/or | ||
| pneumothorax). Lunit | ||
| INSIGHT CXR Triage uses an | ||
| artificial intelligence algorithm | ||
| to analyze images for features | ||
| suggestive of critical findings | ||
| and provides case-level output | ||
| available in the | ||
| PACS/workstation for worklist | ||
| prioritization or triage. As a | ||
| passive notification for | ||
| Manufacturer | Bering Ltd | Lunit Inc. |
| Trade Name | BraveCX | Lunit INSIGHT CXR Triage |
| alert directly to the appropriately | ||
| trained medical specialists. | ||
| BraveCX is not intended to direct | ||
| attention to specific portions of an | ||
| image or to anomalies other than | ||
| pleural effusion and/or | ||
| pneumothorax. Its results are not | ||
| intended to be used on a stand- | ||
| alone basis for clinical decision- | ||
| making. | prioritization-only software | |
| tool within standard of care | ||
| workflow, Lunit INSIGHT | ||
| CXR Triage does not send a | ||
| proactive alert directly to the | ||
| appropriately trained medical | ||
| specialists. Lunit INSIGHT | ||
| CXR Triage is not intended to | ||
| direct attention to specific | ||
| portions of an image or to | ||
| anomalies other than pleural | ||
| effusion and/or pneumothorax. | ||
| Its results are not intended to | ||
| be used on a stand-alone basis | ||
| for clinical decision-making. | ||
| Notification- | ||
| only, parallel | ||
| workflow tool | Yes | Yes |
| User | Appropriately trained medical | |
| specialists who are qualified to | ||
| interpret chest radiographs | Appropriately trained medical | |
| specialists who are qualified to | ||
| interpret chest radiographs | ||
| Targeted | ||
| clinical | ||
| condition, | ||
| anatomy, and | ||
| modality | Pleural effusion, pneumothorax | |
| Chest/Lung Frontal Chest X-ray | Pleural effusion, pneumothorax | |
| Chest/Lung Frontal Chest X- | ||
| ray | ||
| Algorithm for | ||
| pre-specified | ||
| critical findings | ||
| detection | BraveCX is a Deep Learning | |
| Artificial Intelligence (AI) software | ||
| that was trained to detect pleural | ||
| effusion and pneumothorax in chest | ||
| X-Ray images. BraveCX uses a | ||
| vendor agnostic algorithm | ||
| compatible with DICOM chest X- | ||
| Ray images | Lunit INSIGHT CXR is deep | |
| learning based software that | ||
| assists radiologists or clinicians | ||
| in the interpretation of chest x- | ||
| ray. AI algorithm designed to | ||
| detect pleural effusion and | ||
| pneumothorax in chest X-ray | ||
| images. Lunit INSIGHT CXR | ||
| Triage uses a vendor agnostic | ||
| algorithm compatible with | ||
| DICOM chest X-ray images |
Table 5B - Comparison of Characteristics
8
9
| Manufacturer | Bering Ltd | Lunit Inc. |
|---|---|---|
| Trade Name | BraveCX | Lunit INSIGHT CXR Triage |
| Radiological | ||
| images format | DICOM | DICOM |
| Computational | ||
| Platform | BraveCX is supplied as a licensed | |
| Application Programming Interface | ||
| (API) that can be deployed either as | ||
| a cloud-based service, directly on | ||
| premises, or integrated with third- | ||
| party systems. | Lunit INSIGHT CXR Triage is | |
| designed as a software module | ||
| that can be deployed on several | ||
| computing and X-ray imaging | ||
| platforms such as radiological | ||
| imaging equipment, PACS, On | ||
| Premise or On Cloud. | ||
| Device output in | ||
| case of positive | ||
| detection | BraveCX automatically runs after | |
| image acquisition. | ||
| The user may prioritize reporting | ||
| tasks by grouping images flagged by | ||
| BraveCX together. Results are | ||
| displayed through the worklist | ||
| interface of PACS/workstation | Lunit INSIGHT CXR Triage | |
| automatically runs after image | ||
| acquisition and prioritizes and | ||
| displays the analysis result | ||
| through the worklist interface | ||
| of PACS/workstation. | ||
| No markup on original image. | ||
| Secondary capture of the finding. | No markup on original image. | |
| Secondary capture of the finding. | ||
| Upon image acquisition from other | ||
| radiological imaging equipment | ||
| (e.g. X-ray systems), an on-device, | ||
| technologist notification indicating | ||
| which cases were flagged in the | ||
| Secondary Capture image by Brave | ||
| CX in PACS, is generated 15 | ||
| minutes after interpretation by the | ||
| user. | Upon image acquisition from | |
| other radiological imaging | ||
| equipment (e.g. X-ray systems), | ||
| an on-device, technologist | ||
| notification indicating which | ||
| cases were flagged in the | ||
| Secondary Capture image by | ||
| Lunit INSIGHT CXR Triage in | ||
| PACS, is generated 15 minutes | ||
| after interpretation by the user. | ||
| The on-device notification is | ||
| contextual and does not provide any | ||
| diagnostic information. It is not | ||
| intended to inform any clinical | ||
| decision, prioritization, or action to | ||
| the technologist. | The on-device notification is | |
| contextual and does not provide | ||
| any diagnostic information. It is | ||
| not intended to inform any | ||
| clinical decision, prioritization, | ||
| or action to the technologist. | ||
| Manufacturer | Bering Ltd | Lunit Inc. |
| Trade Name | BraveCX | Lunit INSIGHT CXR Triage |
| Notification | ||
| (i.e., recipient, | ||
| timing and | ||
| means of | ||
| notification) | Passive notification. | |
| Images with suspicion are flagged | ||
| in PACS/workstation. | Passive notification. | |
| Images with suspicion | ||
| are | ||
| flagged in PACS/workstation. | ||
| Where | ||
| generated | ||
| results (i.e., | ||
| DICOM files) | ||
| are stored | Picture Archiving and | |
| Communications (PACS) system or | ||
| some other local storage platform | PACS/Workstation | |
| Performance | ||
| level – Timing | ||
| of notification | The average time taken for the | |
| notification to travel from the | ||
| BraveCX API to the point at which | ||
| the result is displayed in the | ||
| destination PACS/RIS/EPR | ||
| worklist is 10.4 seconds. | The average time taken for the | |
| notification to travel from the | ||
| Lunit INSIGHT CXR Triage to | ||
| the point at which the result is | ||
| displayed in the destination | ||
| PACS/RIS/EPR worklist is | ||
| 14.66 seconds. | ||
| Performance | ||
| level - | ||
| accuracy of | ||
| classification | Pleural Effusion | |
| ROC AUC > 0.95 | ||
| AUC: 0.988 (95% CI: [0.988, | ||
| 0.9887] | ||
| Sensitivity 92.62% (95% CI : | ||
| [90.67%, 94.27%]) | ||
| Specificity 98.11% (95% CI | ||
| [97.33%, 98.71%]) | ||
| Pneumothorax | ||
| ROC AUC > 0.95 | ||
| AUC : 0.972 (95% CI: [0.9727, | ||
| 0.9729]) | ||
| Sensitivity 93.38% (95% CI: | ||
| [92.23%, 94.40%]) | ||
| Specificity 97.27% (95%CI: | ||
| [96.49%-97.92%]) | Pleural Effusion | |
| ROC AUC > 0.95 | ||
| AUC: 0.9686 (95% CI: | ||
| [0.9547, 0.9824]) | ||
| Sensitivity 89.86% (95% CI: | ||
| [86.72, 93.00]) | ||
| Specificity 93.48% (95% CI: | ||
| [91.06, 95.91]) | ||
| Pneumothorax | ||
| ROC AUC > 0.95 | ||
| AUC: 0.9630 (95% CI: | ||
| [0.9521, 0.9739]) | ||
| Sensitivity 88.92% (95% CI: | ||
| [85.60, 92.24]) | ||
| Specificity 90.51% (95% CI: | ||
| [88.18, 92.83]) |
10
9. Non-Clinical Performance Data
11
The performance of BraveCX was validated by non-clinical tests. All verification testing met the (passed), demonstrating that the software fulfills its requirement acceptance criteria specifications.
As part of demonstrating safety and effectiveness of BraveCX and in showing substantial equivalence to the predicate devices that are subject to these 510(k) submissions, Bering Ltd completed non-clinical performance tests.
The BraveCX meets all the requirements confirming that the design output meets the design inputs and specifications for the device. BraveCX passed all the testing of the subject device according to Software verification and validation testing per IEC 62304/FDA Guidance.
The company conducted an external independent testing to assess the performance of BraveCX. The studies were conducted with MIMIC Chest X-ray (MIMIC-CXR) Database v2.0.020, NIH Chest X-Ray dataset (NIH-CXR), and CheXpert dataset (Stanford Hospital) that represent the US population.
The datasets contained 867 cases for pleural effusion and 2,114 cases for pneumothorax obtained from Beth Israel Deaconess Medical Center in Boston, MA, NIH Clinical Center, and Stanford Hospital. In all cases, each image corresponded to a single patient. Patients with multiple studies were excluded from the performance validation process. All images were manually labelled by three board-certified Radiologists with at least 10 years of experience in specialty radiology training.
Demographic characteristics of images used in clinical validation studies of BraveCX are shown in the tables (Table 5C, 5D) and reflect a diverse range of ages, gender, and ethnicities.
| Characteristic | MIMIC (Pleural Effusion) | NIH (Pleural Effusion | CheXpert (Pleural Effusion) |
|---|---|---|---|
| Gender | |||
| Male | 54% | 58% | 55% |
| Female | 46% | 42% | 45% |
| Ethnicity | |||
| Asian | 3% | N/A | 12% |
Table 5C - Demographic characteristics of patients in the clinical validation cohort of the Pleural Effusion classifier.
12
| Black | 10% | N/A | 6% |
|---|---|---|---|
| Hispanic | 3% | N/A | 4% |
| Other/Unknown | 30% | N/A | 25% |
| White | 54% | N/A | 53% |
| Age | |||
| 18-25 | 2% | 9% | 9% |
| 25-35 | 4% | 16% | 8% |
| 35-65 | 33% | 63% | 47% |
| >65 | 12% | 12% | 36% |
Table 5D - Demographic characteristics of patients in the clinical validation cohort of the Pneumothorax classifier.
| Characteristic | MIMIC
(Pneumothorax) | NIH
(Pneumothorax) | CheXpert
(Pneumothorax) |
|----------------|-------------------------|-----------------------|----------------------------|
| Gender | | | |
| Male | 60% | 58% | 53% |
| Female | 40% | 42% | 47% |
| Ethnicity | | | |
| Asian | 5% | N/A | 10% |
| Black | 10% | N/A | 7% |
| Hispanic | 3% | N/A | 2% |
| Other/Unknown | 5% | N/A | 25% |
| White | 77% | N/A | 56% |
| Age | | | |
| 18-25 | 4% | 9% | 8% |
13
| 25-35 | 7% | 16% | 7% |
|---|---|---|---|
| 35-65 | 48% | 63% | 43% |
| >65 | 41% | 12% | 42% |
Summary of results:
For pleural effusion, the results are as follows:
A total of n=2,509 images were included in the analysis (n=867 Pleural Effusion). ROC AUC 0.988 (95% CI:0.9885-0.9887) Sensitivity 92.62% (95% CI:90.67%-94.27%) Specificity 98.11% (97.33%-98.71%).
Model performance was unaffected by chest X-ray View Position (anteroposterior or posteroanterior), patient sex, age quartiles (18-53, 53-65, 65-75, ≥75), BMI quartiles (8-25, 25-30, 30-35, and >35), and ethnicities (Asian, Black, Hispanic, Other, and White) DeLong's p-values 0.89 - 0.98
The performance for the predicate device indicated for pleural effusion (Lunit INSIGHT CXR Triage, K211733) are as follows: ROC AUC 0.9686 (95% CI: 0.9547 - 0.9824), sensitivity 89.86% (95% CI: 86.72 - 93.00) and specificity 93.48% (95% CI: 91.06 - 95.91).
For pneumothorax, the results are as follows:
A total of n=3,245 images were included in the analysis (n=2,114 Pneumothorax). ROC AUC 0.972 (95% CI:0.9727-0.9729) Sensitivity 93.38% (95% CI:92.23%-94.40%) Specificity 97.27% (96.49%-97.92%).
Model performance was unaffected by chest X-ray View Position (anteroposterior or posteroanterior), patient sex, age quartiles (18-53, 53-65, 65-75, ≥75), BMI quartiles (8-25, 25-30, 30-35, and ≥35), and ethnicities (Asian, Black, Hispanic, Other, and White) DeLong's pvalues 0.07 - 0.98.
As compared the device performance time of the BraveCX with the predicate device (Lunit INSIGHT CXR Triage, K211733), the result was comparable with the predicate device.
The performance data show that the lower bound of ROC AUC exceeds 0.95 and the lower bounds of both sensitivity and specificity are above 0.85 for both pleural effusion and pneumothorax. Accordingly, the BraveCX is demonstrated to achieve effective image analysis and triage capabilities.
14
With regards to the device performance time, the company assessed the performance time of the BraveCX that reflects the time it takes for the device to analyze the study and send a notification to the worklist.
Time-to-notification of BraveCX was 4.8 seconds-10.4 seconds (95% CI: 4.2-10.41s) for simultaneous prediction of Pleural Effusion and Pneumothorax.
The performance for the predicate device (Lunit INSIGHT CXR Triage, K211733) indicated are as follows: The performance time was 20.76 seconds (95% CI: 20.23 - 21.28) for pleural effusion and 20.45 seconds (95% CI: 19.99 - 20.92) for pneumothorax.
As compared the device performance time of the BraveCX with predicate device (Lunit INSIGHT CXR Triage, K211733), the result was comparable with the predicate device.
In summary, the BraveCX performed successfully in the external standalone study. The BraveCX established standalone adequate detection performance and device performance time for pleural effusion and pneumothorax as compared to the predicate device.
Results of the clinical investigation support the indications for use of BraveCX. External standalone study conclusion confirms that BraveCX is safe and effective as used according to the instructions for use.
10. Statement of Substantial Equivalence
The BraveCX is as safe and effective as the predicate devices. The BraveCX has the same intended uses and similar indications, technological characteristics, and principles of operation as its predicate devices.
The minor technological differences between the BraveCX and its predicate device raise no new issues of safety or effectiveness. The clinical and non-clinical performance data demonstrates that the BraveCX is as safe and effective as the predicate device. Thus, the BraveCX support a decision is substantially equivalent to its predicate devices for triage and notification.
By definition, a device is substantially equivalent to a predicate device when the device has the same intended use and the same technological characteristics as the previously cleared predicate device. Or the device has the same intended use and different technological characteristics that can be demonstrated that the device is substantially equivalent to the predicate device, and that the new device does not raise additional questions regarding its safety and effectiveness as compared to the predicate device.
BraveCX, as designed and manufactured, is determined to be substantially equivalent to the referenced predicate device.