(144 days)
Not Found
Yes
The document explicitly states that the device "uses an artificial intelligence algorithm to analyze images".
No
The device is described as radiological computer-assisted triage and notification software that analyzes images for diagnostic purposes (identifying suspected critical findings like pleural effusion and/or pneumothorax), not for treatment or therapy. Its purpose is to prioritize worklists for medical specialists, not to provide therapeutic intervention.
No
Explanation: The device is described as a "computer-assisted triage and notification software" that analyzes images for "worklist prioritization or triage." It explicitly states that "Its results are not intended to be used on a stand-alone basis for clinical decision-making" and "is not intended to rule out the target conditions or otherwise preclude clinical assessment of X-Ray cases." This indicates it aids in workflow and prioritization rather than providing a definitive diagnosis itself.
Yes
The device description explicitly states it is "radiological computer aided triage and notification software" and processes digital images (DICOM format) received from existing imaging systems (PACS or X-ray systems). There is no mention of accompanying hardware or hardware components included with the device.
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs analyze samples taken from the human body. The definition of an IVD typically involves the examination of specimens such as blood, urine, tissue, etc., in vitro (outside the body).
- This device analyzes medical images. qXR-PTX-PE analyzes chest X-ray images, which are generated by imaging the human body, not by taking a sample from it.
- The intended use is for image analysis and triage. The description clearly states that the device analyzes radiological images for the presence of suspected findings to aid in worklist prioritization. This is a function related to image interpretation and workflow management, not the analysis of biological samples.
Therefore, qXR-PTX-PE falls under the category of medical imaging software or a computer-aided detection/triage device, not an In Vitro Diagnostic.
No
The letter does not state that the FDA has reviewed and approved or cleared a PCCP for this specific device.
Intended Use / Indications for Use
qXR-PTX-PE is a radiological computer-assisted triage and notification software that analyzes adult chest X-ray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). qXR-PTX-PE uses an artificial intelligence algorithm to analyze images for features suggestive of critical findings and provides case-level output available in the PACS/workstation for worklist prioritization or triage.
As a passive notification for prioritization-only software tool within standard of care workflow, gXR-PTX-PE does not send a proactive alert directly to the appropriately trained medical specialists. qXR-PTX-PE is not intended to direct attention to specific portions of an image or to anomalies other than pleural effusion and/or pneumothorax. Its results are not intended to be used on a stand-alone basis for clinical decision-making.
Product codes
QFM
Device Description
qXR-PTX-PE is a radiological computer aided triage and notification software that analyses adult frontal (AP or PA views) CXR images for the presence of pre-specified suspected target conditions (pleural effusion and/or pneumothorax). The algorithm was trained on training data from across the world. The training dataset consisted of 74% of the data from India, 20.04% from the EU, 3.9% from the US, 1.4% from Brazil and 0.63% from Vietnam. The input for qXR-PTX-PE is a frontal chest X-ray (AP and PA view) in digital imaging and communications in medicine (DICOM) format
Chest X-rays are sent to qXR-PTX-PE by the means of transmission within the user's image storage system (e.g., Picture Archiving and Communication System (PACS)) or other radiological imaging equipment (e.g., X-ray systems) and processed by the qXR-PTX-PE for analysis. Following receipt of chest radiographs, the software device automatically analyses each image to detect features suggestive of pneumothorax and/or pleural effusion.
This would allow the appropriately trained medical specialists to group suspicious exams together that may potentially benefit for their prioritization. Chest radiographs without the suspicious findings are placed in the worklist for routine review, which is the standard of care at present. A secondary capture is available for the information on presence of the suspicious findings.
qXR-PTX-PE does not provide any proactive alerts. qXR-PTX-PE is not intended to direct attention to specific portions of the image. The results are not intended to be used on a standalone basis for clinical decision-making nor is it intended to rule out the target conditions or otherwise preclude clinical assessment of X-Ray cases.
Mentions image processing
Yes
Mentions AI, DNN, or ML
qXR-PTX-PE uses an artificial intelligence algorithm to analyze images for features suggestive of critical findings and provides case-level output available in the PACS/workstation for worklist prioritization or triage.
Al algorithm designed to detect pleural effusion and pneumothorax in chest X-ray images. Lunit INSIGHT CXR Triage uses a vendor agnostic algorithm compatible with DICOM chest X-ray images
qXR-PTX-PE uses an AI algorithm to detect pneumothorax and pleural effusion on chest X-ray images. qXR-PTX-PE uses a vendor agnostic algorithm compatible with DICOM chest X-ray images
Input Imaging Modality
Chest X-ray
Anatomical Site
Chest/Lung Frontal
Indicated Patient Age Range
adult
Intended User / Care Setting
Radiologists, clinicians, and other appropriately trained medical specialists qualified to read chest radiographs
Description of the training set, sample size, data source, and annotation protocol
The algorithm was trained on training data from across the world. The training dataset consisted of 74% of the data from India, 20.04% from the EU, 3.9% from the US, 1.4% from Brazil and 0.63% from Vietnam.
Description of the test set, sample size, data source, and annotation protocol
The study for pneumothorax included 613 scans (201 scans with pneumothorax and 412 scans without pneumothorax) from various parts of the US. The dataset was obtained from various hospitals across the US in order to generate evidence on the device function in various subgroups. A data collection protocol was set in place to ensure that there were sufficient numbers of important subgroups. The protocol specifies the inclusion and exclusion criteria and the expected numbers of cases and controls to be included. The dataset consisted of 289 males, 287 females and 37 were without this information available. The cases were aged from 22 years to above 85 years. The dataset was obtained from various hospitals across the US (179 from the Midwest, 152 from the Northeast and 12 from the South, region for 145 were not known) in order to generate evidence on the device function in various subgroups. The dataset consisted of clinical confounders that included opacities, presence of hardware, emphysema, scarring, mediastinal widening and pleural thickening. The dataset was also obtained from various X-ray device manufacturers to ensure consistent performance.
The test set was obtained from sites that were different from the training data sites and therefore this ensured the independence of the test data from training data.
The ground truth was established by 3 ABR thoracic radiologists with a minimum of 10 years of experience.
The study for pleural effusion included 1070 scans (344 scans with pleural effusion and 726 scans without pleural effusion) from various parts of the US. A data collection protocol was set in place to ensure that there were sufficient numbers of important subgroups. The protocol specifies the inclusion criteria and the expected numbers of cases and controls to be included. There were 498 scans from females and 551 from males, 21 samples did not have this information available. The ages ranged from 22 years to greater than 85 years. The samples were from various parts of the US (278 from the Midwest, 213 from the Northeast, 6 from the West. For 265, this information was not known). The dataset consisted of clinical confounders that included opacities, presence of hardware, emphysema, scarring, mediastinal widening and pleural thickening. The dataset was also obtained from various X-ray device manufacturers to ensure consistent performance. The test set was obtained from sites that were different from the training data sites and therefore this ensured the independence of the test data from training data.
The ground truth was established by 3 ABR thoracic radiologists with a minimum of 10 years of experience.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Software verification and validation testing were conducted. Clinical studies were conducted on retrospectively collected Chest X-rays to evaluate the performance of qXR-PTX-PE for triaging of pneumothorax and pleural effusion.
Pneumothorax Study:
Sample Size: 613 scans (201 with pneumothorax, 412 without pneumothorax).
AUC: 0.9894 (95% Cl: 0.9828 - 0.9982).
Sensitivity: 94.53% (95% CI: 90.42-97.24).
Specificity: 96.36% (95% CI: 94.07-97.95).
The device's generalizability for pneumothorax was ensured by performing subgroup analyses by gender and age, showing consistent AUC results across these subgroups.
Pleural Effusion Study:
Sample Size: 1070 scans (344 with pleural effusion, 726 without pleural effusion).
AUC: 0.9890 (95% Cl: 0.9847 - 0.9944).
Sensitivity: 96.22% (95% CI: 93.62-97.97).
Specificity: 94.90% (95% CI: 93.04-96.39).
The device's generalizability for pleural effusion was also ensured by performing subgroup analyses by gender and age, showing consistent AUC results across these subgroups.
Performance Time:
Average time taken for notification: 10 seconds. This is comparable to the predicate device (14.66s).
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Pneumothorax:
ROC AUC: 0.9894 (95% CI: 0.9829, 0.9980)
Sensitivity: 94.53% (95% CI: 90.42, 97.24)
Specificity: 96.36% (95% CI: 94.07, 97.95)
Pleural Effusion:
ROC AUC: 0.989 (95% CI: 0.9847, 0.9944)
Sensitivity: 96.22% (95% CI: 93.62, 97.97)
Specificity: 94.90% (95% CI: 93.04, 96.39)
Predicate Device(s)
Reference Device(s)
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 892.2080 Radiological computer aided triage and notification software.
(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.
0
Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which consists of the letters "FDA" in a blue square, followed by the words "U.S. FOOD & DRUG" in blue, with the word "ADMINISTRATION" underneath.
Qure.ai Technologies % Ayushi Mahendra Senior Regulatory Affairs Specialist Level 7, Commerz II International Business Park Oberoi Garden City, Goregaon (E) Mumbai, Maharashtra 400063 INDIA
Re: K230899
August 22, 2023
Trade/Device Name: qXR-PTX-PE Regulation Number: 21 CFR 892.2080 Regulation Name: Radiological computer aided triage and notification software Regulatory Class: Class II Product Code: QFM Dated: July 24, 2023 Received: July 24, 2023
Dear Ayushi Mahendra:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's
1
requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (OS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Jessica Lamb
Jessica Lamb, Ph.D. Assistant Director Imaging Software Team DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K230899
Device Name qXR-PTX-PE
Indications for Use (Describe)
qXR-PTX-PE is a radiological computer-assisted triage and notification software that analyzes adult chest X-ray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). qXR-PTX-PE uses an artificial intelligence algorithm to analyze images for features suggestive of critical findings and provides case-level output available in the PACS/workstation for worklist prioritization or triage.
As a passive notification for prioritization-only software tool within standard of care workflow, gXR-PTX-PE does not send a proactive alert directly to the appropriately trained medical specialists. qXR-PTX-PE is not intended to direct attention to specific portions of an image or to anomalies other than pleural effusion and/or pneumothorax. Its results are not intended to be used on a stand-alone basis for clinical decision-making.
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
510(k) SUMMARY
Qure.ai's qXR-PTX-PE
1 SUBMITTER
K230899
Qure.ai Technologies Level 7, Commerz II, International Business Park Oberoi Garden City, Goregaon (E), Mumbai 400 063 Phone: +91-9768123013 Primary Contact Person: Ayushi Mahendra Secondary contact person: Sri Anusha Matta
Date Prepared: August 12, 2023
2 DEVICE
| Name of Device: | qXR-PTX-PE |
|---|---|
| Common or Usual Name: | Radiological Computer Assisted Prioritization Software for Lesions |
| Classification Name: | Radiological Computer Aided Triage and Notification Software |
| Regulatory Class: | Class II |
| Regulation Number: | 21 CFR 892.2080 |
| Product Code: | QFM |
3 PREDICATE DEVICE
| Name of Device: | Lunit INSIGHT CXR Triage |
|---|---|
| Manufacturer: | Lunit Inc. |
| 510(k) Number: | K211733 |
4 INTENDED USE / INDICATIONS FOR USE:
qXR-PTX-PE is a radiological computer-assisted triage and notification software that analyzes adult chest Xray images for the presence of pre-specified suspected critical findings (pleural effusion and/or pneumothorax). qXR-PTX-PE uses an artificial intelligence algorithm to analyze images for features
4
suggestive of critical findings and provides case-level output available in the PACS/workstation for worklist prioritization or triage.
As a passive notification for prioritization-only software tool within standard of care workflow, qXR-PTX-PE does not send a proactive alert directly to the appropriately trained medical specialists. qXR-PTX-PE is not intended to direct attention to specific portions of an image or to anomalies other than pleural effusion and/or pneumothorax. Its results are not intended to be used on a stand-alone basis for clinical decisionmaking.
5 DEVICE DESCRIPTION
qXR-PTX-PE is a radiological computer aided triage and notification software that analyses adult frontal (AP or PA views) CXR images for the presence of pre-specified suspected target conditions (pleural effusion and/or pneumothorax). The algorithm was trained on training data from across the world. The training dataset consisted of 74% of the data from India, 20.04% from the EU, 3.9% from the US, 1.4% from Brazil and 0.63% from Vietnam. The input for qXR-PTX-PE is a frontal chest X-ray (AP and PA view) in digital imaging and communications in medicine (DICOM) format
Chest X-rays are sent to qXR-PTX-PE by the means of transmission within the user's image storage system (e.g., Picture Archiving and Communication System (PACS)) or other radiological imaging equipment (e.g., X-ray systems) and processed by the qXR-PTX-PE for analysis. Following receipt of chest radiographs, the software device automatically analyses each image to detect features suggestive of pneumothorax and/or pleural effusion.
This would allow the appropriately trained medical specialists to group suspicious exams together that may potentially benefit for their prioritization. Chest radiographs without the suspicious findings are placed in the worklist for routine review, which is the standard of care at present. A secondary capture is available for the information on presence of the suspicious findings.
qXR-PTX-PE does not provide any proactive alerts. qXR-PTX-PE is not intended to direct attention to specific portions of the image. The results are not intended to be used on a standalone basis for clinical decision-making nor is it intended to rule out the target conditions or otherwise preclude clinical assessment of X-Ray cases.
б COMPARISON OF THE PREDICATE DEVICE
qXR-PTX-PE is technologically similar to the predicate device, Lunit INSIGHT CXR Triage in regards to intended use and technological characteristics. Both are radiological computer-assisted triage and notification software intended to read chest X-rays for the presence of pre-specified target conditions (pneumothorax and pleural effusion). The algorithms function similarly and with the same purpose of workflow notification. There are no notable technological differences between the subject and predicate devices.
In terms of establishing substantial equivalence, the subject and predicate device have intended use, as an image processing tool that triages images for features suggestive of critical findings and produces case-level output. The indications for use proposed for the subject device are similar to those of the predicate device.
5
| Predicate Device | Subject Device | |
|---|---|---|
| Lunit INSIGHT CXR Triage | qXR-PTX-PE | |
| Device Name | Lunit INSIGHT CXR Triage | qXR-PTX-PE |
| 510(k) Number | K211733 | |
| Regulation | 21 CFR 892.2080 | 21 CFR 892.2080 |
| Regulation Description | Radiological computer aided | |
| triage and notification software | Radiological computer aided triage | |
| and notification software | ||
| Product Code | QFM | QFM |
| Device type | Radiological Computer-Assisted | |
| Prioritization Software For Lesions | Radiological Computer-Assisted | |
| Prioritization Software For Lesions | ||
| Manufacturer | Lunit Inc. | Qure.ai Technologies |
| Intended use / Indications | ||
| for Use | Lunit INSIGHT CXR Triage is a | |
| radiological computer-assisted | ||
| triage and notification software | ||
| that analyzes adult chest X-ray | ||
| images for the presence of pre- | ||
| specified suspected critical | ||
| findings (pleural effusions and/or | ||
| pneumothorax). Lunit INSIGHT | ||
| CXR Triage uses an artificial | ||
| intelligence algorithm to analyze | ||
| images for features suggestive of | ||
| critical findings and produce case- | ||
| level output available in the | ||
| PACS/workstation for worklist | ||
| prioritization or triage. | ||
| As a passive notification for | ||
| prioritization-only software tool | ||
| within standard of care workflow, | ||
| Lunit Insight CXR triage does | ||
| not send a proactive alert directly | ||
| to appropriately trained medical | ||
| specialists. Lunit INSIGHT CXR | ||
| Triage is not intended to direct | ||
| attention to specific portions of | ||
| an image. Its results are not | ||
| intended to be used on a | ||
| standalone basis for clinical | ||
| decision-making. | qXR-PTX-PE is a radiological | |
| computer-assisted triage and | ||
| notification software that analyzes | ||
| adult chest X-ray images for the | ||
| presence of pre-specified suspected | ||
| critical findings (pleural effusion | ||
| and/or pneumothorax). qXR-PTX-PE | ||
| uses an artificial intelligence | ||
| algorithm to analyze images for | ||
| features suggestive of critical findings | ||
| and provides case-level output | ||
| available in the PACS/workstation for | ||
| worklist prioritization or triage. | ||
| As a passive notification for | ||
| prioritization-only software tool | ||
| within standard of care workflow, | ||
| qXR-PTX-PE does not send a proactive | ||
| alert directly to the appropriately | ||
| trained medical specialists. qXR-PTX- | ||
| PE is not intended to direct attention | ||
| to specific portions of an image or to | ||
| anomalies other than pleural effusion | ||
| and/or pneumothorax. Its results are | ||
| not intended to be used on a stand- | ||
| alone basis for clinical decision- | ||
| making | ||
| Intended User | Appropriately trained medical | |
| specialists who are qualified to | ||
| interpret chest radiographs. | Radiologists, clinicians, and other | |
| appropriately trained medical | ||
| specialists qualified to read chest | ||
| radiographs | ||
| Modality | Chest X-ray | Chest X-ray |
| Predicate Device | Subject Device | |
| Lunit INSIGHT CXR Triage | qXR-PTX-PE | |
| Target clinical conditions | Pleural effusion, Pneumothorax | |
| on Chest/Lung Frontal Chest X-ray | Pneumothorax, Pleural effusion on | |
| Chest/Lung Frontal Chest X-rays | ||
| Algorithm for pre- | ||
| specified critical findings | ||
| detection | Al algorithm designed to detect | |
| pleural effusion and | ||
| pneumothorax in chest X-ray | ||
| images. Lunit INSIGHT CXR Triage | ||
| uses a vendor agnostic algorithm | ||
| compatible with DICOM chest X- | ||
| ray images | qXR-PTX-PE uses an AI algorithm to | |
| detect pneumothorax and pleural | ||
| effusion on chest X-ray images. qXR- | ||
| PTX-PE uses a vendor agnostic | ||
| algorithm compatible with DICOM | ||
| chest X-ray images | ||
| Notification only/Parallel | ||
| workflow | Yes | Yes |
| Input format | DICOM | DICOM |
| Device output in case of | ||
| positive detection | When deployed on other | |
| radiological imaging equipment, | ||
| Lunit INSIGHT CXR Triage | ||
| automatically runs after image | ||
| acquisition and prioritizes and | ||
| displays the analysis result | ||
| through the worklist interface of | ||
| PACS/workstation. | ||
| No markup on original image. | ||
| Secondary capture of the finding. | ||
| Upon image acquisition from | ||
| other radiological imaging | ||
| equipment (e.g. X-ray systems), | ||
| an on-device, technologist | ||
| notification indicating which cases | ||
| were flagged by Lunit INSIGHT | ||
| CXR Triage in PACS, is generated. | ||
| The on device notification is | ||
| contextual and does not provide | ||
| any diagnostic information. It is | ||
| not intended to inform any clinical | ||
| decision, prioritization, or action | ||
| to the technologist | When deployed on other radiological | |
| imaging equipment, qXR-PTX-PE will | ||
| automatically run after image | ||
| acquisition to perform triage. It | ||
| displays the analysis result through | ||
| the worklist interface of | ||
| PACS/workstation. | ||
| No markup of the conditions will be | ||
| done on the original image. | ||
| Secondary capture of the device will | ||
| indicate the presence of findings | ||
| suspicious of pneumothorax or | ||
| pleural effusion. | ||
| Upon image acquisition from other | ||
| radiological imaging equipment (e.g. | ||
| X-ray systems) a passive notification | ||
| is generated. | ||
| Notification (i.e., | Passive notification. Images with | Passive notification. Images with |
| recipient, timing and | ||
| means of notification) | suspicion of pleural effusion | |
| and/or pneumothorax are flagged | ||
| in PACS/workstation. | suspicion of pneumothorax and/or | |
| pleural effusion are flagged in | ||
| PACS/workstation/DICOM viewer. | ||
| Where generated results | ||
| (i.e., DICOM files) are | ||
| stored | PACS/Workstation | PACS/Workstation/DICOM viewer |
Table 1 Comparison between qXR-PTX-PE and the Predicate Device
6
7
| Predicate Device | Subject Device | |||
|---|---|---|---|---|
| Lunit INSIGHT CXR Triage | qXR-PTX-PE | |||
| Performance metrics of the predicate device and qXR-PTX-PE | ||||
| Performance level - | ||||
| Timing of notification | The average time taken for the | |||
| notification to travel from the | ||||
| Lunit INSIGHT CXR Triage to the | ||||
| point at which the result is | ||||
| displayed in the destination | ||||
| PACS/RIS/EPR worklist is 14.66 | ||||
| seconds. | The average time taken for the | |||
| notification to travel from qXR-PTX- | ||||
| PE to the point at which the result is | ||||
| displayed in the destination Picture | ||||
| Archiving and Communication System | ||||
| (PACS) or workstation/digital | ||||
| radiographic processing system (ex. | ||||
| digital radiography, digital X-ray | ||||
| system etc.) is 10 seconds. | ||||
| Performance level - | ||||
| accuracy of classification | Pleural Effusion | |||
| ROC AUC > 0.95 | ||||
| AUC: 0.9686 (95% Cl: [0.9547, | ||||
| 0.9824]) | ||||
| Sensitivity 89.86% (95% CI: [86.72, | ||||
| 93.00]) | ||||
| Specificity 93.48% (95% CI: [91.06, | ||||
| 95.91]) | ||||
| Pneumothorax | ||||
| ROC AUC > 0.95 | ||||
| AUC: 0.9630 (95% Cl: [0.9521, | ||||
| 0.9739]) | ||||
| Sensitivity 88.92% (95% CI: [85.60, | ||||
| 92.24]) Specificity 90.51% (95% CI: | ||||
| [88.18, 92.83]) | Pneumothorax | |||
| ROC AUC > 0.95 | ||||
| AUC: 0.9894 (95% CI: [0.9829, | ||||
| 0.9980]) | ||||
| Sensitivity 94.53% (95% CI: [90.42, | ||||
| 97.24]) | ||||
| Specificity 96.36% (95% CI: [94.07, | ||||
| 97.95]) | ||||
| Pleural Effusion | ||||
| ROC AUC > 0.95 | ||||
| AUC: 0.989 (95% Cl: [0.9847, 0.9944]) | ||||
| Sensitivity 96.22% (95% CI: [93.62, | ||||
| 97.97]) | ||||
| Specificity 94.90% (95% CI: [93.04, | ||||
| 96.39]) |
7 TESTING
Software
Software verification and validation testing were conducted, and documentation was provided as recommended by FDA's Guidance for Industry and FDA Staff, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices." The software for this device has a Moderate level of concern.
Performance Testing - Clinical
The performance of qXR-PTX-PE was validated by clinical tests. All the safety parameters of the device were verified in accordance with the software specifications and applicable performance standards and met the acceptance criteria (Device shows > 95% AUC) (passed), demonstrating that the software fulfills all its requirement specifications.
Clinical studies were conducted on retrospectively collected Chest X-rays to evaluate the performance of qXR-PTX-PE for triaging of pneumothorax and pleural effusion.
8
The study for pneumothorax included 613 scans (201 scans with pneumothorax and 412 scans without pneumothorax) from various parts of the US. The dataset was obtained from various hospitals across the US in order to generate evidence on the device function in various subgroups. A data collection protocol was set in place to ensure that there were sufficient numbers of important subgroups. The protocol specifies the inclusion and exclusion criteria and the expected numbers of cases and controls to be included. The dataset consisted of 289 males, 287 females and 37 were without this information available. The cases were aged from 22 years to above 85 years. The dataset was obtained from various hospitals across the US (179 from the Midwest, 152 from the Northeast and 12 from the South, region for 145 were not known) in order to generate evidence on the device function in various subgroups. The dataset consisted of clinical confounders that included opacities, presence of hardware, emphysema, scarring, mediastinal widening and pleural thickening. The dataset was also obtained from various X-ray device manufacturers to ensure consistent performance.
The algorithm was trained on training data from across the world. The training dataset consisted of 74% of the data from India, 20.04% from the US, 1.4% from Brazil and 0.63% from Vietnam. The test set was obtained from sites that were different from the training data sites and therefore this ensured the independence of the test data from training data.
The ground truth was established by 3 ABR thoracic radiologists with a minimum of 10 years of experience. The AUC of the device in triaging scans with findings suspicious of pneumothorax exceeded the success criteria with (AUC 98.94 95% Cl (98.28 - 99.82)), Sensitivity 94.53 (90.42-97.24) and Specificity 96.36 (94.07-97.95). The predicate Lunit INSIGHT CXR Triage's performance was ROC AUC 0.9630 (95% Cl: 0.9521 - 0.9739), Sensitivity 88.92% (95% Cl: 85.60 - 92.24) and Specificity 90.51% (95% Cl: 88.18 - 92.83).
| AUC (95% CI) | Sensitivity (95% CI), TP/P | Specificity (95% CI), TN/N |
|---|---|---|
| 98.94 (98.28 - 99.82) | 94.53 (90.42-97.24), 190/201 | 96.36 (94.07-97.95), 397/412 |
Table 2 Overall Results of Accuracy Testing of qXR-PTX-PE for Pneumothorax
The device's generalizability was ensured by performing subgroup analyses. The results for pneumothorax were consistent in both genders and the AUC was 98.67 (97.63-100) in male, 99.3(98.64-100) in female. An AUC of 92.47(84.95-100) was observed in the scans with unknown gender. The device's results were also found to be consistent in a wide range of ages: with AUC of 99.12 (98.34-100) in the 22-44 age group, 99.17 (98.37-100) in the 45-64 age group, 98.69 (97.61-100) in the 65-84 age group and 98.62 (97.25-100.00) in the 85 or greather age group age group.
The study for pleural effusion included 1070 scans (344 scans with pleural effusion and 726 scans without pleural effusion) from various parts of the US. A data collection protocol was set in place to ensure that there were sufficient numbers of important subgroups. The protocol specifies the inclusion criteria and the expected numbers of cases and controls to be included. There were 498 scans from females and 551 from males, 21 samples did not have this information available. The ages ranged from 22 years to greater than 85 years. The samples were from various parts of the US (278 from the Midwest, 213 from the Northeast, 6 from the West. For 265, this information was not known). The dataset consisted of clinical confounders that included opacities, presence of hardware, emphysema, scarring, mediastinal widening and pleural thickening. The dataset was also obtained from various X-ray device manufacturers to ensure consistent performance. The algorithm was training data from across the world. The training dataset consisted of 74% of the data from India, 20.04% from the EU, 3.9%
9
from the US, 1.4% from Brazil and 0.63% from Vietnam. The test set was obtained from sites that were different from the training data sites and therefore this ensured the independence of the test data from training data.
The ground truth was established by 3 ABR thoracic radiologists with a minimum of 10 years of experience. The AUC of the device in triaging scans with findings suspicious of pleural effusion with AUC 98.90 (98.47 -99.44)), sensitivity 96.22 (93.62-97.97) and specificity 94.90 (93.04-96.39). The predicate, Lunit INSIGHT CXR Triage, reported a performance of ROC AUC 0.9686 (95% Cl: 0.9547 - 0.9824), sensitivity 89.86% (95% CI: 86.72 - 93.00) and specificity 93.48% (95% CI: 91.06 - 95.91).
| AUC (95% CI) | Sensitivity (95% CI), TP/P | Specificity (95% CI), TN/N |
|---|---|---|
| 98.90 (98.47 - 99.44) | 96.22 (93.62-97.97), 331/344 | 94.90 (93.04-96.39), 689/726 |
The device's generalizability for triaging scans with pleural effusion was ensured by performing subgroup analyses. The results for pleural effusion were consistent in both genders and the AUC was 98.73 (98.00-99.74) in female, 99.11(98.62-99.72) in male. An AUC of 96.3(92.59-100) was observed in the scans with unknown gender. The device's results were also found to be consistent in a wide range of ages: with AUC of 99.27 (98.56-100) in the 22-44 age group, 98.1 (96.83-99.95) in the 45-64 age group, 98.78 (98.09-99.62) in the 65-84 age group and 99.2 (98.39-100.00) in the 85 or greater age group. The device's performance time was also assessed and for qXR-PTX-PE. It was the time to analyze the study and send the notification to the worklist. The performance time averaged at 10s. This is comparable to the performance of the predicate performance of 14.66s. This is also comparable to other commercially cleared products with the similar intended use.
8 CONCLUSION
The comparison in Table 1 and the software and performance testing presented above demonstrate that the qXR-PTX-PE device is substantially equivalent to the predicate device. The qXR-PTX-PE is a software only device, similar to the predicate (Lunit INSIGHT CXR Triage). It is as safe and effective as the predicate device. The qXR-PTX-PE has the same intended users and similar indications, technological characteristics, and principles of operation as the predicate device. There are no differences in the indications, therefore there no is risk of its safety and effectiveness being affected when used as labelled. Both devices operate in parallel to the standard of care workflow. The performance testing demonstrates that the qXR-PTX-PE performs as intended and is therefore substantially equivalent to the predicate. Software and Clinical testing supports that the device performs in according with the device requirements.