The device is intended to assist hospital networks and trained medical specialists in workflow triage by flagging the following suspected positive findings of pathologies in head CT images: intracranial hemorrhage, mass effect, midline shift and cranial fracture.

qER uses an artificial intelligence algorithm to analyze images on a standalone cloud-based application in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for cases with suspected findings. Notifications include non-diagnostic preview images that are meant for informational purposes only. The device does not alter the original medical image and is not intended to be used as a diagnostic device.

The results of the device are intended to be used in conjunction information and based on professional judgment, to assist with triage/prioritization of medical images. Notified clinicians are responsible for viewing full images per the standard of care.

Device Description

Qure.ai Head CT scan interpretation software, qER, is a deep-learning-based software device that analyses head CT scans for signs of intracranial hemorrhage, midline shift, mass effect or cranial fractures in order to prioritize them for clinical review. The standalone software device consists of an on-premise module and a cloud module. qER accepts non-contrast adult head CT scan DICOM files as input and provides a priority flag indicating critical scans. Additionally, the software has the preview of critical scans to the medical specialist.

AI/ML Overview

Here's a breakdown of the acceptance criteria and the study proving the qER device meets them, based on the provided text:

Acceptance Criteria and Device Performance

The core purpose of the qER device is workflow triage by identifying suspected positive findings of pathologies in head CT images. The performance data presented focuses on the device's ability to accurately detect these pathologies in a standalone setting.

1. Table of Acceptance Criteria and Reported Device Performance

While the document doesn't explicitly state "acceptance criteria" as numerical thresholds beyond "exceeded the predefined success criteria, as well as the required performance criteria for triage and notification software as per the special controls for QAS," the reported sensitivities and specificities for each pathology effectively serve as the demonstrated "acceptance" level the device achieved.

Abnormality	Acceptance Criteria (Implied Success)	Reported Device Performance (Sensitivity [95% CI])	Reported Device Performance (Specificity [95% CI])	Reported Device Performance (AUC [95% CI])
Intracranial Hemorrhage	High sensitivity & specificity for triage	96.98 (95.32 - 98.17)	93.92 (91.87 - 95.58)	98.53 (98.00 - 99.15)
Cranial Fracture	High sensitivity & specificity for triage	96.77 (93.74 - 98.60)	92.72 (91.00 - 94.21)	97.66 (96.88 - 98.57)
Mass Effect	High sensitivity & specificity for triage	96.39 (94.28 - 97.88)	96.00 (94.45 - 97.21)	99.09 (98.73 - 99.52)
Midline Shift	High sensitivity & specificity for triage	97.34 (95.30 - 98.67)	95.36 (93.79 - 96.64)	99.09 (98.74 - 99.51)
Any of the 4 target abnormalities	High sensitivity & specificity for triage	98.53 (97.45 - 99.24)	91.22 (88.39 - 93.55)	NA

Additionally, a key performance metric for a triage device is the time to notification:

Parameter	Acceptance Criteria (Implied improvement over std. care)	Reported Device Performance (Mean [95% CI])	Reported Device Performance (Median [95% CI])
Time to open exam in the standard of care	Benchmark for comparison	65.54 (59.14 - 71.76) min	60.01 (54.57 - 77.63) min
Time-to-notification with qER	Significantly lower than standard of care	2.11 (1.45 - 2.61) min	1.21 (1.12 - 1.25) min

Study Details

2. Sample Size Used for the Test Set and Data Provenance

Sample Size: 1320 head CT scans.
Data Provenance: Retrospective, multicenter study. Data originated from multiple locations within the United States.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications

Number of Experts: 3 board-certified radiologists.
Qualifications: The document explicitly states "board-certified radiologists." No further details on years of experience are provided.

4. Adjudication Method for the Test Set

The text states that the ground truth was established by "3 board-certified radiologists reading the scans." It does not explicitly mention an adjudication method (e.g., 2+1, 3+1 consensus). It is implied that their readings defined the ground truth, but the process of resolving discrepancies among the three readers is not detailed.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, and what was the effect size of how much human readers improve with AI vs without AI assistance

The provided text does not describe an MRMC comparative effectiveness study where human readers' performance with and without AI assistance was directly compared. The study primarily focuses on the standalone performance of the qER algorithm and its ability to reduce the "time to notification" compared to standard of care "time to open." While the "time-to-notification" analysis suggests a significant workflow improvement when using qER for triage (2.11 mins vs. 65.54 mins), this is not a direct measure of human reader diagnostic accuracy improvement with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance study was done. The "Performance Data" section explicitly states, "A retrospective, multicenter, blinded clinical study was conducted to test the accuracy of qER at triaging head CT scans... Sensitivity and specificity exceeded the predefined success criteria... demonstrating the ability of the qER device to effectively triage studies containing one of these conditions." The results in Table 2 are for the qER algorithm's accuracy independently.

7. The Type of Ground Truth Used

Expert Consensus: The ground truth for the pathologies (Intracranial hemorrhage, cranial fractures, mass effect, midline shift, and absence of these abnormalities) was established by "3 board-certified radiologists reading the scans." This indicates an expert consensus approach to defining the ground truth.

8. The Sample Size for the Training Set

The document does not specify the sample size used for the training set. It mentions that the qER software uses "a pre-trained artificial intelligence algorithm" and "a pre-trained classification convolutional neural network (CNN) that has been trained to detect a specific abnormality from head CT scan images." However, the size of the dataset used for this training is not disclosed in the provided text.

9. How the Ground Truth for the Training Set was Established

The document does not explicitly describe how the ground truth for the training set was established. It only states that the CNN was "pre-trained" on medical images to detect specific abnormalities. It is common practice for such training to also rely on expert annotations, but this is not detailed for the training set in this document.

Summary

{0}------------------------------------------------

June 17, 2020

Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: on the left, there is the Department of Health and Human Services logo, which features an abstract image of a human figure. To the right of this is the text "FDA U.S. FOOD & DRUG ADMINISTRATION" in blue font. The word "FDA" is in a larger, bolder font than the rest of the text.

Qure.ai Technologies % Pooja Rao Head, Research and Development Level 7, Commerz II, International Business Park, Oberoi Garden City, Goregaon East Mumbai, Maharashtra 400063 INDIA

Re: K200921

Trade/Device Name: qER Regulation Number: 21 CFR 892.2080 Regulation Name: Radiological computer-assisted triage and notification software Regulatory Class: Class II Product Code: QAS Dated: June 11, 2020 Received: June 11, 2020

Dear Pooja Rao:

We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part

{1}------------------------------------------------

801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4. Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely,

For

Thalia T. Mills, Ph.D. Director Division of Radiological Health OHT7: Office of In Vitro Diagnostics and Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

{2}------------------------------------------------

Indications for Use

510(k) Number (if known) K200921

Device Name qER

Indications for Use (Describe)

qER is a radiological computer aided triage and notification software in the analysis of non-contrast head CT images.

Type of Use (Select one or both, as applicable)
☑ Prescription Use (Part 21 CFR 801 Subpart D)	☐ Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED## CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{3}------------------------------------------------

510(k) SUMMARY

Qure.ai's qER

Submitter:

K200921

Qure.ai Technologies
Level 7, Commerz II,
International Business Park
Oberoi Garden City, Goregaon (E),
Mumbai- 400 063
Phone: +91-9820474098

Contact Person: Pooja Rao

Date Prepared:	June 11, 2010
Name of Device:	qER
Common or Usual Name:	Radiological Computer-Assisted Tria
Classification Name:	Radiological Computer-Assisted Tria

Date Prepared:	June 11, 2018
Name of Device:	qER
Common or Usual Name:	Radiological Computer-Assisted Triage and Notification Software
Classification Name:	Radiological Computer-Assisted Triage and Notification Software
Regulatory Class:	Class II
Product Code:	QAS (21 C.F.R. 892.2080)
Predicate Device:	Aidoc's Briefcase Software (K180647)
Reference Device:	Aidoc's Briefcase Software (for CSF Triage) (K190896)

Device Description:

Intended Use / Indications for Use:

The qER device is a radiological computer aided triage and notification software indicated for use in the analysis of non-contrast head CT images.

{4}------------------------------------------------

The results of the device are intended to be used in conjunction with other patient information and based on professional judgment, to assist with triage/prioritization of medical images. Notified clinicians are responsible for viewing full images per the standard of care.

Technological Characteristics

The qER software consists of two major components: 1) a configurable gateway as an interface with the client systems such as the PACS and worklist; and 2) analysis module that hosts the pre-trained artificial intelligence algorithms. qER software can be deployed completely on client premise) or on cloud servers (oncloud). For on-cloud processing mode, an on-premise gateway is deployed that interfaces with the HIPAAcompliant cloud server(s) where the analysis is performed.

Additionally, a configuration module allows control over triage settings and output formats.

The deep learning analysis module underlying qER consists of a independent algorithms. The core component of each algorithm is a pre-trained classification convolutional neural network (CNN) that has been trained to detect a specific abnormality from head CT scan images. This core component is coupled with a preprocessing module that transforms the CT scan series to a set of images and a post-processing module that combines slice-level outputs to a scan-level triage result for each abnormality.

A predefined threshold is applied to each of the 4 scan-level outputs, to determine the presence (positive, 1) or absence (negative, 0) of the abnormality for purposes of triage for further review. If one or more of the 4 abnormalities are present, the scan is positive for 'critical abnormality' and is triaged by placing an indicator on the worklist with the condition for which it was triaged identified.

Principles of Operation

qER is an artificial intelligence software device that screens head CT scans for signs of ICH, mass effect, midline shift and cranial fractures in order to prioritize them for clinical review. Each target abnormality is detected using an independent underlying pre-trained deep learning algorithm. Users can control the triage process by selecting one or more of the four target critical abnormalities identified by the software and can turn triage on or off for each abnormality. A dedicated, secure user settings webpage is to be made accessible to users for this purpose

The qER device is intended for use in parallel to the standard of care workflow. With the addition of the subject device to the workflow, the user is able to prioritize scans based on a suspected critical abnormality detected and flagged by qER.

Once deployed in conjunction with a PACS or CT scanner, the device automatically processes incoming CT scans and produces triage results that appear on the user's worklist as a priority flag. For every head CT scan identified by qER as containing one or more target abnormalities, a notification is sent to the worklist. For positive head CT scans, non-diagnostic preview images are returned to the PACS. These preview images are smaller in size than the original CT scan image and have a limited dynamic range. The notification and the preview image may be used as a reference by the end-user when deciding which study to read next.

{5}------------------------------------------------

Substantial Equivalence

The above indications for use statement for the subject device is substantially similar to the cleared indications for Aidoc's Briefcase device. Although the subject device is intended to flag mass effect, midline shift and cranial fractures in addition to ICH whereas the predicate flags intracranial hemorrhages only, this difference does not raise new questions of safety or effectiveness. Additionally, the reference device, Aidoc's Briefcase (for CSF Triage) (K190896) is indicated for use in the analysis of cervical spine CT images and is intended to assist radiologists by flagging linear lucencies in the cervical spine bone in patterns compatible with fractures. The overall effect of both the subject and predicate devices, namely the workflow triaging of CT images by flagging and communication of suspected findings of pathologies found in head CT images, remains the same.

Further, although the predicate device is intended to assist radiologists specifically as compared to the subject device, which is intended to assist trained medical specialists more generally, this difference does not raise new questions of safety or effectiveness as the function of the device is to simply triage CT images with the ultimate clinical decision making function continuing to the radiologists. The subject device triages head CT scans with any of the target findings; the user is provided with full control over which critical findings will be flagged through a settings menu.

In addition, the technological characteristics of both devices are very similar and the patient populations that the qER could be used for can be expected to overlap with that of the predicate device (i.e., patients suspected of having ICH abnormalities could also have mass effect, midline shift and/or a cranial fracture). In short, the target populations for which the qER and the predicate would be used for are largely the same.

In summary, the differences in the indications for use for the intended effect of the device as compared to the predicate device or raise any new questions of safety or effectiveness. Thus, the company maintains that the qER meets the first requirement of substantial equivalence.

Both the subject and predicate device consist of a module that handles the incoming studies, de-identifies and pushes them to the cloud module, receives results after processing on the analysis module, and re-identifies the studies and sends the results to the radiology PACS and worklist used by medical specialist. Both devices also consist of a module that performs the analysis using a pre-trained artificial intelligence algorithm.

The qER software and the predicate device are radiological computer aided triage and notification software systems that are indicated for use in the analysis of non-contrast-enhanced head CT images. Both devices are intended to assist hospital networks and trained medical specialists in workflow triage by flagging and communication of suspected positive findings of pathologies identified in head CT images.

At a high level, the subject and predicate devices are based on the following same technological elements:

. Both the subject and predicate device use an artificial intelligence algorithm to analyze images and highlight cases with detected pathologies in parallel to the ongoing standard of care image interpretation. As part of both the subject and predicate software, the user is presented with notifications for cases with suspected findings. Notifications include preview images that are meant for informational purposes only and not intended for diagnostic use beyond notification.
Further, both the subject and predicate device process CT images using similar techniques and a similar artificial intelligence algorithm. Specifically, the subject and predicate deep

{6}------------------------------------------------

learning algorithm which has been trained on medical images. The deep-learning process allows for high accuracy in the detection of initial suspect locations. As a system, the qER raises the same types of safety and effectiveness questions as the predicate device, namely, accurate detection of findings within the reviewed and processed study on which a physician can base a clinically useful triage/prioritization assessment considering all available clinical data.

Importantly, like the predicate, the device does not remove cases from a reading queue. It is also important to note that, similar to predicate, the subject device does not mark, highlight, or direct users' attention to a specific location in the original image. Further, both the subject and predicate software does not alter the original medical image, is not intended to be used as a diagnostic device and are to be used in parallel with the standard of care, which remains the default option for all cases.
A table comparing the key features of the subject and predicate devices is provided below.

{7}------------------------------------------------

Parameter	qER DeviceSubject Device	Aidoc Briefcase Software(K180647)
IndicationsFor Use	qER is a radiological computer aided triage andnotification software indicated for use in theanalysis of non-contrast head CT images.The device is intended to assist hospital networksand trained medical specialists in workflow triageby flagging the following suspected positivefindings of pathologies in head CT images:intracranial hemorrhage, mass effect, midline shiftand cranial fracture.qER uses an artificial intelligence algorithm toanalyze images on a standalone cloud-basedapplication in parallel to the ongoing standard ofcare image interpretation. The user is presentedwith notifications for cases with suspectedfindings. Notifications include non-diagnosticpreview images that are meant for informationalpurposes only. The device does not alter theoriginal medical image and is not intended to beused as a diagnostic device.The results of the device are intended to be usedin conjunction with other patient information andbased on professional judgment, to assist withtriage/prioritization of medical images. Notifiedclinicians are responsible for viewing full imagesper the standard of care.	BriefCase is a radiological computer aided triageand notification software indicated for use in theanalysis of non-enhanced head CT images.The device is intended to assist hospitalnetworks and trained radiologists in workflowtriage by flagging and communication ofsuspected positive findings of pathologies inhead CT images, namely IntracranialHemorrhage (ICH).BriefCase uses an artificial intelligence algorithmto analyze images and highlight cases withdetected ICH on a standalone desktopapplication in parallel to the ongoing standard ofcare image interpretation. The user is presentedwith notifications for cases with suspected ICHfindings. Notifications include compressedpreview images that are meant for informationalpurposes only and not intended for diagnosticuse beyond notification. The device does notalter the original medical image and is notintended to be used as a diagnostic device.The results of BriefCase are intended to be usedin conjunction with other patient informationand based on professional judgment, to assistwith triage/prioritization of medical images.Notified clinicians are responsible for viewing fullimages per the standard of care.
Classification /Product Code	21 CFR 892.2080/QAS	21 CFR 892.2080/QAS
DeviceComponents	1. An on-premise module that:a. Performs de-identification of studies and pushesthem to the cloud module.b. Receives results after processing on the cloudmodule.c. Re-identifies the studies and sends them to theradiology PACS and radiology worklist.2. A cloud module that performs the analysis usinga pre-trained artificial intelligence algorithm	1. An on-premise module that:a. Performs de-identification of studies andpushes them to the cloud module.b. Receives results after processing on the cloudmodule.c. Re-identifies the studies and sends them tothe radiology PACS and radiology worklist.2. A cloud module that performs the analysisusing a pre-trained artificial intelligence
Parameter	qER DeviceSubject Device	Aidoc Briefcase Software(K180647)
Anatomicalregion ofinterest	Head	Head
Dataacquisitionprotocol	Non contrast CT scan of the head	Non contrast CT scan of the head or neck
View DICOMdata	DICOM Information about the patient, study and current image	DICOM Information about the patient, study and current image
Segmentation of region ofinterest	No; device does not mark, highlight, or direct users' attention to a specific location in the original image.	No; device does not mark, highlight, or direct users' attention to a specific location in the original image.
Algorithm	Artificial intelligence algorithm with database of images.	Artificial intelligence algorithm with database of images.
Notification/Prioritization	Yes, with controls for the user to select the finding or combination of findings for triage	Yes
Previewimages	Presentation of a preview of the study for initial assessment not meant for diagnostic purposes. The device operates in parallel with the standard of care, which remains the default option for all cases.	Presentation of a preview of the study for initial assessment not meant for diagnostic purposes. The device operates in parallel with the standard of care, which remains the default option for all cases.
Alteration oforiginalimage	No	No
Removal ofcases fromworklistqueue	No	No
Abnormalities triaged	Four findings: Intracranial haemorrhage, Mass effect, midline shift, cranial fracture and ICH	One finding: Intracranial hemorrhage
User controlover triage	Triages non-contrast CT scan when any of the 4 abnormalities are identifiedNotification/ prioritization with user provided control for configuring the triage finding or combination of finding(s)	Non contrast CT scan of the head or neckSingle configuration
Previewimageinformation	Preview images returned to the PACSPreview images are reduced in size and their dynamic range is limited	Preview images on hover over the worklistPreview images are compressed and/or reduced in size

Table 1. Comparison of Technical Characteristics with Predicate Device

{8}------------------------------------------------

{9}------------------------------------------------

Performance Data:

A retrospective, multicenter, blinded clinical study was conducted to test the accuracy of qER at triaging head CT scans containing one of the conditions listed below and to establish the clinical benefit of such triage.

The study used 1320 head CT scans from multiple Unites States containing one or more of the following conditions: Intracranial hemorrhage (n=629), cranial fractures (n=248), mass effect (n=471), midline shift (n=414), and none of the target abnormalities (n=501) with 3 board-certified radiologists reading the scans to obtain ground truth. Overall, 48.03% of scans in the testing dataset had a slice thickness less than or equal to 3.5 mm.

Sensitivity and specificity exceeded the predefined success criteria, as well as the required performance criteria for triage and notification software as per the special controls for QAS, for all the 4 conditions independently and in combination, demonstrating the ability of the qER device to effectively triage studies containing one of these conditions.

Abnormality	Sensitivity (95% CI),TP/P	Specificity (95% CI),TN/N	AUC (95% CI)
Intracranial Hemorrhage	96.98 (95.32 - 98.17),610/629	93.92 (91.87 - 95.58),649/691	98.53 (98.00 - 99.15)
Cranial Fracture	96.77 (93.74 - 98.60),240/248	92.72 (91.00 - 94.21),994/1072	97.66 (96.88 - 98.57)
Mass Effect	96.39 (94.28 - 97.88),454/471	96.00 (94.45 - 97.21),815/849	99.09 (98.73 - 99.52)
Midline Shift	97.34 (95.30 - 98.67),403/414	95.36 (93.79 - 96.64),864/906	99.09 (98.74 - 99.51)
Any of the 4 targetabnormalities	98.53 (97.45 - 99.24),807/819	91.22 (88.39 - 93.55),457/501	NA

Table 2. Primary Endpoint Results in qER Standalone Performance Study

The control group for specificity calculations and AUC includes all scans in the dataset not containing the target abnormality listed in that row. Control groups contain the other 3 target abnormalities and nontarget abnormalities. TP = true positives, P = all positives, TN = true negatives, N = all negatives

The clinical benefit of using gER to prioritize these scans was quantified by comparing the scan was opened by a radiologist in the Standard of Care (TTO) versus the time that qER notification was received (TTN) for all the eligible head CT scans included in the standalone study.

Since scans not containing any target abnormality would not be prioritized by qER, scans containing one or more of the target abnormalities were used for this analysis. Of the 819 studies containing one or more target abnormalities, the requisite timestamp data was available for 386 scans, of which 378 were correctly identified and notified by qER, and 8 were false negatives. These timestamps were collected and used for the clinical benefit analysis (n=386).

The mean TTO was 65.54 mins (95% Cl 59.14 – 71.76 mins) for Standard of Care and the mean TTN was 2.11 mins (95% Cl 1.45 - 2.61 mins) for qER. qER TTN was substantially lower than standard of care TTO. Thus, study prioritization by qER could substantially shorten the time that elapses before a critical head CT scan is read and diagnosed.

{10}------------------------------------------------

Table 3. Comparison between Standard of Care Time-to-Open and qER Time-to-Notify a Head CT Scan Containing One of the Target Abnormalities

Parameter	Mean (95% CI) in minutes	Median (95% CI) in minutes
Time to open exam in the standard of care	65.54 (59.14 - 71.76)	60.01 (54.57 - 77.63)
Time-to-notification with qER	2.11 (1.45 - 2.61)	1.21 (1.12 - 1.25)

Conclusion:

qER is as safe and effective as Aidoc's Briefcase software. qER has the same intended uses and similar indications, technological characteristics, and principles of operation as its predicate device. The minor differences in indications do not alter the intended use of the device and do not affect its safety and effectiveness when used as labeled. Thus, qER is substantially equivalent to the predicate device.

Regulation Number and Section

§ 892.2080 Radiological computer aided triage and notification software.

(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.