(116 days)
Rapid ICH is a radiological computer aided triage and notification software in the analysis of non-enhanced head CT images. The device is intended to assist hospital networks and trained radiologists in workflow triage by flagging and communication of suspected positive findings of pathologies in head CT images, for IPH, IVH, SAH, and SDH Intracranial Hemorrhages (CH).
Rapid ICH uses an artificial intelligence algorithm to analyze images and highlight cases with detected ICH on a server or standalone desktop application in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for cases with suspected ICH findings. Notifications include compressed preview images, which are meant for informational purposes only and not intended for diagnostic use beyond notification. The device does not alter the original medical image and is not is a a diagnostic device.
The results of Rapid ICH are intended to be used in conjunction and based on professional judgment, to assist with trage /prioritization of medical images. Notified radiologists are responsible for viewing full images per the standard of care.
Rapid ICH is a radiological computer-assisted triage and notification software device. The Rapid ICH module is a non-enhanced CT (NCCT) processing module which operates within the integrated Rapid Platform to provide triage and notification of suspected intracranial hemorrhage. The Rapid ICH module is an AI/ML module. The output of the module is a priority notification to clinicians indicating the suspicion of ICH based on positive findings. The Rapid ICH module uses the basic services supplied by the Rapid Platform including DICOM processing, job management, imaging module execution and imaging output including the notification and compressed image.
Here's a breakdown of the acceptance criteria and study details for the Rapid ICH device, based on the provided text:
Acceptance Criteria and Device Performance
The primary performance goals for Rapid ICH were defined by sensitivity and specificity thresholds.
Acceptance Criteria Table and Reported Device Performance:
| Parameter | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Overall Sensitivity | >80% | 96.8% (95% CI: 92.6% - 98.6%) |
| Overall Specificity | >80% | 100% (95% CI: 97.7% - 100%) |
| AUC (Using Rapid Estimated Volume as predictor of Suspected ICH) | Not explicitly stated as a pass/fail criterion, but reported | 0.98632 |
| Time to Notification (Compared to Time to Open Exam in Standard of Care) | Significantly faster than standard of care | Rapid ICH: 0.65 minutes (95% CI 0.63 - 0.67) |
| Standard of Care: 72.58 minutes (95% CI 45.02 - 100.14) |
Study Details
2. Sample Size and Data Provenance:
- Test Set Sample Size: 314 cases (148 ICH positive, 166 ICH negative).
- Data Provenance: Retrospective, multicenter, multinational study. Specific countries are not detailed, but "multinational" implies diverse geographical origins.
3. Number of Experts and Qualifications for Ground Truth:
- Number of Experts: Not explicitly stated how many individual experts established the ground truth. The document mentions "expert reader truthing of the data," suggesting one or more experts.
- Qualifications of Experts: The document states "trained radiologists" are intended users and mentions "expert reader truthing." However, specific qualifications such as years of experience, board certification, or subspecialty are not provided for the ground truth experts.
4. Adjudication Method for the Test Set:
- The document implies ground truth was established by "expert reader truthing of the data," but does not specify an adjudication method (e.g., 2+1, 3+1, consensus review process if multiple readers were involved).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- No, an MRMC comparative effectiveness study was NOT mentioned for evaluating human readers' improvement with AI assistance. The study focused on the standalone performance of the AI algorithm (accuracy) and the time-to-notification benefit.
6. Standalone Performance (Algorithm Only):
- Yes, a standalone performance study was done. The reported sensitivity, specificity, and AUC values directly reflect the algorithm's performance in identifying ICH presence. The study evaluated the software's performance in identifying abnormalities, and the "time to notification" indicates the speed of the algorithm's output.
7. Type of Ground Truth Used:
- Expert Consensus: The ground truth for the test set was established through "expert reader truthing of the data." This implies a clinical expert (radiologist) determined the presence or absence of ICH.
8. Sample Size for the Training Set:
- The document states that the "minor change causing this filing, is the use of additional data for training and validation," implying the training set for this iteration of the device included more data than the predicate. However, the specific sample size of the training set is not provided in the summary.
9. How the Ground Truth for the Training Set was Established:
- Similar to the test set, the document indicates that the device was trained and validated using "retrospective case data based on expert reader truthing of the data." This suggests the ground truth for the training set was also established by expert review/diagnosis by clinical experts.
§ 892.2080 Radiological computer aided triage and notification software.
(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.