(126 days)
JBS-LVO is a notification-only, parallel workflow tool for use by hospital networks and trained clinicians to identify and communicate images of specific patients to a specialist, independent of standard of care workflow.
JBS-LVO uses an artificial intelligence algorithm to analyze images for findings suggestive of a pre-specified clinical condition and to notify an appropriate medical specialist of these findings in parallel to standard of care image interpretation. Identification of suspected positive findings is not for diagnostic use beyond notification. Specifically, the device analyzes CT angiogram images of the brain acquired in the acute setting and sends notifications to a neurovascular specialist that a suspected large vessel occlusion has been identified and recommends a review of those images. Images can be previewed through a mobile application. JBS-LVO is intended to analyze terminal ICA and MCA-M1 vessels for LVOs.
Images that are previewed through the mobile application are compressed and for informational purposes only. They are not intended for diagnostic use beyond notification. The JBS-LVO device does not alter the original medical image. Notified clinicians are responsible for viewing non-compressed images on a diagnostic viewer and engaging in appropriate patient evaluation and relevant discussion with a treating physician before making care-related decisions or requests. JBS-LVO is limited to the analysis of imaging data and should not be used in-lieu of full patient evaluation or relied upon to make or confirm a diagnosis.
Limitations:
The device does not process scans containing metallic artifacts.
JBS-LVO is a radiological computer aided triage and notification (CAD) software package compliant with the DICOM standard. IBS-VV is a notification-only, parallel workflow tool for use by hospital networks and trained clinicians to analyze computed tomography angiography (CTA) images for findings suggestive of a suspected large vessel occlusion (LVO) and to notify an appropriate medical specialist of these findings in parallel to standard of care image interpretation. Specifically, JBS-LVO is optimized to evaluate occlusions of the intracranial carotid artery (ICA) and proximal middle cerebral artery (MCA-M1 segment). It is important to clarify that this quantification is solely used within the device's Al module to facilitation process. The output provided to heathcare professionals is stiritly a flag indicating the presence (positive) of an LVO, in accordance with regulatory guidelines.
JBS-LVO is a combination of software modules that allow for detection and notification of patients with a suspected LVC. JBS-LVO consists of an algorithm and mobile application software module.
The JBS-LVO Image Analysis Algorithm (LVO Detection Algorithm) is a locked, artificial intelligence (Al) software algorithm utilizing convolutional neural network (CNN) that analyzes CTA images of the brain for a suspected LVO. The LVO Detection Algorithm is hosted on the ILK-server and analyzes applicable CTA images of the brain that are acquired on CT scanners and are automatically transmitted to the ILK-server. Upon detection of a suspected LVO, the LVO notification module sends a notification of the suspected finding.
The JBS-LVO notification functionality enable medical professionals and clinicians to preview compressed and informational images through via mobile application notification. Image viewing through the mobile application interface is for informational purposes only and is not for diagnostic use.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
1. Table of Acceptance Criteria and Reported Device Performance
| Performance Metric | Acceptance Criteria (Target) | Reported Device Performance (JBS-LVO) |
|---|---|---|
| Sensitivity | > 80% | 91.8% (95% CI: 85.8% - 95.8%) |
| Specificity | > 80% | 92.8% (95% CI: 87.2% - 96.5%) |
| Area Under the Curve (AUC) | Not explicitly stated as a target, but reported | 95% CI: 93.0% - 98.1% |
| CTA to Notification Time | ≤ 3.5 minutes (compared to predicate) | Ranged from 2.32 to 3.29 minutes (95% CI: 2.89 - 3.02) |
Notes:
- The document implies that the sensitivity and specificity acceptance criteria were "exceeded," suggesting a >80% threshold was the minimum.
- The AUC is reported as a performance metric, indicating its importance, even if a specific numerical acceptance criterion wasn't explicitly listed alongside the other two.
- The CTA to notification time criterion is established by comparison to a reference predicate device (Rapid LVO, K221248).
Study Details
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: Not explicitly stated as a numerical count of cases or patients in the provided text. The document mentions "a retrospective study" was conducted and "each case output from the JBS-LVO device was compared with a ground truth."
- Data Provenance: Retrospective study. The origin of the data (e.g., country of origin) is not explicitly mentioned for the test set, but it states that "the images used to train the algorithm were sourced from datasets that included equipment from various manufacturers, such as Siemens, Philips, Toshiba, and GE", implying a diverse source for the overall dataset.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Number of Experts: Three (two initial ground truthers, with a third intervening in cases of disagreement).
- Qualifications of Experts: All truthers were US board-certified neuroradiologists.
4. Adjudication Method for the Test Set
- Adjudication Method: 2+1 adjudication. The ground truth was determined by two ground truthers, with a third ground truther intervening in cases of disagreement.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
- MRMC Study: No, an MRMC comparative effectiveness study was not conducted. The document describes a "standalone performance evaluation" of the algorithm only.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- Standalone Performance Study: Yes, a standalone performance evaluation of the algorithm without human-in-the-loop performance was conducted. The text states: "The performance of the device's AI algorithms was validated in a standalone performance evaluation, utilizing an independent dataset different from the one used for algorithm training." and "JLK, Inc. performed a standalone performance with the §892.2080 special controls to show acceptance of the clinical performance of the JBS-LVO module."
7. The Type of Ground Truth Used
- Ground Truth Type: Expert consensus. Specifically, ground truth was "established by US board-certified neuro-radiologists."
8. The Sample Size for the Training Set
- Training Set Sample Size: Not explicitly stated as a numerical count in the provided text. The document refers to it as "datasets."
9. How the Ground Truth for the Training Set Was Established
- Training Set Ground Truth: The training data was "labeled by trained radiologists to identify the presence of LVO." The specific number or adjudication method for these "trained radiologists" is not detailed, but it implies expert labeling.
§ 892.2080 Radiological computer aided triage and notification software.
(a)
Identification. Radiological computer aided triage and notification software is an image processing prescription device intended to aid in prioritization and triage of radiological medical images. The device notifies a designated list of clinicians of the availability of time sensitive radiological medical images for review based on computer aided image analysis of those images performed by the device. The device does not mark, highlight, or direct users' attention to a specific location in the original image. The device does not remove cases from a reading queue. The device operates in parallel with the standard of care, which remains the default option for all cases.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the notification and triage algorithms and all underlying image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, how the algorithm affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide effective triage (
e.g., improved time to review of prioritized images for pre-specified clinicians).(iii) Results from performance testing that demonstrate that the device will provide effective triage. The performance assessment must be based on an appropriate measure to estimate the clinical effectiveness. The test dataset must contain sufficient numbers of cases from important cohorts (
e.g., subsets defined by clinically relevant confounders, effect modifiers, associated diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals for these individual subsets can be characterized with the device for the intended use population and imaging equipment.(iv) Stand-alone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; description of verification and validation activities including system level test protocol, pass/fail criteria, and results).(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use;
(ii) A detailed description of the intended user and user training that addresses appropriate use protocols for the device;
(iii) Discussion of warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality for certain subpopulations), as applicable;(iv) A detailed description of compatible imaging hardware, imaging protocols, and requirements for input images;
(v) Device operating instructions; and
(vi) A detailed summary of the performance testing, including: test methods, dataset characteristics, triage effectiveness (
e.g., improved time to review of prioritized images for pre-specified clinicians), diagnostic accuracy of algorithms informing triage decision, and results with associated statistical uncertainty (e.g., confidence intervals), including a summary of subanalyses on case distributions stratified by relevant confounders, such as lesion and organ characteristics, disease stages, and imaging equipment.