(168 days)
CINA-ASPECTS is a computer-aided diagnosis (CADx) software device used to assist the clinician in the assessment and characterization of brain tissue abnormalities using CT image data.
The Software automatically reorients images, segments and analyzes ASPECTS Regions of Interest (ROIs). CINA-ASPECTS extracts image data for the ROI(s) to provide analysis and computer analytics based on morphological characteristics. The imaging features are then synthesized by an artificial intelligence algorithm into a single ASPECT (Alberta Stroke Program Early CT) Score.
CINA-ASPECTS is indicated for evaluation of patients presenting for diagnostic imaging workup with known MCA or ICA occlusion, for evaluation of extent of disease. Extent of disease refers to the number of ASPECTS regions affected which is reflected in the total score. This device provides information that may be useful in the characterization of early ischemic (acute) brain tissue injury during image interpretation.
CINA-ASPECTS provides a comparative analysis to the ASPECTS standard of care radiologist assessment using the ASPECTS region definitions and highlighting ROIs and numerical scoring.
Limitations:
- CINA-ASPECT is not intended for primary interpretation of CT images. It is used to assist physician evaluation.
- CINA-ASPECT has been validated in patients with known MCA or ICA unilateral occlusion prior to ASPECTS scoring.
- CINA-ASPECTS is not suitable for use on brain scans displaying neurological pathologies other than acute stroke, such as tumors or abscesses, traumatic brain injuries, hemorrhagic transformation and hematoma.
- Use of CINA-ASPECT in clinical settings other than brain ischemia within 12 hours from time last known well, caused by known ICA or MCA occlusions has not been tested.
- CINA-ASPECTS has only been validated and is intended to be used in patient populations aged over 21.
- CINA-ASPECTS has been validated and is intended to be used with images acquired with Canon Medical Systems Corporation, GE Healthcare, Philips Healthcare and Siemens Healthineers scanners.
Contraindications/Exclusions/Cautions:
- Patient motion: Excessive patient motion leading to artifacts that make the scan technically inadequate.
- Important streak artifacts and noisy images: Presence of important streak artifact and significant noise within the NCCT images that make the scan technically inadequate.
- Hemorrhagic Transformation, Hematoma.
CINA-ASPECTS is a standalone computer-aided diagnosis (CADx) software that processes noncontrast head CT (NCCT).
CINA-ASPECTS is a standalone executable program that may be run directly from the commandline or through integration, deployment and use with medical image communications devices. The software does not interface directly with any CT scanner or data collection equipment; instead, the software receives non-contrast head CT (NCCT) scans identified by medical image communications devices, processes them using algorithmic methods involving execution of multiple computational steps to provide an automatic ASPECT score based on the case input file for the physician.
The score includes which ASPECT regions are identified based on regional imaging features derived from non-contrast computed tomography (NCCT) brain image data and overlaid onto brain scan images. The results are generated based on the Alberta Stroke Program Early CT Score (ASPECTS) guidelines and provided to the clinician for review and verification. At the discretion of the clinician, the scores may be adjusted based on the clinician's judgment.
Series are processed by running the CINA-ASPECTS Image Processing Applications on noncontrast head CT images (NCCT) to perform the:
- Reorientation, tilt-correction of the input imaging data;
- Delineation of predefined regions of interest on the corrected input data and computing numerical values characterizing underlying voxel values within those regions;
- Visualizing the voxels which have contributed to the ASPECTS score (also referred to as a 'heat map'); and
- Labeling of these delineated regions and providing a summary score reflecting the number of regions with early ischemic change as per ASPECTS guidelines.
The CINA-ASPECTS User Interface Agent provides the ASPECTS information to the clinician to review and edit. It also requires the confirmation by a clinician that a Large Vessel Occlusion (LVO) is detected. This confirmation is used by the CINA-ASPECTS to limit the detection of areas of early ischemic changes to the infarcted brain hemisphere selected by the user. The final summary score together with the regions selected and underlying voxel values are then stored in DICOM format to be transferred by the medical image communications device for output to a Picture Archiving and Communication System (PACS) or workstation.
The CINA-ASPECTS device is made of two components:
- The CINA-ASPECTS image processing application which reads the input file and generates an automatic ASPECT score and the applications outputs
- A CINA-ASPECTS UI Agent which provides the ASPECTS information to the clinician to review and edit for final archiving.
Here's a breakdown of the acceptance criteria and study details for the CINA-ASPECTS device, based on the provided FDA 510(k) summary:
CINA-ASPECTS Device Acceptance Criteria and Performance Study
1. Table of Acceptance Criteria and Reported Device Performance
The provided document details two main studies: a Standalone Performance Testing and a Clinical Multi-Reader Multi-Case (MRMC) Performance Study. The acceptance criteria aren't explicitly listed as a separate table with pass/fail metrics in the summary, but rather are demonstrated through the successful outcomes of these studies. The performance metrics reported are measures of the device's accuracy and utility.
Note: The FDA 510(k) summary typically presents a high-level overview. Specific numerical acceptance thresholds (e.g., "sensitivity must be > X%") are often detailed in the full submission but are not fully elaborated here. Instead, the document states that the device "met all design requirements and specifications" and "achieved its primary endpoint," implying successful adherence to pre-defined acceptance criteria.
| Acceptance Criterion (Inferred from Study Goals) | Reported Device Performance (CINA-ASPECTS) |
|---|---|
| Standalone Performance | |
| Accurate representation of key processing parameters under a range of clinical parameters. | Demonstrated accurate representation. "The Standalone Performance Testing demonstrated that the proposed device provides accurate representation of key processing parameters under a range of clinically relevant parameters." "The CINA-ASPECTS device performed properly and matched with the ground truth." |
| Generalizability across patient demographics, clinical parameters, ASPECTS regions, and image acquisition parameters. | Achieved primary endpoint and established generalizability. "The Standalone Performance Testing study demonstrated that CINA-ASPECTS achieved its primary endpoint and established that CINA-ASPECTS performances generalize to a range of typical patient demographics, Clinical parameters, ASPECTS regions, and image acquisition parameters encountered in multiple clinical sites and scanner makers and models." |
| Safety and effectiveness. | "The performance testing of the CINA-ASPECTS device establishes that the subject device is safe and effective, meets its intended use statement and is compatible with clinical use." |
| Clinical Performance (MRMC Study) | |
| Improve agreement between readers (with AI assist) and reference standard for ASPECTS scoring. | Readers agreed with "almost ½ a region (4.1%, [95% Cl: 3.3% -4.9%]) more per scan with CINA-ASPECTS than without." "The clinical data demonstrates that CINA-ASPECTS shows a significant improvement in the agreement between the readers and a reference standard when using the CINA-ASPECTS software compared to routine clinical practice." |
| Improve overall reader ROC AUC. | Overall readers' ROC AUC improved significantly from 0.75 (Unaided arm) to 0.79 (Aided arm). |
| Reduce variation in performance between different readers. | The range in the ROC AUC between users was narrower when assisted by the software. |
| Reduce mean time spent per case. | The mean time spent per case among all readers was significantly reduced when using CINA-ASPECTS. |
| Substantial equivalence for improving reader accuracy compared to the predicate device. | "This study demonstrates substantial equivalence of the CINA-ASPECTS software for improving reader accuracy, compared to the predicate device. The results showed statistically significant improvement in the readers' accuracy when using the software compared to the conventional manual method used in routine clinical practice." "With CINA-ASPECTS the readers agreed, on average, with almost ½ a region (4.1%, [95% Cl: 3.3% -4.9%]) more per scan than without CINA-ASPECTS. These findings are similar to the results reported for the predicate device." |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: 200 clinical anonymized NCCT cases.
- Data Provenance: Retrospective, multinational, multi-vendor dataset from 5 clinical sites in two countries (US and France). Acquired by 4 different scanner makers (GE, Siemens, Canon, Philips) and 27 different scanner models.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document mentions that the MRMC study evaluated the performance of "8 clinical readers" and that the "clinical data demonstrates that CINA-ASPECTS shows a significant improvement in the agreement between the readers and a reference standard." However, it does not explicitly state the number or qualifications of experts used to establish the ground truth specifically for the standalone performance test.
For the MRMC study readers, it states: "The panel of readers consisted of 4 expert neuroradiologists and 4 non-experts from different specialties (stroke neurologist, general radiologist, neurointensivist, vascular neurologist), representing the intended use population." While these readers contributed to the "aided" and "unaided" performance evaluation, they are not explicitly designated as the ground truth setters for the test set. The term "reference standard" is used, implying a separate, likely expert-derived, ground truth, but its specifics are not detailed here.
4. Adjudication Method for the Test Set
The document does not explicitly state the adjudication method (e.g., 2+1, 3+1) used to establish the ground truth for the test set. It mentions agreement with a "reference standard" in the context of the MRMC study, but not how that reference standard was formed.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Was an MRMC study done? Yes, a retrospective, multinational, multi-vendor, and blinded Clinical Multi-Reader Multi-Case (MRMC) Performance study was conducted.
- Effect size of how much human readers improve with AI vs without AI assistance:
- Agreement with reference standard: With CINA-ASPECTS, readers agreed, on average, with almost ½ a region (4.1%, [95% Cl: 3.3% -4.9%]) more per scan than without CINA-ASPECTS.
- Overall ROC AUC: Improved significantly from 0.75 (Unaided arm) to 0.79 (Aided arm).
- Reduced variation: The range in the ROC AUC between users was narrower when assisted by the software.
- Time spent: Mean time spent per case among all readers was significantly reduced when using CINA-ASPECTS.
6. Standalone Performance (Algorithm Only without Human-in-the-Loop Performance)
- Was a standalone study done? Yes. "Standalone performance testing was conducted to comply with special controls for this device type."
7. Type of Ground Truth Used
The document states that in the standalone performance testing, "The CINA-ASPECTS device performed properly and matched with the ground truth." For the MRMC study, it refers to improvement in "agreement between the readers and a reference standard."
However, the specific methodology for establishing this "ground truth" or "reference standard" (e.g., expert consensus of several independent radiologists, pathology results, outcomes data) is not explicitly detailed in the provided text. It is implied to be expert-derived, given the context of radiological assessment.
8. Sample Size for the Training Set
The document states, "The validation dataset was separated from the one used for the algorithm training/testing and has never been used in any way in the development of the software device." However, the sample size for the training set is not provided in this summary.
9. How the Ground Truth for the Training Set was Established
The document describes how the validation dataset was separated from the training/testing data but does not specify how the ground truth for the training set was established.
§ 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.
(a)
Identification. A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will improve reader performance as intended.
(iii) Results from performance testing protocols that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain sufficient numbers of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Standalone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; and description of verification and validation activities including system level test protocol, pass/fail criteria, results, and cybersecurity).(2) Labeling must include:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and recommended user training.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations, including situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Detailed instructions for use.
(viii) A detailed summary of the performance testing, including: Test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders (
e.g., lesion and organ characteristics, disease stages, and imaging equipment).