Search Results
Found 1 results
510(k) Data Aggregation
(95 days)
Intended context:
Annalise Enterprise is a device designed to be used in the medical care environment to aid in triage and prioritization of studies with features suggestive of the following findings:
- acute infarct*
See Additional Information, next page.
The device analyzes studies using an artificial intelligence algorithm to identify findings. It makes study-level output available to an order and imaging management system for worklist prioritization or triage.
The device is not intended to direct attention to specific portions of an image and only provides notification for suspected findings.
Its results are not intended:
- to be used on a standalone basis for clinical decision making
- to rule out specific findings, or otherwise preclude clinical assessment of non-contrast computed tomography brain
Intended modality:
Annalise Enterprise identifies suspected findings in non-contrast brain CT studies.
Intended user:
The device is intended to be used by trained clinicians who are qualified to interpret CTB studies as part of their scope of practice.
Intended patient population:
The intended population is patients who are 22 years or older.
Additional information:
The device includes acute infarct of the cerebral hemispheres or cerebellum, also including early signs of acute middle cerebral artery (MCA) infarct such as insular ribbon sign and disappearing basal ganglia sign.
The infarct must be a completed infarct (i.e. include an ischemic core of ≥5mL). The device also includes hyperdense artery in the anterior circulation but does not include lacunar infarcts, brainstem infarcts or venous infarcts.
The radiological device definition of acute infarct includes the following territories and regions:
- anterior cerebral artery (ACA)
- middle cerebral artery (MCA)
- posterior cerebral artery (PCA)
- cerebellum
- basilar artery occlusions
- watershed regions
Specificity may be reduced in the presence of infarcts of <5mL.
Annalise Enterprise is a software workflow tool which uses an artificial intelligence (AI) algorithm to identify suspected findings on non-contrast brain CT studies in the medical care environment. The finding identified by the device is acute infarct.
Radiological findings are identified by the device using an AI algorithm – a convolutional neural network trained using deep-learning techniques. Images used to train the algorithm were sourced from datasets that included a range of patient demographics and technical characteristics, including different equipment manufacturers and machines. This dataset, which contained over 200,000 CT brain imaging studies, was labelled by trained radiologists regarding the presence of the findings of interest.
The performance of the device's AI algorithm was validated in a standalone performance evaluation, in which the case-level output from the device was compared with a reference standard ('ground truth'). This was determined by two ground truthers, with a third truther used in the event of disagreement. All truthers were US board-certified neuroradiologists or neurologists.
The device interfaces with image and order management systems (such as PACS/RIS) to obtain non-contrast brain CT studies for processing by the AI algorithm. Following processing, if any of the clinical findings of interest are identified in the study, the device provides a notification to the image and order management system for prioritization of that study in the worklist. This enables users to review the studies containing features suggestive of these clinical findings earlier than in the standard clinical workflow. It is important to note that the device will never decrease a study's existing priority in the worklist. This ensures that worklist items will never have their priorities downgraded based on AI results.
The device workflow is performed parallel to and in conjunction with the standard clinical workflow for interpretation of non-contrast brain CT studies. The device is intended to aid in prioritization and triage of radiological medical images only.
Here's a summary of the acceptance criteria and the study proving the device meets those criteria, based on the provided FDA 510(k) Clearance Letter for Annalise Enterprise:
Acceptance Criteria and Device Performance
The acceptance criteria for Annalise Enterprise are established through its performance in detecting acute infarct on non-contrast brain CT studies. The device's performance is demonstrated through its Area Under the Curve (AUC), sensitivity, and specificity at various operating points for different slice thicknesses.
Table of Acceptance Criteria and Reported Device Performance (Acute Infarct Detection)
| Finding | Product Code | Slice Thickness | Acceptance Criteria Value (AUC) | Reported Device Performance (AUC) (95% CI) | Acceptance Criteria Value (Sensitivity %) (at various operating points) | Reported Device Performance (Sensitivity %) (95% CI) | Acceptance Criteria Value (Specificity %) (at various operating points) | Reported Device Performance (Specificity %) (95% CI) |
|---|---|---|---|---|---|---|---|---|
| Acute Infarct | QAS | ≤ 1.5mm | Implicit: Demonstrate high AUC, Sensitivity, and Specificity deemed substantially equivalent to predicate. | 0.952 (0.937, 0.965) | (various operating points) | 89.2 (85.8,92.6) to 84.5 (80.5,88.5) | (various operating points) | 84.1 (81.5,86.9) to 93.1 (91.1,95.0) |
| Acute Infarct | QAS | > 1.5 & ≤5.0mm | Implicit: Demonstrate high AUC, Sensitivity, and Specificity deemed substantially equivalent to predicate. | 0.933 (0.917, 0.949) | (various operating points) | 85.7 (81.9,89.2) to 78.1 (73.8,82.5) | (various operating points) | 83.2 (80.3,85.9) to 91.9 (89.8,93.9) |
| Triage Effectiveness | N/A | N/A | Implicit: Demonstrate clinically effective triage turnaround time, substantially equivalent to predicate. | 81.6 (95% CI: 80.3 – 82.9) seconds | N/A | N/A | N/A | N/A |
Study Proving Device Meets Acceptance Criteria
The device performance was assessed in four performance studies, including standalone performance and triage effectiveness evaluations. The primary study described in detail for acute infarct detection is a standalone performance evaluation.
1. Sample Size Used for the Test Set and Data Provenance
-
Acute Infarct Detection (Standalone Performance Evaluation):
- Slice Thickness ≤ 1.5mm cohort: 977 cases.
- Slice Thickness > 1.5mm & ≤ 5.0mm cohort: 1050 cases.
- Total: 2027 cases.
- Data Provenance: Retrospective, anonymized cases collected consecutively from five US hospital network sites. The test dataset was newly acquired and independent from the training dataset. The datasets included a range of patient demographics (gender, age, ethnicity, race) and technical parameters (imaging equipment make and model: GE Healthcare, NeuroLogica, Siemens, and Toshiba CT scanners).
-
Triage Effectiveness (Turn-around Time):
- Sample Size: n=277 cases.
- Data Provenance: Cases positive for any of the findings eligible for prioritization, collected from multiple data sources spanning a variety of geographical locations, patient demographics and technical characteristics. Internal bench study.
2. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Ground Truthers: At least two experts for each case, with a third adjudicator in case of disagreement.
- Qualifications:
- For cases with advanced imaging: ABR-certified and protocol-trained neuroradiologists.
- For negative cases with chart-based interpretations: ABR-certified, protocol-trained neuroradiologists and/or neurologists.
3. Adjudication Method for the Test Set
The adjudication method used was 2+1, meaning:
- Each deidentified case was annotated in a blinded fashion by at least two ground truthers.
- Consensus was determined by the two ground truthers.
- In the event of disagreement between the first two ground truthers, a third ground truther adjudicated to establish the final ground truth.
4. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was done
No, an MRMC comparative effectiveness study was not explicitly described as being done to measure how much human readers improve with AI vs without AI assistance. The performance evaluation focuses on the standalone performance of the AI algorithm and its triage effectiveness (turnaround time effectiveness) compared to the predicate device and standard of care.
5. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, a standalone performance evaluation was done for acute infarct detection. The case-level output from the device's AI algorithm was compared directly with the established ground truth.
6. The Type of Ground Truth Used
- For positive cases (acute infarct): Ground truth was established by expert consensus of ABR-certified neuroradiologists, utilizing advanced imaging data for deidentified cases.
- For negative cases: Ground truth was established by expert consensus of ABR-certified neuroradiologists and/or neurologists, potentially referencing chart-based interpretations.
7. The Sample Size for the Training Set
The training set contained over 200,000 CT brain imaging studies.
8. How the Ground Truth for the Training Set was Established
The images used to train the algorithm were sourced from datasets where the presence of the findings of interest (acute infarct) was labelled by trained radiologists. The document does not specify the number of radiologists per case or an adjudication method for the training data ground truth, nor does it detail their specific qualifications beyond "trained radiologists."
Ask a specific question about this device
Page 1 of 1