(123 days)
BriefCase-Quantification of Midline Shift (MLS) is a radiological image management and processing system software intended for automatic measurement of brain midline shift in non-contrast head CT (NCCT) images, in adults or transitional adolescents aged 18 years and older.
The device is intended to assist appropriately trained medical specialists by providing the user with an automated current manual process of measuring midline shift.
The device provides midline shift measurement from NCCT images acquired at a single time point, and can additionally provide an output with comparative analysis of two or more images that were acquired in the same individual at multiple time points.
The device does not alter the original medical image and is not intended to be used as a diagnostic device. The BriefCase-Quantification results are not intended to be used on a stand-alone basis for clinical decision-making or otherwise preclude clinical assessment of cases. Clinicians are responsible for viewing full images per the standard of care.
BriefCase-Quantification is a radiological image management and processing device. The software consists of a single module based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.
The BriefCase-Quantification receives filtered DICOM Images, and processes them chronologically by running the algorithm on relevant series to quantify the extent of midline shift. Following the Al processing, the output of the algorithm analysis is transferred to an image review software (the PACS or a desktop application).
The device generates a summary report that includes a preview image of the slice with the largest midline shift. The preview image includes the measured shift, the annotation of the midline, and the annotation of the largest perpendicular distance between the midline and septum pellucidum. Also, the summary report includes a table and a graph showing the measured midline shift over time for patients with multiple scans.
Here's a detailed breakdown of the acceptance criteria and study information for the BriefCase-Quantification device, based on the provided document:
Acceptance Criteria and Device Performance
| Criteria | Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Primary Endpoint: Mean Absolute Error (MAE) | Mean absolute error estimate must be lower than prespecified performance goal. | 0.94 mm (95% CI: 0.74 mm, 1.14 mm) (Lower than prespecified goal) |
| Secondary Endpoint: Bias (Bland-Altman plot) | Little to no bias between ground truth and algorithm output. | Mean difference of -0.15 mm (Little to no bias) |
| Secondary Endpoint: MAE for multiple time points (First Case) | Mean absolute error estimate must be lower than prespecified performance goal. | 1.16 mm (95% CI: 0.61 mm, 1.71 mm) (Lower than prespecified goal) |
| Secondary Endpoint: MAE for multiple time points (Follow-up Cases) | Mean absolute error estimate must be lower than prespecified performance goal. | 1.28 mm (95% CI: 0.68 mm, 1.88 mm) (Lower than prespecified goal) |
Study Details
-
Sample size for the test set and data provenance:
- Sample Size: 284 cases from 228 unique patients.
- Data Provenance: Retrospective, multi-center study from 6 US-based clinical sites (both academic and community centers). The cases were distinct in time or center from the cases used to train the algorithm, indicating independent test data.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Three neuroradiologists.
- Qualifications: The document states "appropriately trained medical specialists" and specifically "three neuroradiologists," implying they are qualified experts in the field. Specific experience (e.g., "10 years of experience") is not provided.
-
Adjudication method for the test set:
- Method: The reference standard (ground truth) was created as the mean of all three independent measurements by the neuroradiologists. This implies a "consensus by average" approach rather than a specific 2+1 or 3+1 voting method.
-
If a multi-reader, multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- MRMC Study: No, an MRMC comparative effectiveness study was not explicitly stated as performed with human readers and AI assistance. The study described focuses on the standalone performance of the AI algorithm against a neuroradiologist-established ground truth.
- Effect Size of Human Improvement with AI: This information is not provided because an MRMC study comparing human readers with and without AI assistance was not detailed.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Standalone Performance: Yes, a standalone performance study was done. The reported performance metrics (MAE, Bland-Altman) directly compare the algorithm's output to the ground truth established by experts, without human intervention in the device's measurement process. The device is intended to assist specialists by providing an automated process, but its performance evaluation here is purely algorithmic.
-
The type of ground truth used:
- Ground Truth Type: Expert consensus. Specifically, the "mean of all three [neuroradiologist] measurements."
-
The sample size for the training set:
- Training Set Sample Size: Not explicitly stated. The document only mentions that the "cases collected for the pivotal dataset were all distinct in time or center from the cases used to train the algorithm."
-
How the ground truth for the training set was established:
- Training Set Ground Truth: Not explicitly stated. Given the nature of a supervised learning algorithm, it is implied that the training data also had established ground truth measurements, likely derived through expert review, but the specific method or number of experts for the training data is not detailed in this summary.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).