(192 days)
The i-STAT TBI Plasma test is a panel of in vitro diagnostic immunoassays for the quantitative measurements of glial fibrillary acidic protein (GFAP) and ubiquitin carboxyl-terminal hydrolase L1 (UCH-L1) in plasma and a semiquantitative interpretation of test results derived from these measurements, using the i-STAT Alinity Instrument. The interpretation of test results is used, in conjunction with other clinical information, to aid in the evaluation of patients, 18 years of age or older, presenting with suspected mild traumatic brain injury (Glasgow Coma Scale score 13-15) within 12 hours of injury, to assist in determining the need for a CT (computed tomography) scan of the head. A 'Not Elevated' test interpretation is associated with the absence of acute traumatic intracranial lesions visualized on a head CT scan.
The test is to be used with plasma prepared from EDTA anticoagulated specimens in clinical laboratory settings by a healthcare professional. The i-STAT TBI Plasma test is not intended to be used in point of care settings.
The i-STAT TBI Plasma cartridge is a multiplex immunoassay that contains assays for both ubiquitin carboxyl-terminal hydrolase L1 (UCH-L1) and glial fibrillary acidic protein (GFAP). The assays test for the presence of these biomarkers in a plasma sample and yield a semi-quantitative test interpretation based on measurements of both UCH-L1 and GFAP in approximately 15 minutes. The i-STAT TBI Plasma cartridge is designed to be run only on the i-STAT Alinity instrument.
The i-STAT Alinity instrument is a handheld, in vitro diagnostic device designed to run only i-STAT test cartridges. The instrument is the main user interface of the i-STAT System and functions as the electro-mechanical interface to the test cartridge. The instrument executes the test cycle, acquires and processes the electrical sensor signals converting the signals into quantitative results. These functions are controlled by a microprocessor.
The i-STAT Alinity System is comprised of the i-STAT Alinity instrument, the i-STAT test cartridges and accessories (i-STAT Alinity Base Station, Electronic Simulator and Printer).
Here's a breakdown of the acceptance criteria and study information for the i-STAT TBI Plasma cartridge with the i-STAT Alinity System, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria in a table format. However, it presents clinical performance parameters for sensitivity, specificity, and negative predictive value (NPV). The implied "acceptance criteria" are derived from comparison to the predicate device and the clinical utility for reducing unnecessary CT scans.
| Performance Parameter | Acceptance Criteria (Implied) | Reported Device Performance (Pivotal Study, N=1901) | Reported Device Performance (Supplemental Fresh Specimen Study, N=88) |
|---|---|---|---|
| Clinical Sensitivity | Comparable to predicate device and high enough to identify true positive cases of intracranial lesions. | 95.8% (95% CI: 90.6%, 98.2%) | 100.0% (95% CI: 88.3%, 100.0%) |
| Clinical Specificity | Comparable to predicate device and sufficient to potentially reduce unnecessary CT scans. | 40.4% (95% CI: 38.2%, 42.7%) | 23.7% (95% CI: 14.7%, 36.0%) |
| Negative Predictive Value (NPV) | High enough to confidently rule out the absence of acute traumatic intracranial lesions when the test is 'Not Elevated'. | 99.3% (95% CI: 98.5%, 99.7%) | 100.0% (95% CI: 80.2%, 100.0%) (Adjusted NPV at 6% prevalence: 100.0% (95% CI: 96.9%, 100.0%)) |
| False Negative Rate | Low, especially for lesions requiring surgical intervention. | 4.2% (5/120). No FN for surgical intervention cases. | 0% (0/29) |
| False Positive Rate | Tolerable given the clinical benefit of potentially reducing unnecessary CT scans. | 59.6% (1061/1781) | 76.2% (45/59) |
Note: The document explicitly states that the device was deemed "substantially equivalent" to the predicate, and a "benefit-risk assessment was performed," suggesting that the performance metrics achieved were considered acceptable for its intended use and comparative to the existing predicate.
2. Sample Size Used for the Test Set and Data Provenance
- Pivotal Study:
- Sample Size: 1901 subjects (120 with positive CT, 1781 with negative CT).
- Data Provenance: Prospectively collected and archived (frozen) plasma specimens. Subjects enrolled at 22 clinical sites in three countries: United States, Germany, and Hungary.
- Supplemental Fresh Specimen Study:
- Sample Size: 88 subjects (29 with positive CT, 59 with negative CT).
- Data Provenance: Freshly collected plasma specimens. Subjects enrolled across 4 clinical sites of the TRACK-TBI study in the United States.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Number of Experts: At least two neuroradiologists, with a third neuroradiologist for adjudication if necessary.
- Qualifications of Experts: Neuroradiologists (specific years of experience or subspecialty certification not detailed, but implied by the term "neuroradiologist").
4. Adjudication Method for the Test Set
The adjudication method used was consensus interpretation between two neuroradiologists, with adjudication by a third neuroradiologist if necessary. This is commonly referred to as a "2+1" or "multiple reader, with adjudication" method.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study comparing human readers with AI assistance versus without AI assistance was not described in this document. The clinical studies focused on the standalone diagnostic performance of the device itself (i-STAT TBI Plasma test interpretation) against a CT scan ground truth, not on evaluating human reader performance with or without the device. The device's output is an "interpretation of test results...to aid in the evaluation of patients...to assist in determining the need for a CT scan," suggesting it's designed to be used by a healthcare professional as an aid, but the study design doesn't directly measure the improvement of human readers through its use.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, the clinical studies presented here (Pivotal and Supplemental) are effectively standalone performance studies for the i-STAT TBI Plasma test. The results (sensitivity, specificity, NPV) are reported for the device's interpretation ("Elevated" or "Not Elevated") directly against the CT scan ground truth, without measuring the impact of a human healthcare professional's subsequent decision-making. The device provides "a semiquantitative interpretation of test results derived from these measurements," which is then used "in conjunction with other clinical information, to aid in the evaluation of patients...to assist in determining the need for a CT scan." So, while it's an aid to a human, the performance metrics reported are for the device's output itself.
7. The Type of Ground Truth Used
The primary ground truth used for the clinical studies was the presence or absence of acute traumatic intracranial lesions visualized on a head CT (Computed Tomography) scan. This ground truth was established by consensus interpretation of neuroradiologists.
8. The Sample Size for the Training Set
- Assay Cutoff Determination: A training set of 420 subjects (274 males and 146 females) with suspected mild TBI was used to determine the assay cutoffs for GFAP and UCH-L1.
9. How the Ground Truth for the Training Set Was Established
For the 420 subjects in the training set used to establish assay cutoffs:
- Subjects had suspected mild traumatic brain injury (Glasgow Coma Scale score of 13-15).
- Blood was drawn within 12 hours of injury.
- A head CT scan determination was performed.
- The ground truth would have been established by the head CT scan results (presence or absence of acute traumatic intracranial lesions), similar to the clinical study's ground truth, though the specific process of expert review for these 420 cases is not detailed beyond "head CT scan determination." It's reasonable to infer a similar process of expert radiologist interpretation.
§ 866.5830 Brain trauma assessment test.
(a)
Identification. A brain trauma assessment test is a device that consists of reagents used to detect and measure brain injury biomarkers in human specimens. The measurements aid in the evaluation of patients with suspected mild traumatic brain injury in conjunction with other clinical information to assist in determining the need for head imaging per current standard of care.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The 21 CFR 809.10(b) compliant labeling must include detailed descriptions of and results from performance testing conducted to evaluate precision, accuracy, linearity, analytical sensitivity, interference, and cross-reactivity. This information must include the following:
(i) Performance testing of device precision must, at minimum, use one unmodified clinical specimen from the intended use population with concentration of the brain injury biomarker(s) near the medical decision point. Contrived specimens that have been generated from pooling of multiple samples or spiking of purified analyte to cover the measuring range may be used, but the contrived samples must be prepared to mimic clinical specimens as closely as possible. This testing must evaluate repeatability and reproducibility using a protocol from an FDA-recognized standard.
(ii) Device performance data must be demonstrated through a clinical study and must include the following:
(A) Data demonstrating clinical validity including the clinical sensitivity and specificity, and positive and negative predictive value of the test in the intended use population of patients with suspected mild traumatic brain injury (
i.e., Glasgow Coma Score (GCS) of 13-15), or equivalent standard of care for determination of severity of traumatic brain injury (TBI).(B) Study must be performed using the operators and in settings that are representative of the types of operators and settings for which the device is intended to be used.
(C) All eligible subjects must meet the well-defined study inclusion and exclusion criteria that define the intended use population. The prevalence of diseased or injured subjects in the study population must reflect the prevalence of the device's intended use population, or alternatively, statistical measures must be used to account for any bias due to enrichment of subpopulations of the intended use population.
(D) All eligible subjects must have undergone a head computerized tomography (CT) scan or other appropriate clinical diagnostic standard used to determine the presence of an intracranial lesion as part of standard of care and must also be evaluated by the subject device. All clinical diagnostic standards used in the clinical study must follow standard clinical practice in the United States.
(E) Relevant demographic variables and baseline characteristics including medical history and neurological history. In addition, head injury characteristics, neurological assessments, and physical evidence of trauma must be provided for each subject. This information includes but is not limited to the following: Time since head injury, time from head injury to CT scan, time from head injury to blood draw, GCS score or equivalent, experience of loss of consciousness, presence of confusion, episodes of vomiting, post-traumatic amnesia characteristics, presence of post-traumatic seizures, drug or alcohol intoxication, mechanism of injury, acute intracranial lesion type, neurosurgical lesion, and cranial fracture.
(F) Each CT scan or other imaging result must be independently evaluated in a blinded manner by at least two board-certified radiologists to determine whether it is positive or negative as defined by the presence or absence of acute intracranial lesions. This independent review must be conducted without access to test results of the device. Prior to conducting the review, the criteria and procedures to be followed for scoring the images must be established, including the mechanism for determining consensus.
(G) All the clinical samples must be tested with the subject device blinded to the TBI status and the neurological-lesion-status of the subject.
(H) Details on how missing values in data are handled must be provided.
(I) For banked clinical samples, details on storage conditions and storage period must be provided. In addition, a specimen stability study must be conducted for the duration of storage to demonstrate integrity of archived clinical samples. The samples evaluated in the assay test development must not be used to establish the clinical validity of the assays.
(iii) Performance testing of device analytical specificity must include the most commonly reported concomitant medications present in specimens from the intended use population. Additionally, potential cross-reacting endogenous analytes must be evaluated at the highest concentration reported in specimens from the intended use population.
(iv) Expected/reference values generated by testing a statistically appropriate number of samples from apparently healthy normal individuals.
(2) The 21 CFR 809.10(a) and (b) compliant labeling must include the following limitations:
(i) A limiting statement that this device is not intended to be used a stand-alone device but as an adjunct to other clinical information to aid in the evaluation of patients who are being considered for standard of care neuroimaging.
(ii) A limiting statement that reads “A negative result is generally associated with the absence of acute intracranial lesions. An appropriate neuroimaging method is required for diagnosis of acute intracranial lesions.”
(iii) As applicable, a limiting statement that reads “This device is for use by laboratory professionals in a clinical laboratory setting.”