(259 days)
The EyeBOX is intended to measure and analyze eye movements as an aid in the diagnosis of concussion, also known as mild traumatic brain injury (mTBI), in patients 5 through 67 years of age in conjunction with a standard neurological assessment of concussion.
A negative BOX Score classification may correspond to eye movement that is consistent with a lack of concussion within one week of injury.
A positive BOX Score classification corresponds to eye movement that may be present in both patients with or without concussion within one week of injury.
The EyeBOX provides a set of eye-tracking metrics in relation to a reference database of uninjured individuals.
Oculogica's EyeBOX EBX-4.1 is an eye-tracking device with custom software. The device is comprised of a host PC with integrated touchscreen interface for the operator, eye tracking camera, LCD stimulus screen and head-stabilizing rest (chin rest and forehead rest) for the patient, and data processing algorithm. The data processing algorithm detects subtle changes in eye movements resulting from concussion. The eye tracking task takes about 4 minutes to complete and involves watching a video move around the perimeter of a screen positioned in front of the patient while a high speed near-infrared (IR) camera records gaze positions 500 times per second. The post-processed data are analyzed automatically to produce one or more outcome measures. The device contains a rechargeable battery, which makes it possible to use without a direct connection to an active power source. The device has Wi-Fi and Ethernet capabilities, which optionally can provide the user with the ability to upload scans to a remote server, provide over-the-air software updates, and assist with customer support.
The provided FDA 510(k) Clearance Letter for EyeBOX EBX-4.1 (K242116) primarily focuses on demonstrating substantial equivalence to a predicate device (EyeBOX EBX-4, K212310) rather than presenting a detailed clinical study for novel acceptance criteria. The document states that the EyeBOX EBX-4.1's core diagnostic algorithm (BOX score) and hardware are "exactly the same" as its primary predicate. Therefore, the "acceptance criteria" and "device performance" in relation to a new clinical study are not explicitly defined or rigorously proven as they would be for a de novo device or a device with significant changes affecting its diagnostic performance.
However, the document does describe some performance testing related to additional eye-tracking metrics and software updates. It does NOT include a Multi-Reader Multi-Case (MRMC) comparative effectiveness study or a standalone algorithm performance study for the primary diagnostic function (BOX score).
Based on the provided text, here's an attempt to outline the requested information, acknowledging the limitations inherent in a 510(k) submission focused on substantial equivalence:
Acceptance Criteria and Device Performance (Based on "Additional Eye-tracking Metrics Performance Testing")
The document describes testing for newly included eye-tracking metrics. While not explicit "acceptance criteria" in the strict sense for diagnostic performance, the following table summarizes the reported performance for these new metrics, which are stated to be for "contextual clinical information" and "not intended to aid in diagnosis." The key acceptance criterion for these metrics appears to be their reliability as demonstrated by test-retest performance.
1. Table of Acceptance Criteria and Reported Device Performance
| Feature/Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Accuracy & Precision (General eye-tracking measurements) | Functioned as intended (qualitative statement) | Accuracy and precision testing completed for timestamp, gaze position, pupil size (diameter), and blink duration. Accuracy testing also conducted for blink count and saccade detection. (No quantitative results provided) |
| Reliability (Test-Retest for Eye-tracking Metrics) | Expected variation between repeated tests represented by Bland-Altman 95% limits of agreement (LoA); "functioned as intended." | Bland-Altman 95% LoA values were determined for each metric. These values are incorporated into the device output. "In all instances, the EyeBOX Model EBX-4.1 functioned as intended." (No specific LoA values provided) |
2. Sample Size and Data Provenance
- Test Set Sample Size: 30 healthy individuals.
- Data Provenance: Retrospective, drawn "from the EyeBOX normative database." (No specific country of origin mentioned, but implies data collected during previous EyeBOX studies for normative data). The study itself was prospective in design for the "test-retest" aspect (i.e., participants underwent two tests for this specific study).
3. Number of Experts and Qualifications for Ground Truth
- Not Applicable (N/A) for this specific testing. The "Additional Eye-tracking Metrics Performance Testing" described is primarily a technical validation (accuracy, precision, test-retest reliability) of the eye-tracking measurements themselves, not a diagnostic performance study requiring expert ground truth on concussion. The device's diagnostic capability (BOX score) is deemed substantially equivalent to the predicate, K212310, and thus, its ground truth establishment would refer to the studies for that earlier clearance.
4. Adjudication Method for the Test Set
- None. As this was a technical validation of eye-tracking metrics and not a diagnostic accuracy study, expert adjudication was not relevant.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No. An MRMC study was not described in this 510(k) submission. The document focuses on demonstrating that changes (software updates, additional output metrics) don't adversely impact the performance established by the predicate device. The primary diagnostic algorithm (BOX score) is stated to be identical to the predicate.
6. Standalone (Algorithm Only) Performance
- Not explicitly highlighted as a separate study in this document. The BOX score algorithm's performance as a standalone diagnostic aid would have been established during the clearance of the predicate device (K212310). This 510(k) asserts that the BOX score algorithm itself is "exactly the same" and its "principles of operation have not changed." The "Output Report Metric Selection for Concussion" is mentioned as a performed activity, but details of a standalone study for the EBX-4.1's diagnostic performance are not provided.
7. Type of Ground Truth Used
- Not explicitly detailed for the EBX-4.1 in this document. For the diagnostic performance (BOX score), the ground truth for the predicate device's studies (K212310) would have been crucial for establishing its effectiveness as an "aid in the diagnosis of concussion." The current submission emphasizes the stability of the BOX score algorithm and its output compared to the predicate.
- For the additional eye-tracking metrics (Pupil Size, Blinks, Disconjugacy, etc.) that are for "contextual clinical information" and "not intended to aid in diagnosis," the "ground truth" for the performance testing cited would be the true physical eye movements/features being measured (e.g., actual blink duration, actual pupil diameter for accuracy). This often involves calibrated phantoms or controlled experimental setups to assess measurement correctness. The "test-retest" reliability used the same healthy individuals observed twice.
8. Sample Size for the Training Set
- Not provided in this document. The document states that the EyeBOX EBX-4.1's diagnostic algorithm (BOX score) and hardware are "exactly the same" as the predicate (EyeBOX EBX-4). Therefore, the training data for the BOX score algorithm would be associated with the development and validation of the K212310 device, not explicitly detailed here.
- The "reference database of uninjured individuals" mentioned for the new output metrics is based on 30 healthy individuals for the test-retest study, but this is the testing set for reliability, not a training set for an algorithm. The actual size of the "normative database" (a "reference database of uninjured individuals") for contextual comparison is not specified as a training set, but as a general reference for physicians.
9. How the Ground Truth for the Training Set Was Established
- Not provided in this document for the reasons stated in point 8. This information would be critical for the original K212310 submission for the EyeBOX EBX-4.
- For the newly added "reference database of uninjured individuals," the "ground truth" is simply that these individuals were uninjured. This likely involved a screening process to confirm their healthy status, but no details are given.
§ 882.1455 Traumatic brain injury eye movement assessment aid.
(a)
Identification. A traumatic brain injury eye movement assessment aid is a prescription device that uses a patient's tracked eye movements to provide an interpretation of the functional condition of the patient's brain. This device is an assessment aid that is not intended for standalone detection or diagnostic purposes.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance data under anticipated conditions of use must evaluate tracked eye movement in supporting the indications for use and include the following:
(i) Evaluation of sensitivity, specificity, positive predictive value, and negative predictive value using a reference method of diagnosis;
(ii) Evaluation of device test-retest reliability; and
(iii) A description of the development of the reference method of diagnosis, which may include a normative database, to include the following:
(A) A discussion of how the clinical work-up was completed to establish the reference method of diagnosis, including the establishment of inclusion and exclusion criteria; and
(B) If using a normative database, a description of how the “normal” population was established, and the statistical methods and model assumptions used.
(2) Software verification, validation, and hazard analysis must be performed. Software documentation must include a description of the algorithms used to generate device output.
(3) Performance testing must demonstrate the electrical safety and electromagnetic compatibility (EMC) of the device.
(4) The patient-contacting components of the device must be demonstrated to be biocompatible.
(5) A light hazard assessment must be performed for all eye-tracking and visual display light sources.
(6) Labeling must include:
(i) A summary of clinical performance testing conducted with the device, including sensitivity, specificity, positive predictive value, negative predictive value, and test-retest reliability;
(ii) A description of any normative database that includes the following:
(A) The clinical definition used to establish a “normal” population and the specific selection criteria;
(B) The format for reporting normal values;
(C) Examples of screen displays and reports generated to provide the user results and normative data;
(D) Statistical methods and model assumptions; and
(E) Any adjustments for age and gender.
(iii) A warning that the device should only be used by trained healthcare professionals;
(iv) A warning that the device does not identify the presence or absence of traumatic brain injury or other clinical diagnoses;
(v) A warning that the device is not a standalone diagnostic; and
(vi) Any instructions to convey to patients regarding the administration of the test and collection of test data.