(371 days)
Not Found
Not Found
No
The summary describes a system that measures eye movements, calculates a score based on these measurements, and applies a simple threshold (above or below 10) to classify the result. There is no mention of AI, ML, or any complex learning algorithms being used for the analysis or classification. The processing mentioned is for blinks and normalization, which are standard image processing techniques, not necessarily indicative of AI/ML.
No
The device is intended for diagnosis, not treatment, management, or prevention of a disease.
Yes
The "Intended Use / Indications for Use" states that the device is "intended to measure and analyze eye movements as an aid in the diagnosis of concussion."
No
The device description explicitly states that the system consists of an integrated stand, eye-tracking camera, video stimulus display screen, and computer, indicating it includes hardware components beyond just software.
Based on the provided information, the EyeBOX device is an In Vitro Diagnostic (IVD).
Here's why:
- Intended Use: The intended use explicitly states that the EyeBOX is intended to "measure and analyze eye movements as an aid in the diagnosis of concussion." This clearly indicates a diagnostic purpose.
- Device Description: The device measures gaze, calculates a score, and provides a classification ("positive EyeBOX classification" or "negative EyeBOX classification") based on these measurements. This process of analyzing biological signals (eye movements) to provide information for diagnosis is characteristic of an IVD.
- Performance Studies: The performance studies evaluate the device's ability to aid in the diagnosis of concussion by comparing its results to a clinical reference standard. The metrics reported (Sensitivity, Specificity, NPV, PPV) are standard performance measures for diagnostic devices.
While the device analyzes eye movements, which are physiological signals, the analysis and classification are performed in vitro by the device's software and hardware, not directly within the patient's body. The output is a classification based on the processed eye movement data, which is then used by a physician as an aid in diagnosis.
Therefore, the EyeBOX fits the definition of an In Vitro Diagnostic device.
N/A
Intended Use / Indications for Use
The EveBOX is intended to measure and analyze eve movements as an aid in the diagnosis of concussion within one week of head injury in patients 5 through 67 years of age in conjunction with a standard neurological assessment of concussion.
A negative EyeBOX classification may correspond to eye movement that is consistent with a lack of concussion.
A positive EyeBOX classification corresponds to eye movement that may be present in both patients with or without concussion.
Product codes (comma separated list FDA assigned to the subject device)
QEA
Device Description
The Oculogica EyeBOX system consists of an integrated stand, eye-tracking camera, video stimulus display screen, and computer programmed for analysis of eye movements. It is intended to detect abnormal eye movement that may be related to a concussion. The device measures gaze, calculates a score on a 0-20 scale based on these measurements, and displays an EyeBOX classification based upon whether the scale value is above 10 or not. Scale values of 10 or more yield a positive EyeBOX classification, while scale values under 10 yield a negative EyeBOX classification.
Mentions image processing
Not Found
Mentions AI, DNN, or ML
Not Found
Input Imaging Modality
Not Found
Anatomical Site
Not Found
Indicated Patient Age Range
5 through 67 years of age
Intended User / Care Setting
Used by physicians or under the direction of physicians who have been trained to use the device.
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
N=293 subjects who were screened met the study inclusion criteria. Of these subjects, 10 were excluded from analysis because the user did not save the eye-tracking scan and one subject was excluded because of missing concussion symptom question responses. There are complete data for analysis from 282 enrolled subjects assessed with the device within 2 weeks of injury; however, 263 of the 282 subjects were assessed with the device within 1 week of injury, yielding insufficient evidence of device effectiveness beyond 1 week after injury.
Reference Standard: Initially, a 3-clinician panel was the clinical reference standard. Because 84.4% of the first N=199 adjudications had at least one clinician render a recommendation of "uncertain" for the patient's concussion status, investigators became concerned that the resulting clinical reference standard would not be interpretable. At this point in the study, they revised the clinical reference standard. Using the revised standard, a subject had a concussion if they exhibited (a) alteration of consciousness (AOC) or altered mental status (AMS) and scored less than 23 on the SCAT 3 Standardized Assessment of Concussion (SAC) and greater than 25 on the SCAT3 Symptom Severity Score (SSS), or (b) if they did not exhibit AOC/AMS but scored SAC32 on the SCAT 3. The presence of AOC/AMS was based on any of the following: self-report, witness report or the following responses to the SCAT3 SSS: "Difficulty Remembering" ≥ 4 or "Confusion" ≥ 4; or Child-SCAT3: "I get confused" > 2 or "I forget things" ≥ 2. The results of the device output remained blinded for analysis when the reference standard was changed and remained blinded until the final study analysis.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Study Type: Pivotal Clinical Study
Sample Size: 282 subjects
Key Results:
- No adverse events (device-related or unrelated) were reported in the study.
- Of the 282 subjects included in the study analysis, 46 met the revised clinical reference standard definition of concussion.
- The EyeBOX identified 37 of these as positive for concussion (score 10 or higher), resulting in a measured sensitivity of 80.4% (66.1%, 91.9%).
- This corresponds to a False Negative Rate (FNR) of 19.6% (9/46).
- Of the 236 subjects who did not meet the clinical reference standard definition of concussion, EyeBOX identified 156 as negative for concussion (score less than 10) resulting in a measured specificity of 66.1% (59.7%, 72.1%).
- The negative predictive value (NPV) of the device was 94.5% (89.9%, 97.5%).
- The positive predictive value (PPV) was 31.6% (23.3%, 40.9%).
- For PPV and NPV reference, the study prevalence of clinical classification of concussion was 16.3% (46/282).
- Observed device performance in pediating patients was comparable to the observed performance in adults.
Test-retest reliability:
- The proportion of equivalent outcomes (i.e., both tests result in a EyeBOX score > 10, or both tests results in a EveBOX score
§ 882.1455 Traumatic brain injury eye movement assessment aid.
(a)
Identification. A traumatic brain injury eye movement assessment aid is a prescription device that uses a patient's tracked eye movements to provide an interpretation of the functional condition of the patient's brain. This device is an assessment aid that is not intended for standalone detection or diagnostic purposes.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance data under anticipated conditions of use must evaluate tracked eye movement in supporting the indications for use and include the following:
(i) Evaluation of sensitivity, specificity, positive predictive value, and negative predictive value using a reference method of diagnosis;
(ii) Evaluation of device test-retest reliability; and
(iii) A description of the development of the reference method of diagnosis, which may include a normative database, to include the following:
(A) A discussion of how the clinical work-up was completed to establish the reference method of diagnosis, including the establishment of inclusion and exclusion criteria; and
(B) If using a normative database, a description of how the “normal” population was established, and the statistical methods and model assumptions used.
(2) Software verification, validation, and hazard analysis must be performed. Software documentation must include a description of the algorithms used to generate device output.
(3) Performance testing must demonstrate the electrical safety and electromagnetic compatibility (EMC) of the device.
(4) The patient-contacting components of the device must be demonstrated to be biocompatible.
(5) A light hazard assessment must be performed for all eye-tracking and visual display light sources.
(6) Labeling must include:
(i) A summary of clinical performance testing conducted with the device, including sensitivity, specificity, positive predictive value, negative predictive value, and test-retest reliability;
(ii) A description of any normative database that includes the following:
(A) The clinical definition used to establish a “normal” population and the specific selection criteria;
(B) The format for reporting normal values;
(C) Examples of screen displays and reports generated to provide the user results and normative data;
(D) Statistical methods and model assumptions; and
(E) Any adjustments for age and gender.
(iii) A warning that the device should only be used by trained healthcare professionals;
(iv) A warning that the device does not identify the presence or absence of traumatic brain injury or other clinical diagnoses;
(v) A warning that the device is not a standalone diagnostic; and
(vi) Any instructions to convey to patients regarding the administration of the test and collection of test data.
0
DE NOVO CLASSIFICATION REQUEST FOR EYEBOX
REGULATORY INFORMATION
FDA identifies this generic type of device as:
Traumatic brain injury eye movement assessment aid. A traumatic brain injury eye movement assessment aid is a prescription device that uses a patient's tracked eye movements to provide an interpretation of the functional condition of the patient's brain. This device is an assessment aid that is not intended for standalone detection or diagnostic purposes.
NEW REGULATION NUMBER: 21 CFR 882.1455
CLASSIFICATION: Class II
PRODUCT CODE: QEA
BACKGROUND
DEVICE NAME: EyeBOX
SUBMISSION NUMBER: DEN170091
DATE DE NOVO RECEIVED: December 22, 2017
Oculogica, Inc. CONTACT: 33 Irving Place New York, New York 10003
INDICATIONS FOR USE
The EveBOX is intended to measure and analyze eve movements as an aid in the diagnosis of concussion within one week of head injury in patients 5 through 67 years of age in conjunction with a standard neurological assessment of concussion.
A negative EyeBOX classification may correspond to eye movement that is consistent with a lack of concussion.
A positive EyeBOX classification corresponds to eye movement that may be present in both patients with or without concussion.
LIMITATIONS
For prescription use only.
1
The EveBOX device should not be used as a standalone assessment of concussion.
The safety and effectiveness of the EyeBOX device has not been established in patients suspected of having moderate or severe TBI, including patients with a Glasgow Coma Scale score less than 13 and patients with evidence of structural injury or intracranial hemorrhage as determined by imaging modalities.
The safety and effectiveness of the EyeBOX device has not been established in patients who have incurred an injury more than 1 week before assessment with the EyeBOX.
The safety and effectiveness of the EyeBOX device has not been established in patients who have any of the following:
- Have visual acuitv worse than 20/80 ●
- Have a history of disordered eye movement (including strabismus, diplopia, and . amblvopia)
- . Have conditions that affect the eye tissue, including retinal degeneration, cataracts, and corneal scarring
- . Have abnormal function of cranial nerves III, IV, or IV
- . Are intoxicated or are under the influence of medication, drugs, or alcohol
- Have attention deficit hyperactivity disorder .
The EyeBOX should only be used by physicians or under the direction of physicians who have been trained to use the device.
The device should not be used as a substitute for a CT scan or as a stand-alone diagnostic device.
REFER TO THE LABELING FOR A COMPLETE LIST OF WARNINGS, PRECAUTIONS AND CONTRAINDICATIONS.
DEVICE DESCRIPTION
The Oculogica EyeBOX system consists of an integrated stand, eye-tracking camera, video stimulus display screen, and computer programmed for analysis of eye movements. It is intended to detect abnormal eye movement that may be related to a concussion. The device measures gaze, calculates a score on a 0-20 scale based on these measurements, and displays an EyeBOX classification based upon whether the scale value is above 10 or not. Scale values of 10 or more yield a positive EyeBOX classification, while scale values under 10 yield a negative EyeBOX classification.
2
Image /page/2/Picture/0 description: The image shows a medical device with several labeled components. The device includes features such as chin and forehead rests, a stimulus display, and a camera with a mirror for patient positioning and monitoring. It also has an operator console, a handle, a power button, and a main power input for operation and control.
Figure 1: Oculogica EyeBOX device
Eye-tracking camera
The device eve-tracking camera detects eye motion events and computes the gaze coordinates for each eye over a period of 220 seconds. The camera is a commercially available system that uses an infrared illumination technique to capture 500 frames of gaze data per second for each eye to track the movement of the pupil. Patient-specific calibration is not used for tracking of pupillary movement.
Stimulus screen
The stimulus screen is used to display a video that lasts 220 seconds. The video is one of several pre-determined videos that may be selected. These videos include music videos, clips from children's movies, sports clips, talent performances, and other television clips. The video aperture is square, approximately one ninth the size of the LCD screen; one third each area dimension resulting in one ninth total stimulus screen area. The trajectory around the Stimulus Screen follows a predefined discrete path of 5 cycles along the perimeter of the stimulus screen.
Software analysis
During eye tracking, EyeBOX collects 220 seconds of binocular gaze data at 500Hz as the patient watches the video stimulus go around the screen five times. The first and last ten seconds of data are discarded. The data are processed for blinks and normalized. A score between 0 and 20 is the calculated from the normalized data. Scores of 10 or more are presented as consistent with the possible presence of concussion.
3
Figure 2, below, shows a sample clinical report for a normal patient. The five plot pairs on the left-hand side (L1 through R5) are the left and right eye tracking for each of the five cycles as the eyes follow the aperture around the monitor. The two plots at the top right (L1-5 and R1-5) are all five cycles for the left and right eye superimposed. The two boxes below that (L Avg and R Avg) are the averages of five cycles. The three line plots on the lower right are the differences in the x direction for the left and right eye, the differences in the y direction for the left and right eye, and an overlay plot of the individual left and right x and y plots on the same timeline.
Image /page/3/Figure/1 description: This image shows an EyeBOX report from Oculogica. The report includes patient information such as name, date of birth, gender, and test date. The report also includes results for various metrics, such as BOX score (0.74), right distance top (0.0010539748), and left area median (6.020644209). Additionally, the report contains a series of images labeled L1 through L5, R1 through R5, L Avg, R Avg, X Delta, Y Delta, and Summary.
Figure 2: EyeBOX clinical report
4
SUMMARY OF NONCLINICAL/BENCH STUDIES
BIOCOMPATIBILITY/MATERIALS
The device does not have patient-contacting materials because the device is intended for use with biocompatible drapes for the patient chinrest and headrest. Therefore, a full biocompatibility evaluation according to ISO 10993-1 was not needed in accordance with the FDA 2016 Guidance Document Use of International Standard ISO 10993-1, "Biological evaluation of medical devices - Part 1: Evaluation and testing within a risk management process."
STERILITY
The device is provided non-sterile. Cleaning instructions are provided in the user manual.
ELECTROMAGNETIC CAPABILITY & ELECTROMAGNETIC SAFETY
Performance testing was conducted to demonstrate conformance to the following standards:
- ANSI/AAMI ES60601-1: 2005 and A1:2012, Medical electrical equipment Part ● 1: General requirements for basic safety and essential performance
- IEC 60601-1-2:2014 (4th edition), Medical electrical equipment - Part 1-2: General requirements for basic safety and essential performance – Collateral Standard: Electromagnetic Compatibility - Requirements and Tests
Modifications were made to the device to conform to the requirements of EN 61000-4-2:2008, Electrostatic Discharge Immunity by adding a non-conductive coating to the device power button to mitigate the risks of device shutdown after electrostatic discharge. Marketed versions of the device (b) (4)
SOFTWARE
A failure or latent flaw in the software for the EyeBOX could indirectly result in patient injury; therefore, the software of this device is considered to have a "Moderate" level of concern. The submission contained all the elements of software documentation corresponding to the "Moderate" level of concern, as outlined in the FDA guidance document "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices." Adequate documentation describing the software, firmware, software specifications, architecture design, software development environment, traceability, revision level history, and unresolved anomalies provide the foundation for the conclusion that the software will operate in the manner described in the specifications. Hazard analysis characterized software risks including device malfunction and measurement-related errors. The submission describes verification and
5
validation testing to address the potential hazards with satisfactory results. The analysis algorithm was provided describing how the data are collected, how the data are preprocessed (including artifact removal and normalization), the underlying model that is applied to the processed data, and how the final device output(s) are calculated from the processed data.
PERFORMANCE TESTING - BENCH
Testing was performed to demonstrate conformance with the following standards:
- IEC 62471:2006 Photobiological Safety of Lamps and Lamp Systems ●
- ISO 15004-2:2007 Ophthalmic Instruments Fundamental Requirements and . Test Methods Part 2: Light Hazard Protection
SUMMARY OF CLINICAL INFORMATION
Study overview
In the pivotal clinical study, EyeBOX results were compared to a trial-specific clinical reference standard for concussion because there is no "gold standard" method to diagnose concussion. Initially, a 3-clinician panel was the clinical reference standard. Because 84.4% of the first N=199 adjudications had at least one clinician render a recommendation of "uncertain" for the patient's concussion status, investigators became concerned that the resulting clinical reference standard would not be interpretable. At this point in the study, they revised the clinical reference standard. Using the revised standard, a subject had a concussion if they exhibited (a) alteration of consciousness (AOC) or altered mental status (AMS) and scored less than 23 on the SCAT 3 Standardized Assessment of Concussion (SAC) and greater than 25 on the SCAT3 Symptom Severity Score (SSS), or (b) if they did not exhibit AOC/AMS but scored SAC32 on the SCAT 3. The presence of AOC/AMS was based on any of the following: self-report, witness report or the following responses to the SCAT3 SSS: "Difficulty Remembering" ≥ 4 or "Confusion" ≥ 4; or Child-SCAT3: "I get confused" > 2 or "I forget things" ≥ 2. The results of the device output remained blinded for analysis when the reference standard was changed and remained blinded until the final study analysis.
N=293 subjects who were screened met the study inclusion criteria. Of these subjects, 10 were excluded from analysis because the user did not save the eye-tracking scan and one subject was excluded because of missing concussion symptom question responses. There are complete data for analysis from 282 enrolled subjects assessed with the device within 2 weeks of injury; however, 263 of the 282 subjects were assessed with the device within 1 week of injury, vielding insufficient evidence of device effectiveness beyond 1 week after injury.
Inclusion criteria
-
- Ages 4 through 67 years (inclusive).
-
- Diagnosis of traumatic brain injury with a potential for concussion.
6
- Baseline vision correctable to within 20/500 bilaterally. 3.
- No prior history of diagnosed ocular motility disorder. 4.
- Able to provide a complete ophthalmologic, medical, and neurologic history and to list any 5. medications, non-prescribed drugs, or alcohol consumed within the 24 hours prior to tracking.
Exclusion criteria
- Penetrating head trauma. 1.
- CT scan determined by the attending radiologist to demonstrate evidence of acute brain 2. injury including subdural, epidural or intraparenchymal hemorrhage, edema, or mass effect.
- Burns, anoxic injury, multiple injuries, or extensive injuries resulting in medical, surgical, or 3. hemodynamic instability.
- Prior history of ocular motility dysfunction. 4.
-
- Prior extensive eye surgery.
- Physical or mental injury or baseline disability rendering task completion difficult. 6.
- Intoxicated or have blood alcohol level greater than 0.2. 7.
Co-primary effectiveness endpoint
The primary endpoint of the study is the sensitivity and specificity of the device in discriminating the presence or absence of concussion in head-injured patients per the eligibility criteria above. A lower one-sided 95% confidence limit greater than 70% for sensitivity and a lower one-sided 95% confidence limit greater than 70% for specificity were defined as the prespecified performance goals. These goals were not met in the pivotal clinical study. To supplement the primary analysis, post-hoc analyses of positive predictive value (PPV) and negative predictive value (NPV) were conducted without defined performance goals. Although the primary endpoint was not met. the effectiveness analyses described above demonstrated that the probable benefit of the device outweighs the probable risk, as discussed in the Benefit-Risk Determination section below.
Results
No adverse events (device-related or unrelated) were reported in the study.
Of the 282 subjects included in the study analysis, 46 met the revised clinical reference standard definition of concussion. The EyeBOX identified 37 of these as positive for concussion (score 10 or higher), resulting in a measured sensitivity of 80.4% (66.1%, 91.9%). This corresponds to a False Negative Rate (FNR) of 19.6% (9/46). Of the 236 subjects who did not meet the clinical reference standard definition of concussion, EyeBOX identified 156 as negative for concussion (score less than 10) resulting in a measured specificity of 66.1% (59.7%, 72.1%).
| EyeBOX
| Classification | Clinical Classification | Total | |
|---|---|---|---|
| Concussion | No Concussion | ||
| Positive | 37 | 80 | 117 |
| Negative | 9 | 156 | 165 |
| Total | 46 | 236 | 282 |
Table 1. Classification results of the EyeBOX device versus Clinical Classification
7
The negative predictive value (NPV) of the device was 94.5% (89.9%, 97.5%). The positive predictive value (PPV) was 31.6% (23.3%, 40.9%). For PPV and NPV reference, the study prevalence of clinical classification of concussion was 16.3% (46/282). Observed device performance in pediating patients was comparable to the observed performance in adults. A summary of the device performance measures is provided in Table 2 below.
| | All patients | Adults
21 years old | Pediatrics
10, or both tests results in a EveBOX score