ImPACT is a neurocognitive test battery that provides healthcare professionals with objective measure of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 12-59.

ImPACT Pediatric is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion.

ImPACT Pediatric is a neurocognitive test battery that provides healthcare professionals with objective measure of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 5-11.

Device Description

ImPACT® (Immediate Post-Concussion Assessment and Cognitive Testing) and ImPACT Pediatric are computer-based neurocognitive test batteries for use as an assessment aid in the management of concussion.

ImPACT and ImPACT Pediatric are software-based tools that allows healthcare professionals to conduct a series of neurocognitive tests that provide data related to the neurocognitive functioning of the test taker. This computerized neurocognitive test battery measures various aspects of neurocognitive functioning including reaction time, memory, attention, and spatial processing speed. It also records symptoms of concussion in the test taker.

ImPACT and ImPACT Pediatric provide healthcare professionals with a set of well-developed and researched neurocognitive tasks that have been medically accepted as state-of-the-art best practices. The devices are intended to be used as part of a multidisciplinary approach to concussion assessment and patient management.

AI/ML Overview

This response summarizes the acceptance criteria and supporting studies for the ImPACT and ImPACT Pediatric devices, as detailed in the provided document.

Acceptance Criteria and Device Performance

The acceptance criteria for ImPACT and ImPACT Pediatric are primarily demonstrated through the establishment of their psychometric properties: construct validity, test-retest reliability, and the development of a robust normative database. The device performance is reported through the results of various peer-reviewed studies and clinical data supporting these properties.

Table 1: Acceptance Criteria and Reported Device Performance

Acceptance Criteria Category	Specific Criteria/Outcome Demonstrated	Reported Device Performance (Summary)
Construct Validity	Demonstrate the device measures what it purports to measure (e.g., specific cognitive functions).	ImPACT: Significant correlations with traditional neuropsychological measures (SDMT, WAIS-R, NFL Battery components) for processing speed, reaction time, memory. Demonstrated good convergent and discriminant validity. ImPACT Pediatric: Significant correlations (20/24 potential comparisons) with WRAML-2 subtests (e.g., Story Memory, Design Recall), indicating it measures important aspects of memory.
Test-Retest Reliability	Demonstrate consistent results over time for multiple administrations.	ImPACT: Robust test-retest reliability with ICCs generally ranging from 0.46 to 0.88 across various intervals (30 days to 2 years) for Verbal Memory, Visual Memory, Visual Motor Speed, and Reaction Time. ImPACT Pediatric: ICCs ranging from 0.46 to 0.89 across test modules (e.g., Word List, Memory Touch, Stop & Go) over one-week intervals, indicating adequate to excellent stability for most.
Normative Database	Establish a representative database to compare patient performance against "normal" population.	ImPACT: Standardization sample of 17,013 individuals (ages 10-59 years, diverse gender/age breakdown). Data collected by trained professionals in supervised settings. ImPACT Pediatric: Normative database of 915 children (ages 5-12 years) from multiple clinical sites (Atlanta, Annapolis, Marquette, Pittsburgh, Guelph, ON). Age-stratified, considering gender differences.
Reliable Change Index (RCI)	Provide a statistical calculation to determine if a change in score is clinically meaningful and not due to measurement error or practice effects.	ImPACT: RCI calculation provided to indicate clinically significant improvement, reducing adverse impact of measurement error. ImPACT Pediatric: RCI calculated to highlight score changes not due to practice effects or measurement error, displayed in red on reports.
Validity Index	Provide an index to identify invalid baseline examinations.	ImPACT: Algorithm-based index identifies invalid tests based on sub-optimal performance on specific subtests (e.g., X's and O's Total Incorrect > 30, Word Memory Learning Pct Correct < 69%). Automated flagging in reports. Not explicitly detailed for ImPACT Pediatric, but importance of valid tests is implied.

Study Details Proving Device Meets Acceptance Criteria

The device relies on a collection of previously published peer-reviewed studies and additional clinical data submitted with the de novo request. The FDA's determination is based on the adequacy of this compiled evidence rather than a single, large prospective clinical trial specifically for this de novo submission.

1. Sample Sizes Used for the Test Set and Data Provenance:

ImPACT:
- Construct Validity Studies: Sample sizes for individual studies ranged from N=30 to N=100.
  - Iverson et al. (2005): N=72 athletes from PA high schools.
  - Maerlender et al. (2010): N=54 varsity athletes from Dartmouth College.
  - Schatz & Putz (2006): N=30 college students from St. Joseph's University, Philadelphia.
  - Allen & Gfeller (2011): N=100 psychology students from a private university (Midwestern).
- Reliability Studies: Sample sizes for individual studies ranged from N=25 to N=369.
  - Schatz (2013): N=25
  - Cole (2013): N=44 (active duty military population)
  - Nakayama (2014): N=85
  - Elbin (2011): N=369 high school athletes
  - Schatz (2010): N=95
- Normative Database: N=17,013 individuals. Data collected from high schools and colleges across the US, and adult athlete populations/coaches/administrators.
- Data Provenance: Predominantly retrospective analysis of published literature and existing clinical data. Studies are from various locations within the United States (PA, Dartmouth, Philadelphia, Midwestern university) and some multinational data for normative database (South African sample mentioned for cross-cultural norming). The military population study (Cole, 2013) also indicates a specific cohort. Data is gathered from both concussed and concussion-free individuals for different study purposes (validity vs. norming).
ImPACT Pediatric:
- Normative Database: N=915 children. Data collected from clinical sites in Atlanta, GA; Annapolis, MD; Marquette, MI; Pittsburgh, PA; and Guelph, Ontario, Canada.
- Reliability Study: N=100 children (ages 5-12 years) participating in youth soccer and hockey leagues. (Unpublished study reported).
- Construct Validity Study: N=83 participants (ages 5-12 years).
- Data Provenance: A mix of clinical sites across the US and Canada. Primarily retrospective analysis of previously collected data.

2. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications:

The document does not specify a fixed number of experts for adjudicating individual cases within a test set, as this is not a diagnostic device relying on expert consensus on image interpretation.
Instead, the "ground truth" for these performance studies is established by:
- Clinical Standards: For construct validity, ImPACT was compared to traditional neuropsychological measures (e.g., Symbol Digit Modalities Test, WAIS-R, NFL Battery components). These traditional tests are themselves well-established and interpreted by qualified neuropsychologists or clinicians.
- Normative Data Collection: "Professionals who were specifically trained to administer the tests," including "Neuropsychologists, Psychologists and Neuropsychology/Psychology graduate students, Certified Athletic Trainers and Athletic Training Graduate Students and Nurses." Their expertise ensures correct test administration and data collection for establishing norms.
- Reference Standards: For the ImPACT construct validity studies, diagnoses of concussion were made using "then-applicable clinical standards including grading of concussion using AAN guidelines." For ImPACT Pediatric, construct validity was assessed against the "Wide Range Assessment of Memory and Learning-2 (WRAML-2)," a recognized neuropsychological battery.

3. Adjudication Method for the Test Set:

Not applicable in the typical sense of a diagnostic medical device relying on image interpretation and expert consensus for ground truth. The "adjudication" is inherent in the established methodologies of traditional neuropsychological testing and statistical correlations.

4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

No MRMC study was performed in the traditional sense of comparing human readers with and without AI assistance for interpretation.
This device is a Computerized Cognitive Assessment Aid, not an AI for image interpretation. Its function is to administer a neurocognitive test battery and provide objective measures. The "AI" component is more akin to sophisticated algorithms for scoring, norming, and identifying reliable change or invalid tests, rather than a system that assists human interpretation of complex medical imagery.
The benefit is the "Ability to have access to a non-invasive cognitive assessment battery that can be used to compare pre-injury (baseline cognitive performance) to post-injury cognitive performance" and "Ability to compare cognitive test performance to a large normative database in the absence of baseline testing," replacing or augmenting less standardized manual methods.

5. Standalone Performance (Algorithm Only without Human-in-the-Loop Performance):

The device inherently operates as an "algorithm only" in generating scores and reports. However, it is explicitly not a standalone diagnostic device.
The "performance" of the algorithm is demonstrated through its ability to accurately measure cognitive functions (construct validity), consistently produce results (reliability), and correctly apply the normative and change index calculations.
The output (scores, RCIs, validity flags) is then interpreted by a healthcare professional. The device itself does not make a diagnosis. Therefore, its "standalone performance" is specifically in generating the objective data, which is then used by a human clinician as an aid.

6. Type of Ground Truth Used:

Neuropsychological Test Scores: For construct validity, the device's scores were correlated with scores from established, traditional paper-and-pencil neuropsychological assessment batteries. These batteries serve as the "ground truth" for what cognitive functions are being measured.
Clinical Standards/Diagnosis: For some ImPACT studies, comparison was made to individuals diagnosed with concussion based on "then-applicable clinical standards."
Normative Data: The "normal" population for the normative database was established through "clinical work-up... including the establishment of inclusion and exclusion criteria" by trained professionals, ensuring subjects did not have medical conditions affecting test performance.

7. Sample Size for the Training Set:

ImPACT: The standardization sample (normative database) served as the primary basis for the algorithm's "training" or establishment of "normal" values and internal psychometric properties. This sample consisted of 17,013 individuals.
ImPACT Pediatric: The normative database for ImPACT Pediatric comprised 915 children.
These are not "training sets" in the modern machine learning sense (where a model learns iteratively), but rather the large datasets used to establish the statistical properties and normative values from which the device's algorithms derive meaning (e.g., percentile rankings, Reliable Change Index).

8. How the Ground Truth for the Training Set (Normative Database) Was Established:

ImPACT:
- Data collected from high schools and colleges nationwide, ensuring representation of the intended use population.
- Older adults included coaches, school administrators, nurses, and adult athletes.
- Testing administered by specifically trained professionals: Neuropsychologists, Psychologists, Neuropsychology/Psychology graduate students, Certified Athletic Trainers, Athletic Training Graduate Students, and Nurses.
- Tests completed in a supervised setting.
- Data uploaded to a secure HIPAA-compliant server and de-identified.
- Participants were English speakers, not reported to have underlying intellectual/developmental data, and not currently concussed or suffering from medical conditions that might affect performance.
ImPACT Pediatric:
- Large, age-stratified sample of children (5-12 years) from multiple clinical sites.
- Tests administered by a researcher, clinician, or educational professional trained in the use of ImPACT Pediatric.
- Tests taken on an iPad in a one-on-one basis (no group testing).
- Children instructed to respond by touching the screen.
- "Ground truth" for "normal" performance was derived from statistical analysis (means, standard deviations, t-tests for gender differences) of this large, carefully collected, and screened normative dataset. Factor analysis was conducted on a subset to derive relevant score clusters.

Summary

{0}------------------------------------------------

DE NOVO CLASSIFICATION REQUEST FOR IMPACT AND IMPACT PEDIATRIC

REGULATORY INFORMATION

FDA identifies this generic type of device as:

Computerized Cognitive Assessment Aid for Concussion. The computerized cognitive assessment aid for concussion is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an indication of the current level of cognitive function in response to concussion. The computerized cognitive assessment aid for concussion is used only as an assessment aid in the management of concussion to determine cognitive function for patients after a potential concussive event where other diagnostic tools are available and does not identify the presence or absence of concussion. It is not intended as a stand-alone diagnostic device.

NEW REGULATION NUMBER: 882.1471

CLASSIFICATION: CLASS II

PRODUCT CODE: POM

BACKGROUND

DEVICE NAME: IMPACT AND IMPACT PEDIATRIC

SUBMISSION NUMBER: DEN 150037

DATE OF DE NOVO: AUGUST 11, 2015

CONTACT: IMPACT APPLICATIONS, INC. 2000 TECHNOLOGY DRIVE, SUITE 150 PITTSBURGH, PA 15219

INDICATIONS FOR USE

ImPACT:

ImPACT is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion.

{1}------------------------------------------------

ImPACT Pediatric:

ImPACT Pediatric is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion.

LIMITATIONS

For prescription use only.

The safety and effectiveness of ImPACT for individuals under the age of 12 years and over the age of 59 years has not been established. The safety and effectiveness of ImPACT Pediatric for individuals under the age of 5 years and over the age of 11 years has not been established.

ImPACT and ImPACT Pediatric are intended be used by medical professionals qualified to interpret the results of a concussion assessment examination and aid in the management of concussion.

The devices are not intended to be used as a stand-alone diagnostic device.

PLEASE REFER TO THE LABELING FOR A MORE COMPLETE LIST OF WARNINGS, PRECAUTIONS AND CONTRAINDICATIONS.

DEVICE DESCRIPTION

{2}------------------------------------------------

ImPACT

ImPACT (ages 12-59) is administered using the following sequence:

1. Demographics
1. Symptom Scale
1. Word Memory
1. Design Memory
1. X's and O's
1. Symbol Match
1. Color Match
1. Three Letters

An example of one of the tests, the Symbol Match, is shown below in Figure 1. This particular test evaluates processing speed, learning, and memory.

Figure 1: Example Symbol Match Task

Upon successful completion of the test battery, ImPACT generates a report with the following information:

Demographic Information

Background Information and Native Language
Education and Special Needs as reported by test taker ●
Concussion and Sport Background ●
. Medical Information as reported by test taker

ImPACT Composite Scores

Verbal Memory Composite, Visual Memory Composite, Visual Motor Speed ● Composite, Reaction Time Composite, Impulse Control Composite, Total Symptom Score, Cognitive Efficiency Index (CEI)

{3}------------------------------------------------

Test Battery Modules

Word Memory, Design Memory, X's and O's, Symbol Match, Color Match, Three Letters
Post-Concussion Symptom Checklist
. Individual Symptom Scores
Graphic Representation of Composite Scores and Symptoms
. Graphs depicting: Verbal Memory, Visual Memory, Visual Motor Speed, and Reaction Time Composite Scores, Total Symptom Score
Table 1 shows an example test score report, which identifies the overall composite score table for each cognitive area tested.

Composite Scores	Percentile scores if available are listed in small type
Memory composite (verbal)	82	49%	57	<1%
Memory composite (visual)	78	60%	48	2%
Visual motor speed	32.83	32%	22.78	<1%
Reaction time composite	0.61	46%	1.12	<1%
Impulse control composite	3		4
Total Symptom Score	0		38
Cognitive Efficiency Index	0.33		0.17

Table 1: Example ImPACT Score Report

The Reliable Change Index (RCI) methodology allows the clinician to reduce the adverse impact of measurement error on test interpretation. To represent clinically significant improvement, the change score should be statistically reliable. However, the converse is not true; a statistically reliable change does not necessarily guarantee a clinically meaningful change. It is also important to emphasize that an RCI score does not provide a diagnosis. ImPACT shows the score in red if it surpasses the RCI as shown in the figure above. Test score change over repeated administrations is to be expected. The issue for healthcare professionals is to determine when this change is significant and clinically meaningful. ImPACT provides RCIs for each Module and Composite. The table also shows percentile scores to the right of the composite score. These percentile scores indicate the patient's performance compared to the age- and gender- matched normative database.

The Cognitive Efficiency Index (CEI) measures the interaction between accuracy (percentage correct) and speed (reaction time) in seconds on the Symbol Match test. This score was not developed to make return-to-activity decisions but it can be helpful in determining the extent to which the individual tried to work very fast on symbol match (decreasing accuracy) or attempted to improve their accuracy by taking a more deliberate and slow approach (jeopardizing speed). A higher score indicates that the individual did well in both the speed and memory domains on the Symbol Match test. A low score (below .20) means that the individual performed poorly on both

{4}------------------------------------------------

the speed and accuracy component. If this score is a negative number, the individual performed very poorly on the reaction time component.

ImPACT also provides a validity index designed to aid in identifying invalid baseline examinations. This index is based on the following algorithm:

X's and O's Total Incorrect > 30 OR
Impulse Control Composite >30 OR
Word Memory Learning Pct Correct <69% OR ●
Design Memory Learning Pct Correct <50% OR ●
Three Letters Total Letters Correct <8

If any of these criteria are reached for a given baseline test, the ImPACT Score Report will automatically print a sentence that identifies the test results as being of questionable validity. If this is the case, the test administrator is encouraged to repeat the baseline exam, only after discussing the test results with the patient and identifying the reasons for the invalid test (e.g. difficulty understanding one or more of the modules, not taking the test seriously, etc.).

ImPACT Pediatric

ImPACT Pediatric is administered in a similar manner and has a similar output to ImPACT. ImPACT Pediatric is administered to children (ages 5-11) and includes the following assessments:

1. Demographics
1. Symptom Scale as answered by test taker
1. Symptom Scale as answered by parent or guardian
1. Word Memory
1. Design Memory
1. Stop and Go!
1. Memory Touch
1. Picture Match

The image in Figure 2 shows an example of the Picture Match task:

Image /page/4/Picture/19 description: The image shows a memory match game with 16 cards. Two of the cards are showing a peach. The game also shows the number of taps is 30 and the time is 27.

Figure 2: Example Picture Match Task

{5}------------------------------------------------

Upon successful completion of the test battery, ImPACT Pediatric generates a report with the following information:

Demographic Information and Score Overview

Background Information and Native Language ●
Concussion / Injury Background
Test Dates
. Factor Scores for each of four cognitive domains

Medical History

Detailed Medical History for the Child Including Pre-existing Conditions .
Symptom History
Symptoms Reported by Test Taker
Symptoms Reported by Parent .

Test Battery Modules

Word List, Design Rotation, Stop & Go, Memory Touch, Picture Match, Color ● Match
Table 2 shows an example of the score output from ImPACT Pediatric. Note that ImPACT Pediatric does not provide a composite score table like ImPACT. However, the scores for individual tests in the battery are presented, and values which surpass the Reliable Change Index (RCI) are presented in red.

{6}------------------------------------------------

Word List
Skipped this module	No	No
Immediate Recall Num. Correct	5	1
Immediate Recall Num. Incorrect	0	2
Immediate Recall Percent Correct	100	20
Delayed Recall Number Correct	4	0
Delayed Recall Number Incorrect	0	2
Delayed Recall Percent Correct	100	20
Del. Recognition Number Correct	7	3
Del. Recognition Percent Correct	70	30
Time Between Tests (in seconds)	294.33	342.36
Design Rotation
Skipped this module	No	No
Number Correct	10	10
Percent Correct	100	100
Average Time (in seconds)	1.47	1.86
Stop & Go
Skipped this module	No	No
Number Correct	10	10
Average Correct Answer Time	0.62	1.05
Number Incorrect Red Lights	0	0
Number Incorrect Green Lights	0	0
Number Incorrect Yellow Lights	0	2
Number of Omissions	0	0

Table 2: Example ImPACT Pediatric Report

SUMMARY OF NONCLINICAL/BENCH STUDIES

SOFTWARE

The software for ImPACT and ImPACT Pediatric are consistent with a 'MODERATE' level of concern, as discussed in the FDA document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices," issued May 11, 2005. Appropriate documentation was provided as part of the de novo request.

SUMMARY OF CLINICAL INFORMATION

ImPACT

The sponsor supported the safety and effectiveness of the ImPACT device with clinical data from over 250 peer-reviewed articles that have been published with more than half representing the results of independent research. An adequate representative sample of participants was included in the clinical studies to support the age range for the device. Ten of the published studies were provided by the sponsor as the main studies in support of the test-reliability, construct validity and development of the normative database as detailed below. These research publications analyze the psychometric properties of ImPACT based on studies that address test validity, reliability, clinical utility and sensitivity/specificity or provide indirect support for using the ImPACT test battery in evaluation of concussion which is synonymous with the term, mild traumatic brain injury.

ImPACT's design is based on the traditional neurocognitive testing. Traditional neurocognitive (also known as neuropsychological) testing standards are defined by the American Academy of

{7}------------------------------------------------

Clinical Neuropsychology and the National Academy of Neuropsychology. These organizations released a position paper with recommendations on appropriate standards and conventions for computerized neuropsychological assessment devices (Bauer et. al. Archives of Clinical Neuropsychology Vol. 27 (2012) pp362-373). The position paper includes standards for the psychometric development issues, especially reliability, and validity. In addition, on June 2-3. 2011, the FDA hosted a workshop co-sponsored by three clinical professional societies: Academy of Neurology, American Epilepsy Society, and National Academy of Neuropsychology, to discuss issues related to the validation and labeling of devices used to assess seizures, cognitive function, traumatic brain injury (TBI) and concussion. During the workshop, topics related to standards and conventions for computerized neuropsychological assessment devices were discussed, similar to the topics described in the position paper (Eastman P. FDA Workshop: Guidance Offered for Evaluating Devices for Seizure Detection, TBI/Concussion, and Cognitive Function. Neurology Today. 2011:11(13):42-43.). Development of the ImPACT test battery and its validation is based on the guidelines described in the position paper and discussed during the workshop.

1. Construct Validity of the ImPACT Battery. The sponsor provided abstracts from 4 published studies in support of the validity of the ImPACT test battery:
- Iverson GL, Lovell MR, Collins MW. Validity of ImPACT for measuring . processing speed following sports-related concussion. J Clin Exp Neuropsychol. 2005;27(6):683-689.
- Maerlender A, Flashman L, Kessler A, et al. Examination of the Construct . Validity of ImPACT computerized Test, Traditional and Experimental Neuropsychological Measures. The Clinical Neuropsychologist. 2010; 24(8):1309T1325.
- . Schatz P, Putz BO. Cross-validation of measures used for computer-based assessment of concussion. Appl Neuropsychol. 2006:13(3):151-159.
- Allen BJ, Gfeller JD. The immediate post-concussion assessment and cognitive . testing battery and traditional neuropsychological measures: A construct and concurrent validity study. Brain Inj. 2011;25(2):179-191.

Table 3 summarizes the results of the testing. which supports the validity of ImPACT.

{8}------------------------------------------------

Table 3: Summary of Studies Used to Support Construct Validity of ImPACT

Study	Sample Description	Inclusion/ExclusionCriteria	Clinical ReferenceStandard	Analysis Performed	Summary of Results
Iverson,Lovell &Collins(2005)	N=72. Athletes randomly extractedfrom larger study. These athletes werefrom high schools in PA.Average age of 17.1 years. Averageeducation of 10.5 years. 83.8% maleand 16.2% female. For 59% this wasfirst documented concussion, 21% hadone prior concussion and 20% had 2+prior concussionsSample size was selected based uponplanned t tests and PCFA analysis.	Athletes had to havesuccessfullycompleted a baselineassessment pre-participation. Had tohave been diagnosedby a physician orathletic trainer with aconcussion and seenin clinic within 21days of concussion	All subjects werediagnosed with aconcussion usingthen-applicableclinical standardsincluding grading ofconcussion usingAAN guidelines	Comparison of theImPACT to atraditionalneuropsychologicalmeasure, the SymbolDigit Modalities Test(SDMT)Correlational analysis,t-tests and PrincipleComponents FactorAnalysis	SDMT correlated 0.70 withProcessing Speed and -0.60with Reaction Time. PairwiseT-tests revealed strongestrelationships were betweenSDMT and Processing Speedand Reaction Time versusVerbal, or Visual Memory orTotal Symptoms. The PCFAconfirms a three factor solutionof Speed/Reaction TimeMemory and Total Symptoms
Marlander,Flashman,(2010)	N=54. Varsity athletes randomlyselected from a larger study. Allathletes were tested at DartmouthCollege.40 from football team and theremainder from non- contact sports:golf, cross country and crew.42 reported no prior concussions.13 reported at least 1 prior concussion.Sample size was selected based uponplanned analysis	All subjects wereconcussion free atthe time of testingand agreed to take acompleteneuropsych battery	A comprehensiveNeuropsychologicalbattery designed tobe sensitive tocognitive functionsknown to showdeficits or changes inthe context of TBI(see Table 2 in thearticle)	Correlational analysisrelating ImPACTComposites andNeuropsychologicaldomains	Overall ImPACT was shownto have good convergent anddiscriminant validity. ImPACTdomains correlated withexpected experimentalmeasures and traditional paperand pencil measures ofmemory and processing speedDue to cross-loadings of somesubtests authors caution thatImPACT is best used as ascreening tool.
Schatz &Putz(2006)	N=30. College students whovolunteered as part of courserequirement. All subjects were testedat St. Joseph's University inPhiladelphia.Subjects were 14 men, 16 women,non- varsity athletes Ages were 18-23(mean =21) and they were concussionfree in prior 6 months.Sample size was selected based uponplanned analysis	All subjects wereconcussion free attime of testing andwere participating aspart of courserequirement	Trail Making Test A& B (TMT),Digit Symbol subtestof the WechslerAdult IntelligenceScale-Revised(WAIS-R)	Correlational analysisrelating ImPACTComposites to TrailMaking A-B.D-2 testand Digit Symbolfrom WAIS-R	ImPACT Reaction time wassignificantly correlated withTrails A-B (.61, .44respectively) and Digit Symbolfrom the WAIS (.46). DigitSymbol was also significantlycorrelated with ImPACTProcessing Speed Index -. 51.
Allen &Gfeller(2011)	N=100. Students recruited fromundergraduate psychology class at aprivate university (Midwestern).Subjects included 56 females, 44males. 76% Caucasian, 9% Asian, 8%Asian Indian, 4% African American,1% Pacific Islander and 2% Other.Mean age of 19.69 years with meaneducation of 13.21.Sample size selected based uponplanned analysis.	Excluded werestudents whoreported a history ofADHD, major TBIwithin last 3 monthsor ESL. Additionally.subjects had to passa Validity Index	Neuropsychologicalbattery used by theNFL. (See Table 1 inthe article)	Correlational analysiswith ImPACT andNFL Battery (HVLT-R, BVMT-R, TrailMaking Test A-B,COWA and SymbolSearch and Digit SpanForward andBackward from theWAIS-III	Significant correlations wereobtained between many of theImPACT composite scoresand components of the NFLbattery. Correlations tended tobe in the .3 to .45 range andlikely could have beensuppressed by a restrictedrange in the sample

PCFA: principal component factor analysis; AAN: American Academy of Neurology; SDMT: Symbol Digit Modalities Test; ADHD: Attention Deficit Hyperactivity Disorder; ESL: English as a second language; HVLT-R: Hopkins Verbal Learning Test - Revised; BVMT-R: Benton Visual Learning Test -Revised; COWA: Controlled Oral Word Association; WAIS-R: Wechsler Adult Intelligence Scale -Revised; WAIS-III: Wechsler Adult Intelligence Scale - 3rd edition.

{9}------------------------------------------------

1. Reliability. The sponsor provided five published studies, which assessed the reliability of the ImPACT test battery using different intervals between assessments ranging from 30 days to 2 years between tests:
- Schatz P. Long term test-retest reliability of baseline cognitive assessments using ● ImPACT. Am J Sports Med. 2010:38(1):47-53.
- Schatz P. Ferris CS. One-month test-retest reliability of the ImPACT test battery. Arch Clin Neuropsychol. 2013 Aug; 28(5):499-504. Epub 2013 May 23.
- Nakayama Y. Covassin T. Schatz P. Nogle S. Kovan J. Examination of the Test-● Retest Reliability of a Computerized Neurocognitive Test Battery. Am J Sports Med. 2014 Jun 6;42(8):2000-2005.
- . Cole WR, Arrieux JP, Schwab K, Ivins BJ, Oashu FM, Lewis SC. Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population. Arch Clin Neuropsychol. 2013 Nov:28(7):732-42. Epub 2013 Jul 2.
- . Elbin. Schatz. Covassin. One-Year Test-Retest Reliability of the Online Version of ImPACT in High School Athletes. Am J Sports Med. 2011 Nov: 39 (11):2319-24.

Table 4 below summarizes the published studies in support of the stability of the various subtests administered by ImPACT (e.g., test-retest reliability). The table demonstrates relatively robust test-retest reliability of the ImPACT modules over a variety of time intervals up to two years.

Variable	Interval Between Assessments
	30 daysSchatz (2013)n=25	30 daysCole (2013)n=44	0-45 daysNakayama (2014)n=85	0-50 daysNakayama (2014)n=85	45-50Nakayama (2014)n=85	1 yearElbin (2011)n=369	2 yearsSchatz (2010)n=95
Verbal Memory
ICC	0.79	0.6	0.76	0.65	0.69	0.62	0.46
r	0.66	0.61	-	-	-	0.45	0.3
Visual Memory
ICC	0.6	0.5	0.72	0.6	0.69	0.7	0.65
r	0.43	0.49	-	-	-	0.55	0.49
Visual Motor Speed
ICC	0.88	0.83	0.87	0.85	0.88	0.82	0.74
r	0.78	0.86	-	-	-	0.74	0.6
Reaction Time
ICC	0.77	0.53	0.67	0.71	0.81	0.71	0.68
r	0.63	0.53	-	-	-	0.62	0.52

Table 4: Summary of the obtained test-retest reliability coefficients

*ICC - intraclass correlation coefficient, a measure of test-retest reliability

1. Standardization/Normative Database. The sponsor provided information on the standardization of the ImPACT test battery and the development of the normative database. Standardization of the current version of ImPACT was accomplished through participation of test subjects from high schools and colleges from around the country that are representative of the intended use population. Older adults were drawn from adult athlete populations or were coaches, school administrators, nurses. Although not keyed specifically to the US Census, the sample was inclusive of minorities at a rate that

{10}------------------------------------------------

reflected the composition of the school systems involved. It is important to note that published data have not indicated significant differences between minority and Caucasian athletes on the ImPACT test scores , nor in a large multi-racial South African sample .

Normative data were collected by research partners acknowledged in the user manual. All testing was completed by professionals who were specifically trained to administer the tests. These professionals consisted of Neuropsychologists, Psychologists and Neuropsychology/Psychology graduate students, Certified Athletic Trainers and Athletic Training Graduate Students and Nurses. All testing was completed in a supervised setting and data were later uploaded onto a secure HIPPA-compliant server. Data were deidentified and placed in a database for analysis. All participants in the resulting studies were English speakers and were not reported to have underlying intellectual or developmental data and were not currently concussed or suffering from any other medical condition that might affect their test performance.

The standardization sample consisted of 17,013 individuals who underwent baseline ImPACT testing. The older subjects represented teachers, coaches, school administrators, and adult athletes. Athletes who participated in the normative sample were participants in the following sports: Tackle football (males only), soccer (males and females), lacrosse (males and females), wrestling (males only), baseball (males only), softball (females only), swimming/diving (males and females), cheerleading (females only), crew/rowing (males and females), volleyball (males and females), track and field (males and females), field hockey (females only) and cross country (males and females). The specific age and gender breakdown is presented in Table 5.

Age Range	Male	Female
10-12 years 11 months	321	129
13-15 years 11 months	4,359	1,851
16-18 years 11 months	4,804	1,767
19-29 years 11 months	2,061	626
30-39 years 11 months	443	127
40-49 years 11 months	192	96
50-59 years 11 months	153	82
Totals	12,335	4,678

Table 5: Age and Gender Breakdown for ImPACT Normative Sample

1. Reliable Change Index (RCI). The ImPACT software calculates a RCI, which provides information regarding if a change in the ImPACT score from baseline to post-injury is a change that is not due to either practice effects or the result of measurement error. The RCI method for interpreting change on neurocognitive tests is a method for determining change. This method relies on the standard error of the difference score.
  1 Kontos, A.P., Elbin, R.J., Covasin, T., Larson, E. (2010), Exploring differences in neurocognitive concussion testing in African American and White athletes. Archives of Clinical Neuropsycholog, 25(8), 734-744.

" Shuttleworth-Edwards AB1, Whitefield-Alexander VJ, Radloff SE, Taylor AM, Lovell MR. Computerized neuropsychological profiles of South African versus US athletes: a basis for commentary on cross-cultural norming issues in the sports concussion arena. Phys Sportmed. 2009:37(4),45-52.

{11}------------------------------------------------

The RCI is a statistical calculation to demonstrate the change in score is not due to expected test-retest variance. The RCI calculation provides additional information to the clinician in determining if the change in test scores is clinically meaningful and not solely due to measurement error and provides important information to the clinician both diagnostically and prognostically.

1. Validity Index. The ImPACT test battery provides an index designed to aid in identifying invalid baseline examinations. The validity index is based upon sub-optimal performance on the five subtests, which comprise the ImPACT test battery and include the published research cut-off values for each subtest:
- X's and O's Total Incorrect > 30 OR
- Impulse Control Composite > 30 OR
- Word Memory Learning Pct Correct < 69% OR
- Design Memory Learning Pct Correct < 50% OR ●
- Three Letters Total Letters Correct < 8

If any of these criteria are reached for a given baseline test, the ImPACT report will automatically print a sentence that identifies the test results as being of questionable validity. If this is the case, the test administrator is encouraged to repeat the baseline exam, only after discussing the test results with the patient and identifying the reasons for the invalid test (e.g., difficulty understanding one or more of the modules, not taking the test seriously, etc.).

ImPACT Pediatric

Research data have been collected through the collaborative efforts of a number of independent organizations and institutions that volunteered to participate in various aspects of the overall research project. These organizations were recruited based on the interest in participating and their involvement with children from 5 to 12 years of age.

Specific research projects and results are presented below:

1. Development of the Normative Database. A large, age-stratified sample of children ages 5 through 12 years old that are representative of the intended use population were tested utilizing ImPACT Pediatric to generate normal score ranges for the test. An adequate representative sample of participants was included in the clinical studies to support the age range for the device. Means and standard deviations were calculated across age by gender and are reported within. The subjects were 915 children between the ages of 5 and 12 years as depicted in Table 6:

Dataset	n
Children's Hospital of Atlanta – Atlanta, Georgia	312
Right Time Pediatrics – Annapolis, Maryland	230
Northern Michigan University – Marquette, Michigan	199
Mount Lebanon School District – Pittsburgh, Pennsylvania	94
Shift Concussion Management – Guelph, Ontario, Canada	80
Total	915

	Table 6: Clinical Sites and Numbers of Subjects
--	-------------------------------------------------	--

{12}------------------------------------------------

ImPACT Pediatric tests were administered by a researcher, clinician, or educational professional trained in the use of ImPACT Pediatric. Tests were taken on an iPad 2 or above with the device flat on the table and instructions read out loud to the participant. Testing typically takes between 10 and 20 minutes. The children were instructed to respond by touching the screen with the pointing (first) finger of the hand that they write with. No "group testing" was conducted and all testing was done on a one-on-one basis.

a. Results. Examination of scores for Pediatric ImPACT involved standard calculation of means, standard deviations, and number of participants for each test response item, at each age group, by gender. There was the expected trend of older children performing better (faster with fewer mistakes) compared to younger children. Significant gender differences were uncovered between males and females through ttests on the Word Memory Immediate, Word Memory Delayed, and Picture Match Average Taps scores. For the Word Memory scores differences were observed for ages 7 through 12 years old (p<0.05). However, Picture Match Average Taps showed different performance by gender only from ages 9 to 12 years old (p<0.05).
b. Development of Factor Scores. A factor analysis was conducted on 712 participants from the ImPACT Pediatric normative database to ascertain whether relevant score clusters exist and for the purpose of improving the interpretability and utility of the test. Data were derived from a subset of the normative database that had completed all specific subtests.

This analysis yielded a four-factor solution. Factors include what can be termed an attention and sequencing factor, a word memory factor, a visual memory factor and a reaction time factor. Once agreement was achieved on the resulting factors, each standardization case was rescored on these factors using the raw score to T score conversion and these data were then used to calculate the normative database. T scores were calculated with a mean of 50 and a standard deviation of 10. These standardized scores allow direct comparison of factor scores to one another, as they are all provided utilizing the same metric.

1. Reliable Change Index (RCI). The ImPACT Pediatric also calculates a RCI in a similar fashion to ImPACT. Reliable change data provide evidence for the stability of the test measures. Any test output that exceeds the RCI is displayed in red, indicating that the difference in scores shows a change that is not due to practice effects or measurement error. The RCI is a statistical calculation to demonstrate the change in score is not due to expected test-retest variance. The RCI calculation provides additional information to the clinician in determining if the change in test scores is clinically meaningful and not solely due to measurement error and provides important information to the clinician both diagnostically and prognostically.
1. Reliability. The sponsor provided the results of a clinical study (unpublished) of 100 children between the ages of 5 and 12 years (mean = 7.8 yrs) who were participants in youth soccer and hockey leagues. The test-retest reliability of ImPACT Pediatric was measured at one week and one month. Intraclass correlation coefficients were calculated

{13}------------------------------------------------

for the one week interval and revealed highly significant (p<.001) correlations for each test. Two items showed poor stability (ICC= .46, .54), five demonstrated adequate to good stability (ICC=.61 .63, .67, .71, .72) and the remaining five showed excellent stability (ICC=.79, 81, .82, .83, .89). The ICC reliability coefficients are depicted in Table 7 below:

Test Module	ICC's
Word List Immediate Recall Number Correct	.83
Word List Delayed Recall Number Correct	.82
Word List Delayed Recognition Number Correct	.89
Design Rotation Number Correct	.67
Design Rotation Average Time	.61
Memory Touch Number Correct	.81
Memory Touch Total Sequences Correct	.79
Memory Touch Highest Sequence	.72
Stop and Go Number Correct	.63
Stop and Go Average Time	.71
Picture Match Average Taps	.46
Picture Match Average Time	.54

Table 7: Test-retest reliability coefficients (ICCs)

1. Construct Validity. To assess construct validity of ImPACT Pediatric, 83 participants ages 5 through 12 were given a modified battery of the Wide Range Assessment of Memory and Learning-2 (WRAML-2) and Pediatric ImPACT. Significant correlations were revealed for 20 of the 24 potential test comparisons. Pearson's correlation coefficients were calculated between the ImPACT Pediatric scores, and comparable counterparts in the WRAML-2. ImPACT Pediatric correlates significantly with relevant WRAML-2 subtests. Small but significant Pearson's correlation coefficients were documented for all measures, except for: 1) Word Memory Recall and both WRAML-2 Design Recall and Recognition, and 2) Design Rotation and WRAML-2 Design Recognition. Therefore, ImPACT Pediatric appears to measure important aspects of memory. Note that negative correlations represent a relationship where one variable increases (i.e., higher/better Story Memory scores) and another decreases (i.e., lower/better Picture Match Average Time). These results are depicted in Table8 and Table9 below. These data support the concurrent validity is therefore considered to be adequate between the examined cognitive domains of these two tests.

Table 8: The relationship between WRAML-2 Story Memory and ImPACT Pediatric Word List test

	Word Memory Recall	Word Memory Recognition
WRAML-2 Story Memory	.49**	.30**
WRAML-2 Story Memory Del.	.52**	.42**
WRAML-2 Design Recall	.48 **	.22
WRAML-2 Design Recognition	.23*	.16

Note: ** denotes correlation is significant at the p<. 01 level, * at the p<. 05 level.

{14}------------------------------------------------

Table 9: The relationship between WRAML-2 Story memory and ImPACT Pediatric Word List test. All correlations were significant at P<0.01 with the exception between Design Memory (WRAML) and Word Memory Recognition.

Test	Design RotationTime	Design RotationCorrect	Picture MatchAverage Taps	Picture MatchAverage Time
WRAML-2 Story Memory	-.26*	.27*	-.33**	-.34**
WRAML-2 Story Memory-Delay	-.31**	.26*	-.36**	-.43
WRAML-2 Design Recall	-.34**	.29*	-.22	-.33**
WRAML-2 Design Recognition	-.34**	.11	-.21*	-.24*

Note: ** denotes correlation is significant at the p<. 01 level, * at the p<. 05 level.

LABELING

The labeling for the ImPACT and ImPACT Pediatric meets the requirements of 21 CFR 801.109 for prescription devices. The labeling includes the following information to mitigate the risks of user discomfort and incorrect result (false positive and false negative):

1. A summary of the testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. This includes a description of the device output and clinical interpretation, information about device repeatability and reproducibility of the device output, construct validity of the device output, and a description of the normative database.
A warning that the device should only be used by healthcare professionals who are 2. trained in concussion management.
A warning that the device does not identify the presence or absence of concussion or 3. other clinical diagnoses.
1. A warning that the device is not a stand-alone diagnostic.
1. Instructions that test administrators must convey to patients regarding the administration of the test and collection of cognitive test data.

RISKS TO HEALTH

Table 0 identifies the risks to health that may be associated with use of Computerized Cognitive Assessment Aid for Concussion and the measures necessary to mitigate these risks.

Table 10: Risk Mitigation Table
Identified Risk	Mitigation Measure
User discomfort (e.g., visual or mental fatigue)	• Labeling
Incorrect result, inclusive of:• False positive – cognitive impairmentfrom concussion when in fact none ispresent• False negative – cognitive impairmentfrom concussion is not noted when infact cognitive impairment is present	• Clinical performance testing• Software verification, validation,and hazard analysis• Labeling

Table 10: Rick Mitigation Table

{15}------------------------------------------------

SPECIAL CONTROLS:

In combination with the general controls of the FD&C Act, the Computerized Cognitive Assessment Aid for Concussion is subject to the following special controls:

1. Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Software verification, validation, and hazard analysis must be performed.
1. Clinical performance data must be provided that demonstrates how the device functions as an interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. The testing must:
- a. Evaluate device output and clinical interpretation.
- b. Evaluate device test-retest reliability of the device output.
- Evaluate construct validity of the device cognitive assessments. C.
- d. Describe the construction of the normative database, which includes the following:
  - i. How the clinical work-up was completed to establish a "normal" population, including the establishment of inclusion and exclusion criteria.
  - ii. Statistical methods and model assumptions used.
1. The labeling must include:
- a. A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. The summary of testing must include the following:
  - i. Device output and clinical interpretation.
  - ii. Device test-retest reliability of the device output.
  - iii. Construct validity of the device cognitive assessments.
  - iv. A description of the normative database, which includes the following:
    - 1. How the clinical work-up was completed to establish a "normal" population, including the establishment of inclusion and exclusion criteria.
    - 1. How normal values will be reported to the user.
    - 1. Representative screen shots and reports that will be generated to provide the user results and normative data.
    - 1. Statistical methods and model assumptions used.
    - 1. Whether or not the normative database was adjusted due to differences in age and gender.
- b. A warning that the device should only be used by healthcare professionals who are trained in concussion management.
- A warning that the device does not identify the presence or absence of concussion or C. other clinical diagnoses.
- d. A warning that the device is not a stand-alone diagnostic.
- Any instructions technicians must convey to patients regarding the administration of e. the test and collection of cognitive test data.

{16}------------------------------------------------

BENEFIT/RISK DETERMINATION

The probable benefits of ImPACT and ImPACT Pediatric are based on data collected in a group of studies published in literature and/or submitted in support of this de novo request as described above.

. Psychometric properties of the ImPACT system, including adequate demonstration of testretest reliability, construct validity via comparison to traditional neuropsychological tests, calculation of the reliable change index when baseline testing is available, and a validity index to detect suboptimal test performance.
Extensive normative database consisting of individuals broken down by age and gender for determining level of cognitive function in the absence of baseline testing.
. Patient benefit value: Ability to have access to a non-invasive cognitive assessment battery that can be used to compare pre-injury (baseline cognitive performance) to postinjury cognitive performance. Ability to compare cognitive test performance to a large normative database in the absence of baseline testing.

The risks of the device are based on data collected from literature analysis of subjects and/or data submitted in support of this de novo request who were administered ImPACT or ImPACT Pediatric as described above.

A false positive result may influence a clinician's decision to diagnose a patient with a . concussion when there is none. However, this risk to the patient is considered to be low as the current guidelines regarding treatment of mild traumatic brain injury (mTBI) is rest and observation. Therefore, a false positive result would support a conservative treatment approach (i.e., not allowing a patient to return to activity in which they are at risk for a second concussive head injury). In addition, a false positive result would not expose the patient to any unnecessary intervention.
A false negative result may influence a clinician's decision that the subject has not . experienced a concussive head injury when indeed one is present. This would support a clinical decision not to treat the patient and possibly allow the individual to return to activity in which they are at risk for a second concussive head injury, introducing risk of a serious adverse event (i.e., second impact syndrome) that may have significant morbidity and mortality implications.
The risks of false positive and false negative results are mitigated by the fact that ● ImPACT and ImPACT Pediatric are not indicated for use as a stand-alone diagnostic system. The device classification should be integrated into the clinician's decision making process alongside information from other evaluative tests.
. Subjects may experience user discomfort due to visual or mental fatigue. This can be mitigated through proper test administration as described in the labeling.

{17}------------------------------------------------

Additional factors to be considered in determining probable risks and benefits for ImPACT and ImPACT Pediatric include:

The clinical performance data in support of this de novo submission were not based upon . a prospective clinical validation study and rely totally upon published literature and/or other clinical data submitted in support of this de novo request. The literature and/or data provided in support of the psychometric properties of the ImPACT and ImPACT Pediatric, including test-retest reliability and construct validity are considered to be adequate and consistent with results obtained from traditional paper and pencil psychological and neuropsychological tests.
Development of the normative database was based upon several published studies, which ● used various inclusion/exclusion criteria to determine that a subject did not have any medical conditions that might influence the result of the cognitive assessment. The normative database is considered adequate.

Patient Perspectives

This submission did not include specific information on patient perspectives for this device.

Benefit/Risk Conclusion

In conclusion, given the available information above, the data support that for use as an assessment aid and in the management of concussion, the probable benefits outweigh the probable risks for ImPACT and ImPACT Pediatric. The devices provide substantial benefits and the risks can be mitigated by the use of general and the identified special controls.

CONCLUSION

The de novo request for ImPACT and ImPACT Pediatric is granted and the devices are classified under the following:

Product Code: POM Device Type: Computerized Cognitive Assessment Aid for Concussion Class: II Regulation: 21 CFR 882.1471

Regulation Number and Section

§ 882.1471 Computerized cognitive assessment aid for concussion.

(a)
Identification. The computerized cognitive assessment aid for concussion is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an indication of the current level of cognitive function in response to concussion. The computerized cognitive assessment aid for concussion is used only as an assessment aid in the management of concussion to determine cognitive function for patients after a potential concussive event where other diagnostic tools are available and does not identify the presence or absence of concussion. It is not intended as a stand-alone diagnostic device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function, must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Software verification, validation, and hazard analysis must be performed.
(2) Clinical performance data must be provided that demonstrates how the device functions as an interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. The testing must:
(i) Evaluate device output and clinical interpretation.
(ii) Evaluate device test-retest reliability of the device output.
(iii) Evaluate construct validity of the device cognitive assessments.
(iv) Describe the construction of the normative database, which includes the following:
(A) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.
(B) Statistical methods and model assumptions used.
(3) The labeling must include:
(i) A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. The summary of testing must include the following:
(A) Device output and clinical interpretation.
(B) Device test-retest reliability of the device output.
(C) Construct validity of the device cognitive assessments.
(D) A description of the normative database, which includes the following:
(
1 ) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.(
2 ) How normal values will be reported to the user.(
3 ) Representative screen shots and reports that will be generated to provide the user results and normative data.(
4 ) Statistical methods and model assumptions used.(
5 ) Whether or not the normative database was adjusted due to differences in age and gender.(ii) A warning that the device should only be used by health care professionals who are trained in concussion management.
(iii) A warning that the device does not identify the presence or absence of concussion or other clinical diagnoses.
(iv) A warning that the device is not a stand-alone diagnostic.
(v) Any instructions technicians must convey to patients regarding the administration of the test and collection of cognitive test data.