(117 days)
ImPACT Quick Test is intended for use as a computerized cognitive test aid in the assessment and management of concussion in individuals ages 12-70.
ImPACT Quick Test (ImPACT) QT is a brief computerized neurocognitive test designed to assist trained healthcare professionals in determining a patient's status after a suspected concussion. ImPACT QT provides basic data related to neurocognitive functioning, including working memory, processing speed, reaction time, and symptom recording.
ImPACT QT is designed to be a brief 5-7 minute iPad-based test to aid sideline personnel and first responders in determining if an athlete/individual is in need of further evaluation or is able to immediately return to activity. ImPACT QT is not a substitute for a full neuropsychological evaluation or a more comprehensive computerized neurocognitive test (such as ImPACT).
Here's an analysis of the acceptance criteria and the study that proves the device meets the acceptance criteria, based on the provided text:
Acceptance Criteria and Device Performance
The document does not explicitly state a table of pre-defined acceptance criteria for the ImPACT Quick Test itself, in terms of specific performance thresholds (e.g., a minimum correlation coefficient). Instead, the studies aim to demonstrate construct validity, concurrent validity, and test-retest reliability as evidence of the device's performance, aiming to show substantial equivalence to the predicate device.
The reported device performance aligns with these goals:
| Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|
| Concurrent Validity | Correlations between ImPACT Quick Test and the predicate device (ImPACT) were in the moderate to high range (0.32-0.63, all p<.001). This suggests that although the two instruments measure similar constructs, the fact that ImPACT Quick Test contains a subset of ImPACT tests as well as unique content explains the moderate relationship between the two instruments. |
| Construct Validity | ImPACT Quick Test measures (Attention Tracker and Motor Speed) correlated more highly with established neuropsychological tests (BVMT-R, CTT) that assess similar constructs (attention and motor speed). Correlations varied from 0.28 to 0.61 (many significant at p<.001), demonstrating expected relationships with external measures. Lower correlation for Memory scale with BVMT-R was attributed to significant differences in format and task demands. |
| Test-Retest Reliability | Test-retest correlations for composite scores were: Memory (r=0.18), Attention Tracker (r=0.73), and Motor Speed (r=0.82). All significant at p<.001 or beyond. The majority of correlations were in the 0.6 to 0.8 range, reflecting "considerable stability" across the re-test period. Reliable Change Index (RCI) also indicated the percentage of cases falling outside confidence intervals. |
| Clinical Acceptability | The device provides a "reliable measure of cognitive function to aid in assessment of concussion" and is "substantially equivalent to the Predicate Device." |
| Software Validation & Risk Management | ImPACT QT software was developed, validated, and documented according to IEC 62304 and FDA Guidance. Risk Management (ISO 14971) was conducted, with all risks appropriately mitigated. |
| Normative Data | A normative database was developed from 772 subjects, representative of ages 12-70 based on 2010 U.S. Census for age, gender, and race. |
Study Details
The provided document describes several studies to support the predicate device equivalence and performance of the ImPACT Quick Test.
-
Sample sizes used for the test set and the data provenance:
- Concurrent Validity Study: 92 subjects (41 males, 51 females; average age 36.5 years, range 12-76 years).
- Construct Validity Study: 118 subjects (73 females, 45 males; average age 32.5 years, range 18-79 years).
- Test-Retest Reliability Study: 76 individuals.
- Normative Database Development: 772 subjects.
- Data Provenance: All subjects were recruited from 11 sites across the United States. All subjects completed an IRB approved consent form and met eligibility criteria. The studies appear to be prospective in nature, collecting new data for these specific analyses.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The concept of "ground truth" for the test set in this context applies differently. For the concurrent and construct validity studies, the "ground truth" or reference standards were the predicate device (ImPACT) and established traditional neuropsychological tests (BVMT-R, CTT, SDMT). The performance of these reference tests themselves is considered the standard.
- For the administration of the tests and data collection, the document states: "All testing was completed by professionals who were specifically trained to administer the test. These professionals consisted of neuropsychologists, physicians, psychology/psychology/psychology graduate students, certified athletic trainers and athletic training graduate students. All testing was completed in a supervised setting." While this describes the administrators, it doesn't specify how many "experts" established a singular ground truth for any given case, as the outputs are quantitative cognitive scores.
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- The document does not describe an adjudication method for the test set as would typically be seen in diagnostic studies where expert consensus determines a disease state. The outputs of the device and the comparison tests are quantitative scores, not subjective interpretations requiring adjudication.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study focusing on human reader improvement with AI assistance was not done or reported in this document. The device is a "computerized cognitive assessment aid" providing quantitative scores, not an AI that directly assists human interpretation in the MRMC sense. It's an aid for trained healthcare professionals, but the study doesn't quantify their improvement with the aid versus without it.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, the studies evaluating concurrent validity, construct validity, and test-retest reliability are essentially standalone performance evaluations. The ImPACT Quick Test (algorithm/device) generated cognitive scores which were then compared to other tests or re-administered. While trained professionals administered the test, their role was in test administration, not in altering the device's output or performing an "in-the-loop" interpretation that influenced the device's score generation. The "performance" being measured is the scores produced by the device itself.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The "ground truth" was established by comparison to established, validated neuropsychological tests and a predicate device (ImPACT).
- For concurrent validity, the predicate device (ImPACT) was the reference standard.
- For construct validity, traditional neuropsychological tests (Brief Visuospatial Motor Test (BVMT-R), Color Trails Test (CTT), Symbol Digit Modalities Test (SDMT)) served as the reference standards.
- For test-retest reliability, the device's own repeated measurements served as the basis for evaluation, with consistency across measurements being the goal.
- The "ground truth" was established by comparison to established, validated neuropsychological tests and a predicate device (ImPACT).
-
The sample size for the training set:
- The normative database used to establish percentiles for the ImPACT Quick Test was developed using 772 subjects. This serves as a "training set" in the sense that it provides the reference data against which individual patient's scores are compared. It is not a software machine learning training set in the typical sense, but rather a reference population.
- The document states that the new device "reports symptoms and displays test results in a form of composites score percentiles based on normative data" and that "The standardization sample was developed to be representative of the population of individuals ages 12-70 years".
-
How the ground truth for the training set was established:
- For the normative database (which serves a similar function to a training set here), the "ground truth" was simply the measured performance of a large, representative sample of healthy individuals on the ImPACT Quick Test itself. These subjects were free from concussion within one year, neurological disorders, and psychoactive medication. Their scores define the "normal" range for different age, gender, and race stratifications, against which subsequent patient scores are compared. All subjects completed IRB approved consent and met eligibility criteria. Testing was completed by trained professionals in a supervised setting to ensure standardized administration.
{0}------------------------------------------------
Public Health Service
Food and Drug Administration 10903 New Hampshire Avenue Document Control Center - WO66-G609 Silver Spring, MD 20993-0002
June 21, 2017
Impact Applications, Inc. Michael Zagorski Director of Regulatory Affairs 9665 Granite Ridge Drive. Suite 550 San Diego, California 92123
Re: K170551
Trade/Device Name: ImPACT Ouick Test Regulation Number: 21 CFR 882.1471 Regulation Name: Computerized Cognitive Assessment Aid For Concussion Regulatory Class: Class II Product Code: POM Dated: May 22, 2017 Received: May 23, 2017
Dear Mr. Zagorski:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device
{1}------------------------------------------------
related adverse events) (21 CFR 803); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820); and if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
If you desire specific advice for your device on our labeling regulation (21 CFR Part 801), please contact the Division of Industry and Consumer Education at its toll-free number (800) 638-2041 or (301) 796-7100 or at its Internet address
http://www.fda.gov/MedicalDevices/ResourcesforYou/Industry/default.htm. Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to
http://www.fda.gov/MedicalDevices/Safety/ReportaProblem/default.htm for the CDRH's Office of Surveillance and Biometrics/Division of Postmarket Surveillance.
You may obtain other general information on your responsibilities under the Act from the Division of Industry and Consumer Education at its toll-free number (800) 638-2041 or (301) 796-7100 or at its Internet address
http://www.fda.gov/MedicalDevices/ResourcesforYou/Industry/default.htm.
Sincerely,
Michaelリ. Hoffmann -S
for
Carlos L. Peña, PhD, MS Director Division of Neurological and Physical Medicine Devices Office of Device Evaluation Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K170551
Device Name ImPACT Quick Test
Indications for Use (Describe)
ImPACT Quick Test is intended for use as a computerized cognitive test aid in the assessment and management of concussion in individuals ages 12-70.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| ------------------------------------------------- | -- |
X Prescription Use (Part 21 CFR 801 Subpart D)
| Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
Image /page/3/Picture/0 description: The image is a logo for ImPACT Applications, Inc. The logo features a stylized image of a human head in profile, with a brain inside the head. The brain is colored in a light blue color. To the right of the head is the text "ImPACT" in a combination of dark blue and light blue colors. Below the word "ImPACT" is the text "APPLICATIONS, INC."
Section 5 - 510(k) Summary
Submission Date: May 22, 2017
Submitter Information:
| Company: | ImPACT Applications, Inc.9665 Granite Ridge Drive, Suite 150San Diego, CA 92123 |
|---|---|
| Official Correspondent: | Michael ZagorskiDirector of Regulatory Affairs |
Device Information:
| Trade Name: | ImPACT Quick Test™™ |
|---|---|
| Classification Name: | Computerized Cognitive Assessment Aid for Concussion |
| Device Class: | II |
| Device Classification: | 21 CFR 882.1471, Product Code POM |
| Review Panel | Neurology |
ImPACT (DEN150037)
ImPACT Applications, Inc. Tel: 412-567-8400 x.939
Email: mzagorski@impacttest.com
Predicate Device:
Indications for Use:
lmPACT Quick Test is intended for use as a computerized cognitive test to aid in the assessment and management of concussion in individuals ages 12-70
Device Description:
lmPACT Quick Test (ImPACT) QT is a brief computerized neurocognitive test designed to assist trained healthcare professionals in determining a patient's status after a suspected concussion. ImPACT QT provides basic data related to neurocognitive functioning, including working memory, processing speed, reaction time, and symptom recording.
lmPACT QT is designed to be a brief 5-7 minute iPad-based test to aid sideline personnel and first responders in determining if an athlete/individual is in need of further evaluation or is able to immediately return to activity. ImPACT QT is not a substitute for a full neuropsychological evaluation or a more comprehensive computerized neurocognitive test (such as ImPACT).
{4}------------------------------------------------
Image /page/4/Picture/2 description: The image is a logo for ImPACT Applications, Inc. The logo features a stylized image of a human head in profile, with the brain outlined in blue. To the right of the head is the word "ImPACT" in blue, with the "I" in a darker shade of blue than the rest of the word. Below "ImPACT" is the phrase "APPLICATIONS, INC." in a smaller, darker blue font.
Comparison to Predicate Device:
lmPACT QT is substantially equivalent to ImPACT, manufactured by ImPACT Applications, Inc., and cleared under DEN150037. ImPACT QT and Predicate Device ImPACT share the same intended use as Computerized Cognitive Assessment Aid for Concussion. Both devices enable administration of neuropsychological test taker's cognitive state. They are also similar in terms of technological characteristics as both administer neurocognitive test modules on a standard off-the-shelf computing device and record objective cognitive performance measurements as the test taker responds to stimuli presented on the screen.
lmPACT QT differs from Predicate Device in the content and design of the cognitive tasks, and the test reported for analysis, including different normative data set.
| Table 1 - Substantial Equivalence Comparison | |||
|---|---|---|---|
| Characteristic | New Device: ImPACT QuickTest | Predicate Device: ImPACT(DEN150037) | Reference Device: ImPACTPediatric (DEN150037) |
| Indications of Use | Similar to the Predicate andthe Reference Device.ImPACT QT is intended foruse as a computerizedcognitive test to aid in theassessment and managementof concussion in individualsages 12-70. | ImPACT is intended for use asa computer-basedneurocognitive test batteryto aid in the assessment andmanagement of concussion.ImPACT is a neurocognitivetest battery that provideshealthcare professionals withobjective measure ofneurocognitive functioning asan assessment aid and in themanagement of concussionin individuals ages 12-59. | ImPACT is intended for use asa computer-basedneurocognitive test batteryto aid in the assessment andmanagement of concussion.ImPACT is a neurocognitivetest battery that provideshealthcare professionals withobjective measure ofneurocognitive functioning asan assessment aid and in themanagement of concussionin individuals ages 5-11. |
| PatientPopulation | Similar to the PredicateImPACT QT age range is 12-70 | 12-59 | 5-11 |
| Technology | Similar to Predicate, and thesame as Reference Device. | Stand-alone software applicationdelivered on desktop/laptopcomputers | Stand-alone software applicationdelivered on iPad |
| Neurocognitivetasksadministered | Similar to Predicate andReference DevicesImPACT QT includes:Demographic data, (age,gender);Symptoms list;Neurocognitive test batteryconsisting of 3 modules:Module 1: Symbol MatchModule 2: Three lettermemoryModule 3: AttentionTrackerModules 1 and 2 are identical toModules 3 and 4 of the PredicateDevice. | 1. Demographic data, (age,gender, concussion history);2. Symptoms list;3. Neurocognitive test batteryconsisting of 6 modules:Module 1: Word Memoryand Delayed MemoryRecognitionModule 2: Design Memoryand Delayed DesignRecognitionModule 3: X's and O'sModule 4: SymbolMatchingModule 5: Color MatchModule 6: Three LetterMemory | 1. Demographic data, (age,gender, concussion history);2. Symptoms list;3. Neurocognitive test batteryconsisting of 6 modules:Module 1: Word Memoryand Delayed MemoryRecognitionModule 2: Design Memoryand Delayed DesignRecognitionModule 3: Stop and GoModule 4: Memory TouchModule 5: Picture MatchModule 6: Color Match |
{5}------------------------------------------------
| Image: IMPACT APPLICATIONS, INC. | ImPACT Quick Test - Traditional 510(k) | Section 5 - 510(k) Summary | |
|---|---|---|---|
| Reporting | Similar to Predicate andReference Devices.The new device reportssymptoms and displays testresults in a form of compositesscore percentiles based onnormative data. | • Symptom scores• Raw Scores (subscales)• Composite scores withpercentiles based onnormative data.• Reliable Change Index. | • Symptom scores• Raw Scores• Factor Scores withpercentiles based onnormative data. |
| ResultsInterpretation | Same as Predicate andReference Devices. | ImPACT does not provide arecommendation that thepatient is impaired vs.unimpaired.Clinical interpretation of theresults includes comparison withthe normative database. | ImPACT Pediatric does notprovide a recommendation thatthe patient is impaired vs.unimpaired.Clinical interpretation of theresults includes comparison withthe normative database. |
| Stimuluspresentation | Same as Predicate Device | Computer screen | Tablet (mobile device) screen |
| Test takerresponse capture | Same as Reference Device | Computer mouse or keyboard | User input (finger touch) viatablet touch-screen |
| Psychometricproperties | Same as Predicate andReference Devices | Demonstrates construct validitywith traditionalneuropsychological tests andtest-retest reliability. | Demonstrates construct validitywith traditionalneuropsychological tests andtest-retest reliability. |
| Standards used | ISO 14971IEC 62304 | ISO 14971IEC 62304 | ISO 14971IEC 62304 |
Summary of Non-Clinical Testing:
lmPACT QT software was developed, validated, and documented in accordance with IEC 62304 and FDA Guidance "General Principles of Software Validation." Software verification activities including code reviews, design reviews, evaluations, analyses, traceability assessment, and manual software testing were executed to demonstrate device performance and functionality. All activities were completed successfully and met the required acceptance criteria.
Risk Management activities, conducted in accordance on ISO 14971, assure that all risks related to use of a computerized neurocognitive test, including use related risks and cybersecurity risks, are appropriately mitigated.
Summary of Clinical Testing:
The 510(k) included the results of clinical studies that examined the construct and concurrent validity of ImPACT QT as a clinical tool by documenting correlations with ImPACT and traditional neuropsychological tests. Clinical data was also collected to examine test-retest reliability and to construct a normative database.
Normative Database:
To develop a normative data set, 772 subjects were recruited by utilizing 11 sites across the United States. All subjects completed an IRB approved consent form and met eligibility criteria before testing. The standardization sample was developed to be representative of the population of individuals ages 12-70 years, based upon the 2010 U.S. Census, approximating the targets for age, gender and race.
All subjects completed an IRB approved consent form and met eligibility criteria before testing. All testing was completed by professionals who were specifically trained to administer the test. These professionals consisted of neuropsychologists, physicians, psychology/psychology/psychology graduate students, certified athletic trainers and athletic training graduate students. All testing was completed in a supervised setting.
{6}------------------------------------------------
Image /page/6/Picture/2 description: The image is a logo for ImPACT Applications, Inc. The logo features a stylized head with a brain inside, followed by the word "ImPACT" in blue letters. Below the word "ImPACT" is the phrase "APPLICATIONS, INC." in smaller, blue letters. The logo is clean and professional, and it is likely used to represent a company that specializes in brain-related applications.
Concurrent Validity
A study was completed comparing ImPACT Quick Test to ImPACT, the test from which it was derived. This study compared 92 subjects who were given both ImPACT Quick Test. Tests were administered in counterbalanced order across subjects by trained examiners. The sample consisted of 41 (45%) males and 51 (55%) females with an average age of 36.5 years (S.D. = 19.8 years, range = 12-76 years).
The correlations between the two instruments tend to be in the moderate to high range, 0.32-0.63. This suggests that although the two instruments measure similar constructs, the fact that ImPACT Quick Test contains a subset of ImPACT tests as well as unique content explains the moderate relationship between the two instruments.
| Table 2. Concurrent Validity Between ImPACT and ImPACT Quick Test. | |
|---|---|
| Visual Motor Speed | |
| Three Letters Counting Average Correct | .63 |
| Reaction Time | |
| Three Letters Average Time First Click | .47 |
| Attention Tracker Rectangular Average Time Correct | .44 |
| Attention Tracker Figure Eight Average Time Correct | .40 |
| Attention Tracker Complex Average Time Correct | .43 |
| Symbol Match Correct Visible Average Answer Time | .61 |
| Symbol Match Correct Hidden Average Answer Time | .36 |
| Symbol Match Incorrect Hidden Average Answer Time | .32 |
| All correlations p<.001 |
The results of these studies demonstrate that ImPACT QT provides a reliable measure of cognitive function to aid in assessment of concussion, and is therefore substantially equivalent to the Predicate Device.
Construct Validity
A study was completed comparing ImPACT Quick Test to the Brief Visuospatial Motor Test (BVMT-R) (Benedict, 1997), a measure of visual-spatial memory, the Color Trails Test (CTT) (D'Elia et al., 1998), a measure of attention and sequencing, and the Symbol Digit Modalities Test (SDMT) (Smith, 1973), a measure of attention, visual scanning and motor speed).
The study compared the results of 118 subjects; the tests were administered in counterbalanced order across subjects by trained examiners. The sample consisted of 73 (62%) females and 45 (38%) males with an average age of 32.5 years (S.D. = 16.7 years, range = 18-79 years).
The results suggest that Attention Tracker and Motor Speed correlate more highly with the BVMT-R, and CTT, than does the Memory Scale. These results are consistent with what one would expect as the BVMT-R, and CTT, provide measures of attention and motor speed. Although the correlation between the Memory Scale and the BVMT-R, was lower than expected, the fact that the format and task demands are significantly different can explain the lower than expected correlations.
{7}------------------------------------------------
Image /page/7/Picture/0 description: The image is a logo for ImPACT Applications, Inc. The logo features a stylized image of a human head with a brain inside, followed by the text "ImPACT" in a blue gradient, and the text "APPLICATIONS, INC." in a smaller font below. The logo is clean and professional, and it conveys a sense of intelligence and innovation.
| Table 3 - Construct validity: correlation of ImPACT Quick Test and neuropsychological test measures | ||||||||
|---|---|---|---|---|---|---|---|---|
| Attention Tracker | r | Sig r | Motor Speed | r | Sig r | Memory | r | Sig r |
| BVMT-R Trial 1 | .29 | .07 | BVMT-R Trial 1 | .32 | .001 | BVMT-R Trial 1 | .10 | .30 |
| BVMT-R Trial 2 | .05 | .58 | BVMT-R Trial 2 | .18 | .05 | BVMT-R Trial 2 | .14 | .14 |
| BVMT-R Trial 3 | -.01 | .93 | BVMT-R Trial 3 | .17 | .06 | BVMT-R Trial 3 | .13 | .16 |
| BVMT-R Total Score | .10 | .27 | BVMT-R Total Score | .30 | .001 | BVMT-R Total Score | .13 | .16 |
| Color Trails-R Trial 1 Time | .37 | .001 | Color Trails-R Trial 1 Time | .50 | .001 | Color Trails-R Trial 1 Time | .03 | .75 |
| Color Trails-R Trial 2 Time | .28 | .003 | Color Trails-R Trial 2 Time | .61 | .001 | Color Trails-R Trial 2 Time | .16 | .09 |
| Color Trails-S Trial 1 Time | -.35 | .05 | Color Trails-S Trial 1 Time | .36 | .04 | Color Trails-S Trial 1 Time | .12 | .53 |
| SDMT Correct | .19 | .29 | SDMT Correct | .20 | .28 | SDMT Correct | .04 | .85 |
| Legend: r=Pearson's correlation coefficient; Sig r=significance r |
Test-Retest Reliability
A sample of 76 individuals were tested twice for this study. During their initial visit, informed consent was obtained from each participant after which they were given ImPACT Quick Test and asked to return to take the test 1-4 weeks in the future. None of the test subjects had a history of concussion within one year of participating in the study and none had concussion symptoms during their participation. No subject was included in the study if he or she had a history of epilepsy or other neurological disorders or were taking psychoactive medication at the time of the study. All testing was completed under the direction of a trained test administrator. Pearson's Correlations for the Composite scores presented in the table were calculated to examine test-retest reliability for the subtests across the first two test sessions. All test-retest correlations (r) were significant at the p<.001 level or beyond, with the composite score correlations 0.18 for Memory, 0.73 for Attention Tracker, and 0.82 for Motor Speed. With the large majority of the correlations in the .6 to .8 range and the RCI scores total percent change at the various levels, this reflects considerable stability across the re-test period.
| Table 4 – Test-retest reliability results for ImPACT Quick Test Composite Scores | |||||||
|---|---|---|---|---|---|---|---|
| Composite Scores | r | aReliable Change (RCI) | bRCI Total Percent Change | ||||
| 80% CI | 90% CI | 95% CI | 80% CI | 90% CI | 95% CI | ||
| Attention Tracker | .73 | △.56/14%/3% | △.72/8%/1% | △.86/5%/1% | 17% | 9% | 6% |
| Motor Speed | .83 | △4.08/14%/5% | △5.22/12%/4% | △6.24/8%/3% | 19% | 16% | 11% |
| Memory | .18 | △3/15%/12% | △4/9%/5% | △5/1%/1% | 28% | 14% | 2% |
| aReliable Change (RCI) reflects the delta (△) required for "reliable" change, and the percentage of cases declining and improving beyond that delta. | |||||||
| bReliable Change Total Percent Change reflects the total percentage of cases falling outside the confidence interval, summing both improvement and decline. |
Conclusion:
ImPACT QT falls within the generic type of device regulated under 21 CFR 882.1471, Computerized Cognitive Assessment Aid for Concussion, Product Code POM. The differences between the two devices described above do not affect either the safety or effectiveness of ImPACT QT for its intended use and do not raise new questions of safety and effectiveness; therefore, ImPACT QT is substantially equivalent to the Predicate Device ImPACT.
§ 882.1471 Computerized cognitive assessment aid for concussion.
(a)
Identification. The computerized cognitive assessment aid for concussion is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an indication of the current level of cognitive function in response to concussion. The computerized cognitive assessment aid for concussion is used only as an assessment aid in the management of concussion to determine cognitive function for patients after a potential concussive event where other diagnostic tools are available and does not identify the presence or absence of concussion. It is not intended as a stand-alone diagnostic device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function, must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Software verification, validation, and hazard analysis must be performed.
(2) Clinical performance data must be provided that demonstrates how the device functions as an interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. The testing must:
(i) Evaluate device output and clinical interpretation.
(ii) Evaluate device test-retest reliability of the device output.
(iii) Evaluate construct validity of the device cognitive assessments.
(iv) Describe the construction of the normative database, which includes the following:
(A) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.
(B) Statistical methods and model assumptions used.
(3) The labeling must include:
(i) A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. The summary of testing must include the following:
(A) Device output and clinical interpretation.
(B) Device test-retest reliability of the device output.
(C) Construct validity of the device cognitive assessments.
(D) A description of the normative database, which includes the following:
(
1 ) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.(
2 ) How normal values will be reported to the user.(
3 ) Representative screen shots and reports that will be generated to provide the user results and normative data.(
4 ) Statistical methods and model assumptions used.(
5 ) Whether or not the normative database was adjusted due to differences in age and gender.(ii) A warning that the device should only be used by health care professionals who are trained in concussion management.
(iii) A warning that the device does not identify the presence or absence of concussion or other clinical diagnoses.
(iv) A warning that the device is not a stand-alone diagnostic.
(v) Any instructions technicians must convey to patients regarding the administration of the test and collection of cognitive test data.