Search Results

ImPACT is intended for use as a computer-based neurocognitive test battery to aid in the assessment and management of concussion.

ImPACT is a neurocognitive test battery that provides healthcare professionals with objective measures of neurocognitive functioning as an assessment aid and in the management of concussion in individuals ages 12-80.

ImPACT® (Immediate Post-Concussion Assessment and Cognitive Testing) is a computer-based neurocognitive test battery that allows healthcare professionals to conduct a series of tests on individuals to gather data related to the neurocognitive functioning of the test subject. This test battery measures various aspects of neurocognitive functioning including reaction time, memory, attention, spatial processing speed, and records symptoms of a test subject. ImPACT Version 4 is similar to the paper-and-pencil neuropsychological tests that have long been used by psychologists to evaluate cognition, and memory related to a wide variety of disabilities.

The device is not intended to provide a direct diagnosis or a return-to-activity recommendation, it does not directly manage or provide any treatment recommendations, and any interpretation of the results should be made only by qualified healthcare professional. The neurocognitive assessment represents only one aspect of assisting healthcare professionals in evaluating and managing individuals with cognitive function impairment related to TBI (concussion).

Here's a breakdown of the acceptance criteria and the study information for the ImPACT Version 4 device, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The FDA submission primarily focuses on demonstrating substantial equivalence to a predicate device (ImPACT Version 3.3.0) rather than listing specific, quantitative acceptance criteria for each functional aspect. The document indicates that all software verification and validation tests met the required acceptance criteria, but these criteria are described generally rather than with specific metrics.

However, based on the summary of performance testing and clinical data, the implicit acceptance criteria relate to:

• Software Functionality: The software performs as intended, modifications did not affect existing functionality.
• Safety and Effectiveness: The device modifications do not raise new questions of safety and effectiveness.
• Test-Retest Reliability (for extended age range): Cognitive performance should remain stable over time.
• Construct Validity (for extended age range): ImPACT Version 4 scores should correlate with established neuropsychological tests.
• Normative Database Quality: The normative data should be accurately established for the specified age ranges, differentiating between input device types.

2. Sample Sizes Used for the Test Set and Data Provenance

The document describes several "clinical studies" or investigations which serve as test sets for different aspects (normative data, reliability, validity).

• For the 12-59 age range (normative database update):
  • Sample Size: 71,815 subjects.
  • Data Provenance: Retrospective, de-identified data from the Company's test database of 766,093 subjects. Subjects were from the United States of America.
• For the 60-80 age range (normative database, test-retest reliability, construct validity):
  • Normative Sample Size: 554 subjects (174 males, 380 females).
  • Test-retest Reliability Sample Size: 93 individuals (a subset of the normative extension sample, 64.5% females, 35.5% males).
  • Construct Validity Sample Size: 71 individuals (ages 60-80, 63.4% females, 36.6% males).
  • Data Provenance: Prospective clinical investigation. Data collected from 8 different sites across the United States (universities, hospitals, clinics, private medical practices). Data collection began in 2017 and ended in 2020.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• The document does not explicitly state the number of experts used to establish a "ground truth" for the test set in the sense of adjudicating concussion status for individual subjects within the studies.
• For the construct validity study, the document states: "All measures were administered by Neuropsychologists trained in test administration as part of the study to collect data for normative dataset described above." These Neuropsychologists administered the comparative tests (HVLT, BVMT-R, SDMT) which served as the "ground truth" or reference standard for comparison, but they were not adjudicating the ImPACT scores or concussion status. They were experts in administering and interpreting the comparative neuropsychological tests.

4. Adjudication Method for the Test Set

• The document does not describe an "adjudication method" in the typical sense (e.g., 2+1 rater adjudication) for determining ground truth of concussion or for evaluating the ImPACT scores.
• Instead, the studies used established clinical methods:
  • For the normative database, subjects were selected based on self-reported criteria (no recent concussion, no neurological issues, no ADHD/LD diagnosis) and, for the 60-80 age range, MMSE scores (>24).
  • For construct validity, the reference standards were scores from other validated neuropsychological tests administered by trained neuropsychologists.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

• No, the document does not describe a multi-reader multi-case (MRMC) comparative effectiveness study evaluating how human readers (healthcare professionals) improve with ImPACT Version 4 assistance versus without it.
• ImPACT is presented as an "aid" in assessment and management, providing objective measures, but the study design did not assess the human reader's diagnostic performance with and without the device.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

• Yes, the performance testing described for ImPACT Version 4 is in a standalone capacity. The device itself (the neurocognitive test battery) collects and processes data, and its outputs (scores, validity indicators) are what were evaluated for reliability and validity against other tests or over time. The "performance" being assessed is that of the computerized test generating consistent and correlated results, not a human using the device to make a decision.

7. The Type of Ground Truth Used

• For the normative database: The "ground truth" for subject inclusion was based on self-reported health status (no recent concussion, no neurological issues, no ADHD/LD) and, for the older age group, a basic cognitive screen (MMSE score > 24). The purpose was to establish norms for a healthy population.
• For test-retest reliability: The "ground truth" was the expectation that cognitive performance in a stable individual should not significantly change over a short period.
• For construct validity: The "ground truth" was established by scores from other widely utilized and previously validated traditional neuropsychological tests (HVLT, BVMT-R, SDMT).

8. The Sample Size for the Training Set

• The document does not explicitly mention a "training set" in the context of machine learning model development. ImPACT is described as a "computer-based neurocognitive test battery" and its modifications primarily involve updating the normative database and output scores, not training a new predictive algorithm.
• The "normative database" could be considered analogous to a reference dataset that the device's scoring relies upon.
  • For ages 12-59: The normative database was constructed from 71,815 subjects.
  • For ages 60-80: The normative database was constructed from 554 subjects.

9. How the Ground Truth for the Training Set Was Established

• As mentioned above, there isn't a "training set" for a machine learning model described.
• For the normative databases, which serve as the reference for interpreting individual test results, the "ground truth" for inclusion was:
  • For 12-59 age range: De-identified data selected from a large company database. Subjects had to meet criteria such as no concussion in the past 6 months, no other neurological issues, no ADHD/LD diagnosis. These were self-reported or existing medical history.
  • For 60-80 age range: Prospective data collection where subjects met strict inclusion criteria: ages 60-80, primary English speaking, not in a skilled nursing facility, not suffering from/being treated for a concussion, no known impairing physical/neurological/behavioral/psychological conditions, corrected hearing/vision within normal limits, and a Mini-Mental State Examination (MMSE) score of 24 or greater. This was established through screening and clinical assessment at 8 different sites.

Ask a specific question about this device