(303 days)
The ANAM® Test System: Military, Military Expanded, Core, and Sports Batteries are intended for use as computerbased neurocognitive test batteries to aid in the assessment of mild traumatic brain injury. The ANAM Test System: Military, Military Expanded, Core, and Sports Batteries are neurocognitive test batteries that provide healthcare professionals with objective measure of neurocognitive functioning as assessment aids and in the management of mild traumatic brain injury in individuals ages 13-65.
The ANAM Test System should only be used as an adjunctive tool for evaluating cognitive function.
The ANAM Test System is a software only device that provides clinicians with objective measurements of cognitive performance in populations, to aid in the assessment and management of concussion. ANAM measures various aspects of neurocognitive functioning including reaction time, memory, attention, and spatial processing speed. It also records symptoms of concussion in the test taker.
The software is downloaded from the Vista LifeSciences website and is for use on Dell Inspiron 15 3000 Series or similar Windows PC model or Android Samsung Galaxy tablet or similar Android device. The hardware is not provided as part of the device but is purchased separately by the user. Each ANAM battery consists of a collection of pre-selected modules that are administered in a sequential manner.
Specific modules included in the ANAM Test System:
-
- Questionnaires
-
- Demographics
-
- Mood Scale
-
- Neurobehavioral Symptom Inventory (NSI)
-
- PTSD Checklist (PCL)
-
- Sleepiness Scale
-
- Symptoms Checklist
-
- TBI Questionnaire
Performance Tests
-
- Code Substitution Learning
-
- Code Substitution Delayed
-
- Go/No-Go"
-
- Matching to Sample*
-
- Mathematical Processing
-
- Memory Search
-
- Procedural Reaction Time
-
- Simple Reaction Time*
-
- Spatial Processing*
- *Available for tablet platform.
The tests and questionnaires can be combined into custom batteries or users can choose from pre-configured standardized batteries. The standardized batteries include ANAM Core, ANAM Sports. ANAM Military, and ANAM Military-Expanded. These standardized batteries have fixed test settings and parameters to ensure standardized presentation and enable comparison to normative data.
Here's a breakdown of the requested information regarding the acceptance criteria and the study that proves the device meets them, based on the provided FDA 510(k) summary for the ANAM Test System.
Please note: The provided document is a 510(k) summary, which focuses on demonstrating substantial equivalence to a predicate device rather than detailing specific acceptance criteria and the full study methodology in the way a clinical study report would. Therefore, some information, particularly regarding specific acceptance criteria values, sample sizes for test and training sets, number and qualification of experts, adjudication methods, and MRMC study specifics, is not explicitly stated in this document. The document primarily highlights the "numerous studies" that have examined concurrent validity.
Acceptance Criteria and Reported Device Performance
Given the nature of the 510(k) submission, the "acceptance criteria" are implicitly tied to demonstrating substantially equivalent performance to the predicate device, ImPACT, in providing an objective measure of neurocognitive functioning to aid in the assessment and management of mild traumatic brain injury (mTBI).
| Acceptance Criteria (Inferred from Substantial Equivalence) | Reported Device Performance (Summary from Document) |
|---|---|
| Intended Use Equivalence: Aid in assessment and management of mTBI by providing objective measures of neurocognitive functioning. | The ANAM Test System's intended use is identical to the predicate device: "intended for use as computer-based neurocognitive test batteries to aid in the assessment and management of mild traumatic brain injury." (p. 5) The ANAM Test System measures various aspects of neurocognitive functioning, including reaction time, memory, attention, and spatial processing speed, similar to the predicate's measurement of verbal and visual memory, visual motor speed, impulse control, and reaction time. |
| Safety and Effectiveness Equivalence: Reliable measure of cognitive function comparable to the predicate device. | The 510(k) submission states, "The 510(k) submission includes the results of numerous studies that have examined the concurrent validity of ANAM as a clinical tool by documenting correlations with traditional neuropsychological tests with both normal and concussed populations. The results of these studies demonstrate that ANAM provides a reliable measure of cognitive function for use as an assessment aid and in the management of concussion and is therefore substantially equivalent to the predicate device." (p. 5) |
| Patient Population Equivalence: Individuals aged 13-65. | The ANAM Test System is indicated for individuals aged 13-65, which is largely similar to the predicate device's age range of 12-59 years. |
| Fundamental Neurocognitive Functions Measured: Assess core cognitive domains relevant to mTBI. | ANAM measures "response speed, attention/concentration, immediate and delayed memory, spatial processing, inhibition, and decision processing speed and efficiency." This aligns with the types of functions measured by the predicate device (verbal and visual memory, visual motor speed, impulse control, and reaction time). (p. 5) |
| Reporting and Interpretation Features: Provide meaningful data for clinical interpretation, including comparison to normative data and reliable change indices. | ANAM provides raw scores, standard scores (from a normative database), and Reliable Change Indices (RCI) for individual tests and an ANAM Composite Score (ACS). This is comparable to the predicate device's provision of composite scores, percentile scores, and RCIs. (p. 5) |
Study Proving Device Meets Acceptance Criteria
The document states that the substantial equivalence determination is based on "numerous studies" that have examined the concurrent validity of ANAM.
-
Sample sizes used for the test set and the data provenance:
- Sample Size: Not explicitly stated for specific test sets within this summary. It mentions "numerous studies" with "normal and concussed populations."
- Data Provenance: Not specified (e.g., country of origin). The document implies the studies are part of the broader clinical evidence for the device. The studies examined "concurrent validity" by correlating ANAM with "traditional neuropsychological tests." This suggests retrospective or prospective clinical data involving real patient populations.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Not specified within the 510(k) summary. The ground truth for "concurrent validity" studies would typically involve the results from the "traditional neuropsychological tests" administered by qualified neuropsychologists or clinicians.
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- Not specified. The studies focused on "concurrent validity" which suggests a comparison to established, validated neuropsychological tests rather than an adjudication process between human readers for diagnostic consensus.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- Not applicable/Not mentioned. The ANAM Test System is described as a "computer-based neurocognitive test battery" and a "software only device." It functions as an adjunctive tool to aid healthcare professionals in assessment and management. This type of device typically provides quantitative measures of cognitive function directly, rather than assisting human 'readers' in interpreting medical images or complex data in an MRMC study setup. The studies focus on the validity of the test itself in measuring cognitive function.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, implicitly. The "performance data" section (p. 5) discusses the results of studies examining the "concurrent validity of ANAM as a clinical tool by documenting correlations with traditional neuropsychological tests." This refers to the standalone performance of the ANAM test in measuring cognitive function and its correlation with established measures, without direct human-in-the-loop assistance for interpreting the ANAM results, though a human clinician ultimately uses the results in their assessment.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc):
- The ground truth relies on the scores and results derived from "traditional neuropsychological tests" administered to "normal and concussed populations." This implies that expert diagnosis of concussion and the established measurements from these traditional gold-standard tests served as the reference for determining concurrent validity.
-
The sample size for the training set:
- Not specified in the 510(k) summary. Given that it's a "computer-based neurocognitive test battery" and not a typical AI/ML algorithm that requires a discrete training dataset in the same way, any "training" would likely refer to the iterative development and validation of the test components and norming data based on large population studies, rather than a separate "training set" for an explicit AI model. The document mentions "normative data" and a "normative database."
-
How the ground truth for the training set was established:
- For the establishment of "normative data" (which would serve a similar function to a training set for a traditional AI model), the ground truth would be established through the collection of ANAM test performance data from large, healthy populations across different demographics, age groups, and potentially educational levels. This process involves statistical analysis to define "normal" ranges and establish a reliable baseline against which individual performance can be compared. The summary mentions "standard scores (calculated with the normative database)" and "the summed T-score of a normative control group."
§ 882.1471 Computerized cognitive assessment aid for concussion.
(a)
Identification. The computerized cognitive assessment aid for concussion is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an indication of the current level of cognitive function in response to concussion. The computerized cognitive assessment aid for concussion is used only as an assessment aid in the management of concussion to determine cognitive function for patients after a potential concussive event where other diagnostic tools are available and does not identify the presence or absence of concussion. It is not intended as a stand-alone diagnostic device.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function, must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Software verification, validation, and hazard analysis must be performed.
(2) Clinical performance data must be provided that demonstrates how the device functions as an interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. The testing must:
(i) Evaluate device output and clinical interpretation.
(ii) Evaluate device test-retest reliability of the device output.
(iii) Evaluate construct validity of the device cognitive assessments.
(iv) Describe the construction of the normative database, which includes the following:
(A) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.
(B) Statistical methods and model assumptions used.
(3) The labeling must include:
(i) A summary of any clinical testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function in a patient that has recently received an injury that causes concern about a possible concussion. The summary of testing must include the following:
(A) Device output and clinical interpretation.
(B) Device test-retest reliability of the device output.
(C) Construct validity of the device cognitive assessments.
(D) A description of the normative database, which includes the following:
(
1 ) How the clinical workup was completed to establish a “normal” population, including the establishment of inclusion and exclusion criteria.(
2 ) How normal values will be reported to the user.(
3 ) Representative screen shots and reports that will be generated to provide the user results and normative data.(
4 ) Statistical methods and model assumptions used.(
5 ) Whether or not the normative database was adjusted due to differences in age and gender.(ii) A warning that the device should only be used by health care professionals who are trained in concussion management.
(iii) A warning that the device does not identify the presence or absence of concussion or other clinical diagnoses.
(iv) A warning that the device is not a stand-alone diagnostic.
(v) Any instructions technicians must convey to patients regarding the administration of the test and collection of cognitive test data.