(217 days)
The ANAM Test System: Military Battery provides clinicians with objective measurements of cognitive performance in military populations ages 18 to 65 years, to aid in the assessment of an individual's level of cognitive functioning. The ANAM Test System should only be used as an adjunctive tool for evaluating cognitive function.
ANAM Test System: Military Battery is a software only device that provides clinicians with objective measurements of cognitive performance in military populations, to aid in the assessment of an individual's level of cognitive function. The software is downloaded from the Vista LifeSciences website and is for use on a Dell Latitude E6440 Laptop. The laptop is not provided as part of the device, but is purchased separately by the user. Each ANAM battery consists of a collection of pre-selected modules that are administered in a sequential manner.
The ANAM Test System: Military Battery is a software-only device designed to provide objective measurements of cognitive performance in military populations (ages 18 to 65 years) as an adjunctive tool for evaluating cognitive function.
Here's an analysis of the provided information regarding its acceptance criteria and the supporting study:
1. Table of Acceptance Criteria and Reported Device Performance
The provided document does not explicitly state formal acceptance criteria with numerical thresholds. Instead, the "Performance Data" section discusses the general demonstration of concurrent validity with traditional neuropsychological tests. The "Summary / Conclusion of Substantial Equivalence Rationale" further states that "ANAM provides a reliable measure of cognitive function."
To present this in a tabular format based on the given information:
| Acceptance Criterion (Implicit) | Reported Device Performance |
|---|---|
| Demonstrate Concurrent Validity with Traditional Neuropsychological Tests | The 510(k) included results of numerous studies demonstrating correlations with traditional neuropsychological tests. |
| Provide a Reliable Measure of Cognitive Function | The results of these studies "demonstrate that ANAM provides a reliable measure of cognitive function." |
| Aid in the assessment of an individual's level of cognitive functioning | The device is indicated to "aid in the assessment of an individual's level of cognitive functioning" and is considered "substantially equivalent to the predicate device." |
2. Sample Size Used for the Test Set and Data Provenance
The document states, "The 510(k) included the results of numerous studies that examined the concurrent validity of ANAM as a clinical tool by documenting correlations with traditional neuropsychological tests."
- Sample Size: The exact sample size for the test set(s) is not specified. The mention of "numerous studies" suggests multiple datasets were used, but no specific numbers are given.
- Data Provenance: The document does not specify the country of origin of the data or whether the studies were retrospective or prospective. The target population is "military populations ages 18 to 65 years," suggesting the data would likely be from military personnel.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications of Experts
The document does not provide information on the number of experts used to establish ground truth or their qualifications. The ground truth appears to be derived from "traditional neuropsychological tests," implying that the interpretation of these tests by qualified neuropsychologists or clinicians would form the reference standard, but this is not explicitly stated or detailed.
4. Adjudication Method for the Test Set
The document does not specify any adjudication method for the test set. Since the "performance data" describes correlations with traditional neuropsychological tests, it's likely that the results of these established tests served as the reference, rather than a separate adjudication process of the ANAM results.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study is not mentioned. The evaluation focused on the device's standalone performance in correlation with traditional neuropsychological tests to establish substantial equivalence, not on how human readers' performance improves with or without AI assistance.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, a standalone performance evaluation was done. The "Performance Data" section describes "the results of numerous studies that examined the concurrent validity of ANAM as a clinical tool by documenting correlations with traditional neuropsychological tests." This implies that the device's output (scores) was directly compared or correlated with the output of established cognitive tests, operating as a standalone algorithm. The device "provides clinicians with objective measurements," and "ANAM does not provide a recommendation that the patient is impaired vs. unimpaired. Clinical interpretation... includes comparison with the normative database." This further reinforces its role as a standalone measurement tool.
7. The Type of Ground Truth Used
The ground truth used appears to be the results or scores from "traditional neuropsychological tests." The studies aimed to establish "concurrent validity" by showing correlations between ANAM's measurements and these established tests, which serve as the accepted standard for assessing cognitive function.
8. The Sample Size for the Training Set
The document does not provide information on the sample size used for the training set. It focuses on the validation studies that demonstrate concurrent validity with established tests. The device utilizes a "military normative database" for interpreting results, but the size or development of this database is not detailed regarding "training."
9. How the Ground Truth for the Training Set Was Established
The document does not provide information on how the ground truth for any training set was established. The focus is on the validation of the ANAM system by correlating its output with "traditional neuropsychological tests." If a "military normative database" was used for training or calibration, the method for establishing its ground truth (e.g., through large-scale testing of healthy military populations by neuropsychologists) is not described.
§ 882.1470 Computerized cognitive assessment aid.
(a)
Identification. The computerized cognitive assessment aid is a prescription device that uses an individual's score(s) on a battery of cognitive tasks to provide an interpretation of the current level of cognitive function. The computerized cognitive assessment aid is used only as an assessment aid to determine level of cognitive functioning for which there exists other valid methods of cognitive assessment and does not identify the presence or absence of clinical diagnoses. The computerized cognitive assessment aid is not intended as a stand-alone or adjunctive diagnostic device.(b)
Classification. Class II (special controls). Except when the computerized cognitive assessment aid is intended for diagnostic assessment of specific diseases or conditions and relies on inputs from visual cues, auditory cues, and/or functional use of the hand, the device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter subject to the limitations in § 882.9. The special control(s) for this device are:(1) The technical parameters of the device's hardware and software must be fully characterized and be accompanied by appropriate non-clinical testing:
(i) Hardware specifications must be provided. Appropriate verification, validation, and hazard analysis must be performed.
(ii) Software, including any proprietary algorithm(s) used by the device to arrive at its interpretation of the patient's cognitive function, must be described in detail in the software requirements specification (SRS) and software design specification (SDS). Appropriate software verification, validation, and hazard analysis must be performed.
(2) The device must be designed and tested for electrical safety.
(3) The labeling must include:
(i) A summary of any testing conducted to demonstrate how the device functions as an interpretation of the current level of cognitive function. The summary of testing must include the following, if available: Any expected or observed adverse events and complications; any performance measurements including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) per the devices intended use; a description of the repeatability of measurements; a description of how the cut-off values for categorization of measurements were determined; and a description of the construct validity of the device.
(ii) A warning that the device does not identify the presence or absence of clinical diagnoses.
(iii) A warning that the device is not a stand-alone diagnostic.
(iv) The intended use population and the intended use environment.
(v) Any instructions technicians must convey to patients regarding the administration of the test and collection of cognitive test data.