(84 days)
For clinical use the NeuroGuide Analysis system is to be used by qualified medical and qualified clinical professionals for the post-hoc statistical evaluation of the human electroencephalogram (EEG).
The NeuroGuide Analysis System (NAS) is a software program for the post-hoc statistical analysis of the human electroencephalogram (EEG). EEG recorded on a separate device (i.e., the host system) is transferred to the NAS for display and user-review. The system requires that the user select reliable samples of artifact-free, eyes-closed or eyes open, resting digital EEG for purposes of analysis. Analysis consists of the Fast-Fourier Transformation (FFT) of the data to extract the spectral power for each of the four primary frequency bands (delta, theta, alpha, and beta), and frequency information from the EEG. The results of this analysis are then subjected to univariate, bivariate, and multivariate statistical analyses and displayed in statistical tables and topographical brain maps of absolute and relative power, power asymmetry, and coherence for 19 monopolar and 171 selected bipolar derivations of the EEG. In all over 1,200 measures are derived for comparison against a carefully constructed and statistically controlled age-regressed, normative database in which the variables have been transformed and confirmed for their Gaussian distribution. Each variable extracted by the analysis is compared to the database using parametric statistical procedures that express the differences between the patient and an appropriate age-matched reference group in the form of Z-scores. Multivariate features are compared to the normative database using Gaussian Univariate and Multivariate Distance Statistics. The Gaussian multivariate Distance statistic controls for the interrelationship of the measures of brain cortical function in the feature set, and provides an accurate estimate of their difference from normal. The multivariate measures permit an evaluation of regional indices of brain function that reflect the perfusion fields of the brain. Extracted feature sets are further analyzed to determine if the pattern of 'hits' (statistically significant feature score values identified for the patient) are consistent with patterns of 'hits' identified in prior neuroguide evaluations of clinical patients with known disorders. A step-wise discriminate analysis program classifies the patient in terms of their similarity to known neuroguide-defined patterns of abnormality, providing a probability estimate of the patient's profile with the average profile of groups of individuals constituting the normative and clinical database. The discriminant classification program is restricted by confining potential outcomes to specific patient symptoms derived from the patient history profile. Established discriminant functions were evaluated through the use of Receiver Operating Characteristic (ROC) curves for their sensitivity and specificity. The outcome of the statistical analysis is presented in report form that includes (a) patient demographic and history information, (b) selected EEG epochs, (c) statistical tables of monopolar, bipolar, and multivariate extracted feature values, and topographical brain mans. This information is to be read and interpreted within the context of the current clinical assessment of the patient by the attending physician/clinician. The decision to accept or reject the results of the neuroguide analysis, and incorporate these results into their clinical appraisal of the patient, is dependent upon the judgment of the attending physician or clinician.
The NeuroGuide Analysis System (NAS) is a software program designed for the post-hoc statistical analysis of the human electroencephalogram (EEG). It extracts spectral power, frequency information, and performs statistical analyses, comparing results against a normative database. The system is intended to be used by qualified medical and clinical professionals as an adjunct to traditional visually-appraised EEG.
Here's an analysis of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance:
| Acceptance Criteria (Stated Goal of Study) | Reported Device Performance |
|---|---|
| Non-Clinical Testing: Algorithms and statistical methods used for data analysis must be accurate. Conformity between host digital EEG system and NAS for frequency and power analysis. Reproduce sampling frequency of host system and visualize/evaluate EEG waveform accuracy between host system and NAS translation. Consistency and accuracy of NAS analysis of stored subject data with prior analyses (using same methods). | Control signals (generated waveforms) were analyzed for frequency and power, confirming accuracy. EEG signals were analyzed for conformity between the host digital EEG system and the NAS. The NAS includes a feature to reproduce the host system's sampling frequency and visualize/evaluate EEG waveform accuracy. Data from previous NeuroGuide analysis implementations were evaluated for consistency and accuracy, and the NAS's analysis of stored subject data conformed to these prior analyses, indicating reproducibility. |
| Clinical Testing: Accurate translation and presentation of EEGs from clinical patients. Agreement between NAS analysis results (statistical tables and topographical brain maps) and results from the host system used at the Applied Neuroscience Laboratory (ANL). Discriminant analysis outcome on NAS must be consistent with the host system, without errors of misclassification. Reproducibility of results within acceptable variation, consistent with reliability estimates from normative studies, when analyzing eyes-closed resting, artifact-free EEG. | Non-clinical testing confirmed the ability of the NAS to accurately translate and present EEGs. The results of the analysis (statistical tables and topographical brain maps) were in agreement with the results of the analysis conducted on the host system used for processing patient information at ANL. The outcome of the discriminant analysis was consistent, not resulting in errors of misclassification, aligning with the host system's performance. These tests confirmed that when eyes-closed resting, and artifact-free EEG was selected for analysis, the results were reproducible within an acceptable degree of variation consistent with reliability estimates identified in the normative studies. |
| Overall Conclusion (Safety & Effectiveness): Safe and effective for the quantitative analysis of eyes-closed resting EEG in alert human subjects, and to help determine if EEG is normal or abnormal. If abnormal, to statistically characterize the distribution of selected derived features by their probability of being similarly distributed in specified groups of clinical patients. Provides information that complements and supplements traditional EEG analysis, when used as a safe and effective adjunctive aid to diagnosis, treatment planning, and follow-up. | The non-clinical and clinical testing conducted over 25 years demonstrates that the NAS is both safe and effective for the quantitative analysis of eyes-closed resting EEG in alert human subjects. It can help determine if the EEG is normal or abnormal, and if abnormal, statistically characterize feature distribution. This information complements and supplements traditional EEG analysis, and when used properly with other clinical tests, serves as a safe and effective adjunctive aid for diagnosis, treatment planning, and follow-up of neurologic and psychiatric patients. |
2. Sample Size Used for the Test Set and Data Provenance:
- Test Set Sample Size: The document states that subjects ranged in age from "2 months to 82 years". It does not provide a specific number for the test set sample size. It refers to "stored subject data" and "clinical patients".
- Data Provenance: The data used for both the normative and clinical databases were developed over a "25-year effort" at the Applied Neuroscience Laboratory (ANL) at the University of Maryland. These included "numerous government and privately funded normative and clinical database projects." Subjects were either volunteers or clinical patients referred to ANL by the Department of Psychiatry University of Maryland School of Medicine, and/or Shock Trauma and the Applied Neuroscience Institute at the University of Maryland Eastern Shore. This indicates a retrospective and prospective collection, likely primarily from the United States (University of Maryland).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:
- The document does not explicitly state the number of experts used to establish the ground truth for the test set.
- It mentions that "the results of the analysis were conveyed to the referring physician or Ph.D. clinician who was asked to use the information as an adjunct to their clinical interpretation of the patient's traditional EEG." This implies that the referring physicians/Ph.D. clinicians were involved in evaluating the NAS's output in comparison to their traditional EEG assessment. The qualifications of these individuals are stated as "referring physician or Ph.D. clinician."
4. Adjudication Method for the Test Set:
- The document describes the evaluation as a comparison where the results of the NAS analysis (statistical tables and topographical brain maps) "had to be in agreement with the results of the analysis conducted on the host system used in the processing of patient information at the Applied Neuroscience Laboratory." Additionally, the discriminant analysis outcome "had to be consistent, not resulting in errors of misclassification" with the host system.
- This suggests a comparison against a previously established "host system" output and clinical interpretation by referring physicians/Ph.D. clinicians. It does not explicitly mention a formal expert adjudication method like 2+1 or 3+1. The "consistency" with the host system and the clinical interpretation by referring physicians/Ph.D. clinicians served as the implicit "adjudication."
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done:
- No, an MRMC comparative effectiveness study was not explicitly mentioned. The study focuses on verifying the NAS's consistency with a prior "host system" and its adjunct utility for clinicians, rather than directly measuring human reader improvement with AI assistance. The device is explicitly stated to be an "adjunct" and not a standalone diagnostic system.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- No, a standalone performance study was not performed or intended. The device is explicitly developed and tested as a tool for "post-hoc statistical evaluation" to be used by "qualified medical and qualified clinical professionals" as an "adjunct to the traditional visually-appraised EEG." The application contraindicates using the NeuroGuide Analysis System as a stand-alone diagnostic system.
7. The Type of Ground Truth Used:
- The ground truth for the device's validation appears to be a combination of:
- Expert Consensus/Clinical Agreement: The "referring physician or PhD. clinician" determined the relevance of the NAS information to their "clinical evaluation and diagnosis or treatment." The NAS results had to be consistent with the "host system" analysis, which itself would have been based on established methods and clinical understanding.
- Normative and Clinical Database: The extensive 25-year effort at ANL built a "viable normative and clinical database." This database, based on a combination of healthy volunteers and patients with known disorders, served as the statistical ground truth against which individual patient data is compared to identify deviations.
- Internal Consistency: Non-clinical testing verified algorithmic accuracy and consistency with prior analyses conducted using the same methods on the "host system."
8. The Sample Size for the Training Set:
- The document states that the NAS's design was based on a "25-year effort to construct a viable normative and clinical database" at the Applied Neuroscience Laboratory. This database serves as the foundation for the algorithms and statistical comparisons within NAS. While specific numbers are not given for the training set per se, this "normative and clinical database" effectively acts as the large-scale training/reference data. It involved subjects ranging from "2 months to 82 years" and included "numerous government and privately funded normative and clinical database projects."
9. How the Ground Truth for the Training Set Was Established:
- The ground truth for the "normative and clinical database" (which serves as the basis for the device's analysis and "training") was established through:
- Extensive Data Collection: A 25-year effort involving data from volunteers and clinical patients.
- Clinical Diagnosis/Categorization: Clinical patients were "referred for neuroguide evaluation to the Applied Neuroscience Laboratory by the Department of Psychiatry University of Maryland School of Medicine, and/or Shock Trauma and the Applied Neuroscience Institute." This implies that the clinical categories of these patients (e.g., specific disorders) were established through traditional medical diagnostic processes by these departments.
- Careful Construction and Statistical Control: The database was "carefully constructed and statistically controlled" with age-regressed variables confirmed for Gaussian distribution. This indicates a robust statistical methodology in building the reference data for what constitutes "normal" and "abnormal" patterns based on known clinical groups.
- Prior NeuroGuide Evaluations: The "patterns of 'hits' identified in prior neuroguide evaluations of clinical patients with known disorders" were used to develop discriminant analysis capabilities, effectively using previously established clinical-EEG correlations as a form of ground truth for classification.
§ 882.1400 Electroencephalograph.
(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).