(90 days)
Automatic scoring of sleep EEG data to identify stages of sleep according the American Academy of Sleep Medicine definitions, rules and guidelines. It is to be used with adult populations.
The Neurosom EEG Assessment Technology (NEAT) is a medical device software application that allows users to perform sleep staging post-EEG acquisition. NEAT allows users to review sleep stages on scored MFF files and perform sleep scoring on unscored MFF files.
NEAT software is designed in a client-server model and comprises a User Interface (UI) that runs on a Chrome web browser in the client computer and a Command Line Interface (CLI) software that runs on a Forward-Looking Operations Workflow (FLOW) server.
The user interacts with the NEAT UI through the FLOW front-end application to initiate the NEAT workflow on unscored MFF files and visualize sleep-scoring results. Sleep stages are scored by the containerized neat-cli software on the FLOW server using the EEG data. The sleep stages are then added to the input MFF file as an event track file in XML format. Once the new event track file is created, the NEAT UI component retrieves the sleep events from the FLOW server and displays a hypnogram (visual representation of sleep stages over time) on the screen, along with sleep statistics and other subject details. Additionally, a summary of the sleep scoring is automatically generated and added to the same participant in the FLOW server in PDF format.
The FDA 510(k) Clearance Letter for NEAT 001 provides information about the device's acceptance criteria and the study conducted to prove its performance.
Acceptance Criteria and Device Performance
The core acceptance criteria for NEAT 001, as demonstrated by the comparative clinical study, are based on its ability to classify sleep stages (Wake, N1, N2, N3, REM) with performance comparable to the predicate device, EnsoSleep, and within the variability observed among expert human raters.
Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined numerical "acceptance criteria" for each metric (Sensitivity, Specificity, Overall Agreement) that NEAT 001 had to meet. Instead, the approach was a comparative effectiveness study against a predicate device (EnsoSleep), with the overarching criterion being "substantial equivalence" as interpreted by performance falling within the range of differences expected among expert human raters.
Therefore, the "acceptance criteria" are implied by the findings of substantial equivalence. The "reported device performance" is given in terms of the comparison between NEAT and EnsoSleep, and their differences relative to human agreement variability.
| Metric / Sleep Stage | NEAT Performance (vs. Predicate EnsoSleep) | Acceptance Criteria (Implied) |
|---|---|---|
| Wake (Wa) | Equivalent performance (1-2% difference) | Difference within range of human agreement variability |
| REM (R) | EnsoSleep performed better (3-4% difference) | Difference within range of human agreement variability (stated as 3% for CSF dataset) |
| N1 (Overall Performance) | EnsoSleep better (4-7%) | Difference within range of human agreement variability (only in BEL data set was this difference bigger than human agreement) |
| N1 (Sensitivity) | NEAT substantially better (8-20%) | Not a primary equivalence metric, but noted as an area where NEAT excels. |
| N1 (Specificity) | EnsoSleep better (5-9%) | Not a primary equivalence metric, but noted. |
| N2 (Overall Performance) | EnsoSleep marginally better (5%) for BEL data set | Difference within range of human agreement variability |
| N2 (Sensitivity) | EnsoSleep more sensitive (22%) | Not a primary equivalence metric, but noted. |
| N2 (Specificity) | EnsoSleep less specific (9-11%) | Not a primary equivalence metric, but noted. |
| N3 (Overall Performance) | Equivalent (1% difference overall) | Difference within range of human agreement variability |
| N3 (Sensitivity) | NEAT substantially better (15-39%) | Not a primary equivalence metric, but noted as an area where NEAT excels. |
| N3 (Specificity) | EnsoSleep marginally better (3-4%) | Not a primary equivalence metric, but noted. |
| General Conclusion | Statistically significant differences, but practically within the range of differences expected among expert human raters. | Substantial equivalence to predicate device. |
Study Details
Here's a breakdown of the study details based on the provided text:
1. Sample Size and Data Provenance
- Test Set Sample Size: The exact number of participants or EEG recordings in the test set is not explicitly stated. The document refers to "two data sets" (referred to as "BEL data set" and "CSF data set") used for testing both NEAT and EnsoSleep. The large resampling number (R=2000 resamples for bootstrapping) suggests a dataset size sufficient to yield small confidence intervals.
- Data Provenance:
- Country of Origin: Not explicitly stated.
- Retrospective or Prospective: Not explicitly stated, but the mention of "All data files were scored by EnsoSleep" and "All data files were scored by NEAT" implies these were pre-existing datasets, making them retrospective.
2. Number of Experts and Qualifications for Ground Truth
- Number of Experts: Not explicitly stated. The study refers to "established gold standard" and "human agreement variability" among "expert human raters," implying multiple experts.
- Qualifications of Experts: Not explicitly stated beyond "expert human raters." No details are provided regarding their specific medical background (e.g., neurologists, sleep specialists), years of experience, or board certifications.
3. Adjudication Method for the Test Set
- Adjudication Method: Not explicitly stated. The document simply refers to "the established gold standard." It does not mention whether this gold standard was derived from a single expert, consensus among multiple experts, or a specific adjudication process (like 2+1 or 3+1).
4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- Was an MRMC study done? A direct MRMC comparative effectiveness study involving human readers assisting with AI vs. without AI assistance was not explicitly described. The study primarily focuses on comparing the standalone performance of NEAT (the AI) against the standalone performance of the predicate device (EnsoSleep), and then interpreting these differences in the context of human-to-human agreement variability.
- Effect Size of Human Reader Improvement: Since a direct MRMC study with human readers assisting AI was not detailed, there is no information provided on the effect size of how much human readers improve with AI vs. without AI assistance.
5. Standalone Performance (Algorithm Only)
- Was a standalone study done? Yes. The study evaluated the "segment-by-segment" performance of NEAT and EnsoSleep algorithms directly against the "established gold standard." This is a measure of the algorithm's standalone performance without human input during the scoring process.
6. Type of Ground Truth Used
- Type of Ground Truth: The ground truth for the test set was based on an "established gold standard" for sleep stage classification. This strongly implies expert consensus or expert scoring of the EEG data according to American Academy of Sleep Medicine definitions, rules, and guidelines. Pathology or outcomes data were not used for sleep staging ground truth.
7. Training Set Sample Size
- Training Set Sample Size: The sample size for the training set is not explicitly stated in the provided document.
8. How Ground Truth for Training Set Was Established
- How Ground Truth for Training Set Was Established: The document states that
neat-cli"leverages Python libraries for identifying stages of sleep on MFF files using Machine Learning (ML)." However, it does not explicitly describe how the ground truth for the training set was established. Typically, for ML models, the training data's ground truth would also be established by expert annotation or consensus, similar to the test set ground truth, but this is not confirmed in the provided text.
FDA 510(k) Clearance Letter - NEAT 001
Page 1
U.S. Food & Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
www.fda.gov
Doc ID # 04017.07.05
April 10, 2025
Brain Electrophysiology Laboratory Company, LLC
Roman Shusterman
Chief Technology Officer
440 E Broadway, Suite 200
Eugene, Oregon 97401
Re: K250058
Trade/Device Name: NEAT 001
Regulation Number: 21 CFR 882.1400
Regulation Name: Electroencephalograph
Regulatory Class: Class II
Product Code: OLZ
Dated: January 10, 2025
Received: January 10, 2025
Dear Roman Shusterman:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device"
Page 2
(https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medical-device-reporting-mdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Page 3
Sincerely,
Jay R. Gupta -S
Jay Gupta
Assistant Director
DHT5A: Division of Neurosurgical,
Neurointerventional, and
Neurodiagnostic Devices
OHT5: Office of Neurological and
Physical Medicine Devices
Office of Product Evaluation and Quality
Center for Devices and Radiological Health
Enclosure
Page 4
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Food and Drug Administration
Indications for Use
Form Approved: OMB No. 0910-0120
Expiration Date: 07/31/2026
See PRA Statement below.
Submission Number (if known)
Device Name
NEAT 001
Indications for Use (Describe)
Automatic scoring of sleep EEG data to identify stages of sleep according the American Academy of Sleep Medicine definitions, rules and guidelines. It is to be used with adult populations.
Type of Use (Select one or both, as applicable)
☒ Prescription Use (Part 21 CFR 801 Subpart D) ☐ Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services
Food and Drug Administration
Office of Chief Information Officer
Paperwork Reduction Act (PRA) Staff
PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
Page 5
510(K) Summary
1. SUBMITTER
| Field | Information |
|---|---|
| Submitter Name: | Brain Electrophysiology Laboratory Company, LLC |
| Address: | 440 E Broadway, Eugene, OR 97401 |
| Phone Number: | 541-653-9797 |
| Contact Person: | Dr. Phan Luu |
| Date Prepared: | April 07 2025 |
2. DEVICE
| Field | Information |
|---|---|
| Device Trade Name: | NEAT 001 |
| Common Name: | Automatic Event Detection Software For Polysomnograph With Electroencephalograph |
| Classification Name, Number & Product Code: | Electroencephalograph, 21 CFR 882.1400, OLZ |
| Class: | II |
| Classification Panel: | Neurology |
3. PREDICATE DEVICES
| Field | Information |
|---|---|
| Primary Predicate Device: | K162627 |
| Intended use: | EnsoSleep is intended for use for the diagnostic evaluation by a physician to assess sleep quality and as an aid for the diagnosis of sleep and respiratory related sleep disorders in adults only. EnsoSleep is a software-only medical device to be used under the supervision of a clinician to analyze physiological signals and automatically score sleep study results, including the staging of sleep, detection of arousals, leg movements, and sleep disordered breathing events including obstructive apneas. All automatically scored events are subject to verification by a qualified clinician. Central apneas, mixed apneas, and hypopneas must be manually marked within records. |
The primary predicate device has not been subject to a design-related recall.
Page 6
4. DEVICE DESCRIPTION
The Neurosom EEG Assessment Technology (NEAT) is a medical device software application that allows users to perform sleep staging post-EEG acquisition. NEAT allows users to review sleep stages on scored MFF files and perform sleep scoring on unscored MFF files.
NEAT software is designed in a client-server model and comprises a User Interface (UI) that runs on a Chrome web browser in the client computer and a Command Line Interface (CLI) software that runs on a Forward-Looking Operations Workflow (FLOW) server.
The user interacts with the NEAT UI through the FLOW front-end application to initiate the NEAT workflow on unscored MFF files and visualize sleep-scoring results. Sleep stages are scored by the containerized neat-cli software on the FLOW server using the EEG data. The sleep stages are then added to the input MFF file as an event track file in XML format. Once the new event track file is created, the NEAT UI component retrieves the sleep events from the FLOW server and displays a hypnogram (visual representation of sleep stages over time) on the screen, along with sleep statistics and other subject details. Additionally, a summary of the sleep scoring is automatically generated and added to the same participant in the FLOW server in PDF format.
Figure 1. Architecture Overview of NEAT for post-acquisition mode.
Modules description
NEAT UI:
The graphical user interface for the NEAT workflow is accessible through the FLOW frontend application, which is served by FLOW's web server. NEAT UI communicates with the FLOW server through a RESTful API to run the NEAT workflow and retrieve sleep staging results.
Page 7
RESTful API:
The FLOW API runs on the FLOW server and is the primary back end for the application. It is a RESTful API and communicates over HTTP.
neat-cli:
The neat-cli software runs on the FLOW server as an executable Docker container and leverages Python libraries for identifying stages of sleep on MFF files using Machine Learning (ML). The neat-cli container interacts with the FLOW API to start the NEAT workflow and store sleep staging results on the database.
FLOW Database:
All data that the NEAT workflow uses to perform ML sleep staging is stored in a Mongo DB database on the FLOW server.
5. INDICATIONS FOR USE
Automatically scoring of sleep EEG data to identify stages of sleep according to the American Academy of Sleep Medicine definitions, rules and guidelines. It is to be used with adult populations.
6. COMPARISON OF TECHNOLOGICAL CHARACTERISTICS WITH PREDICATE DEVICE
| Characteristic | New Device | Predicate Device |
|---|---|---|
| Device Name | NEAT | EnsoSleep |
| 510(k) number | K162627 | |
| Manufacturer | BELCo, LLC | EnsoData, Inc. |
| Regulation name | Electroencephalograph | Electroencephalograph |
| Product code | OLZ | OLZ |
| Regulatory class | Class II | Class II |
Page 8
| Characteristic | New Device | Predicate Device |
|---|---|---|
| Regulation number | 21 CFR 882.1400 | 21 CFR 882.1400 |
| Software only | Yes | Yes |
| Indication for use | Automatically scoring of sleep EEG data to identify stages of sleep according to the American Academy of Sleep Medicine definitions, rules and guidelines. It is to be used with adult populations. | EnsoSleep is intended for use for diagnostic evaluation by a physician to assess sleep quality and as an aid for the diagnosis of sleep and respiratory related sleep disorders in adults only. EnsoSleep is a software-only medical device to be used under the supervision of a clinician to analyze physiological signals and automatically score sleep study results, including the staging of sleep, detection of arousals, leg movements, and sleep disordered breathing events including obstructive apneas. All automatically scored events are subject to verification by a qualified clinician. Central apneas, mixed apneas, and hypopneas must be manually marked within records. |
| Environment for Use | Physician office or at home | |
| Derived Signals | Sleep stages: Rapid eye movement (REM), non-REM (N1, N2, N3) and wake | Sleep stages: Rapid eye movement (REM), non-REM (N1, N2, N3) and wake |
| Sleep Measures | • Sleep, REM and N3 onset• Total sleep and recording times• Sleep efficiency• % time by sleep stage | |
| Sleep staging | Based on one forehead EEG signals to differentiate Wake (W), REM (R), NREM stage 1 (N1), NREM stage 2 (N2) and slow wave sleep (N3, includes both stages 3 and 4) | Not known if classification of sleep stages can operate on forehead EEG. Automated method, however, can differentiate Wake (W), REM (R), NREM stage 1 (N1), NREM stage 2 (N2) and slow wave sleep (N3, includes both stages 3 and 4) |
Table 1: Comparison of the new device to the predicate device
Page 9
7. PERFORMANCE DATA
Summary of Non-Clinical Testing
Validation testing involved algorithm testing, which validated NEAT's accuracy. The product was deemed fit for clinical use.
NEAT was designed and developed as recommended by the FDA's Guidance, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Device".
According to the AAMI/ANSI/IEC 62304 Standard, NEAT safety classification has been set to Class B. "Basic Documentation Level" applied to this device.
Summary of Clinical Testing
NEAT and EnsoSleep Performance Evaluation Method
EnsoSleep: All data files were scored by EnsoSleep and sleep scores for each 30 second epoch were returned in JSON format. EnsoData uses a software-as-a-service model, where a client uploads PSG data to a cloud server and annotation files are output on the cloud server for the client to download.
NEAT: All data files were scored by NEAT and sleep scores for each 30-second epoch were labeled with a sleep stage.
Once the files were scored, performance on a segment-by-segment basis was compared against the established gold standard. Evaluation was performed by computing sensitivity performance (positive agreement--PA), specificity (negative agreement--NA), and overall agreement (OA) metrics for each data set (i.e., for different EEG systems). For each metric, we compute the bootstrapped (R=2000 resamples) median-point estimates and 95% confidence intervals. The 95% confidence interval was defined as the threshold values cutting off the top and bottom 2.5% of values.
Sensitivity was calculated by dividing the number of true positives (e.g., number of segments that NEAT correctly classified as N3) by the number of real positive cases (e.g., total number of segments classified as N3). Specificity (negative agreement--NA) was calculated by dividing the number of true negatives (e.g., number of segments which NEAT classified as not N3) by the number of real negative cases (e.g., number of epochs not classified as N3). OA was defined as the sum of all positive epochs (i.e., number of epochs that was correctly classified as a particular sleep stage) and negative epochs (i.e., number of epochs correctly classified as not being part of a particular sleep stages) divided by the total number of epochs.
NEAT vs EnsoSleep Overall Performance Comparisons
Page 10
Figure 2 shows confusion matrices for EnsoSleep (left) and NEAT (right) across the two data sets. NEAT was more accurate at classifying Wake, N1, and N3 sleep stages. On the other hand, EnsoSleep is better at classifying REM (R) and N2 sleep stages.
Figure 2. Confusion matrices for EnsoSleep (left) and NEAT (right) performance by sleep stage for across data sets. Median performance and 95% confidence interval (CI) are shown. Wa=Wake, R=REM.
In general, although there were different accuracies for different sleep stages for NEAT versus ENSO Sleep, these were largely within the range of differences that would be expected among expert human raters. We conclude that there is substantial equivalence between NEAT and the predicate ENSO Sleep.
The more specific findings are:
-
NEAT and EnsoSleep performed in an equivalent manner for correctly classifying Wake state EEG (1-2% difference depending on data set). This difference was within the range of human agreement variability.
-
EnsoSleep performed better (3-4% difference depending on data set) classifying REM state EEG.
-
EnsoSleep was better than NEAT in overall performance (4-7%) and specificity (5-9%) for classifying N1 sleep. However, only in the BEL data set was this difference bigger than the difference observed for human agreement. Moreover, sensitivity was substantially worse than NEAT (8-20%).
-
EnsoSleep was marginally better (5%) at classifying N2 sleep only for the BEL data set when compared to human agreement variability. Results showed EnsoSleep is more sensitive (22%) but less specific (9-11%) than NEAT.
Page 11
-
EnsoSleep and NEAT were equivalent (1% difference) in overall performance for N3 classification, with EnsoSleep being better (1%) for the BEL data set and NEAT being better (1%) for the CSF data set. Sensitivity is substantially better for NEAT across the two data sets (15-39%) compared to EnsoSleep, whereas specificity was marginally better for EnsoSleep (3-4%).
-
EnsoSleep sensitivity for N3 sleep is greatly affected by the data set. This may raise concerns about generalizability.
Implications of the Specific Differences Between ENSO Sleep and NEAT
NEAT and EnsoSleep are equivalent in functional features with regard to automated staging of sleep from EEG data. NEAT does not detect leg movements or sleep disordered breathing. With regards to equivalent functional features, although there were statistically significant differences in performance between the two devices, due to the very small confidence interval estimates from the large resampling number, the differences are best interpreted practically. To provide context for practicality, we compare the differences between NEAT and EnsoSleep with the range of human differences (i.e., agreement) and found that for the CSF dataset the only practical difference was for REM. Even in this case, the difference between the two devices is 3%.
In general, although there were different accuracies for different sleep stages for NEAT versus ENSO Sleep, these were largely within the range of differences that would be expected among expert human raters. We conclude that there is substantial equivalence between NEAT and the predicate ENSO Sleep.
8. CONCLUSION
The information discussed above and provided in the 510(k) submission demonstrate that the NEAT device is substantially equivalent to the predicate.
§ 882.1400 Electroencephalograph.
(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).