Search Results

Automatic scoring of sleep EEG data to identify stages of sleep according the American Academy of Sleep Medicine definitions, rules and guidelines. It is to be used with adult populations.

The Neurosom EEG Assessment Technology (NEAT) is a medical device software application that allows users to perform sleep staging post-EEG acquisition. NEAT allows users to review sleep stages on scored MFF files and perform sleep scoring on unscored MFF files.

NEAT software is designed in a client-server model and comprises a User Interface (UI) that runs on a Chrome web browser in the client computer and a Command Line Interface (CLI) software that runs on a Forward-Looking Operations Workflow (FLOW) server.

The user interacts with the NEAT UI through the FLOW front-end application to initiate the NEAT workflow on unscored MFF files and visualize sleep-scoring results. Sleep stages are scored by the containerized neat-cli software on the FLOW server using the EEG data. The sleep stages are then added to the input MFF file as an event track file in XML format. Once the new event track file is created, the NEAT UI component retrieves the sleep events from the FLOW server and displays a hypnogram (visual representation of sleep stages over time) on the screen, along with sleep statistics and other subject details. Additionally, a summary of the sleep scoring is automatically generated and added to the same participant in the FLOW server in PDF format.

The FDA 510(k) Clearance Letter for NEAT 001 provides information about the device's acceptance criteria and the study conducted to prove its performance.

Acceptance Criteria and Device Performance

The core acceptance criteria for NEAT 001, as demonstrated by the comparative clinical study, are based on its ability to classify sleep stages (Wake, N1, N2, N3, REM) with performance comparable to the predicate device, EnsoSleep, and within the variability observed among expert human raters.

Table of Acceptance Criteria and Reported Device Performance

The document does not explicitly state pre-defined numerical "acceptance criteria" for each metric (Sensitivity, Specificity, Overall Agreement) that NEAT 001 had to meet. Instead, the approach was a comparative effectiveness study against a predicate device (EnsoSleep), with the overarching criterion being "substantial equivalence" as interpreted by performance falling within the range of differences expected among expert human raters.

Therefore, the "acceptance criteria" are implied by the findings of substantial equivalence. The "reported device performance" is given in terms of the comparison between NEAT and EnsoSleep, and their differences relative to human agreement variability.

Study Details

Here's a breakdown of the study details based on the provided text:

1. Sample Size and Data Provenance

• Test Set Sample Size: The exact number of participants or EEG recordings in the test set is not explicitly stated. The document refers to "two data sets" (referred to as "BEL data set" and "CSF data set") used for testing both NEAT and EnsoSleep. The large resampling number (R=2000 resamples for bootstrapping) suggests a dataset size sufficient to yield small confidence intervals.
• Data Provenance:
  • Country of Origin: Not explicitly stated.
  • Retrospective or Prospective: Not explicitly stated, but the mention of "All data files were scored by EnsoSleep" and "All data files were scored by NEAT" implies these were pre-existing datasets, making them retrospective.

2. Number of Experts and Qualifications for Ground Truth

• Number of Experts: Not explicitly stated. The study refers to "established gold standard" and "human agreement variability" among "expert human raters," implying multiple experts.
• Qualifications of Experts: Not explicitly stated beyond "expert human raters." No details are provided regarding their specific medical background (e.g., neurologists, sleep specialists), years of experience, or board certifications.

3. Adjudication Method for the Test Set

• Adjudication Method: Not explicitly stated. The document simply refers to "the established gold standard." It does not mention whether this gold standard was derived from a single expert, consensus among multiple experts, or a specific adjudication process (like 2+1 or 3+1).

4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Was an MRMC study done? A direct MRMC comparative effectiveness study involving human readers assisting with AI vs. without AI assistance was not explicitly described. The study primarily focuses on comparing the standalone performance of NEAT (the AI) against the standalone performance of the predicate device (EnsoSleep), and then interpreting these differences in the context of human-to-human agreement variability.
• Effect Size of Human Reader Improvement: Since a direct MRMC study with human readers assisting AI was not detailed, there is no information provided on the effect size of how much human readers improve with AI vs. without AI assistance.

5. Standalone Performance (Algorithm Only)

• Was a standalone study done? Yes. The study evaluated the "segment-by-segment" performance of NEAT and EnsoSleep algorithms directly against the "established gold standard." This is a measure of the algorithm's standalone performance without human input during the scoring process.

6. Type of Ground Truth Used

• Type of Ground Truth: The ground truth for the test set was based on an "established gold standard" for sleep stage classification. This strongly implies expert consensus or expert scoring of the EEG data according to American Academy of Sleep Medicine definitions, rules, and guidelines. Pathology or outcomes data were not used for sleep staging ground truth.

7. Training Set Sample Size

• Training Set Sample Size: The sample size for the training set is not explicitly stated in the provided document.

8. How Ground Truth for Training Set Was Established

• How Ground Truth for Training Set Was Established: The document states that neat-cli "leverages Python libraries for identifying stages of sleep on MFF files using Machine Learning (ML)." However, it does not explicitly describe how the ground truth for the training set was established. Typically, for ML models, the training data's ground truth would also be established by expert annotation or consensus, similar to the test set ground truth, but this is not confirmed in the provided text.

Ask a specific question about this device

DeepRESP is an aid in the diagnosis of various sleep disorders where subjects are often evaluated during the initiation or follow-up of treatment of various sleep disorders. The recordings to be analyzed by DeepRESP can be performed in a hospital, patient home, or an ambulatory setting. It is indicated for use with adults (22 years and above) in a clinical environment by or on the order of a medical professional.

DeepRESP is intended to mark sleep study signals to aid in the identification of events and annotation of traces; automatically calculate measures obtained from recorded signals (e.g., magnitude, time, frequency, and statistical measures of marked events); infer sleep staging with arousals with EEG and in the absence of EEG. All output is subject to verification by a medical professional.

DeepRESP is a cloud-based software as a medical device (SaMD), designed to perform analysis of sleep study recordings, with and without EEG signals, providing data for the assessment and diagnosis of sleep-related disorders. Its algorithmic framework provides the derivation of sleep staging including arousals, scoring of respiratory events and key parameters such as the Apnea-Hypopnea Index (AHI).

DeepRESP is hosted on a serverless stack. It consists of:

• A web Application Programming Interface (API) intended to interface with a third-party client application, allowing medical professionals to access DeepRESP's analytical capabilities.
• Predefined sequences called Protocols that run data analyses, including artificial intelligence and rule-based models for the scoring of sleep studies, and a parameter calculation service.
• A Result storage using an object storage service to temporarily store outputs from the DeepRESP Protocols.

Here's a breakdown of the acceptance criteria and the study details for the DeepRESP device, based on the provided FDA 510(k) summary:

1. Table of Acceptance Criteria & Reported Device Performance:

The document doesn't explicitly state "acceptance criteria" as a separate table, but it compares DeepRESP's performance against manual scoring and predicate devices. I've extracted the performance metrics that effectively serve as acceptance criteria given the "non-inferiority" and "superiority" claims against established devices.

2. Sample Size Used for the Test Set and Data Provenance:

• Type I/II Studies (EEG present): 2,224 sleep recordings
• Type III Studies (No EEG): 3,488 sleep recordings (including 2,213 Type I recordings and 1,275 Type II recordings, processed to utilize only Type III relevant signals).
• Provenance: Retrospective study. Data originated from sleep clinics in the United States, collected as part of routine clinical work for patients suspected of sleep disorders. The patient population showed diversity in age, BMI, and race/ethnicity (Caucasian or White, Black or African American, Other, Not Reported) and was considered representative of patients seeking medical services for sleep disorders in the United States.

3. Number of Experts and Qualifications for Ground Truth:

The document explicitly states that the studies used "manually scored sleep recordings" but does not specify the number of experts or their specific qualifications (e.g., "radiologist with 10 years of experience"). It implicitly relies on the quality of "manual scoring" from routine clinical work in US sleep clinics as the ground truth.

4. Adjudication Method for the Test Set:

The document does not describe any specific adjudication method (e.g., 2+1, 3+1). It refers to "manual scoring" as the established ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

No, a MRMC comparative effectiveness study was not reported. The study design was a retrospective data analysis comparing the algorithm's performance against existing manual scoring (ground truth) and established predicate devices. There is no information about human readers improving with AI vs. without AI assistance. The device is intended to provide automatic scoring subject to verification by a medical professional.

6. Standalone (Algorithm Only) Performance:

Yes, the study report describes the standalone performance of the DeepRESP algorithm. The reported PPA, NPA, OPA percentages, and ICC values represent the agreement of the automated scoring by DeepRESP compared to the manual ground truth. The device produces output "subject to verification by a medical professional," but the performance metrics provided are for the algorithmic output itself.

7. Type of Ground Truth Used:

The ground truth used was expert consensus (manual scoring). The document states "It used manually scored sleep recordings... The studies were done by evaluating the agreement in scoring and clinical indices resulting from the automatic scoring by DeepRESP compared to manual scoring."

8. Sample Size for the Training Set:

The document does not explicitly state the sample size used for the training set. The clinical validation study is described as a "retrospective study" used for validation, but details about the training data are not provided in this summary.

9. How the Ground Truth for the Training Set Was Established:

The document does not specify how the ground truth for the training set was established. It only describes the ground truth for the validation sets as "manually scored sleep recordings" from routine clinical work.

Ask a specific question about this device

The Dreem 3S is intended for prescription use to measure, record, display, transmit and analyze the electrical activity of the brain to assess sleep and awake in the home or healthcare environment. The Dreem 3S can also output a hypnogram of sleep scoring by 30-second epoch and summary of sleep metrics derived from this hypnogram.

The Dreem 3S is used for the assessment of sleep on adult individuals (22 to 65 years old). The Dreem 3S allows for the generation of user/predefined reports based on the subject's data.

The Dreem 3S headband contains microelectronics, within a flexible case made of plastic, foam, and fabric. It includes 6 EEG electrodes and a 3D accelerometer sensor.

The EEG signal is measured by two electrodes in the frontal position) and two at the back of the head (occipital position), along with one reference electrode and one ground electrode.

The 3D accelerometer is embedded in the top of the headband to ensure accurate measurements of the wearer's head movement during the night. The raw EEG and accelerometer data are transferred to Dreem's servers for further analysis after the night is over.

The device includes a bone-conduction speaker with volume control to provide notifications to the wearer, and a power button circled by a multicolor LED light

The device generates a sleep report that includes a sleep staging for each 30-second epoch during the night. This output is produced using an algorithm that analyzes data from the headband EEG and accelerometer sensors. A raw data file is also available in EDF format.

The provided text is a 510(k) summary for the Dreem 3S device. It does not contain a comprehensive study detailing acceptance criteria and device performance. Instead, it states that no new testing was performed because the current submission is primarily for the inclusion of a Predetermined Change Control Plan (PCCP). It relies on the performance characteristics previously reported for the predicate device (K223539).

Therefore, I cannot provide a table of acceptance criteria with reported performance, or details about the sample sizes and ground truth for a new study, as none was conducted or reported in this document.

However, based on the information for the predicate device, and the intent behind the PCCP, I can infer and summarize what would typically be expected for such a device and what the PCCP aims to maintain:

Inferred Acceptance Criteria based on Predicate Device (K223539) and PCCP:
The document states, "clinical performance validation will also be repeated, and will require that the performance of any modification to Dreem 3S to be non-inferior to the all previously released versions of the Dreem 3S device." This indicates that the primary acceptance criterion for any future algorithmic updates under the PCCP is non-inferiority to the performance established in the original clearance (K223539). While the specific metrics are not detailed in this current summary, for a sleep staging device, these would typically include accuracy metrics like Cohen's Kappa, Sensitivity, Specificity, and overall accuracy for differentiating sleep stages (Wake, NREM1, NREM2, NREM3, REM).

Regarding Study Information (based on the original clearance of K223539, not detailed here):

Since the provided document explicitly states, "No bench testing, animal testing, or clinical testing was performed to support this submission," I cannot fill in the details for a new study. The performance information relates to the predicate device (K223539).

However, based on the Predetermined Change Control Plan (PCCP) section, which outlines how future algorithmic modifications will be validated, I can describe the methodology for future performance validation under that plan:

Inferred Acceptance Criteria and Future Performance Validation Methodology (based on PCCP)

1. Table of Acceptance Criteria and Reported Device Performance:

Note: The actual numerical performance metrics for the predicate device (K223539) are not provided in this document. They would have been part of the original K223539 submission. The PCCP ensures that future algorithmic changes meet these same (or non-inferior) performance levels.

2. Sample Size Used for the Test Set and Data Provenance:

• For future updates under PCCP: The PCCP states, "Recordings that are used for any purpose (e.g., training, tuning, failure analysis, etc.) that might lead to direct or indirect insight regarding the performance of a modified sleep staging algorithm on this recording, other than execution of the clinical performance validation per the methods specified in the PCCP, are excluded from the test dataset." This implies that a new, independent test set will be used for each validation under the PCCP.
• Sample Size: Not specified for future PCCP validations, but it is stated that "Quality checks will ensure that the test data are sufficiently high quality and representative of the intended use population."
• Data Provenance: Not explicitly stated, but for sleep studies, typically involves polysomnography (PSG) data. The "human variability estimated from comparison of expert scoring from 284 American Academy of Sleep Medicine (AASM) compliant polysomnography recordings" suggests a U.S. or internationally recognized standard for data interpretation. The fact that the device assesses adult individuals (22 to 65 years old) means the test set would be composed of data from this age demographic. Retrospective or prospective is not specified, but typically retrospective datasets are used for initial clearances.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:

• For future updates under PCCP: "Non-inferiority margins were selected based on the level of human variability estimated from comparison of expert scoring from 284 American Academy of Sleep Medicine (AASM) compliant polysomnography recordings." This strongly implies that the ground truth for validation (both for K223539 and subsequent PCCP validations) is expert consensus scoring based on AASM guidelines.
• Number of Experts: Not explicitly stated, but "expert scoring" typically implies one or more certified sleep technologists or sleep physicians. The mention of "human variability" often means comparison between at least two independent expert scorings.
• Qualifications: "American Academy of Sleep Medicine (AASM) compliant polysomnography recordings" strongly suggests that the experts would be board-certified sleep physicians or registered polysomnographic technologists (RPSGTs) with experience in AASM sleep staging. The number of years of experience is not specified.

4. Adjudication Method for the Test Set:

• Not explicitly defined in the provided text. However, for "expert scoring" and estimating "human variability," common adjudication methods include:
  • Consensus: Multiple experts independently score, and a final consensus is reached (e.g., by discussion or a third adjudicator if initial scores differ significantly).
  • Majority vote: If more than two experts, the majority decision prevails.
  • Pairwise agreement: Often used to quantify inter-rater variability for tasks like sleep staging.

5. Multi Reader Multi Case (MRMC) Comparative Effectiveness Study:

• The document does not report on an MRMC comparative effectiveness study where human readers improve with AI vs. without AI assistance for this specific submission (K242094). This submission is for a PCCP and relies on the predicate's performance.

6. Standalone (Algorithm Only) Performance Study:

• Yes, the document implies that a standalone performance study was conducted for the predicate device (K223539). The algorithm "analyzes data from the headband EEG and accelerometer sensors" and "uses raw EEG data and accelerometer data to provide automatic sleep staging according to the AASM classification." The PCCP is about maintaining and improving this algorithm's standalone performance.
• The "clinical performance validation will also be repeated, and will require that the performance of any modification to Dreem 3S to be non-inferior" to previous versions. This directly refers to the algorithm's standalone performance.

7. Type of Ground Truth Used:

• Expert Consensus: The phrase "automatic sleep staging according to the AASM classification" and "comparison of expert scoring from 284 American Academy of Sleep Medicine (AASM) compliant polysomnography recordings" strongly indicates that the ground truth is established by expert scoring conforming to AASM guidelines. This is the standard for sleep staging.

8. Sample Size for the Training Set:

• Not specified in this document. This refers to the original training data used for the predicate device (K223539). For future updates, the PCCP mentions "Retraining with an updated training/tuning dataset" but does not specify the size of these datasets.

9. How the Ground Truth for the Training Set Was Established:

• Not explicitly specified for the training set itself, but it is highly probable that the ground truth for the training set was established through expert consensus scoring according to AASM guidelines, similar to how the test set's ground truth is (or will be for PCCP updates) established. This is standard practice for supervised machine learning models in this domain.

Ask a specific question about this device

SleepStageML is intended for assisting the diagnostic evaluation by a qualified clinician to assess sleep quality from level 1 polysomnography (PSG) recordings in a clinical environment in patients aged 18 and older.

SleepStageML is a software-only medical device to be used to analyze physiological signals and automatically score sleep stages. All outputs are subject to review by a qualified clinician.

SleepStageML is an Artificial Intelligence/Machine Learning (Al/ML)-enabled software-only medical device that analyzes polysomnography (PSG) recordings and automatically scores sleep stages. It is intended for assisting the diagnostic evaluation by a qualified clinician to assess sleep quality in patients aged 18 and older.

Qualified clinicians (also referred to as clinical users) such as sleep physicians, sleep technicians, or registered PSG technologists (RPSGTs) who are qualified to review PSG studies, provide PSG recordings in European Data Format (EDF) file format through a secure file transfer system to Beacon Biosignals. SleepStageML automatically analyzes the provided PSG recording and return an EDF file containing the original PSG recording with software-generated sleep stage annotations (i.e., Wake (W), non-REM 1 (N1), non-REM 2 (N2), non-REM 3 (N3), and REM (R)) back to the clinical user. The EDF files containing PSG signals as well as sleep stage annotations are referred to as EDF+. The returned EDF+ files can then be reviewed by the qualified clinicians via the users' PSG viewing software. The recordings processed by SleepStageML are level-1 PSG recordings obtained in an attended setting in accordance with American Association of Sleep Medicine (AASM) recommendations with respect to minimum sampling rate, electroencephalography (EEG) channels, and EEG locations. SleepStageML only uses the EEG signals in provided PSGs and does not consider electromyography (EMG) or electrooculography (EOG) signals when performing sleep staging. The sleep stage outputs of SleepStageML are intended to be comparable to sleep stages as defined by AASM guidelines. SleepStageML software outputs are subject to qualified clinician's review.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) summary for SleepStageML:

Acceptance Criteria and Reported Device Performance

Study Details:

1. Sample sizes used for the test set and data provenance:

   • Test Set Sample Size: 100 patients.
   • Data Provenance: Retrospective pivotal validation study using previously collected clinical polysomnography (PSG) recordings. The recordings were randomly selected from three Level 1 clinical PSG data sources. The document does not specify the country of origin of the data.

2. Number of experts used to establish the ground truth for the test set and their qualifications:

   • Number of Experts: Three (3) registered PSG technologists (RPSGTs).
   • Qualifications: Each RPSGT had at least 5 years of experience in clinical scoring of sleep studies.

3. Adjudication method for the test set:

   • Method: 2/3 majority scoring. Expert consensus sleep stages were constructed using the stage per epoch where at least 2 of the 3 experts agreed. Epochs where all 3 RPSGTs disagreed were excluded.

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:

   • No, an MRMC study comparing human readers with AI vs. without AI assistance was not explicitly detailed. The study focused on the standalone performance of the AI algorithm against human expert consensus to demonstrate non-inferiority to a predicate device. The device's indication for use explicitly states, "All outputs are subject to review by a qualified clinician," indicating a human-in-the-loop design, but the described performance study is primarily a standalone evaluation.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, the clinical validation test evaluated the SleepStageML software's performance "against the expert consensus sleep stages" in a standalone manner. The device's outputs are intended to be reviewed by a clinician, but the performance metrics reported are for the algorithm's direct output compared to ground truth.

6. The type of ground truth used:

   • Type: Expert Consensus. The ground truth was established by three RPSGTs, with a 2/3 majority rule for consensus.

7. The sample size for the training set:

   • The document states, "SleepStageML uses a deep learning algorithm based on convolutional neural networks, which was trained on a large and diverse set of PSG recordings with sleep staging labels." However, a specific sample size for the training set is not provided in the summary.

8. How the ground truth for the training set was established:

   • The document states the training was on "PSG recordings with sleep staging labels." It does not explicitly detail the method for establishing ground truth for the training set (e.g., if it was also expert consensus, single expert, or another method). However, given the nature of sleep staging, it is highly likely that these labels were also derived from expert annotations, similar to the test set, though possibly not with the same rigorous 3-expert consensus and adjudication for every record.

Ask a specific question about this device

The Dreem 3S is intended for prescription use to measure, record, display, transmit and analyze the electrical activity of the brain to assess sleep and awake in the home or healthcare environment.

The Dreem 3S can also output a hypnogram of sleep scoring by 30-second epoch and summary of sleep metrics derived from this hypnogram.

The Dreem 3S is used for the assessment of sleep on adult individuals (22 to 65 years old). The Dreem 3S allows for the generation of user/predefined reports based on the subject's data.

The Dreem 3S headband contains microelectronics, within a flexible case made of plastic, foam, and fabric. It includes 6 EEG electrodes and a 3D accelerometer sensor.

The EEG signal is measured by two electrodes in the frontal position) and two at the back of the head (occipital position), along with one reference electrode and one ground electrode.

The 3D accelerometer is embedded in the top of the headband to ensure accurate measurements of the wearer's head movement during the night. The raw EEG and accelerometer data are transferred to Dreem's servers for further analysis after the night is over.

The device includes a bone-conduction speaker with volume control to provide notifications to the wearer, and a power button circled by a multicolor LED light

The device generates a sleep report that includes a sleep staging for each 30-second epoch during the night. This output is produced using an algorithm that analyzes data from the headband EEG and accelerometer sensors. A raw data file is also available in EDF format.

The algorithm uses raw EEG data and accelerometer data to provide automatic sleep staging according to the AASM classification. The algorithm is implemented with an artificial neural network. Frequency spectrums are computed from raw data and then passed to several neural network layers including recurrent layers and attention layers. The algorithm outputs prediction for several epochs of 30 seconds at the same time, every 30 seconds. The various outputs for a single epoch of 30 seconds are combined to provide robust sleep scoring.

Here's a breakdown of the acceptance criteria and study details for the Dreem 3S device based on the provided text:

Acceptance Criteria and Device Performance

Note: The document primarily presents performance results rather than explicitly stating pre-defined acceptance criteria with numerical thresholds the device needed to meet. The "acceptance criteria" listed above are inferred from the demonstrated performance that supported substantial equivalence.

Study Details

1. Sample size used for the test set and the data provenance:

   • Sample Size: 38 subjects
   • Data Provenance: The study was a "clinical investigation... completed... in a sleep lab setting." Subjects ranged from 23 to 66 years old, equally split between male and female, and included individuals self-identified as White, Black African American, Asian, Hispanic, and some not identified. This suggests a prospective study with diverse participants, likely conducted in a single country, though the specific country of origin is not explicitly stated. The study included a total of 36447 epochs, corresponding to about 303 hours and 43 minutes of sleep.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • The ground truth for sleep staging was established by "expert-scored sleep stages from a cleared device." For EEG data quality, this was assessed by "at least two out of three reviewers qualified to read EEG and/or PSG data." The specific number of experts used for the primary sleep staging ground truth is not explicitly stated (e.g., whether it was one expert, or a consensus of multiple). Their exact qualifications (e.g., years of experience as a radiologist) are also not detailed beyond "expert-scored" and "qualified to read EEG and/or PSG data."

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • For the primary sleep staging (Table 2), the ground truth is referred to as "Consensus from manual staging" or "expert-scored PSG." The specific adjudication method (e.g., 2+1, 3+1) is not detailed.
   • For EEG data quality, acceptability was determined if "at least two out of three reviewers qualified to read EEG and/or PSG data" agreed. This implies a 2-out-of-3 consensus (similar to a 2+1 method if one reviewer was the primary and two others adjudicated).

4. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, a multi-reader multi-case (MRMC) comparative effectiveness study focusing on human reader improvement with AI assistance was not done. This study solely evaluated the standalone performance of the Dreem 3S algorithm against expert-scored PSG.

5. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:

   • Yes, a standalone study was done. The clinical performance evaluation directly compares the "Dreem 3S (Automated analysis)" output to "Consensus from manual staging" (expert-scored PSG), indicating the performance of the algorithm without human intervention in the loop.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • Expert consensus of manual staging from a 510(k)-cleared PSG system.

7. The sample size for the training set:

   • The sample size for the training set is not specified in the provided document. The document only mentions "The algorithm is implemented with an artificial neural network. Frequency spectrums are computed from raw data and then passed to several neural network layers including recurrent layers and attention layers," which implies a training process, but no details on the training data size are given.

8. How the ground truth for the training set was established:

   • How the ground truth for the training set was established is not specified in the provided document.

Ask a specific question about this device

SOMNUM is a computer program (software) intended for use as an aid for the diagnosis of sleep and respiratory related sleep disorders. SOMNUM is intended to be used for analysis (automatic scoring and manual re-scoring), display, redisplay(retrieve), summarize and reports generation of digital data collected by monitoring devices typically used to evaluate sleep and respiratory related sleep disorders. The device is to be used under the supervision of a physician. Use is restricted to files obtained from adult patients.

For respiratory events - Sleep Disordered Breathing (Apneas)- obstructive, central, mixed apneas, and hypopneas must be manually scored by physician. The device does not output specific apnea or hypopnea events and therefore should not be used for management decisions.

SOMNUM is a standalone software application that analyze previously recorded physiological data obtained during level 1 sleep studies, referred to as polysomnography (PSG) records. The SOMNUM software can analyze any EDF files. Automated algorithms are applied to the raw signals in order to identify the occurrence of certain events. The software automates recognition of:

• · Sleep Stage Events : Wake, Stage N1, Stage N2, Stage N3, Stage REM
• Respiratory Events : Sleep Disordered Breathing (device output does not distinguish between Apneas and Hypopneas. Obstructive, central, mixed apneas, and hypopneas must be manually scored by physician)
• · Arousal Events
• · Leg Movement Events : Periodic Leg Movements during Sleep (PLMs)

The SOMNUM software can be used as a stand-alone application for use on Windows 10 operating system platform. All processing, scoring, and analysis of signal data occurs on local desktop PC.

The Honeynaps Co., Ltd. Somnum (v.1.1.2) device underwent a non-clinical performance test to establish substantial equivalence to predicate devices for the analysis of sleep and respiratory-related sleep disorders.

1. Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly stated as numerical thresholds for specific metrics before the results table. However, the "Discussion" sections following the results implicitly define acceptable performance based on comparisons to predicate devices and acceptable error ranges in clinical practice. The reported device performance is shown in the tables below.

Endpoint 1: Performance for Detecting Each Event Type (Sleep Stage, Arousal, SDB, PLMs)

Endpoint 2: Performance for Summary Variables (Absolute Max Difference/LOA)

Conclusion: The study concludes that SOMNUM passed all pass/fail criteria for both Endpoint 1 and Endpoint 2, demonstrating substantial equivalence.

2. Sample Size and Data Provenance

• Sample Size (Test Set): N=48 subjects
• Data Provenance: The data was recorded in a sleep laboratory. The country of origin is not explicitly stated. The study design is described as "cross-sectional experimental design," which implies it was specifically conducted for this evaluation. It is not explicitly stated if it's retrospective or prospective, but the phrasing "representative N=48 subjects of data recorded in the sleep laboratory" suggests it was existing data selected for the study.

3. Number of Experts and Qualifications for Ground Truth

• Number of Experts: Three technologists.
• Qualifications of Experts: "Medical professionals certified on PSG recording and analysis". Specific years of experience are not provided.

4. Adjudication Method

• Adjudication Method: 2/3 majority rule. This means at least two out of the three experts had to agree on the presence of an event within an epoch for it to be considered ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• A MRMC comparative effectiveness study was not conducted to measure the improvement of human readers with AI assistance versus without AI assistance. The study focuses on the standalone performance of the AI.

6. Standalone Performance Study

• Yes, a standalone study was conducted. The performance test evaluated "SOMNUM device performance using a cross-sectional experimental design." The comparison was between SOMNUM's scoring and the ground truth established by expert consensus.

7. Type of Ground Truth Used

• Type of Ground Truth: Expert consensus (2/3 majority rule of three certified medical professionals).

8. Sample Size for the Training Set

• The document does not provide specific details regarding the sample size used for the training set. It only describes the performance test (test set).

9. How Ground Truth for Training Set was Established

• The document does not provide details on how the ground truth for the training set was established. It only describes the ground truth establishment for the test set.

Ask a specific question about this device

SomnoMetry is intended for use for the diagnostic evaluation by a physician to assess sleep quality and as an aid for the diagnosis of sleep and respiratory-related sleep disorders in adults only. SomnoMetry is a software-only medical device to be used to analyze physiological signals and automatically score sleep study results, including the staging of sleep, AHI, and detection of sleepdisordered breathing events including obstructive apneas. It is intended to be used under the supervision of a clinician in a clinical environment. All automatically scored events are subject to verification by a qualified clinician.

The SomnoMetry is an Artificial Intelligent/Machine Learning (AI/ML)-enabled Software as a Medical Device (SaMD) that automatically scores sleep study results by analyzing polysomnography (PSG) signals recorded during sleep studies. It is intended to be used under the supervision of a clinician in clinical environments to aid in the diagnosis of sleep and respiratory related sleep disorders.

All scored events that are analyzed, displayed, and summarized can be manually marked or edited by a qualified clinician during review and verification.

SomnoMetry consists of:

• A web Application Programing Interface (API) to allow authenticated users to upload . PSG files to SomnoMetry Platform
• A database to store the input, intermedium output, final output, and associated data ●
• A database API to access the database and store/retrieve the output ●
• A dashboard to display, retrieve, manage, edit, verify, and summarize the output
• An AI/ML Engine using AI/ML algorithms/approaches to analyze PSG data ●
• A reporting API to generate sleep reports

Here's an analysis of the acceptance criteria and study that proves the device meets them, based on the provided text:

Device: SomnoMetry (Neumetry Medical Inc.)

Regulatory Class: Class II (Product Code: OLZ)
Intended Use: Diagnostic evaluation to assess sleep quality and aid in diagnosing sleep and respiratory-related sleep disorders in adults. Automatically scores sleep study results (sleep staging, AHI, obstructive apneas). Used under clinician supervision; all automatically scored events are subject to verification.

1. Table of Acceptance Criteria and Reported Device Performance

The document doesn't explicitly define 'acceptance criteria' in a numerical table for the SomnoMetry device against specific thresholds (e.g., "accuracy must be > 90%"). Instead, it frames the performance evaluation as demonstrating non-inferiority and substantial equivalence to a predicate device (EnsoSleep K162627) based on clinical performance metrics. The implicit acceptance criteria are that the SomnoMetry's performance for sleep staging and sleep apnea diagnosis is statistically equivalent to, or better than, the predicate device.

Table 1: Implicit Acceptance Criteria and SomnoMetry's Reported Performance (vs. Predicate)

2. Sample Size and Data Provenance for Test Set

• Sample Size: N = 201 adult subjects.
• Data Provenance: Retrospective clinical data obtained from 2 AASM accredited Sleep Testing Facilities in California, USA.
  • The data was verified to meet specified disease spectrum, medical condition, medication, and demographic requirements.
  • Randomized sampling with proportionate allocation across each sleep apnea disease severity quantile (normative, mild, moderate, and severe sleep apnea) and sleep cycles was used.
  • Age of subjects ranged from 20 to 84 years.
  • No race/ethnicity information was collected.

3. Number of Experts and Qualifications for Ground Truth

The document explicitly states the ground truth was established by "manually scored PSG data" which aligns with the "AASM gold standard". While it doesn't specify the number of experts, it implies these were the standard scores provided by the AASM accredited facilities. The qualifications are implicitly that they are "qualified clinicians" following "American Academy of Sleep Medicine scoring manual and guidelines."

4. Adjudication Method for Test Set

The document does not describe a formal expert adjudication method (e.g., 2+1, 3+1). The ground truth is stated as "AASM gold standard of manually scored PSG data," which suggests a single, accepted manual score per PSG study generated by the sleep facility.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a MRMC comparative effectiveness study was not conducted. The study evaluated the standalone performance of the AI algorithm against the AASM gold standard and then compared this algorithm performance to the predicate device's algorithm performance. It did not assess how human readers improve with AI assistance versus without. The device's intended use note ("All automatically scored events are subject to verification by a qualified clinician") implies a human-in-the-loop workflow, but the study itself wasn't designed to measure the impact of AI assistance on human reader performance.

6. Standalone (Algorithm Only) Performance Study

Yes, a standalone (algorithm only) performance study was performed. The "retrospective clinical performance testing" evaluated the SomnoMetry AI/ML algorithms directly, comparing their output to the AASM gold standard. The results presented in Table 2 (Confusion Matrix for Sleep Staging) and Table 3 (Sleep Apnea Diagnostic Agreement) are for the algorithm's performance.

7. Type of Ground Truth Used

The type of ground truth used was expert consensus / AASM (American Academy of Sleep Medicine) gold standard based on "manually scored PSG data." This is considered the clinical standard for sleep study interpretation.

8. Sample Size for the Training Set

The document does not specify the sample size for the training set. It only describes the test set (N=201).

9. How Ground Truth for the Training Set Was Established

The document does not describe how the ground truth for the training set was established. It focuses solely on the clinical evaluation (test set) and states that data was obtained from AASM accredited facilities. It can be inferred that similar "manually scored PSG data" would have been used for training, given the expertise required for such scoring and the device's reliance on AASM guidelines.

Ask a specific question about this device

EnsoSleep is intended for use in the diagnostic evaluation by a physician to assess sleep quality and as an aid for physicians in the diagnosis of sleep disorders and respiratory related sleep disorders in pediatric as follows:

• · Pediatric patients 13 years and older with polysomnography (PSG) tests obtained in a Hospital or Sleep Clinic
• · Adult patients with PSGs obtained in a Hospital or Sleep Clinic
• · Adult patients with Home Sleep Tests
  EnsoSleep is a software-only medical device to be used under the supervision of a clinician to analyze physiological signals and automatically score sleep study results, including the staging of arousals, leg movements, and sleep disordered breathing events including obstructive apneas (OSA), central sleep apneas (CSA), and hypopneas.
  All automatically scored events and physiological signals which are retrieved, analyzed, displayed, and summarized are subject to verification by a qualified clinician. Central sleep apneas (CSA) should be manually reviewed and modified as appropriate by a clinician.
  All events can be manually marked or edited within records during review.
  Photoplethysmography (PPG) total sleep time is not intended for use when electroencephalograph (EEG) data is recorded. PPG total sleep time is not intended to be used as the sole or primary basis for diagnosing any sleep related breathing disorder, prescribing treatment, or determining whether additional diagnostic assessment is warranted.

EnsoSleep is a software-only medical device that analyzes previously recorded physiological signals obtained during sleep. Users of EnsoSleep are consistent with the roles required to run a sleep clinic: sleep physicians, sleep technicians, clinic operations managers, and IT administrators. EnsoSleep can analyze at-home and in-lab sleep studies for both adult and pediatric patients who are at least 13 years old. Automated algorithms are applied to the raw signals in order to derive additional signals and interpret the raw and derived signal information. The software automates recognition of the following: respiratory events, sleep staging events, arousal events, movement events, cardiac events, derived signals, and calculated indices. EnsoSleep does not interpret the results, nor does it suggest a diagnosis. The device only marks events of interest for review by a physician who is responsible for diagnoses. The device does not analyze data that are different from those analyzed by human scorers.
The signals and automated analyses can be visually inspected and edited prior to the results being integrated into a sleep study report.
The software consists of 4 major components:

• The Application Platform runs on local clinic workstations and manages the detection, upload, and download of study records and scoring to and from the Storage Platform
• The Processing Platform accepts raw physiological signals as inputs in order to recognize events, derive signals, and calculate indices
• . The Storage Platform facilitates file and database storage in the EnsoSleep cloud through an API
• The Dashboard is a web-based user interface to support configuration, clinic management, and sleep study scoring

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The text describes that "the acceptance criteria were selected based on PA, NA, OA, MAE, Deming Regression coefficient, Bland-Altman mean difference and limits of agreement performance criteria that validate performance substantially equivalent to, or greater than, but not lesser than by more than 10% or similarly defined criteria in any category of the predicate 510(k) reported device performance across all endpoints respectively."

For detailed performance metrics, the document presents tables for specific endpoints. Below is a structured representation focusing on the key performance indicators mentioned and a comparison to the predicate device where available.

Endpoint 1: Sleep Staging Event Detection (EnsoSleep K210034 vs. Predicate K162627)

Endpoint 2: Sleep Apnea Diagnostic Agreement (Per-Patient AHI)

Endpoint 3: Sleep Scoring Event Detection (EnsoSleep K210034 vs. Predicate K162627 and Reference K112102)

Endpoint 4: Total Sleep Time (TST) and Respiratory Rate (RR)

Ask a specific question about this device

Sleepware G3 is a software application used for analysis (automatic and manual scoring), display, retrieval, summarization, report generation, and networking of data received from monitoring devices used to categorize sleep related events that help aid in the diagnosis of sleep-related disorders. It is indicated for use with adults (18 and older) and infant patients (one year old or less) in a clinical environment by or on the order of a physician.

The optional Somnolyzer scoring algorithms are for use with adults (18 and older) to generate an output that is ready for review and interpretation by a physician. Cardio-Respiratory Sleep Staging (CReSS) is an additionality of Somnolyzer which uses standard Home Sleep Apnea Test HSAT signals (in the absence of EEG signals) to infer sleep stage.

Sleepware G3 software is a polysomnography scoring application, used by trained clinical professionals, for managing data from sleep diagnostic devices using a personal computer. Sleepware G3 is able to configure sleep diagnostic device parameters, transfer data stored in sleep diagnostic device memory to the personal host computer, process and auto-score data to display graphical and statistical analyses, provide aid to clinical professionals for evaluating the physiological data waveforms relevant to sleep monitoring, and create unique patient reports.

Sleepware G3 includes an optional Somnolyzer plug-in. The auto-scoring algorithms of the Somnolyzer Inside software can be used in addition to, or in the place of, the auto-scoring algorithms that are included in Sleepware G3.

Sleepware G3, remains unchanged in function and fundamental scientific technology from Sleepware G3 which was cleared under K142988.

Here's a breakdown of the acceptance criteria and the study proving the device meets those criteria, based on the provided text:

Acceptance Criteria and Reported Device Performance

The acceptance criteria for the Somnolyzer scoring algorithms are based on demonstrating non-inferiority to manual expert scoring. The reported device performance indicates that all primary and secondary endpoints were met.

Detailed Study Information:

1. Sample sizes used for the test set and the data provenance:

   • Test Set Sample Size: A total of 1,204 polysomnography (PSG) and home sleep apnea test (HSAT) files were used in the five clinical studies.
   • Data Provenance: The document does not explicitly state the country of origin. The studies are described as using a "large, diverse sample... collected via a number of different platforms," suggesting diverse sources but not specifying geographical location. The studies were likely retrospective, as they involved validating algorithms against existing manual scoring, but this is not explicitly stated.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Number of Experts: Not explicitly stated how many individual experts were used across all studies. However, the non-inferiority margin for comparisons was "set at the lower-margin of the agreement observed across expert technologists." This implies multiple experts were involved in defining the range of agreement for ground truth.
   • Qualifications of Experts: The experts are referred to as Registered Polysomnographic Technologists (RPSGT). This indicates their professional qualification in sleep study scoring.

3. Adjudication method for the test set:

   • The document implies a form of consensus or agreement among experts was utilized to set the non-inferiority margin, but it does not explicitly describe a specific adjudication method like 2+1 or 3+1 for individual cases within the test set. The focus is on comparing the algorithm's performance against the established range of agreement among experts.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, an MRMC comparative effectiveness study was not the primary focus described. The study design primarily involved a standalone evaluation of the AI algorithm (Somnolyzer) against human expert scoring, demonstrating its non-inferiority.
   • The document states that Somnolyzer's output is "ready for review and interpretation by a physician," implying it assists human readers by providing a pre-scored output. However, it does not quantify the improvement in human reader performance with AI assistance versus without AI assistance.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, a standalone evaluation was performed. The clinical studies "validated the Somnolyzer and CReSS algorithms against manual scoring." The non-inferiority claims ("Somnolyzer scoring... is non-inferior to manual expert scoring") directly refer to the algorithm's performance without human intervention in the scoring process.

6. The type of ground truth used:

   • The ground truth was expert consensus scoring. The document states that the algorithms were validated "against manual scoring" by "expert technologists" (RPSGTs). The non-inferiority margin was based on "the agreement observed across expert technologists."

7. The sample size for the training set:

   • The document does not provide information on the training set sample size. The provided text focuses solely on the clinical performance testing for validation.

8. How the ground truth for the training set was established:

   • As the training set information is not provided, the method for establishing its ground truth is also not described in the document.

Ask a specific question about this device

The Nox Sleep System is used as an aid in the diagnosis of different sleep disorders and for the assessment of sleep.

The Nox Sleep System is used to measure, record, display, organize, analyze, summarize, and retrieve physiological parameters during sleep and wake.

The Nox Sleep System allows the user to decide on the complexity of the study by varying the number and types of physiological signals measured.

The Nox Sleep System allows for generation of user/pre-defined reports based on subject's data.

The user of the Nox Sleep System are medical professionals who have received training in the areas of hospital/clinical procedures, physiological monitoring of human subjects, or sleep disorder investigation.

The intended environments are hospitals, institutions, sleep clinics, or other test environments, including patient's home.

The Nox Sleep System is intended for patients undergoing physiological measurements, for the assessment of sleep quality and the screening for sleep disorders.

The Nox Sleep System does not provide any alarms and is not intended to be used for continuous monitoring where failure to operate can cause injuries or death of the patient.

The basic Nox Sleep System consists of two recording/acquisition devices (Nox A1 Recorder and Nox C1 Access Point), a software running on a PC (Noxturnal PSG), an Android application (Noxturnal App) running on mobile platform, along with sensors and accessories. The system supports full Polysomnography (PSG) studies both in ambulatory and online/attended setups but also more simple sleep study setups, recording only few channels. The ambulatory sleep studies may take place in the clinic or in the home environment, but the online/attended sleep studies are only conducted in the clinical environment.

The Nox A 1 Recorder is a small battery-operated recording unit that is worn by the patient during the study. It records signals from patient applied sensors that connect to the unit but also supports recording of signals from auxiliary devices over Bluetooth. The Nox A1 Recorder allows for communication over Bluetooth with the Noxturnal App during ambulatory setup and with the Nox C1 Access Point during online setup. The recorder is intended to be worn over clothing.

New accessories and sensors as part of this submission are the Nox A 1 EEG 5 Lead Gold Electrode Cable and Nox A1 EEG Head Cable that are used for recording of EEG/EOG. These components are in direct contact with the patient.

The Nox C1 Access Point is a separate mains powered unit located remotely from the patient that allows for recording of signals from auxiliary devices. It supports communication over LAN/Ethernet to the Noxturnal PSG, and communication with the Nox A1 Recorder and Noxturnal App over Bluetooth. The Nox C1 Access Point is only used for online study setup and is thus not intended to be used in the home environment.

The Noxturnal App is used as a mobile interface to the Nox A1 Recorder and Nox C1 Access Point. The communications are via Bluetooth link. The app is normally used in the beginning of a sleep study, for basic tasks such as device configuration, starting a recording, checking the signal quality of signals being recorded and marking events during bio calibration.

The Noxturnal PSG is used for configuration of the Nox recording/acquisition devices, to download a study from ambulatory recording or to collect an online study. The software supports the viewing, retrieving, storing and processing of data recorded/collected, manual and automatic analysis and reporting on the results from the recorded studies. The purpose with the automatic scoring function in Noxturnal PSG is to assist the trained physician in the diagnosis of a patient. It is not intended to provide the trained physician with a diagnostic results. The type of automatic analysis events scored by Noxturnal PSG include: Sleep Stages (Wake, N1, N2, N3, REM), Apneas, Hypopneas, Apnea Cassification (Obstructive, Mixed and Central Apneas), Limb Movements, Periodic Limb Movements, SpO2 Desaturation Events. and potential Bruxism-Related Events.

The result of the automatic analysis/scoring must always be manually verified by the trained physician prior to diagnosis.

The Nox Sleep System is designed to aid in the diagnosis of sleep disorders and assess sleep quality by measuring, recording, displaying, organizing, analyzing, summarizing, and retrieving physiological parameters during sleep and wake. The system includes automatic scoring functionalities for various sleep events, which are intended to assist trained medical professionals in diagnosis.

Here's an analysis of the acceptance criteria and the study proving the device meets them:

1. A table of acceptance criteria and the reported device performance

The document presents separate sections for the performance of different automatic scoring algorithms rather than a single consolidated table. However, the information can be extracted and presented as follows:

Ask a specific question about this device