(218 days)
Sleep Profiler is intended for the diagnostic evaluation by a physician to assess sleep quality in adults only. The Sleep Profiler is a software-only device to be used under the supervision of a clinician to analyze physiological signals and automatically score sleep study results, including the staging of sleep, detection of arousals and snoring.
The Sleep Profiler is a software application that analyzes previously recorded physiological signals obtained during sleep. The Sleep Profiler software can analyze any EDF files meeting defined specifications, including signals acquired with the Advanced Brain Monitoring X4 System which is the subject of a separate 510(k). Automated algorithms are applied to the raw signals in order to derive additional signals and interpret the raw and derived signal information. The software automates recognition of: a) sleep stage, b) snoring frequency and severity, c) pulse rate, d) cortical (EEG), sympathetic (pulse) and behavioral (actigraphy and snoring) arousals. A single channel of electrocardiography, electrooculargraphy, electromyography, or electroencephalography can be optionally presented for visual inspection and interpretation. The software identifies and rejects periods with poor electroencephalography signal quality. The full disclosure recording of derived signals and automated analyses can be visually inspected and edited prior to the results being integrated into a sleep study report. Medical and history information can be input from a questionnaire. Responses are analyzed to provide a pre-test probability of Obstructive Sleep Apnea (OSA) (a condition that cannot be diagnosed with Sleep Profiler) so an appropriate referral to a sleep physician is made. The automated analyses of physiological data are integrated with the questionnaire data, medical and history information to provide a comprehensive report. Several report formats are available depending on whether the user has acquired more than one night of data, wishes to obtain a narrative summary report or provide patient reports.
Here's a breakdown of the acceptance criteria and the study details for the Sleep Profiler device, based on the provided 510(k) summary:
Acceptance Criteria and Device Performance
The acceptance criteria are implied by the comparison to a predicate device, MICHELE (K112102). The goal is to demonstrate "similar" performance. The specific metrics are overall percent agreement and agreement for each sleep stage.
Table 1: Sleep Profiler Performance vs. Predicate Device Performance (Sleep Staging)
| Metric | Sleep Profiler Performance Data (from 44 subjects) | Predicate Device (MICHELE, K112102) Performance Data (from its study) |
|---|---|---|
| Overall % Agreement | Not explicitly stated as an overall value, but individual positive and negative agreements are provided. | 82.6% |
| Agreement by Sleep Stage (Positive Agreement / Sensitivity) | ||
| Wake | 0.79 | 89.9% |
| N1 | 0.40 | 50.4% |
| N2 | 0.80 | 82.9% |
| N3 | 0.76 | 82.9% |
| REM | 0.72 | 89.8% |
| Agreement by Sleep Stage (Negative Agreement / Specificity) | ||
| Wake | 0.95 | 96.4% |
| N1 | 0.91 | 94.7% |
| N2 | 0.83 | 89.6% |
| N3 | 0.97 | 97.5% |
| REM | 0.97 | 98.5% |
The summary states, "The positive and negative percent agreement obtained during clinical validation of the Sleep Profiler are similar to that obtained by the predicate device, MICHELE (K112102), which was validated using a different data set."
Study Details
-
Sample Size used for the test set and the data provenance:
- Test Set Sample Size: 44 subjects.
- Data Provenance: Not explicitly stated, but it's a "clinical validation" comparing to manual observation. It doesn't state whether it's retrospective or prospective, or the country of origin.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Three raters.
- Qualifications: "either sleep technicians or physicians."
-
Adjudication method for the test set:
- The table indicates a "No-Consensus" row for the experts, implying adjudication by consensus was used. Specifically, the "Epochs assigned by Expert Scoring" includes a "No-Consensus" category (653 epochs out of 39191 total), suggesting that if the three raters did not agree, those epochs were excluded from the primary agreement calculations for individual stages. The main performance metrics are likely based on epochs where there was full consensus (3 out of 3, or potentially 2 out of 3 if that's what "consensus" meant here, although the "No Consensus" row suggests agreement was required).
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study was not done. The study's purpose was to validate the "sleep staging algorithms by comparison to sleep staging made by manual observation by three raters." This is a standalone algorithm performance study compared to human experts as ground truth, not a study evaluating human performance with or without AI assistance.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone algorithm performance study was done. The results presented in the table ("Epochs assigned by Sleep Profiler" vs. "Epochs assigned by Expert Scoring") directly report the algorithm's performance without human interaction or modification. The description also states the software "automates recognition" and that the "full disclosure recording of derived signals and automated analyses can be visually inspected and edited prior to the results being integrated into a sleep study report," but the presented validation data is for the automated algorithm's output.
-
The type of ground truth used:
- Expert Consensus. The ground truth for the test set was established by the "manual observation by three raters who were either sleep technicians or physicians."
-
The sample size for the training set:
- Not specified. The document does not provide details about the training set size or how it was established. It only discusses the clinical validation (test set) of the software.
-
How the ground truth for the training set was established:
- Not specified. Since details about the training set are not provided, how its ground truth was established is also not mentioned.
§ 882.1400 Electroencephalograph.
(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).