(163 days)
The NightWare digital therapeutic is indicated to provide vibrotactile feedback on an Apple Watch based on an analysis of heart rate and motion during sleep for the temporary reduction of sleep disturbance related to nightmares in adults 22 years or older who suffer from nightmare disorder or have nightmares from posttraumatic stress disorder (PTSD). It is intended for home use.
The NightWare device consists of a software application and the NightWare server. The device uses an Apple Watch and an Apple iPhone for its platform. Patients must use the prescribed iPhone and Apple Watch pre-provisioned by NightWare, Inc.
The watch monitors physiological parameters that indicate that the patient is experiencing a nightmare. Signals from the watch's heart rate sensor, accelerometer, and gyroscope are input to the onboard algorithms on the watch software and sent to the server and processed by NightWare software. The software on the server calculates a device-specific "Stress Index" from watch measurements of heart rate, rotation, and acceleration. When a "Stress Index" threshold is exceeded, the device provides a vibrotactile stimulation on the patient's wrist intended to interrupt the nightmare but not awaken the patient.
Patients need to have at least an intermittent connection to wireless internet through WiFi in order to transmit data to the NightWare server. However, this wireless internet connection is not needed to receive therapy through the treatment period, as the processing of physiological data and delivery of stimulation occurs on the watch itself.
The "Stress Index" threshold is uniquely calculated for each patient based on an artificial intelligence algorithm. The algorithm differentiates between a patient's normal and abnormal "Stress Index" levels during the night. To create its personalized "Stress Index" threshold, NightWare collects several hours of data during sleep before any intervention is applied. During this initial time, the system monitors the patient's movements and heart rate to delineate the usual low, medium, and high "Stress Index" periods to determine the wearer's usual sleep patterns. The movement and heart rate information is then securely sent to the NightWare server to establish the patient's specific "Stress Index" threshold; when the threshold is reached the vibrotactile stimulation is applied. The "Stress Index" threshold is automatically and periodically updated as needed to accommodate the patient's changing "Stress Index" and thresholds that naturally occur during consistent use.
Here's a breakdown of the acceptance criteria and the study that proves the NightWare device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as numerical targets in the provided text. Instead, they are framed as demonstrating improvement in various sleep and associated disturbance measures compared to a sham system and ensuring no worsening of safety metrics.
| Acceptance Criteria Category | Specific Measure | Reported Device Performance (Active vs. Sham) | Achieved Acceptance? |
|---|---|---|---|
| Primary Effectiveness | Improved sleep as assessed by Pittsburgh Sleep Quality Index (PSQI) | Active arm: 3.2-point improvement; Sham arm: 2.2-point improvement. (Difference not statistically significant, p=0.2606) | No (statistically) |
| Primary Safety | No worsening of daytime sleepiness (Epworth Sleepiness Scale - ESS) | Both Active and Sham arms: 1.2-point decrease (less sleepiness). (Difference not statistically significant, p=0.9739) | Yes (no worsening) |
| Primary Safety | No increase in suicidality (Columbia Suicide Severity Rating Scale - C-SSRS) | Active arm: 0.2-point decrease; Sham arm: no change. (Difference not statistically significant, p=0.2943) | Yes (no worsening) |
| Secondary Effectiveness | Improved sleep disturbance related to nightmares in PTSD (PSQI-Addendum for PTSD - PSQI-A) | Active arm: 3.3-point improvement; Sham arm: 1.4-point improvement. (Difference not statistically significant, p=0.0938) Nevertheless, considered a more useful endpoint for given population. | Yes (beneficial trend observed) |
| Secondary Effectiveness | Changes in PTSD severity (PTSD Checklist - PCL-5) | Active arm: 9.9-point improvement; Sham arm: 6.5-point improvement. (Difference not statistically significant, p=0.2727) | Trend observed |
| Secondary Effectiveness | Changes in nightmare severity (Trauma Related Nightmare Scale - TRNS) | Active arm: 4.8-point improvement; Sham arm: 2.7-point improvement. (Difference not statistically significant, p=0.1909) | Trend observed |
| Adverse Events | No device-related adverse events | Two adverse events reported, neither determined to be device-related. One patient withdrew due to perceived sleep disruption. | Yes |
Note: While statistical significance was not achieved for many primary and secondary endpoints, the FDA's conclusion on benefit-risk acknowledges the observed trends of improvement and the low risk profile of the device, especially for the PSQI-A which is considered more relevant for the target population. Emphasis is placed on the device's acceptable adjunct therapy role and the lack of other cleared non-pharmaceutical options.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size (Test Set): 70 subjects were enrolled (36 in the Active group, 34 in the Sham group). 63 adults completed the study (29 in the Active group, 34 in the Sham group).
- Data Provenance: Prospective, randomized, double-blind, sham-controlled clinical trial. The study was conducted at the Minneapolis Veterans Affairs Health Care System (USA).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The "ground truth" for the effectiveness and safety metrics in this study was established using validated patient-reported questionnaires (e.g., PSQI, ESS, C-SSRS, PCL-5, TRNS, FOSQ-10, VR-12) rather than expert consensus on specific events. A principal investigator or qualified co-investigator conducted risk assessments for suicidal ideation based on PHQ-9 scores, and a licensed psychologist assessed high suicide risk based on C-SSRS responses. The text does not specify the number or detailed qualifications of these individuals beyond "principal investigator or qualified co-investigator" and "licensed psychologist."
4. Adjudication Method for the Test Set
The study was a double-blind trial, meaning neither the subjects nor the study staff directly administering the intervention (Active vs. Sham) knew which treatment arm the subject was in. The outcomes were measured using standardized questionnaires. There is no mention of an independent adjudication committee for the questionnaire results themselves.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not conducted. This type of study is typical for diagnostic devices where human readers interpret medical images or other data with and without AI assistance. The NightWare device is a therapeutic device, and its effectiveness was assessed by measuring changes in patient-reported outcomes over time.
6. Standalone Performance
The study primarily evaluates the device in conjunction with the user (patient-reported outcomes, device providing vibrotactile feedback). The core function of the device is to provide intervention to the human based on physiological data. While the algorithm for "Stress Index" calculation works standalone on the watch and server, its clinical "performance" is inherently tied to its impact on the human user as measured through the listed outcomes. There's no separate "algorithm only" performance reported in terms of accuracy in detecting nightmares or triggering interventions against a specific physiological ground truth.
7. Type of Ground Truth Used
The ground truth used for evaluating the device's effectiveness and safety was patient-reported outcomes obtained through validated questionnaires (e.g., PSQI, ESS, C-SSRS, PSQI-A, PCL-5, TRNS, FOSQ-10, VR-12).
8. Sample Size for the Training Set
The text primarily describes the clinical validation study. It states: "The 'Stress Index' threshold is uniquely calculated for each patient based on an artificial intelligence algorithm. The algorithm differentiates between a patient's normal and abnormal 'Stress Index' levels during the night. To create its personalized 'Stress Index' threshold, NightWare collects several hours of data during sleep before any intervention is applied."
This indicates a patient-specific internal training/calibration phase rather than a large, separate, pre-defined training set for the core AI algorithm parameters itself. The text does not specify a separate, distinct large-scale "training set" sample size for the development of the general AI algorithm within the documentation provided. It focuses on how the algorithm adapts per patient.
9. How the Ground Truth for the Training Set Was Established
For the patient-specific "Stress Index" threshold calibration, the ground truth is established by "collects several hours of data during sleep before any intervention is applied. During this initial time, the system monitors the patient's movements and heart rate to delineate the usual low, medium, and high 'Stress Index' periods to determine the wearer's usual sleep patterns." This constitutes the "ground truth" for that individual's unique sleep profile, allowing the algorithm to personalize its threshold for vibrotactile stimulation.
For the development of the overarching AI algorithm that differentiates normal from abnormal "Stress Index" levels, the document does not explicitly describe how a ground truth was established during its development phase. It implies that the core algorithm learns to differentiate these levels based on physiological data, but the specific process for labeling or curating a dataset for initial algorithm training is not detailed.
§ 882.5705 Digital therapy device to reduce sleep disturbance for psychiatric conditions.
(a) Identification. A digital therapy device to reduce sleep disturbance for psychiatric conditions is a prescription device that is intended to provide stimulation using a general purpose computing platform to reduce sleep disturbance in patients who experience this symptom due to psychiatric conditions such as nightmare disorder or post-traumatic stress disorder.
(b) Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing under the labeled conditions for use must evaluate the following:
(i) The ability of the device to provide therapy for patients with sleep disturbance due to psychiatric conditions, using a validated measure;
(ii) Worsening of any condition-specific symptoms using a validated measure for assessment of the particular condition; and
(iii) Increase in symptoms of disturbed sleep or sleepiness using a validated measure.
(2) Software must clearly describe all features and functions of the software implementing the digital therapy. Software verification, validation, and hazard analysis must also be provided.
(3) The labeling must include the following:
(i) Patient and physician labeling must include instructions for use, including images that demonstrate how to interact with the device;
(ii) Patient and physician labeling must list the minimum operating system and general purpose computing requirements that support the software of the device;
(iii) Patient and physician labeling must include a warning that the digital therapy device is not intended for use as a stand-alone therapeutic device;
(iv) Patient and physician labeling must include a warning that the digital therapy device does not represent a substitution for the patient's medication; and
(v) Physician labeling must include a summary of the clinical performance testing conducted with the device.