Search Results

Daylight is a prescription device delivering Cognitive Behavioral Therapy and can be made available on the order of a licensed healthcare provider. Daylight is a digital therapeutic intended to treat generalized anxiety disorder (GAD) by improving a patient's GAD symptoms as an adjunct to usual care in patients aged 22 years and older.

Daylight is a digital therapeutic designed to address the symptoms of adults with generalized anxiety disorder (GAD) through the use of Cognitive Behavioral Therapy (CBT) techniques. The Daylight program is has been demonstrated to help improve symptoms of GAD, if followed correctly, and is supported by evidence from peer-reviewed studies and clinical trials. The program is delivered digitally through the Daylight iOS/Android apps, giving users easy access to effective techniques.

The document provided focuses on asserting the substantial equivalence of the Daylight device to a predicate device (Somryst) for regulatory clearance, primarily based on the similarity of their intended use, technological characteristics, and a clinical study demonstrating the efficacy of Daylight for its specific indication (Generalized Anxiety Disorder).

This type of submission document (510(k) Summary) does not typically contain detailed information about specific acceptance criteria for device performance in the same way one might describe the performance of a diagnostic AI algorithm against a set of quantitative metrics. Instead, "acceptance criteria" here refer to the overall regulatory requirements for establishing substantial equivalence and demonstrating safety and effectiveness for a digital therapeutic. The primary "proof" of the device meeting these criteria is via the clinical trial results.

Below is an interpretation of "acceptance criteria" in the context of this 510(k) and the information provided to substantiate that the device meets these criteria.

Acceptance Criteria and Device Performance for Daylight

Based on the 510(k) summary, the "acceptance criteria" are implicitly tied to demonstrating safety and effectiveness for the device's intended use and establishing substantial equivalence to a predicate device. For a digital therapeutic, effectiveness is primarily demonstrated through clinical outcomes.

1. Table of Acceptance Criteria and Reported Device Performance

Given the nature of the device (a digital therapeutic delivering CBT for GAD), the acceptance criteria are not in the form of typical quantitative performance metrics like sensitivity/specificity for a diagnostic device. Instead, they relate to:

• Clinical Efficacy: Improvement in GAD symptoms.
• Safety: Acceptable adverse event profile.
• Substantial Equivalence: Alignment with predicate device in terms of intended use, technology, and risk profile.

2. Sample Size and Data Provenance

• Test Set (Clinical Trial Data):
  • Sample Size: 351 adults, randomized 1:1, with 175 assigned to Daylight and 176 to online anxiety psychoeducation.
  • Data Provenance: Participants were recruited from across the United States via social media. The study was a two-arm, parallel group, randomized controlled trial (RCT), indicating prospective data collection.

3. Number of Experts and their Qualifications (for Ground Truth)

• Not applicable in this context. The device is a digital therapeutic for generalized anxiety disorder, and the "ground truth" for its effectiveness is based on patient-reported outcomes (GAD-7, PHQ-8, SCI-8, OASIS), and clinician-reported outcomes (CGI-I, CGI-S) from a Randomized Controlled Trial, rather than expert interpretation of images or other data typically requiring consensus for ground truth. The "experts" involved would be the clinicians conducting the CGI assessments and diagnosing GAD, but the document does not specify their number or qualifications.

4. Adjudication Method for the Test Set

• Not applicable in this context. As the effectiveness is measured through standardized questionnaires and clinical scales in an RCT, there is no need for expert adjudication of cases/data points in the same way as, for example, reviewing medical images.

5. MRMC Comparative Effectiveness Study

• No, this was not an MRMC study. This was a randomized controlled trial (RCT) comparing a digital therapeutic (Daylight) to an active control (online anxiety psychoeducation).
• Effect Size of Human Readers Improvement: This concept is not relevant to this type of device or study. The study investigates the effect of the digital therapeutic on patient symptoms, not how AI assists human readers in a diagnostic task. The "human readers" in this context are the patients interacting with the digital therapeutic.

6. Standalone Performance Study

• Yes, in the sense that the clinical trial evaluated the performance of the Daylight algorithm/program as a standalone therapeutic intervention. The study compared Daylight (the intervention) to a psychoeducation control. The device is a "Software as a Medical Device (SaMD)" intended to treat GAD. Its performance is its ability to improve GAD symptoms.
• The study design evaluated the device's direct therapeutic effect on participants, not its ability to assist a human in performing a diagnostic or interpretive task.

7. Type of Ground Truth Used

• Clinical Outcomes / Patient-Reported Outcomes (PROs) and Clinician-Reported Outcomes (CROs):
  • Primary:
    • Generalized Anxiety Disorder 7-item questionnaire (GAD-7) for patient-reported symptom severity.
    • Clinical Global Impression - Improvement scale (CGI-I) for remission (scores of 1 or 2).
  • Secondary:
    • Patient Health Questionnaire (PHQ-8) for depression symptoms.
    • Sleep Condition Indicator (SCI-8) for insomnia symptoms.
    • Clinical Global Impression - Severity (CGI-S) for anxiety severity.
    • Overall Anxiety Severity and Impairment Scale (OASIS).
  • Diagnosis of GAD at baseline according to DSM-5 criteria.

8. Sample Size for the Training Set

• Not explicitly stated in the 510(k) summary. For digital therapeutics based on established CBT principles, a traditional "training set" for a machine learning model might not be applicable in the same way it is for image-based AI diagnostics. The device's "training" might refer to the development and refinement of the CBT content and delivery structure, which is typically based on clinical psychology principles and past research, rather than a data-driven machine learning training set of patient data to optimize an algorithm. The clinical trial serves as the primary validation.

9. How Ground Truth for the Training Set Was Established

• Not applicable / Not detailed in the document. As mentioned above, the "training set" concept is different for this type of device. The "ground truth" for developing the therapeutic content of Daylight would be the established principles of Cognitive Behavioral Therapy for anxiety, supported by decades of psychological and clinical research. The document highlights that Daylight "is supported by evidence from peer-reviewed studies and clinical trials," implying that its efficacy framework is built upon existing scientific understanding of CBT. The GATE trial served as the definitive evaluation of the product's effectiveness as developed.

Ask a specific question about this device

Sleepio is a digital therapeutic intended for the treatment of chronic insomnia disorder as an adjunct to usual care in patients aged 18 and older. Sleepio is a prescription device delivering Cognitive Behavioral Therapy for Insomnia (CBT-I) and can be made available on the order of a licensed healthcare provider.

Sleepio is a digital therapeutic for the treatment of chronic insomnia / insomnia disorder. Sleepio treats chronic insomnia disorder by delivering evidence-based techniques targeting the cognitive and behavioral factors that maintain insomnia and chronic sleep problems. Patient experience is tailored based on symptoms and daily sleep tracking. In addition to core therapeutic components, there is in-the-moment therapeutic content for help falling asleep. Content is delivered via smartphone and tablet applications (iOS and Android), as well as via web. Sleepio is intended as an adjunct to usual care treatment for chronic insomnia / insomnia disorder by a healthcare provider. Healthcare providers have access to a dashboard to track patient engagement with Sleepio.

Based on the provided text, the device in question is Sleepio, a digital therapeutic for the treatment of chronic insomnia disorder. Here's a breakdown of the acceptance criteria and the study proving the device meets them:

Disclaimer: The provided document is an FDA 510(k) summary, which focuses on demonstrating substantial equivalence to a predicate device rather than setting explicit acceptance criteria for this specific device's performance in a new, de novo fashion. The clinical data presented serves to support the substantial equivalence claim by showing similar or superior performance to standard care (Sleep Hygiene Education, SHE) and implicitly, to the predicate. Therefore, the "acceptance criteria" table below is inferred from the goals of the clinical study and the claims made about the device's efficacy in the context of demonstrating substantial equivalence for an FDA 510(k) submission.

1. Table of inferred acceptance criteria and the reported device performance

2. Sample size used for the test set and the data provenance

• Test set sample size: The clinical effectiveness was evaluated in the CrEDIT trial, a two-arm, parallel group, randomized controlled trial (RCT) with 336 adults.
  • Sleepio group (n=168)
  • Online sleep hygiene education (SHE) group (n=168)
• Data provenance: Participants were recruited from across the United States via social media. The study was prospective (Randomized Controlled Trial).

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The document does not specify the use of human experts to establish ground truth for the test set in the conventional sense of image or diagnostic adjudication. The ground truth for insomnia was established clinically based on self-reported patient assessments using standardized questionnaires (Insomnia Severity Index, ISI) and sleep diaries (SOL, WASO), and a DSM-5 diagnosis of insomnia disorder as criteria for study inclusion. The study evaluated the effectiveness of a therapeutic intervention, not the diagnostic accuracy of an AI model against expert consensus.

4. Adjudication method for the test set

Not applicable. As noted above, this study measures the therapeutic effectiveness of Sleepio based on patient-reported outcomes, not the performance of a diagnostic or classification algorithm that would require an adjudication method for ground truth.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study was not done. This was a clinical trial evaluating a digital therapeutic directly delivered to patients for treatment, not an AI-assisted diagnostic tool for human readers. Therefore, the concept of "human readers improve with AI vs without AI assistance" does not apply.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, in a way. Sleepio is a digital therapeutic that delivers Cognitive Behavioral Therapy for Insomnia (CBT-I) directly to patients via smartphone/tablet apps and web. While it's a "prescription device" and "can be made available on the order of a licensed healthcare provider" who "have access to a dashboard to track patient engagement," the core therapeutic delivery and patient interaction are directly from the algorithm/software. The study design directly assesses the effect of this standalone digital intervention compared to sleep hygiene education.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth or clinical endpoints were based on patient-reported outcomes data derived from:

• Insomnia Severity Index (ISI) scores: A standardized, self-reported questionnaire.
• Sleep diary data: Self-reported sleep onset latency (SOL) and wake after sleep onset (WASO).
• DSM-5 diagnosis of insomnia disorder: Used as an inclusion criterion for participants.
• Sleep Condition Indicator (SCI-8): A secondary outcome measure.

8. The sample size for the training set

The document does not specify a training set sample size in the context of an AI/ML model for this device. Sleepio delivers Cognitive Behavioral Therapy for Insomnia (CBT-I), which is an evidence-based therapeutic approach. The "training" for such a device would typically relate to the development of the CBT-I content and algorithms based on established clinical guidelines and research in psychology and sleep medicine, rather than a machine learning training dataset in the typical sense for image recognition or diagnostic AI. The software verification and validation are noted, but this refers to engineering and quality assurance processes, not a machine learning training curriculum.

9. How the ground truth for the training set was established

Not applicable directly, as this is not described as a device that uses a trained AI model in the sense of predictive or diagnostic capabilities. Sleepio is a digital delivery mechanism for an established therapy (CBT-I). The "ground truth" for its therapeutic approach is the established principles and efficacy of CBT-I as validated in clinical literature and practice over many years. The clinical study (CrEDIT trial) served as a validation study to demonstrate that Sleepio's digital implementation of CBT-I is effective, rather than a study to establish ground truth for a novel AI algorithm's training.

Ask a specific question about this device

MamaLift Plus is a prescription-only digital therapeutic intended to provide neurobehavioral interventions to patients 22 years of age and older, as an adjunct to clinician-managed outpatient care. MamaLift Plus treats mild to moderate postpartum depression by improving a patient's symptoms of depression.

MamaLift Plus is a digital therapy designed to treat symptoms of postpartum depression by delivering evidence-based therapeutic components of Cognitive Behavioral Therapy (CBT) via software on a mobile application (smartphone or tablet). MamaLift Plus is indicated as a behavioral health intervention for patients 22 years of age and older with mild to moderate symptoms of depression by improving their symptoms of depression. As with face- to-face CBT, MamaLift Plus uses personalized cognitive restructuring as the main therapeutic component to improve the symptoms of postpartum depression. This element is mapped onto standard, evidence-based CBT interventions that are developed for and provided by a therapist in a face-toface care setting with a patient. The content is conveyed via a sequence of eight self-guided and interactive treatment modules daily over a period of eight to nine weeks sequentially.

Patients are encouraged to complete all eight modules of the MamaLift Plus application at the rate of one module per week. However, the entire program can last up to 9 weeks, inclusive of the baseline and posttreatment assessment periods. Full engagement by prescribing clinicians and their support staff facilitates effective use by patients of the prescribed treatment and their continued use in conjunction with ongoing monitoring by the clinician.

The content of MamaLift Plus is delivered through a variety of features which include text, personalized goal setting, graphical feedback based on inputted symptoms, and illustrations to enrich comprehension, quizzes to test and enhance user knowledge, video vignettes to promote user identification with material, and video-based expert explanations. Periodic notifications or "nudges" are also sent to increase user engagement and encourage program adherence. Additional features of MamaLift Plus include a daily tracker in which patients can self- monitor and record standardized sleep parameters (e.g., sleep and naps), self-reports of perceived Sleep Quality, and self-reports of perceived Energy Level. In addition, MamaLift Plus offers a daily mood tracker and activity tracker. The application provides personalized cognitive restructuring guidance based on the individual's beliefs, context, and attitudes.

To facilitate its use as an adjunct to outpatient standard of care conducted under the supervision of a qualified health care provider, MamaLift Plus includes a clinician facing dashboard summarizes patient use of the mobile application during the treatment period thereby enabling the clinician to assess and monitor their patient's progress throughout the therapeutic period. Clinical data collected via the patient interface, including self-reports of depressive symptoms/moods, and sleep data, are also displayed via the clinician dashboard. (All data are encrypted and compliant with data privacy and patient confidentiality requirements of the Health Insurance Portability Act.) These features are intended to support the clinician and enable patient follow up, engagement and communication of healthcare decisions. This facilitates treatment adherence and achievement of optimal patient outcomes. These are critical components that demonstrate the effectiveness of MamaLift Plus.

The provided text describes the MamaLift Plus device, its intended use, and the studies conducted to demonstrate its safety and effectiveness.

Here's an analysis based on your requested information:

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for MamaLift Plus are based on improvements in the Edinburgh Postpartum Depression Scale (EPDS) scores.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size:
  • Total Enrolled: 141 participants
  • Intent to Treat (ITT) Analysis Set: MamaLift Plus: 95; Sham Control: 46
  • Full Analysis Set (FAS): MamaLift Plus: 90; Sham Control: 40
  • Evaluable Population (EP): MamaLift Plus: 78; Sham Control: 38
• Data Provenance: The study was a "pivotal, remote, sham controlled, randomized study" conducted in the USA, enrolling participants from 33 states. The study utilized prospective data collection.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The ground truth for participant eligibility (diagnosis of PPD) was established by a "confirmatory clinical diagnosis of PPD that was confirmed by licensed behavioral health therapist or medical professional." The number of these experts involved in confirming the diagnosis for the test set is not specified.

4. Adjudication Method for the Test Set

The document does not explicitly describe an adjudication method like 2+1 or 3+1 for the test set. The clinical diagnosis of PPD was confirmed by licensed professionals, but there's no mention of multiple experts reviewing cases or a specific method for resolving discrepancies.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study Was Done, If So, What Was the Effect Size of How Much Human Readers Improve with AI vs Without AI Assistance

No, a Multi Reader Multi Case (MRMC) comparative effectiveness study was not done. This study focuses on the effectiveness of a digital therapeutic (MamaLift Plus) as an adjunct to clinician-managed outpatient care, not on improving human reader performance with AI assistance.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

The study assessed the performance of MamaLift Plus plus Treatment as Usual (TAU) compared to Sham Control plus TAU. While MamaLift Plus is a digital therapeutic, its intended use is "as an adjunct to clinician-managed outpatient care" and "under the supervision of a clinician." Therefore, it is not a standalone algorithm-only performance study without human-in-the-loop, as ongoing monitoring and clinical oversight are part of its intended use.

7. The Type of Ground Truth Used

The ground truth for the efficacy evaluation was based on clinical assessment scores from the Edinburgh Postpartum Depression Scale (EPDS) and confirmed clinical diagnosis of PPD by licensed behavioral health therapists or medical professionals.

8. The Sample Size for the Training Set

The document does not provide information about a separate training set or its sample size. The clinical study described is a pivotal trial for effectiveness, indicating it's likely the primary method of validating the device's performance rather than a training dataset for an AI model. MamaLift Plus uses "personalized cognitive restructuring as the main therapeutic component," derived from "standard, evidence-based CBT interventions." This suggests the content is based on established therapeutic practices rather than a machine learning model trained on a specific dataset.

9. How the Ground Truth for the Training Set Was Established

Since no training set is explicitly mentioned for an AI model, the method for establishing its ground truth is not applicable based on the provided text. The therapeutic content stems from evidence-based Cognitive Behavioral Therapy (CBT) principles.

Ask a specific question about this device

Rejoyn is a prescription digital therapeutic for the treatment of Major Depressive Disorder (MDD) symptoms as an adjunct to clinician-managed outpatient care for adult patients with MDD aged 22 years and older who are on antidepressant medication. It is intended to reduce MDD symptoms.

Rejoyn (also known as CT-152) is a digital therapeutic smartphone application (app) for the treatment of Major Depressive Disorder (MDD) symptoms. Rejoyn is a prescription smartphone app-based digital therapeutic administered to a user via the user's smartphone device (running Apple iPhone operating system [iOS®] or Android™ operating system [OS]), which delivers a proprietary interactive cognitiveemotional and behavioral therapeutic intervention. The core components of Rejoyn are the Emotional Faces Memory Task (EFMT) exercises, brief cognitive behavioral therapy (CBT)-based lessons to learn and apply key therapeutic skills, and short message service (SMS) text messaging to reinforce CBT-based lesson content and to encourage engagement with the app. It is intended for the treatment of MDD symptoms as an adjunct to clinician-managed outpatient care for adult patients with MDD aged 22 years and older. It is intended to reduce MDD symptoms.

Rejoyn is designed for use as an adjunct to clinician-managed outpatient care over a period of 6 weeks for the treatment of MDD symptoms, followed by a 4-week extension period where CBT-based lesson content will be accessible but no new therapeutic content or EFMT exercises will be available. Rejoyn is not intended to be used as a stand-alone therapy or as a substitution for the patient's clinician prescribed medications.

Here's a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Device Name: Rejoyn™
Regulation Number: 21 CFR 882.5801
Regulation Name: Computerized Behavioral Therapy Device For Psychiatric Disorders
Regulatory Class: Class II

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for Rejoyn are defined by the "Special Controls" for computerized behavioral therapy devices for psychiatric disorders. These special controls mandate clinical data and detailed software documentation. The device's performance is demonstrated through the Mirai trial.

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size (Test Set):

  • Intent-To-Treat (ITT) population: 386 participants (194 Rejoyn, 192 Sham)
  • Modified Intent-To-Treat (mITT) population: 354 participants (177 Rejoyn, 177 Sham) (This was the primary population for the primary efficacy endpoint analysis).
  • Safety Sample: 373 participants (187 Rejoyn, 186 Sham)

• Data Provenance: The Mirai trial (NCT04770285) was a pivotal, multicenter, remote, double-blinded (patients also blinded to hypothesis), randomized, controlled trial. The study was conducted virtually, with participants across multiple centers, implying a prospective and multi-site data collection. No specific country of origin is mentioned, but "multicenter" typically implies multiple sites within a region (e.g., US).

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

The ground truth for the clinical effectiveness was established through commonly used and validated psychiatric assessment scales:

• Montgomery-Asberg Depression Rating Scale (MADRS): This is a clinician-rated scale. The study states the benefit was "consistently rated by independent assessors via the MADRS," indicating multiple clinicians likely contributed to these assessments. Specific number and qualifications are not detailed, but it is implied they are qualified clinicians for psychiatric assessment.
• Clinical Global Impression-Severity Scale (CGI-S): This is also a clinician-rated scale, where benefit was "rated by study investigators via the CGI-S." Again, specific numbers and qualifications of these "study investigators" are not explicitly stated, but they would be medical professionals involved in the clinical trial.

4. Adjudication Method for the Test Set

The text indicates that the trial was "double-blinded (patients also blinded to hypothesis)" and assessments were made by "independent assessors" (for MADRS) and "study investigators" (for CGI-S). There is no explicit mention of an adjudication method like 2+1 or 3+1 for resolving discrepancies in assessments. However, the use of "independent assessors" for the primary outcome measure (MADRS) suggests a measure to reduce bias.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, an MRMC comparative effectiveness study was not done in the context of radiologists or similar image interpretation professions. This device is a digital therapeutic for psychiatric disorders, not an imaging diagnostic tool requiring multiple readers to interpret cases. The effectiveness study compared the device (Rejoyn) to a Sham control group, not human readers with and without AI assistance.

6. Standalone Performance

Yes, a standalone (algorithm only without human-in-the-loop performance) study was effectively done. Rejoyn is a "prescription digital therapeutic" that provides "proprietary interactive cognitive-emotional and behavioral therapeutic intervention" directly to the user via a smartphone app. The trial design assessed the effectiveness of this app-based intervention (Rejoyn) against a Sham app, with both groups continuing "clinician-managed outpatient care" and "antidepressant medication." The primary efficacy endpoint measured the change in MADRS total score directly attributable to the Rejoyn app's use as an adjunct, demonstrating its standalone contribution to reducing MDD symptoms beyond standard care.

7. Type of Ground Truth Used

The ground truth was based on expert clinical assessments and patient-reported outcomes using validated scales:

• Clinician-rated scales: Montgomery-Asberg Depression Rating Scale (MADRS) and Clinical Global Impression-Severity Scale (CGI-S).
• Patient-reported outcomes (PROs): Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7).

These are standard, widely accepted measures for assessing depressive and anxiety symptoms in clinical trials.

8. Sample Size for the Training Set

The provided document describes a pivotal clinical trial (Mirai Trial) used for validation. It does not provide details about a training set for the development of the Rejoyn algorithm itself. Digital therapeutics often undergo iterative development and testing, but the specifics of a "training set" in the machine learning sense are not included in this regulatory summary, which focuses on the clinical validation of the final product.

9. How the Ground Truth for the Training Set Was Established

As mentioned above, the document does not include information about a "training set" for the algorithm itself. The focus is on the clinical validation of the device's effectiveness using the Mirai trial. If Rejoyn's "proprietary interactive cognitive-emotional and behavioral therapeutic intervention" involves machine learning components that were "trained," the methods and ground truth for that training are not detailed in this 510(k) summary. The summary highlights that the software documentation demonstrates Rejoyn "effectively implements the behavioral therapy model," suggesting the model itself is based on established therapeutic principles (EFMT and CBT).

Ask a specific question about this device

Somryst is a prescription-only digital therapeutic intended to provide a neurobehavioral intervention (Cognitive Behavioral Therapy for Insomnia - CBT-I) in patients 22 years of age and older with chromic insomnia. Somryst treats chronic insomnia by improving a patient's insomnia symptoms.

Somryst is a 9-week prescription digital therapeutic (computerized behavioral therapy) used in the treatment of chronic insomnia. Somryst is based on principles of Cognitive Behavioral Therapy (CBT) for Insomnia, Sleep Restriction, and other proven psychosocial treatment elements, which are delivered in a sequence of "cores" of patient education, training and skill building. The therapy is delivered via a mobile application intended to be used on a patient's mobile device and consists of text, video, animation and graphics. Clinicians, as part of a patient's general treatment program, have access to a clinician dashboard that shows patient utilization and engagement with the application.

Here's an analysis of the acceptance criteria and the studies that prove Somryst meets these criteria, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

The FDA clearance document for Somryst does not explicitly list "acceptance criteria" in a typical quantitative pass/fail format with predefined thresholds for performance metrics. Instead, it demonstrates the device's efficacy by comparing its performance to a control group in several clinical studies. The implicit acceptance criterion is that Somryst should significantly improve insomnia symptoms compared to usual care or a control.

Below is a summary of the reported device performance from the two principal clinical studies, showing the statistically significant improvements.

2. Sample Size for Test Set and Data Provenance

The "test set" in this context refers to the participants in the two clinical trials where the effectiveness of Somryst (referred to as SHUTi in the studies) was evaluated.

• "The GoodNight Study" (ANU Study):

  • Sample Size (Test Set):
    • UC+SHUTi group: N=574
    • UC+Control group: N=575
    • Total participants mentioned in some tables (e.g., ISI, Week 9): UC+SHUTi (250), UC+Control (342). These are "available patient data," possibly reflecting those who completed the assessments at that specific timepoint.
  • Data Provenance: Australia (Australian National University, Black Dog Institute, University of Sydney). Prospective randomized controlled trial.

• Pivotal Trial (UVA Study - NCT01438697):

  • Sample Size (Test Set): Total N=303 adults with chronic insomnia.
    • UC+SHUTi group: N=151
    • UC+Control group: N=152
    • Total participants mentioned in some tables (e.g., ISI, Week 9): UC+SHUTi (133), UC+Control (142).
  • Data Provenance: United States (University of Virginia). Prospective randomized controlled trial.

3. Number of Experts and Qualifications for Ground Truth

The document does not describe the use of experts to establish a "ground truth" for the test set in the traditional sense of image annotation or diagnostic consensus. Instead, the ground truth for measuring insomnia severity and outcomes was based on:

• Patient-reported outcomes: Insomnia Severity Index (ISI) questionnaires. The ISI is a validated scale.
• Patient-reported sleep diaries: These were administered online and collected for 10 days within a 2-week window at each assessment time point. These diaries were used to calculate objective-like sleep parameters such as Sleep Onset Latency (SOL) and Wake After Sleep Onset (WASO).

The ISI itself incorporates a defined scoring system for severity and remission, which acts as a standardized "ground truth" established by clinical consensus (as referenced by published psychometric studies of the ISI).

4. Adjudication Method for the Test Set

No explicit adjudication method is mentioned. The primary outcome measures (ISI scores, sleep diary data for SOL and WASO) are quantitative and collected directly from patients. Clinical intervention studies like these typically rely on these validated instruments rather than expert adjudication of individual case outcomes after the trial.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, an MRMC comparative effectiveness study was not done. Somryst is a computerized behavioral therapy device designed to be used by patients, not an AI diagnostic tool that assists human readers in interpreting medical images or data. Therefore, the concept of assessing how much human readers improve with AI assistance is not applicable to this device.

6. Standalone Performance

Yes, a standalone performance study was done. The entire evaluation of Somryst (SHUTi in the studies) was based on its standalone performance in improving insomnia symptoms in patients directly. The comparator was either "Usual Care" plus an attention-matched digital control or an un-tailored digital education program, allowing for the isolation of Somryst's therapeutic effect. The device (SHUTi) itself delivered the intervention without human-in-the-loop assistance for the core therapy delivery. Clinicians had access to a dashboard, but this was for monitoring and management, not direct therapeutic intervention via the device.

7. Type of Ground Truth Used

The ground truth used was primarily patient-reported outcomes and validated psychometric scales:

• Insomnia Severity Index (ISI): A validated questionnaire for assessing insomnia severity. The definitions of "responder" (>=7-point ISI reduction) and "remitter" (ISI score < 8) are based on established clinical criteria derived from psychometric studies of the ISI.
• Sleep diaries: Patient-reported logs of sleep parameters like sleep onset latency (SOL) and wake after sleep onset (WASO). These provide quantitative, if subjective, measures of sleep behavior.

8. Sample Size for the Training Set

The document does not provide a specific "training set" for the Somryst algorithm in the context of machine learning. Somryst is described as a digital therapeutic based on "principles of Cognitive Behavioral Therapy (CBT) for Insomnia, Sleep Restriction, and other proven psychosocial treatment elements." This implies that the therapy logic and content are based on established medical and psychological knowledge (CBT-I protocols) rather than being developed through a data-driven machine learning training process.

The clinical studies described (ANU and UVA studies) are effectiveness studies, essentially serving as a "test set" to validate the established therapeutic approach delivered by the device.

9. How Ground Truth for the Training Set Was Established

As noted above, there isn't a traditional "training set" with ground truth in the machine learning sense. The "ground truth" for the content and principles of Cognitive Behavioral Therapy for Insomnia (CBT-I) and other psychosocial treatment elements, which form the basis of Somryst, would have been established over decades of clinical research, expert consensus, and efficacy studies in the field of sleep medicine and psychology. It represents established medical knowledge and best practices for treating insomnia.

Ask a specific question about this device

reSET-O™ is intended to increase retention of patients with opioid use disorder (OUD) in outpatient treatment by providing cognitive behavioral therapy, as an adjunct to outpatient that includes transmucosal buprenorphine and contingency management, for patients 18 years or older who are currently under the supervision of a clinician. reSET-O is indicated as a prescription-only digital therapeutic.

reSET-O™ is a 12-week interval prescription digital therapeutic for Opioid Use Disorder (OUD). reSET-O™ is modeled on the Community Reinforcement Approach (CRA) and engineered to deliver behavioral therapy for patients with OUD. reSET-O™ delivers CRA therapy as a series of interactive therapy lessons. Each therapy lesson is comprised of a cognitive behavioral therapy component and skill building exercises. Therapy lesson content is delivered primarily via text or audio, and may include videos, animations and graphics.

reSET-O™ is intended as an adjunct to standard of care for patients with OUD. It is limited to persons with a valid prescription from their licensed provider. reSET-O™ supports clinician-patient communication between visits, by providing a means for patients to self-report cravings and triggers, and buprenorphine use/non-use. reSET-O™ reinforces the importance of using buprenorphine for treatment of OUD.

Acceptance Criteria and Study for reSET-O

1. Table of Acceptance Criteria and Reported Device Performance

Note: While the study also evaluated abstinence, it was stated that "The ability of reSET-O to improve abstinence has not been established as clinically significant." Therefore, enhanced abstinence is not listed as an acceptance criterion met by the device.

2. Sample Size and Data Provenance

• Sample Size for Test Set: 170 patients
• Data Provenance: The document does not explicitly state the country of origin. The study was a randomized clinical trial, indicating a prospective study design.

3. Number of Experts and Qualifications for Ground Truth

The document does not provide information on the number of experts used to establish ground truth or their specific qualifications for the test set. The ground truth appears to be based on the clinical trial outcomes (dropout rates, abstinence) rather than expert consensus on individual cases.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for the test set. The outcomes (dropout and abstinence) were measured directly during the randomized clinical trial.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly described in the provided text. The study design was a randomized controlled trial comparing the reSET-O device (referred to as TES) plus Treatment As Usual (TAU) against TAU alone. This is a direct comparison of a treatment arm with the device versus a control arm, not an MRMC study comparing human readers with and without AI assistance.

6. Standalone Performance

The study design assessed the effectiveness of reSET-O (TES) as an adjunct to Treatment As Usual (TAU) compared to TAU alone. It does not evaluate the device's standalone performance without human-in-the-loop (i.e., without the adjunctive standard of care). The device is explicitly indicated as an "adjunct to outpatient treatment."

7. Type of Ground Truth Used

The ground truth used was based on clinical outcomes data from a randomized controlled trial:

• Treatment Dropout: Defined as the percentage of patients who discontinued treatment during the 12-week intervention.
• Abstinence: Defined as the "longest documented period of continuous abstinence from opioids and cocaine for each participant," measured through thrice-weekly urine drug screens (UDS).

8. Sample Size for the Training Set

The document does not provide information about a separate training set or its sample size. The clinical study described served as the validation (test) set for the device's effectiveness.

9. How Ground Truth for the Training Set Was Established

As no separate training set is mentioned, the method for establishing its ground truth is not provided. The provided text focuses on the clinical validation study.

Ask a specific question about this device

reSET is intended to provide cognitive behavioral therapy, as an adjunct to a contingency management system, for patients 18 years of age and older who are currently enrolled in outpatient treatment under the supervision of a clinician. reSET is indicated as a 12 week (90 days) prescription-only treatment for patients with substance use disorder (SUD), who are not currently on opioid replacement therapy, who do not abuse alcohol solely, or who do not abuse opioids as their primary substance of abuse. It is intended to:

• increase abstinence from a patient's substances of abuse during treatment, and ●
• increase retention in the outpatient treatment program.

reSET™ is a digital therapy comprised of a patient application and clinician dashboard intended to deliver cognitive behavioral therapy (CBT) to patients with SUD to increase abstinence from substance use and increase retention in outpatient therapy programs. CBT is a psychosocial intervention that aims to change a patient's thinking and behavior, and it has been studied in psychiatric disorders such as major depressive disorder (Psychiatr Clin North Am. 2010; 33: 537-55). reSET is based on a specialized version of CBT known as the community reinforcement approach (CRA). which was originally developed for alcohol dependence and cocaine use (Behav Res Ther. 1973; 11:91-104; Exp. Clin. Psychopharmacol 2000; 3:205–2). The community reinforcement incorporates a range of therapeutic modalities including CBT to make substance-free lifestyle rewarding, skill building to promote behavioral change, and contingency management to reward and incentivize abstinence and replace the satisfaction obtained from substance abuse. CBT and CRA are considered valid models for substance abuse therapy and other psychiatric disorders.

reSET consists of several therapy lessons (modules) that are intended to teach the user the following skills to aid in the treatment of substance use disorder:

• Identifying situations and triggers that make substance use more likely ●
• . Avoiding substance use.
• . Coping with thoughts about substance use,
• . Recognizing negative thinking and identifying techniques to move to positive thinking
• Making decisions about substance use
• Taking responsibility for choices made and evaluating the consequences of those choices ●

Each therapy lesson is comprised of a cognitive behavioral therapy component and skill building exercises related to the above areas. The content of the therapy lessons is delivered primarily via text, and may include videos, animations and graphics. All text and video within reSET is narrated in English.

Following most therapy lessons, the patient undergoes fluency learning, a method of questioning that intended to promote learning and improve both short-term and long-term retention of material. Within the fluency learning section of reSET, patients are asked between 4-10 multiple choice and fill-in-the-blank questions about the key concepts presented in the lesson. An example of screenshots from the device is provided in the figure below.

When a question is answered incorrectly, the patient is presented with the correct response and the question is recycled back into the queue and asked again. In order to successfully complete the fluency learning section, patients must answer each question correctly three times, providing a repetitive component that reinforces concept mastery. If the patient completes the lesson and demonstrates proficiency, they can "spin the wheel" for virtual rewards that may be incorporated into a contingency management program at the treating physician's clinic.

reSET is comprised of 62 lessons, including one on-screen User Guide that explains how to use the reSET app and 61 therapy lessons. When a patient uses reSET for the very first time, he or she must step through the User Guide session to ensure they understand how to use the app. The 61 therapy lessons are split into 31 core therapy lessons and 30 supplemental therapy lessons. The therapy lessons include categories related to life skills, treatment, mood matters, social connections, sexual health, and hepatitis C and HIV.

The therapy lessons in the core therapy lesson group are focused on building basic cognitive behavioral and relapse prevention skills (e.g., functional analysis of drug use and selfmanagement planning, drug refusal skills). The therapy lessons in the supplemental group cover a range of topics that can be relevant for patients with SUD such as managing relationships, building communication skills, and time management. They also provide more in-depth training on HIV, hepatitis and STI prevention as well as support for those patients living with HIV and Hepatitis C.

Once the initial User Guide lesson has been completed, the patient gains access to the next core therapy lesson. In the core therapy lessons, a patient can only advance to the next lesson after successfully completing the prior lesson. At any time, patients can choose to review a completed lesson. Once a patient successfully completes all the core therapy lessons, they gain access to all the supplemental therapy lessons. These lessons do not have a set order of completion, and patients can choose lessons that are relevant to managing their disease or as recommended by their clinician.

reSET recommends that patients should complete 4 lessons per week. Each lesson is intended to take between 10-20 minutes to complete. Therapy lesson lengths vary, as do the number of fluency assessment questions the patient must take at the end of a lesson. Some therapy lessons have optional worksheets for the patient to complete that are intended to help the patient understand the key concepts taught in the therapy lesson.

The reSET application allows patients to track their own progress on the device's therapy modules. The device additionally has a Patient Self Report interface that allows for patients to track their cravings and substance use. This information is available to the patient's treating physician through the device.

The clinician dashboard for reSET displays the patient's progress. The clinician can view which therapy lessons the patient has completed, as well as view patient-reported substance use, cravings and triggers. The reSET app automatically pushes an assessment every four days to ask if the patient has taken any drugs or alcohol in the past 4 days, and if so, on which days. The patient is also asked to report whether they have had any cravings for drugs and alcohol and if so, to rate the intensity of the cravings. This reSET initiated, self-report data will display on the clinician's dashboard. Patients may also log their drug and alcohol use at any time into reSET along with tracking their cravings and distinct triggers. These use, craving and trigger data are presented to the clinician. The clinician can also enter in-clinic data inputs such as urine drug screens and appointment attendance.

reSET is intended be used in conjunction with a contingency management incentives system. reSET provides virtual "rewards" when the patient completes a lesson successfully as well as when their urine drug screen, or other objective test, is negative for substances. Clinics mav convert the virtual "rewards" into tangible rewards according to their own procedures.

The reSET device is a software-based medical device, and its acceptance criteria and clinical performance are based on a randomized controlled trial. Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance

The acceptance criteria for reSET, a computerized behavioral therapy device for psychiatric disorders, were implicitly defined by the statistically significant improvements shown in the clinical trial. The device was deemed acceptable if it demonstrated a benefit over "treatment as usual" (TAU) in its primary outcome measures, especially in cohorts relevant to its indicated use.

2. Sample size used for the test set and the data provenance

• Test Set (Clinical Study):
  • Cohort 1 (All Comers): 507 participants (252 in TAU, 255 in rTAU + reSET)
  • Cohort 2 (Excluding Primary Opioids): 399 participants (193 in TAU, 206 in rTAU + reSET)
  • Cohort 3 (Excluding All Opioids): 305 participants (153 in TAU, 152 in rTAU + reSET)
• Data Provenance: The study was a multi-site, randomized clinical trial (National Institute of Drug Abuse CTN0044). The text does not explicitly state the country of origin, but "National Institute of Drug Abuse" strongly suggests the United States. The study was prospective as it was a randomized controlled trial designed to characterize reSET's probable benefits and risks.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The ground truth for the clinical study was established primarily through self-report and objective measures like urine drug screens.

• Self-Report: Patient self-reported substance use via Timeline Follow-Back (TLFB).
• Objective Measures: Urine drug screens and breath alcohol tests.
• Expert involvement: While clinicians were supervising patients and there was a "clinician dashboard," the "ground truth" for abstinence and retention was based on a combination of patient self-report and verifiable clinical tests (urine drug screens). The text does not specify a panel of experts explicitly establishing "ground truth" through a consensus process for individual patient outcomes in the test set. Clinical investigators would have interpreted the urine drug screens based on established protocols.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

For abstinence, a combination of self-report and urine drug screens was used. The adjudication method for discrepancies between self-report and urine drug screens was defined in "Table 1: Urine drug screen interpretations." For example, a positive self-report plus a negative urine drug screen resulted in a "positive" interpretation (meaning non-abstinent). A negative self-report plus a positive urine drug screen also resulted in a "positive" interpretation. If both were negative, it was "negative" (abstinent). There was no explicit mention of an expert consensus or multi-reader adjudication method in the traditional sense for image-based diagnostic AI, as this is a behavioral therapy device.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study was not done. The reSET device is a direct patient-facing therapeutic intervention, not a diagnostic aid for human readers. Its effectiveness was evaluated by comparing patient outcomes (abstinence and retention) in two groups: one receiving rTAU + reSET and the other receiving TAU.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

The study was not a "standalone" algorithm performance study in the typical sense for diagnostic AI. The reSET device is designed to be an adjunct to clinician-supervised outpatient treatment. It is a digital therapy tool that patients use, and its performance is evaluated in that human-in-the-loop context (i.e., patient using the app, clinician supervising). The "rTAU + reSET" arm represents the device's performance within its intended use alongside human clinical care. The "control arm" (TAU) did not receive the device.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used was outcomes data, specifically:

• Abstinence: A combination of patient self-report (Timeline Follow-Back) and objective urine drug screens.
• Retention: Time-to-event data (time until last face-to-face contact with the treatment program).

8. The sample size for the training set

The provided text describes a clinical trial (CTN0044) for the evaluation of the reSET device. It does not mention a separate "training set" in the context of machine learning model development. For behavioral therapy, the "training" data typically refers to the aggregated knowledge and clinical evidence that informed the design and content of the therapeutic modules themselves, rather than a specific dataset used to train a statistical model within the device. The device's content is based on established CBT and CRA principles.

9. How the ground truth for the training set was established

As there was no "training set" in the machine learning sense described for this specific clinical trial, there's no ground truth established in that manner. The "ground truth" for the content of reSET (i.e., why certain therapeutic modules and exercises are included) is based on the established evidence and clinical literature supporting Cognitive Behavioral Therapy (CBT) and Community Reinforcement Approach (CRA) as effective interventions for substance use disorder. This underlying scientific and clinical understanding forms the basis for the device's therapeutic content.

Ask a specific question about this device