Search Results

reSET-O™ is intended to increase retention of patients with opioid use disorder (OUD) in outpatient treatment by providing cognitive behavioral therapy, as an adjunct to outpatient that includes transmucosal buprenorphine and contingency management, for patients 18 years or older who are currently under the supervision of a clinician. reSET-O is indicated as a prescription-only digital therapeutic.

reSET-O™ is a 12-week interval prescription digital therapeutic for Opioid Use Disorder (OUD). reSET-O™ is modeled on the Community Reinforcement Approach (CRA) and engineered to deliver behavioral therapy for patients with OUD. reSET-O™ delivers CRA therapy as a series of interactive therapy lessons. Each therapy lesson is comprised of a cognitive behavioral therapy component and skill building exercises. Therapy lesson content is delivered primarily via text or audio, and may include videos, animations and graphics.

reSET-O™ is intended as an adjunct to standard of care for patients with OUD. It is limited to persons with a valid prescription from their licensed provider. reSET-O™ supports clinician-patient communication between visits, by providing a means for patients to self-report cravings and triggers, and buprenorphine use/non-use. reSET-O™ reinforces the importance of using buprenorphine for treatment of OUD.

Acceptance Criteria and Study for reSET-O

1. Table of Acceptance Criteria and Reported Device Performance

Note: While the study also evaluated abstinence, it was stated that "The ability of reSET-O to improve abstinence has not been established as clinically significant." Therefore, enhanced abstinence is not listed as an acceptance criterion met by the device.

2. Sample Size and Data Provenance

• Sample Size for Test Set: 170 patients
• Data Provenance: The document does not explicitly state the country of origin. The study was a randomized clinical trial, indicating a prospective study design.

3. Number of Experts and Qualifications for Ground Truth

The document does not provide information on the number of experts used to establish ground truth or their specific qualifications for the test set. The ground truth appears to be based on the clinical trial outcomes (dropout rates, abstinence) rather than expert consensus on individual cases.

4. Adjudication Method for the Test Set

The document does not describe an adjudication method for the test set. The outcomes (dropout and abstinence) were measured directly during the randomized clinical trial.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly described in the provided text. The study design was a randomized controlled trial comparing the reSET-O device (referred to as TES) plus Treatment As Usual (TAU) against TAU alone. This is a direct comparison of a treatment arm with the device versus a control arm, not an MRMC study comparing human readers with and without AI assistance.

6. Standalone Performance

The study design assessed the effectiveness of reSET-O (TES) as an adjunct to Treatment As Usual (TAU) compared to TAU alone. It does not evaluate the device's standalone performance without human-in-the-loop (i.e., without the adjunctive standard of care). The device is explicitly indicated as an "adjunct to outpatient treatment."

7. Type of Ground Truth Used

The ground truth used was based on clinical outcomes data from a randomized controlled trial:

• Treatment Dropout: Defined as the percentage of patients who discontinued treatment during the 12-week intervention.
• Abstinence: Defined as the "longest documented period of continuous abstinence from opioids and cocaine for each participant," measured through thrice-weekly urine drug screens (UDS).

8. Sample Size for the Training Set

The document does not provide information about a separate training set or its sample size. The clinical study described served as the validation (test) set for the device's effectiveness.

9. How Ground Truth for the Training Set Was Established

As no separate training set is mentioned, the method for establishing its ground truth is not provided. The provided text focuses on the clinical validation study.

Ask a specific question about this device

reSET is intended to provide cognitive behavioral therapy, as an adjunct to a contingency management system, for patients 18 years of age and older who are currently enrolled in outpatient treatment under the supervision of a clinician. reSET is indicated as a 12 week (90 days) prescription-only treatment for patients with substance use disorder (SUD), who are not currently on opioid replacement therapy, who do not abuse alcohol solely, or who do not abuse opioids as their primary substance of abuse. It is intended to:

• increase abstinence from a patient's substances of abuse during treatment, and ●
• increase retention in the outpatient treatment program.

reSET™ is a digital therapy comprised of a patient application and clinician dashboard intended to deliver cognitive behavioral therapy (CBT) to patients with SUD to increase abstinence from substance use and increase retention in outpatient therapy programs. CBT is a psychosocial intervention that aims to change a patient's thinking and behavior, and it has been studied in psychiatric disorders such as major depressive disorder (Psychiatr Clin North Am. 2010; 33: 537-55). reSET is based on a specialized version of CBT known as the community reinforcement approach (CRA). which was originally developed for alcohol dependence and cocaine use (Behav Res Ther. 1973; 11:91-104; Exp. Clin. Psychopharmacol 2000; 3:205–2). The community reinforcement incorporates a range of therapeutic modalities including CBT to make substance-free lifestyle rewarding, skill building to promote behavioral change, and contingency management to reward and incentivize abstinence and replace the satisfaction obtained from substance abuse. CBT and CRA are considered valid models for substance abuse therapy and other psychiatric disorders.

reSET consists of several therapy lessons (modules) that are intended to teach the user the following skills to aid in the treatment of substance use disorder:

• Identifying situations and triggers that make substance use more likely ●
• . Avoiding substance use.
• . Coping with thoughts about substance use,
• . Recognizing negative thinking and identifying techniques to move to positive thinking
• Making decisions about substance use
• Taking responsibility for choices made and evaluating the consequences of those choices ●

Each therapy lesson is comprised of a cognitive behavioral therapy component and skill building exercises related to the above areas. The content of the therapy lessons is delivered primarily via text, and may include videos, animations and graphics. All text and video within reSET is narrated in English.

Following most therapy lessons, the patient undergoes fluency learning, a method of questioning that intended to promote learning and improve both short-term and long-term retention of material. Within the fluency learning section of reSET, patients are asked between 4-10 multiple choice and fill-in-the-blank questions about the key concepts presented in the lesson. An example of screenshots from the device is provided in the figure below.

When a question is answered incorrectly, the patient is presented with the correct response and the question is recycled back into the queue and asked again. In order to successfully complete the fluency learning section, patients must answer each question correctly three times, providing a repetitive component that reinforces concept mastery. If the patient completes the lesson and demonstrates proficiency, they can "spin the wheel" for virtual rewards that may be incorporated into a contingency management program at the treating physician's clinic.

reSET is comprised of 62 lessons, including one on-screen User Guide that explains how to use the reSET app and 61 therapy lessons. When a patient uses reSET for the very first time, he or she must step through the User Guide session to ensure they understand how to use the app. The 61 therapy lessons are split into 31 core therapy lessons and 30 supplemental therapy lessons. The therapy lessons include categories related to life skills, treatment, mood matters, social connections, sexual health, and hepatitis C and HIV.

The therapy lessons in the core therapy lesson group are focused on building basic cognitive behavioral and relapse prevention skills (e.g., functional analysis of drug use and selfmanagement planning, drug refusal skills). The therapy lessons in the supplemental group cover a range of topics that can be relevant for patients with SUD such as managing relationships, building communication skills, and time management. They also provide more in-depth training on HIV, hepatitis and STI prevention as well as support for those patients living with HIV and Hepatitis C.

Once the initial User Guide lesson has been completed, the patient gains access to the next core therapy lesson. In the core therapy lessons, a patient can only advance to the next lesson after successfully completing the prior lesson. At any time, patients can choose to review a completed lesson. Once a patient successfully completes all the core therapy lessons, they gain access to all the supplemental therapy lessons. These lessons do not have a set order of completion, and patients can choose lessons that are relevant to managing their disease or as recommended by their clinician.

reSET recommends that patients should complete 4 lessons per week. Each lesson is intended to take between 10-20 minutes to complete. Therapy lesson lengths vary, as do the number of fluency assessment questions the patient must take at the end of a lesson. Some therapy lessons have optional worksheets for the patient to complete that are intended to help the patient understand the key concepts taught in the therapy lesson.

The reSET application allows patients to track their own progress on the device's therapy modules. The device additionally has a Patient Self Report interface that allows for patients to track their cravings and substance use. This information is available to the patient's treating physician through the device.

The clinician dashboard for reSET displays the patient's progress. The clinician can view which therapy lessons the patient has completed, as well as view patient-reported substance use, cravings and triggers. The reSET app automatically pushes an assessment every four days to ask if the patient has taken any drugs or alcohol in the past 4 days, and if so, on which days. The patient is also asked to report whether they have had any cravings for drugs and alcohol and if so, to rate the intensity of the cravings. This reSET initiated, self-report data will display on the clinician's dashboard. Patients may also log their drug and alcohol use at any time into reSET along with tracking their cravings and distinct triggers. These use, craving and trigger data are presented to the clinician. The clinician can also enter in-clinic data inputs such as urine drug screens and appointment attendance.

reSET is intended be used in conjunction with a contingency management incentives system. reSET provides virtual "rewards" when the patient completes a lesson successfully as well as when their urine drug screen, or other objective test, is negative for substances. Clinics mav convert the virtual "rewards" into tangible rewards according to their own procedures.

The reSET device is a software-based medical device, and its acceptance criteria and clinical performance are based on a randomized controlled trial. Here's a breakdown of the requested information:

1. A table of acceptance criteria and the reported device performance

The acceptance criteria for reSET, a computerized behavioral therapy device for psychiatric disorders, were implicitly defined by the statistically significant improvements shown in the clinical trial. The device was deemed acceptable if it demonstrated a benefit over "treatment as usual" (TAU) in its primary outcome measures, especially in cohorts relevant to its indicated use.

2. Sample size used for the test set and the data provenance

• Test Set (Clinical Study):
  • Cohort 1 (All Comers): 507 participants (252 in TAU, 255 in rTAU + reSET)
  • Cohort 2 (Excluding Primary Opioids): 399 participants (193 in TAU, 206 in rTAU + reSET)
  • Cohort 3 (Excluding All Opioids): 305 participants (153 in TAU, 152 in rTAU + reSET)
• Data Provenance: The study was a multi-site, randomized clinical trial (National Institute of Drug Abuse CTN0044). The text does not explicitly state the country of origin, but "National Institute of Drug Abuse" strongly suggests the United States. The study was prospective as it was a randomized controlled trial designed to characterize reSET's probable benefits and risks.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

The ground truth for the clinical study was established primarily through self-report and objective measures like urine drug screens.

• Self-Report: Patient self-reported substance use via Timeline Follow-Back (TLFB).
• Objective Measures: Urine drug screens and breath alcohol tests.
• Expert involvement: While clinicians were supervising patients and there was a "clinician dashboard," the "ground truth" for abstinence and retention was based on a combination of patient self-report and verifiable clinical tests (urine drug screens). The text does not specify a panel of experts explicitly establishing "ground truth" through a consensus process for individual patient outcomes in the test set. Clinical investigators would have interpreted the urine drug screens based on established protocols.

4. Adjudication method (e.g., 2+1, 3+1, none) for the test set

For abstinence, a combination of self-report and urine drug screens was used. The adjudication method for discrepancies between self-report and urine drug screens was defined in "Table 1: Urine drug screen interpretations." For example, a positive self-report plus a negative urine drug screen resulted in a "positive" interpretation (meaning non-abstinent). A negative self-report plus a positive urine drug screen also resulted in a "positive" interpretation. If both were negative, it was "negative" (abstinent). There was no explicit mention of an expert consensus or multi-reader adjudication method in the traditional sense for image-based diagnostic AI, as this is a behavioral therapy device.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study was not done. The reSET device is a direct patient-facing therapeutic intervention, not a diagnostic aid for human readers. Its effectiveness was evaluated by comparing patient outcomes (abstinence and retention) in two groups: one receiving rTAU + reSET and the other receiving TAU.

6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done

The study was not a "standalone" algorithm performance study in the typical sense for diagnostic AI. The reSET device is designed to be an adjunct to clinician-supervised outpatient treatment. It is a digital therapy tool that patients use, and its performance is evaluated in that human-in-the-loop context (i.e., patient using the app, clinician supervising). The "rTAU + reSET" arm represents the device's performance within its intended use alongside human clinical care. The "control arm" (TAU) did not receive the device.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

The ground truth used was outcomes data, specifically:

• Abstinence: A combination of patient self-report (Timeline Follow-Back) and objective urine drug screens.
• Retention: Time-to-event data (time until last face-to-face contact with the treatment program).

8. The sample size for the training set

The provided text describes a clinical trial (CTN0044) for the evaluation of the reSET device. It does not mention a separate "training set" in the context of machine learning model development. For behavioral therapy, the "training" data typically refers to the aggregated knowledge and clinical evidence that informed the design and content of the therapeutic modules themselves, rather than a specific dataset used to train a statistical model within the device. The device's content is based on established CBT and CRA principles.

9. How the ground truth for the training set was established

As there was no "training set" in the machine learning sense described for this specific clinical trial, there's no ground truth established in that manner. The "ground truth" for the content of reSET (i.e., why certain therapeutic modules and exercises are included) is based on the established evidence and clinical literature supporting Cognitive Behavioral Therapy (CBT) and Community Reinforcement Approach (CRA) as effective interventions for substance use disorder. This underlying scientific and clinical understanding forms the basis for the device's therapeutic content.

Ask a specific question about this device