Search Results

Prismira is a digital therapeutic indicated to improve attention function in adults ages 22-55 years old with primarily inattentive or combined-type Attention Deficit and Hyperactivity Disorder (ADHD). Patients who engage with Prismira demonstrate improvements in a digitally assessed measure of sustained and selective attention, Test of Variables of Attention (TOVA), and may not display benefits in typical behavioral symptoms, such as hyperactivity.

Prismira should be considered for use as part of a therapeutic program that may include clinician directed therapy, medication, and/or educational programs, which further address symptoms of the disorder.

Prismira is software-as-a-medical device (SaMD) that resides on the user's mobile device and can be executed at home.

Prismira is a prescription digital therapeutic indicated to improve attention function in patients 22 and older with primarily inattentive or combined type ADHD. Patients who engage with Prismira demonstrate improvements in attention functioning and may not display benefits in other behavioral symptoms such as hyperactivity. Prismira is intended to be used as part of a therapeutic program that may include clinician-directed therapy, medication, and/or educational programs, which further address symptoms of the disorder.

The Prismira App also includes an Engagement System to promote compliance with the treatment. The Engagement System gives the patient encouragement and feedback on progress at multiple timescales: performance on the most recent gameplay and historical game performance, completion of the daily assignment, progress toward the weekly target in the current week, and historical assignment completion.

Each game includes visual and optional auditory stimulus presentation. The basic input from the patient includes physical interaction with a touch screen accomplished by using the mobile device. The device's feedback within each game includes visual and optional auditory responses to the inputs. Output from each game includes a score earned by playing the game. Each gameplay is brief, typically between 1 and 5 minutes in duration, depending on the game. The treatment program guides patients through an ordered list of games (the "script") that enables subjects to engage in the recommended daily regimen (approximately 15 minutes per day).

Here's a breakdown of the acceptance criteria and study details for the Prismira device, based on the provided FDA 510(k) clearance letter:

1. Table of Acceptance Criteria and Reported Device Performance

Acceptance Criteria (as implied by study success and substantial equivalence claims):

• Primary Endpoint: Statistically significant improvement in TOVA ACS in the LLM-001 therapy arm compared to the control arm.
• Safety: No new or increased safety risks compared to the predicate device, and a favorable safety profile with minimal adverse events.

2. Sample Size and Data Provenance for the Test Set (Clinical Study)

• Sample Size: 560 subjects were enrolled in The GAMES Study.
  • Safety Population: All 560 subjects.
  • Intent-to-Treat (ITT) Population: 456 subjects (224 randomized to LLM-001 active therapy, 232 to control therapy) who successfully completed gameplay treatment and Day 63 follow-up assessments.
• Data Provenance: Not explicitly stated (e.g., country of origin). However, it was a "multi-center" clinical trial, suggesting diverse locations. It was a prospective, randomized, double-blind, parallel-group, sham-controlled clinical trial.

3. Number of Experts and Qualifications for Ground Truth (Test Set)

• The study used blinded clinicians to assess the CGI-I (Clinical Global Impression-Improvement), a global measure of clinical improvement.
• Qualifications of Experts: Not explicitly stated beyond "blinded clinicians." Their expertise would be in diagnosing and assessing ADHD.

4. Adjudication Method for the Test Set

• The primary effectiveness measure (TOVA ACS) was a digitally assessed measure, so it wouldn't typically require human adjudication in the same way imaging studies might.
• For the CGI-I, it was assessed by "blinded clinicians." This implies independent assessment rather than a consensus/adjudication process among multiple experts for each individual case's outcome on this specific measure. The "blinded" nature is the key control here.
• There's no mention of a 2+1, 3+1, or other specific adjudication method for the clinical endpoints.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• No, an MRMC comparative effectiveness study was not done. This type of study typically applies to diagnostic imaging devices where human readers interpret medical images with and without AI assistance.
• This study was a randomized controlled clinical trial evaluating a digital therapeutic, focusing on its direct effect on patient outcomes (attention function) compared to a control.

6. Standalone Performance Study

• Yes, a standalone (algorithm only) performance study was done in the context of the clinical trial.
• The "LLM-001 active therapy" arm represents the device's performance when used by the patient as intended, without direct human-in-the-loop interaction from the algorithm's perspective. The clinicians were involved in diagnosis and overall patient care, but the daily therapeutic delivery was directly from the device to the patient.
• The primary endpoint, "change from baseline in the digitally assessed measure of sustained and selective attention, the Test of Variables of Attention (TOVA®)," is a direct measure of the algorithm's effect on attention function.

7. Type of Ground Truth Used (Clinical Study)

• The primary ground truth for effectiveness was derived from the Test of Variables of Attention (TOVA®) Attention Comparison Score (ACS), a digitally assessed, objective measure of sustained and selective attention. This is a standardized, quantitative neuropsychological assessment tool.
• Secondary ground truths included:
  • ADHD-RS: ADHD Rating Scale, likely a clinician-rated symptom scale or patient-reported outcome.
  • CGI-I: Clinical Global Impression-Improvement, a clinician-assessed global measure of improvement.
  • BRIEF-A: Behavior Rating Inventory of Executive Function – Adult, likely a patient-reported or clinician-rated scale.
  • WFIRS-S: Weiss Functional Impairment Rating Scale – Self Report, a patient-reported outcome.
  • AAQoL: Adult ADHD Quality of Life Questionnaire, a patient-reported outcome.
• Diagnosis of ADHD (combined or inattentive type) was confirmed using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria and confirmed by the Mini-International Neuropsychiatric Interview (MINI) for ADHD adult version.

8. Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set used to develop the Prismira algorithm. The provided clinical study (The GAMES Study) is the pivotal validation study designed to test the already developed device.
• The device description mentions an "Adaptive algorithm and Engagement System." Adaptive algorithms are often trained on large datasets, but the details of that training process and dataset size are not included in this 510(k) summary. References ¹ and ² hint at large-scale cognitive training data and online trials, which might relate to the algorithm's development.

9. How Ground Truth for the Training Set Was Established

• Similar to the training set sample size, the document does not explicitly describe how ground truth for the training set was established.
• Given that it's an "adaptive algorithm," training data would likely involve cognitive performance metrics and potentially diagnostic information from previous patient populations. This could involve using standardized cognitive tests (like TOVA or similar measures) as the "ground truth" to train the algorithm to improve performance on those measures.

Ask a specific question about this device

EndeavorOTC is a digital therapeutic indicated to improve attention as measured by computer-based testing in patients 18 and older with primarily inattentive or combined type ADHD, who have a demonstrated attention issue. Patients who engage with EndeavorOTC demonstrate improvements in a digitally assessed measure, Test of Variables of Attention (TOVA®) of sustained and selective attention and may not display benefits in typical behavioral symptoms such as hyperactivity. EndeavorOTC is not intended to be a replacement for any form of treatment and should be used as part of a therapeutic program that may include clinician-directed therapy, medicational programs, which further address symptoms of the disorder.

EndeavorOTC is software-as-a-medical device (SaMD) that resides on the user's mobile device and can be executed at home. It is an over-the-counter (OTC) digital therapeutic indicated to improve attention function as measured by computer-based testing in patients 18 and older with primarily inattentive or combined type ADHD, who have a demonstrated attention issue. Patients who engage with EndeavorOTC demonstrate improvements in a digitally assessed measure, Test of Variables of Attention (TOVA®) of sustained and selective attention and may not display benefits in typical behavioral symptoms such as hyperactivity. EndeavorOTC is not intended to be a replacement for any form of treatment and should be used as part of a therapeutic program that may include clinician-directed therapy, medication, and/or educational programs, which further address symptoms of the disorder. The device is built on Akili's proprietary, patented, technology platform. EndeavorOTC uses adaptive algorithms (also known as Selective Stimulus Management Engine, SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function. In a closed-loop system, the adaptive SSME™ algorithms automatically adjust the difficulty level for a personalized treatment experience that is tailored to the needs of each individual patient. EndeavorOTC is delivered through a video game experience which leverages art, music, storytelling, and reward cycles to keep patients engaged. The adaptive algorithm constantly pushes patients precisely at predefined performance bounds relative to each individual, such that they are continuously encouraged to exceed their historic performance. The science behind EndeavorOTC was developed at the University of California, San Francisco by Adam Gazzaley, M.D., Ph.D., Founding Director of the University of California San Francisco's Neuroscape and Akili's Chief Science Advisor. The basic program inputs are steering, which is accomplished by using the mobile device's internal accelerometer to measure the degree to which it is tilted, and tapping, which is accomplished using the touch screen to measure correct and incorrect targeting. The basic outputs are the visual display of the game progression along with audio, which is accomplished by using the internal high resolution display and internal speaker. The program includes features to ensure it is used per the recommended regimen (approximately 25 minutes per day, 5 days per week, for 6 weeks).

Here's a summary of the acceptance criteria and the study proving the EndeavorOTC device meets those criteria, based on the provided FDA 510(k) summary:

Device Acceptance Criteria and Performance Study: EndeavorOTC

Device Name: EndeavorOTC
Regulatory Class: Class II
Product Code: QFT
Indication for Use: Digital therapeutic indicated to improve attention as measured by computer-based testing in patients 18 and older with primarily inattentive or combined type ADHD, who have a demonstrated attention issue.

1. Table of Acceptance Criteria and Reported Device Performance

The primary acceptance criteria for EndeavorOTC are based on demonstrating an improvement in attention, as measured by the Test of Variables of Attention (TOVA®) Attention Comparison Score (ACS), and showing a favorable safety profile in the indicated adult population. The 510(k) summary refers to "clinical performance study support[ing] the performance and safety of EndeavorOTC in the adult age range" and "a statistically significant positive mean change from baseline to study day 42 in the TOVA".

Based on the provided document, the key efficacy performance criterion appears to be a statistically significant positive change in the TOVA-ACS.

2. Sample Size and Data Provenance

• Test Set (Clinical Study Population):
  • Safety Population (ITT): 221 participants
  • Efficacy Population (mITT): 153 participants (all enrolled subjects with sufficient data at baseline and exit to calculate change scores)
• Data Provenance: The study was a "multicenter open-label study enrolled 221 subjects... across the US (a mix of institutional sites and private practice centers)." It was a prospective clinical performance study.

3. Number of Experts for Ground Truth and Qualifications

The provided document does not specify the number of experts used to establish the ground truth for the test set or their specific qualifications (e.g., "Radiologist with 10 years of experience").

However, the diagnosis of ADHD was determined in study participants using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria as confirmed by the Mini-International Neuropsychiatric Interview (MINI) for ADHD adult version 7.0.2. This implies the involvement of trained clinicians (likely psychiatrists or psychologists) to administer and interpret these diagnostic tools. The TOVA® test itself is a standardized, computer-based assessment of attention.

4. Adjudication Method for the Test Set

The document does not describe a formal adjudication method (e.g., 2+1, 3+1) for the interpretation of the primary outcomes. The primary outcome (TOVA-ACS) is a quantitative, digitally assessed measure, which generally does not require traditional expert adjudication of results in the same way image interpretations might. The ADHD diagnosis was clinician-confirmed using standardized interviews.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly mentioned or performed. This study was a single-arm, open-label study, primarily evaluating the device's effect on attention in the target population. It was not designed to compare human readers' improvement with or without AI assistance, as the device is a direct therapeutic intervention, not an AI-assisted diagnostic tool.

6. Standalone (Algorithm Only) Performance

EndeavorOTC is a therapeutic device (software-as-a-medical device) that directly interacts with the user as a game-like experience. Its performance is the algorithm's effect on the user's attention, measured by standardized tests like TOVA. Therefore, the clinical study results represent the standalone performance of the device's algorithm in a human-in-the-loop context (the human being the patient using the device). It's not an AI model that outputs a diagnostic interpretation for a human to review.

7. Type of Ground Truth Used

The primary ground truth for efficacy was objective, digitally assessed measures (Test of Variables of Attention - TOVA-ACS). The patient's ADHD diagnosis, which defined the study population, was established by clinician-administered standardized diagnostic interviews (MINI) based on DSM-5 criteria.

8. Sample Size for the Training Set

The document does not specify the sample size for the training set of the adaptive algorithms (Selective Stimulus Management Engine - SSME™). It notes that the science behind EndeavorOTC was developed at the University of California, San Francisco by Adam Gazzaley, M.D., Ph.D. The algorithms are "adaptive" and "in a closed-loop system, automatically adjust the difficulty level," implying continuous adaptation rather than a single, fixed training dataset in the typical machine learning sense.

9. How Ground Truth for Training Set was Established

Given that the device uses "adaptive algorithms (also known as Selective Stimulus Management Engine, SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function," and adjusts difficulty "for a personalized treatment experience," the "ground truth" for the training of these adaptive algorithms is inherently tied to the patient's real-time performance within the game and their physiological responses or attentional engagement.

The document does not detail how the initial parameters or underlying "training" for these adaptive algorithms were established. It mentions the algorithms were "developed at the University of California, San Francisco by Adam Gazzaley, M.D., Ph.D." This suggests a research-based, iterative development process informed by neuroscience and cognitive psychology, rather than a single, labeled dataset typical of supervised machine learning. The "ground truth" for fine-tuning the adaptive difficulty would likely be the user's performance and implicit learning within the game itself.

Ask a specific question about this device

EndeavorRx is a digital therapeutic indicated to improve attention as measured by computer-based testing in children ages 8-17 years old with primarily inattentive or combined-type ADHD, who have demonstrated attention issue. Patients who engage with EndeavorRx demonstrate improvements in a digitally assessed measure Tests of Variables of Attention (TOVA) of sustained and selective attention and may not display benefits in typical behavioral symptoms, such as hyperactivity. EndeavorRx should be considered for use as part of a therapeutic program that may include: clinician-directed therapy, medication, and/ or educational programs, which further address symptoms of the disorder.

EndeavorRx is a modification to the previously granted EndeavorRx (DEN200026) with the primary difference being the expansion of the indicated patient population from 8-12 years old to 8-17 years old. In addition, minor software changes were made to improve app accessibility and user engagement. The core therapeutic software technology was not changed.

EndeavorRx is a prescription-only digital therapeutic software indicated for use in the treatment of attention impairment in pediatric patients (8-17 years of age) with primarily inattentive or combinedtype ADHD. EndeavorRx is a software-as-a-medical device (SaMD) that resides on the user's mobile device and can be executed at home.

EndeavorRx is engineered as a therapeutically active treatment for attention in pediatric patients affected by ADHD. EndeavorRx is built on Akili's proprietary, patented, technology platform and uses adaptive algorithms (also known as Selective Stimulus Management Engine, SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function. In a closedloop system, the adaptive SSME™ algorithms automatically adjust the difficulty level for a personalized treatment experience that is tailored to the needs of each individual patient.

EndeavorRx is delivered through a video game experience which leverages art, music, storytelling, and reward cycles to keep patients engaged. The adaptive algorithm constantly pushes patients precisely at predefined performance bounds relative to each individual, such that they are continuously encouraged to exceed their historic performance. The science behind EndeavorRx was developed at the University of California, San Francisco by Adam Gazzaley, M.D., Ph.D., Founding Director of the University of California San Francisco's Neuroscape and Akili's Chief Science Advisor.

The basic program inputs are steering, which is accomplished by using the mobile device's internal accelerometer to measure the degree to which it is tilted, and tapping, which is accomplished using the touch screen to measure correct and incorrect targeting. The basic outputs are the visual display of the game progression along with audio, which is accomplished by using the internal high resolution display and internal speaker. The program includes features to ensure it is used per the prescribed regimen (approximately 25 minutes per day, 5 days per week, for 4 weeks or as recommended by the health care provider).

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) summary for EndeavorRx (K231337):

Device Name: EndeavorRx
Indication for Use: Digital therapeutic indicated to improve attention as measured by computer-based testing in children ages 8-17 years old with primarily inattentive or combined-type ADHD, who have demonstrated attention issue.

1. Table of Acceptance Criteria and Reported Device Performance

The provided document primarily details efficacy results rather than explicitly stated "acceptance criteria" as pass/fail thresholds for regulatory approval outside of the comparison tables for substantial equivalence. However, the core acceptance criterion for efficacy appears to be a significant positive mean change from baseline in the TOVA-ACS score.

2. Sample Sizes and Data Provenance

• Test Set (Clinical Study for Subject Device EndeavorRx v3.0 - K231337):

  • "Efficacy Population" (Complete Case Analysis - CCA): N = 146
  • "Safety Population" (Intent-to-Treat with Multiple Imputation - ITT with MI): N = 162
  • Provenance: Multi-center open-label study across 14 sites in the US.
  • Retrospective/Prospective: The description "multi-center open-label study enrolled 162 adolescents" indicates a prospective study.

• Predicate Device Study (STARS - DEN200026):

  • "ITT Population": N = 179 (for TOVA-ACS primary endpoint)
  • Provenance: Multi-site study across 20 sites in the US.
  • Retrospective/Prospective: Described as "Randomized, controlled, parallel arm," indicating a prospective study.

3. Number of Experts and Qualifications for Ground Truth (Test Set)

The ground truth for the device's efficacy is based on the objective, computer-based testing measure, TOVA-ACS (Test of Variables of Attention - ADHD Composite Score). This is a normed, standardized test, not subjective expert assessment requiring multiple readers or qualifications.

• Diagnosis of ADHD: Confirmed by Mini-International Neuropsychiatric Interview for Children and Adolescents (MINI-Kid) Version 7.0.2, administered by a trained clinician. The document does not specify the number or specific qualifications (e.g., years of experience) of these clinicians, but implies they are qualified to administer the MINI-Kid and make diagnoses based on DSM-5 criteria.
• High inattention: Measured by baseline TOVA-ACS score ≤ -1.8. This is a predefined numerical threshold.

4. Adjudication Method for the Test Set

Since the primary efficacy outcome (TOVA-ACS) is an objective, computer-based measure, and ADHD diagnosis was established by trained clinicians using a standardized interview (MINI-Kid), there is no mention of a traditional "adjudication method" involving multiple human readers reviewing results and resolving discrepancies. The data is either directly collected from the TOVA test or from structured diagnostic interviews.

The study design for the subject device was a "Single arm, open-label" study. This means there was no control group, and no 2+1 or 3+1 adjudication for comparing against a different intervention or sham.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC study was done. This device is a digital therapeutic, not a diagnostic imaging AI that assists human readers. Its primary claim is to improve attention as measured by a computer-based test, not to aid human interpretation of various cases. The clinical study for the subject device was:

• Standalone (algorithm only without human-in-the-loop performance): Yes, the device itself is the intervention, and its effect is measured directly.
• Human Reader Improvement with AI vs. Without AI Assistance (Effect Size): Not applicable, as this is not an AI-assisted diagnostic tool for human readers. The clinical study evaluated the direct therapeutic effect of the device on patients.

6. Standalone (Algorithm only) Performance

Yes, the primary efficacy endpoint (TOVA-ACS change) reflects the standalone performance of the EndeavorRx device in its effect on attention, as it is a direct therapeutic intervention.

7. Type of Ground Truth Used

• Primary Efficacy: Objective, digitally assessed measure of sustained and selective attention (Test of Variables of Attention - TOVA-ACS). This is a quantitative, standardized neurocognitive test.
• Secondary Outcomes: Clinical scales like ADHD-RS (Attention-Deficit/Hyperactivity Disorder Rating Scale), which are subjective clinician/parent-reported measures designed to assess symptom severity.
• Diagnosis (for inclusion criteria): Standardized diagnostic interview using DSM-5 criteria (MINI-Kid), performed by trained clinicians.

8. Sample Size for the Training Set

The document does not specify the sample size for the training set of the EndeavorRx device's algorithms (Selective Stimulus Management Engine, SSME™). It only describes the adaptive nature of the algorithm that "automatically adjust the difficulty level for a personalized treatment experience." This suggests an on-device adaptive learning system rather than a pre-trained machine learning model in the conventional sense where a large, separate training dataset is explicitly used for model development and then locked. However, the underlying "proprietary, patented, technology platform" and SSME™ algorithms would have been developed and refined using clinical and behavioral data over time, but details on such training data are not provided in this 510(k) summary.

9. How the Ground Truth for the Training Set was Established

As the document does not detail a distinct "training set" in the context of a fixed machine learning model, it also does not explain how ground truth was established for such a set.

The "adaptive algorithms" (SSME™) adjust difficulty based on the individual patient's real-time performance ("closed-loop system," "automatically adjust the difficulty level," "pushes patients precisely at predefined performance bounds relative to each individual"). This implies that the individual patient's performance within the game itself serves as the dynamic "ground truth" to adapt the therapeutic experience, rather than a separate, pre-labeled training dataset.

Ask a specific question about this device

EndeavorRx is a digital therapeutic indicated to improve attention function as measured by computer-based testing in children ages 8-12 years old with primarily inattentive or combinedtype ADHD, who have a demonstrated attention issue. Patients who engage with EndeavorRx demonstrate improvements in a digitally assessed measure Test of Variables of Attention (TOVA) of sustained and selective attention and may not display benefits in typical behavioral symptoms, such as hyperactivity. EndeavorRx should be considered for use as part of a therapeutic program that may include: clinician-directed therapy, medication, and/or educational programs, which further address symptoms of the disorder.

EndeavorRx is a digital therapeutic indicated to improve attention as measured by computerbased testing in children ages 8-12 years old with primarily inattentive or combined-type ADHD, who have a demonstrated an attention issue. The EndeavorRx program is a software-as-medical device (SaMD) that resides on the user's mobile device and can be executed at home. EndeavorRx is engineered as a therapeutically active treatment for attention in pediatric affected by ADHD. EndeavorRx is built on Akili's proprietary, patented, technology platform and uses adaptive algorithms (also known as Selective Stimulus Management Engine. SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function. In a closed-loop system, the adaptive SSME™ algorithms automatically adjust the difficulty level for a personalized treatment experience that is tailored to the needs of each individual patient. EndeavorRx is delivered through a video game experience which leverages art, music, storytelling, and reward cycles to keep patients engaged. The adaptive algorithm constantly pushes patients precisely at predefined performance bounds relative to each individual, such that they are continuously encouraged to exceed their historic performance. The basic program inputs are steering, which is accomplished by using the internal accelerometer to measure the degree to which the mobile device is tilted, and tapping, which is accomplished using the touch screen to measure correct and incorrect targeting. The basic outputs are visual display of the game progression along with audio, which is accomplished by using the internal high-resolution display and internal speaker.

Here's an analysis of the acceptance criteria and the study that proves the EndeavorRx device meets those criteria, based on the provided text:

Acceptance Criteria and Reported Device Performance

The acceptance criteria for the EndeavorRx device are primarily driven by clinical performance demonstrating effectiveness and safety. The text doesn't explicitly state a table of "acceptance criteria" with specific thresholds prior to reporting results; rather, the clinical studies presented here serve as the evidence to establish that the device does meet the necessary criteria for regulatory approval. Based on the "Special Controls" section, the key criteria are demonstrated effectiveness using a validated measure and capture of adverse events.

Here's a table summarizing the implicit acceptance criteria (derived from "Special Controls" and the results presented) and the reported performance:

Study Details

Given the multiple studies described, I'll focus primarily on the STARS-ADHD study as it's identified as the "pivotal" study, and briefly mention key aspects of the others where relevant.

1. Sample Sized Used for the Test Set and Data Provenance:

• Pivotal Study (STARS-ADHD):

  • Total Screened: 857 children
  • Test Set (Randomized): 348 children
    • EndeavorRx group (treatment): 180 children
    • Digital Control group: 168 children
  • Data Provenance: Multi-center randomized controlled trial conducted at 20 sites in the USA.
  • Retrospective/Prospective: Prospective. This was a randomized, double-blind, digital controlled study.

• STARS-Adjunct Study:

  • Total Enrolled: 206 participants (130 "On Stimulants", 76 "No Stimulants")
  • Data Provenance: Not explicitly stated, but likely multi-center and prospective, similar to STARS-ADHD.
  • Retrospective/Prospective: Prospective.

• ADHD Proof of Concept (POC):

  • Analyzed Sample: 40 children in ADHD group, 40 Healthy Controls.
  • Data Provenance: 3 sites in the US.
  • Retrospective/Prospective: Prospective (open-label study).

2. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:

The concept of "ground truth" for a diagnostic accuracy study (e.g., classifying an image) doesn't directly apply here, as EndeavorRx is a therapeutic device, not a diagnostic one. The "test set" in this context refers to the participant cohorts in the clinical trials.

• Diagnosis of ADHD: Inclusion criteria for ADHD diagnoses were based on the DSM V (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition), and confirmed with a MINI-KID (Mini International Neuropsychiatric Interview for Children and Adolescents) and ADHD-RS (ADHD Rating Scale) Total score. TOVA API score was also used to identify a "demonstrated attention issue."
• Experts involved: Clinicians at each of the 20 sites (STARS-ADHD) or 3 sites (ADHD POC) would have made the formal ADHD diagnoses. The document does not specify the number or specific qualifications (e.g., years of experience) of the individual diagnosing clinicians, but presumably, they were qualified medical professionals (e.g., pediatricians, child psychiatrists, clinical psychologists) experienced in diagnosing ADHD.

3. Adjudication Method for the Test Set:

• STARS-ADHD: This was a double-blind study. This means neither the participants nor the investigators/clinicians interacting with them (who would be part of the "adjudication" of symptoms if it were a diagnostic study) knew whether the child was receiving EndeavorRx or the digital control. This inherent blinding serves as a robust form of "adjudication" against bias in assessing primary (TOVA) and secondary (ADHD-RS, IRS, CGI-I, BRIEF) outcomes. The TOVA is a computer-based, objective measure, which reduces subjective interpretation. Secondary measures like ADHD-RS and IRS are parent/clinician-reported, but the blinding helps mitigate bias.
• ADHD POC: This was an open-label study, meaning participants and investigators knew who was receiving EndeavorRx. This is a less rigorous design in terms of blinding.
• STARS-Adjunct: The summary does not specify blinding, but given the nature of the interventions (different game stimuli or no stimulants), full blinding might have been challenging for participants, though clinician assessment could have been blinded.

There is no mention of a formal adjudicated consensus process by an independent panel of experts reviewing cases, as might be done in studies of diagnostic imaging AI, but rather relies on the study design's blinding and validated assessment tools.

4. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size:

• No, an MRMC comparative effectiveness study was not done. MRMC studies are typically used to compare the performance of multiple human readers (e.g., radiologists) with and without AI assistance on a set of cases, primarily in diagnostic imaging. EndeavorRx is a therapeutic device for direct patient use, so this type of study design is not applicable.
• The studies demonstrated the therapeutic effect of the device compared to a control (e.g., digital control or baseline), not a human reader's improvement with AI.

5. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done:

• Yes, the primary evaluation was standalone. EndeavorRx is a "Software-as-Medical Device (SaMD)" that resides on the user's mobile device and is executed at home. The therapeutic effect (improving attention function) is largely driven by the device's adaptive algorithms (Selective Stimulus Management Engine - SSME™) acting directly on the patient. While it's used "as part of a therapeutic program that may include: clinician-directed therapy," the measured improvements in attention function (TOVA API) are attributed directly to the device's use. The studies measured the outcome of the algorithm's therapeutic delivery.

6. The Type of Ground Truth Used:

Again, "ground truth" as a diagnostic reference standard doesn't perfectly fit a therapeutic device. Instead, the "ground truth" for evaluating the effectiveness of EndeavorRx was:

• Objective Computer-Based Testing: The Test of Variables of Attention (TOVA) Attention Performance Index (API) / Attention Composite Score (ACS). This is an FDA-cleared continuous performance test measuring attention. This is considered an objective measure.
• Validated Rating Scales (Parent/Clinician-Reported):
  • ADHD-RS (Total, Inattentive, Hyperactive subscales)
  • ADHD Impairment Rating Scale (IRS)
  • CGI-I (Clinical Global Impression-Improvement)
• Academic Performance Measures: TOSREC (Test of Silent Reading Efficiency and Comprehension) and MFaCTS (Measured Functional Activities of Daily Living - for children).

These measures served as the "truth" against which the device's therapeutic impact was assessed.

7. The Sample Size for the Training Set:

The document does not specify a "training set" in the sense of a dataset used to train the adaptive algorithms (SSME™) of EndeavorRx.

• The device uses "adaptive algorithms (also known as Selective Stimulus Management Engine. SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function." It "automatically adjust the difficulty level for a personalized treatment experience."
• This implies that the individual patient's performance during gameplay serves as the dynamic "training set" for the adaptive algorithm's real-time adjustments for that specific patient.
• The core algorithms themselves were developed at the University of California, San Francisco (by Adam Gazzaley, M.D.), likely drawing on neuroscience research; the initial development would have involved a form of "training" or optimization, but this wouldn't be referred to as a "training set" in the context of general clinical trial reporting for an AI/ML device like this. The effectiveness studies (STARS-ADHD, STARS-Adjunct, etc.) are evaluating the final, deployed algorithm's therapeutic effect.

8. How the Ground Truth for the Training Set Was Established:

Since there isn't a described "training set" in the conventional sense for the algorithm's initial development, the concept of "ground truth for the training set" as it applies to a supervised machine learning model isn't directly applicable here.

• The SSME™ adapts to individual player performance. Therefore, the "ground truth" for its adaptive mechanism is the real-time performance data of the player (e.g., accuracy, reaction time, consistency) which the algorithm uses to adjust difficulty and optimize the therapeutic challenge. This is an internal, dynamic "ground truth" established during each play session.
• The science behind EndeavorRx was developed academically (UCSF), which suggests a foundation in cognitive neuroscience principles guiding the design of the adaptive stimuli and feedback loops. The "truth" for this underlying scientific design would be established through decades of cognitive psychology and neuroscience research on attention and learning.

Ask a specific question about this device