Search Results
Found 4 results
510(k) Data Aggregation
(192 days)
California 95682
Re: K243729
Trade/Device Name: Prismira
Regulation Number: 21 CFR 882.5803
Therapeutic Software for Attention Deficit Hyperactivity Disorder
Regulation Number: 21 CFR § 882.5803
Disorder | Same |
| Product Code | QFT | QFT | QFT | Same |
| Regulation Number | 21 CFR § 882.5803
| 21 CFR § 882.5803 | 21 CFR § 882.5803 | Same |
| Intended use | Digital therapeutic adaptive stimulus
Moreover, Prismira complies with the same special controls as the predicate device set forth in 21 CFR § 882.5803
Prismira is a digital therapeutic indicated to improve attention function in adults ages 22-55 years old with primarily inattentive or combined-type Attention Deficit and Hyperactivity Disorder (ADHD). Patients who engage with Prismira demonstrate improvements in a digitally assessed measure of sustained and selective attention, Test of Variables of Attention (TOVA), and may not display benefits in typical behavioral symptoms, such as hyperactivity.
Prismira should be considered for use as part of a therapeutic program that may include clinician directed therapy, medication, and/or educational programs, which further address symptoms of the disorder.
Prismira is software-as-a-medical device (SaMD) that resides on the user's mobile device and can be executed at home.
Prismira is a prescription digital therapeutic indicated to improve attention function in patients 22 and older with primarily inattentive or combined type ADHD. Patients who engage with Prismira demonstrate improvements in attention functioning and may not display benefits in other behavioral symptoms such as hyperactivity. Prismira is intended to be used as part of a therapeutic program that may include clinician-directed therapy, medication, and/or educational programs, which further address symptoms of the disorder.
The Prismira App also includes an Engagement System to promote compliance with the treatment. The Engagement System gives the patient encouragement and feedback on progress at multiple timescales: performance on the most recent gameplay and historical game performance, completion of the daily assignment, progress toward the weekly target in the current week, and historical assignment completion.
Each game includes visual and optional auditory stimulus presentation. The basic input from the patient includes physical interaction with a touch screen accomplished by using the mobile device. The device's feedback within each game includes visual and optional auditory responses to the inputs. Output from each game includes a score earned by playing the game. Each gameplay is brief, typically between 1 and 5 minutes in duration, depending on the game. The treatment program guides patients through an ordered list of games (the "script") that enables subjects to engage in the recommended daily regimen (approximately 15 minutes per day).
Here's a breakdown of the acceptance criteria and study details for the Prismira device, based on the provided FDA 510(k) clearance letter:
1. Table of Acceptance Criteria and Reported Device Performance
Acceptance Criteria (as implied by study success and substantial equivalence claims):
- Primary Endpoint: Statistically significant improvement in TOVA ACS in the LLM-001 therapy arm compared to the control arm.
- Safety: No new or increased safety risks compared to the predicate device, and a favorable safety profile with minimal adverse events.
Metric / Endpoint | Acceptance Criteria (Implied) | Reported Device Performance (Prismira / LLM-001) |
---|---|---|
Primary Endpoint: TOVA ACS Change from Baseline at Day 63 | Statistically significant difference between active and control arms. | - LLM-001 Change from Baseline Mean: 1.1 points (SD=3.6) |
- Control Change from Baseline Mean: 0.3 points (SD=3.6)
- LSM Difference (LLM-001 - Control): 0.78 points (SE=0.32)
- LSM Difference p-value: 0.0149 (statistically significant)
- LSM improvement of 1.09 points was less than the recognized MCID of 1.4 points, but notably greater than the predicate device (0.93 points). |
| Secondary Endpoint: CGI-I | Improvement over control (desirable for clinical benefit). | Superior clinical benefit for the LLM-001 arm over control (p=0.0087). (For LLM-001 subjects, 32.1% were assessed as "very much improved" on the CGI-I). |
| Secondary Endpoint: ADHD-RS Total Score | Improvement over control (desirable for clinical benefit). | 30.4% of LLM-001 subjects experienced greater than 30% reduction in their ADHD symptoms based on ADHD-RS total score. (Numerically favored LLM-001, but no statistically meaningful differences between groups for the continuous measure). |
| Adverse Events (AEs) | Minimal; no serious AEs; low treatment-related AE rate. | - No serious adverse events reported. - Only two treatment-related AEs (frustration) out of 275 LLM-001 treated subjects (AE rate of 0.7%).
- No subjects exited early for medical or gameplay-related complaints. |
| Treatment Compliance | High, demonstrating feasibility of use. | Mean of 874.7 minutes (97.2%) of 900 expected minutes. |
2. Sample Size and Data Provenance for the Test Set (Clinical Study)
- Sample Size: 560 subjects were enrolled in The GAMES Study.
- Safety Population: All 560 subjects.
- Intent-to-Treat (ITT) Population: 456 subjects (224 randomized to LLM-001 active therapy, 232 to control therapy) who successfully completed gameplay treatment and Day 63 follow-up assessments.
- Data Provenance: Not explicitly stated (e.g., country of origin). However, it was a "multi-center" clinical trial, suggesting diverse locations. It was a prospective, randomized, double-blind, parallel-group, sham-controlled clinical trial.
3. Number of Experts and Qualifications for Ground Truth (Test Set)
- The study used blinded clinicians to assess the CGI-I (Clinical Global Impression-Improvement), a global measure of clinical improvement.
- Qualifications of Experts: Not explicitly stated beyond "blinded clinicians." Their expertise would be in diagnosing and assessing ADHD.
4. Adjudication Method for the Test Set
- The primary effectiveness measure (TOVA ACS) was a digitally assessed measure, so it wouldn't typically require human adjudication in the same way imaging studies might.
- For the CGI-I, it was assessed by "blinded clinicians." This implies independent assessment rather than a consensus/adjudication process among multiple experts for each individual case's outcome on this specific measure. The "blinded" nature is the key control here.
- There's no mention of a 2+1, 3+1, or other specific adjudication method for the clinical endpoints.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, an MRMC comparative effectiveness study was not done. This type of study typically applies to diagnostic imaging devices where human readers interpret medical images with and without AI assistance.
- This study was a randomized controlled clinical trial evaluating a digital therapeutic, focusing on its direct effect on patient outcomes (attention function) compared to a control.
6. Standalone Performance Study
- Yes, a standalone (algorithm only) performance study was done in the context of the clinical trial.
- The "LLM-001 active therapy" arm represents the device's performance when used by the patient as intended, without direct human-in-the-loop interaction from the algorithm's perspective. The clinicians were involved in diagnosis and overall patient care, but the daily therapeutic delivery was directly from the device to the patient.
- The primary endpoint, "change from baseline in the digitally assessed measure of sustained and selective attention, the Test of Variables of Attention (TOVA®)," is a direct measure of the algorithm's effect on attention function.
7. Type of Ground Truth Used (Clinical Study)
- The primary ground truth for effectiveness was derived from the Test of Variables of Attention (TOVA®) Attention Comparison Score (ACS), a digitally assessed, objective measure of sustained and selective attention. This is a standardized, quantitative neuropsychological assessment tool.
- Secondary ground truths included:
- ADHD-RS: ADHD Rating Scale, likely a clinician-rated symptom scale or patient-reported outcome.
- CGI-I: Clinical Global Impression-Improvement, a clinician-assessed global measure of improvement.
- BRIEF-A: Behavior Rating Inventory of Executive Function – Adult, likely a patient-reported or clinician-rated scale.
- WFIRS-S: Weiss Functional Impairment Rating Scale – Self Report, a patient-reported outcome.
- AAQoL: Adult ADHD Quality of Life Questionnaire, a patient-reported outcome.
- Diagnosis of ADHD (combined or inattentive type) was confirmed using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria and confirmed by the Mini-International Neuropsychiatric Interview (MINI) for ADHD adult version.
8. Sample Size for the Training Set
- The document does not explicitly state the sample size for the training set used to develop the Prismira algorithm. The provided clinical study (The GAMES Study) is the pivotal validation study designed to test the already developed device.
- The device description mentions an "Adaptive algorithm and Engagement System." Adaptive algorithms are often trained on large datasets, but the details of that training process and dataset size are not included in this 510(k) summary. References ¹ and ² hint at large-scale cognitive training data and online trials, which might relate to the algorithm's development.
9. How Ground Truth for the Training Set Was Established
- Similar to the training set sample size, the document does not explicitly describe how ground truth for the training set was established.
- Given that it's an "adaptive algorithm," training data would likely involve cognitive performance metrics and potentially diagnostic information from previous patient populations. This could involve using standardized cognitive tests (like TOVA or similar measures) as the "ground truth" to train the algorithm to improve performance on those measures.
Ask a specific question about this device
(228 days)
Rd 7th Floor Boston, MA 02110
Re: K233496
Trade/Device Name: EndeavorOTC Regulation Number: 21 CFR 882.5803
Software for Attention Deficit
Hyperactivity Disorder |
| Regulation Number: | 21 CFR § 882.5803
|
| Regulation
Number | 21 CFR § 882.5803
| 21 CFR § 882.5803
complies with the same special controls as the predicate Endeavor Rx (DEN200026) set forth in 21 CFR § 882.5803
EndeavorOTC is a digital therapeutic indicated to improve attention as measured by computer-based testing in patients 18 and older with primarily inattentive or combined type ADHD, who have a demonstrated attention issue. Patients who engage with EndeavorOTC demonstrate improvements in a digitally assessed measure, Test of Variables of Attention (TOVA®) of sustained and selective attention and may not display benefits in typical behavioral symptoms such as hyperactivity. EndeavorOTC is not intended to be a replacement for any form of treatment and should be used as part of a therapeutic program that may include clinician-directed therapy, medicational programs, which further address symptoms of the disorder.
EndeavorOTC is software-as-a-medical device (SaMD) that resides on the user's mobile device and can be executed at home. It is an over-the-counter (OTC) digital therapeutic indicated to improve attention function as measured by computer-based testing in patients 18 and older with primarily inattentive or combined type ADHD, who have a demonstrated attention issue. Patients who engage with EndeavorOTC demonstrate improvements in a digitally assessed measure, Test of Variables of Attention (TOVA®) of sustained and selective attention and may not display benefits in typical behavioral symptoms such as hyperactivity. EndeavorOTC is not intended to be a replacement for any form of treatment and should be used as part of a therapeutic program that may include clinician-directed therapy, medication, and/or educational programs, which further address symptoms of the disorder. The device is built on Akili's proprietary, patented, technology platform. EndeavorOTC uses adaptive algorithms (also known as Selective Stimulus Management Engine, SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function. In a closed-loop system, the adaptive SSME™ algorithms automatically adjust the difficulty level for a personalized treatment experience that is tailored to the needs of each individual patient. EndeavorOTC is delivered through a video game experience which leverages art, music, storytelling, and reward cycles to keep patients engaged. The adaptive algorithm constantly pushes patients precisely at predefined performance bounds relative to each individual, such that they are continuously encouraged to exceed their historic performance. The science behind EndeavorOTC was developed at the University of California, San Francisco by Adam Gazzaley, M.D., Ph.D., Founding Director of the University of California San Francisco's Neuroscape and Akili's Chief Science Advisor. The basic program inputs are steering, which is accomplished by using the mobile device's internal accelerometer to measure the degree to which it is tilted, and tapping, which is accomplished using the touch screen to measure correct and incorrect targeting. The basic outputs are the visual display of the game progression along with audio, which is accomplished by using the internal high resolution display and internal speaker. The program includes features to ensure it is used per the recommended regimen (approximately 25 minutes per day, 5 days per week, for 6 weeks).
Here's a summary of the acceptance criteria and the study proving the EndeavorOTC device meets those criteria, based on the provided FDA 510(k) summary:
Device Acceptance Criteria and Performance Study: EndeavorOTC
Device Name: EndeavorOTC
Regulatory Class: Class II
Product Code: QFT
Indication for Use: Digital therapeutic indicated to improve attention as measured by computer-based testing in patients 18 and older with primarily inattentive or combined type ADHD, who have a demonstrated attention issue.
1. Table of Acceptance Criteria and Reported Device Performance
The primary acceptance criteria for EndeavorOTC are based on demonstrating an improvement in attention, as measured by the Test of Variables of Attention (TOVA®) Attention Comparison Score (ACS), and showing a favorable safety profile in the indicated adult population. The 510(k) summary refers to "clinical performance study support[ing] the performance and safety of EndeavorOTC in the adult age range" and "a statistically significant positive mean change from baseline to study day 42 in the TOVA".
Based on the provided document, the key efficacy performance criterion appears to be a statistically significant positive change in the TOVA-ACS.
Acceptance Criteria (Implicit from Study Design & Outcomes) | Reported Device Performance (STARS-Adult Study, K233496) |
---|---|
Statistically significant positive change in TOVA-ACS from baseline to exit | Mean change in TOVA-ACS: 6.460 (SD 6.9522) |
95% CI: [5.349, 7.570] | |
P-value: |
Ask a specific question about this device
(219 days)
Road 7th Floor Boston, MA 02210
Re: K231337
Trade/Device Name: EndeavorRx Regulation Number: 21 CFR 882.5803
Software for Attention Deficit
Hyperactivity Disorder |
| Regulation Number: | 21 CFR § 882.5803
|
| Regulation Number | 21 CFR § 882.5803
| 21 CFR § 882.5803
complies with the same special controls as the predicate Endeavor Rx (DEN200026) set forth in 21 CFR § 882.5803
EndeavorRx is a digital therapeutic indicated to improve attention as measured by computer-based testing in children ages 8-17 years old with primarily inattentive or combined-type ADHD, who have demonstrated attention issue. Patients who engage with EndeavorRx demonstrate improvements in a digitally assessed measure Tests of Variables of Attention (TOVA) of sustained and selective attention and may not display benefits in typical behavioral symptoms, such as hyperactivity. EndeavorRx should be considered for use as part of a therapeutic program that may include: clinician-directed therapy, medication, and/ or educational programs, which further address symptoms of the disorder.
EndeavorRx is a modification to the previously granted EndeavorRx (DEN200026) with the primary difference being the expansion of the indicated patient population from 8-12 years old to 8-17 years old. In addition, minor software changes were made to improve app accessibility and user engagement. The core therapeutic software technology was not changed.
EndeavorRx is a prescription-only digital therapeutic software indicated for use in the treatment of attention impairment in pediatric patients (8-17 years of age) with primarily inattentive or combinedtype ADHD. EndeavorRx is a software-as-a-medical device (SaMD) that resides on the user's mobile device and can be executed at home.
EndeavorRx is engineered as a therapeutically active treatment for attention in pediatric patients affected by ADHD. EndeavorRx is built on Akili's proprietary, patented, technology platform and uses adaptive algorithms (also known as Selective Stimulus Management Engine, SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function. In a closedloop system, the adaptive SSME™ algorithms automatically adjust the difficulty level for a personalized treatment experience that is tailored to the needs of each individual patient.
EndeavorRx is delivered through a video game experience which leverages art, music, storytelling, and reward cycles to keep patients engaged. The adaptive algorithm constantly pushes patients precisely at predefined performance bounds relative to each individual, such that they are continuously encouraged to exceed their historic performance. The science behind EndeavorRx was developed at the University of California, San Francisco by Adam Gazzaley, M.D., Ph.D., Founding Director of the University of California San Francisco's Neuroscape and Akili's Chief Science Advisor.
The basic program inputs are steering, which is accomplished by using the mobile device's internal accelerometer to measure the degree to which it is tilted, and tapping, which is accomplished using the touch screen to measure correct and incorrect targeting. The basic outputs are the visual display of the game progression along with audio, which is accomplished by using the internal high resolution display and internal speaker. The program includes features to ensure it is used per the prescribed regimen (approximately 25 minutes per day, 5 days per week, for 4 weeks or as recommended by the health care provider).
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided FDA 510(k) summary for EndeavorRx (K231337):
Device Name: EndeavorRx
Indication for Use: Digital therapeutic indicated to improve attention as measured by computer-based testing in children ages 8-17 years old with primarily inattentive or combined-type ADHD, who have demonstrated attention issue.
1. Table of Acceptance Criteria and Reported Device Performance
The provided document primarily details efficacy results rather than explicitly stated "acceptance criteria" as pass/fail thresholds for regulatory approval outside of the comparison tables for substantial equivalence. However, the core acceptance criterion for efficacy appears to be a significant positive mean change from baseline in the TOVA-ACS score.
Acceptance Criterion (Inferred from Study Design & Outcomes) | Reported Device Performance (Subject Device: EndeavorRx v3.0, K231337) |
---|---|
Significant positive mean change from baseline to Day 28 in TOVA-ACS (Primary Efficacy Endpoint) | Mean Change: 2.639 (SD 3.7986) |
95% CI: 2.018, 3.261 | |
p-value: |
Ask a specific question about this device
(60 days)
NEW REGULATION NUMBER: 21 CFR 882.5803
CLASSIFICATION: Class II
PRODUCT CODE: QFT
BACKGROUND
Type: Digital therapy device for Attention Deficit Hyperactivity Disorder
Regulation Number: 21 CFR 882.5803
EndeavorRx is a digital therapeutic indicated to improve attention function as measured by computer-based testing in children ages 8-12 years old with primarily inattentive or combinedtype ADHD, who have a demonstrated attention issue. Patients who engage with EndeavorRx demonstrate improvements in a digitally assessed measure Test of Variables of Attention (TOVA) of sustained and selective attention and may not display benefits in typical behavioral symptoms, such as hyperactivity. EndeavorRx should be considered for use as part of a therapeutic program that may include: clinician-directed therapy, medication, and/or educational programs, which further address symptoms of the disorder.
EndeavorRx is a digital therapeutic indicated to improve attention as measured by computerbased testing in children ages 8-12 years old with primarily inattentive or combined-type ADHD, who have a demonstrated an attention issue. The EndeavorRx program is a software-as-medical device (SaMD) that resides on the user's mobile device and can be executed at home. EndeavorRx is engineered as a therapeutically active treatment for attention in pediatric affected by ADHD. EndeavorRx is built on Akili's proprietary, patented, technology platform and uses adaptive algorithms (also known as Selective Stimulus Management Engine. SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function. In a closed-loop system, the adaptive SSME™ algorithms automatically adjust the difficulty level for a personalized treatment experience that is tailored to the needs of each individual patient. EndeavorRx is delivered through a video game experience which leverages art, music, storytelling, and reward cycles to keep patients engaged. The adaptive algorithm constantly pushes patients precisely at predefined performance bounds relative to each individual, such that they are continuously encouraged to exceed their historic performance. The basic program inputs are steering, which is accomplished by using the internal accelerometer to measure the degree to which the mobile device is tilted, and tapping, which is accomplished using the touch screen to measure correct and incorrect targeting. The basic outputs are visual display of the game progression along with audio, which is accomplished by using the internal high-resolution display and internal speaker.
Here's an analysis of the acceptance criteria and the study that proves the EndeavorRx device meets those criteria, based on the provided text:
Acceptance Criteria and Reported Device Performance
The acceptance criteria for the EndeavorRx device are primarily driven by clinical performance demonstrating effectiveness and safety. The text doesn't explicitly state a table of "acceptance criteria" with specific thresholds prior to reporting results; rather, the clinical studies presented here serve as the evidence to establish that the device does meet the necessary criteria for regulatory approval. Based on the "Special Controls" section, the key criteria are demonstrated effectiveness using a validated measure and capture of adverse events.
Here's a table summarizing the implicit acceptance criteria (derived from "Special Controls" and the results presented) and the reported performance:
Acceptance Criterion (Implicit) | Reported Device Performance |
---|---|
1. Clinical Performance Testing: | |
a. Use a validated measure to evaluate effectiveness of device to provide therapy for ADHD or any of its individual symptoms. | STARS-ADHD Study (Pivotal): |
- Primary Endpoint Met: Statistically significant improvement from baseline in the TOVA API compared to digital control (0.93 vs 0.03; p=0.006).
- Secondary Endpoints: Numerically favored EndeavorRx over control for ADHD-RS Total, ADHD-RS Inattention, and IRS. Trend observed in favor of EndeavorRx on IRS.
- Clinically Meaningful Improvement: 34.5% of EndeavorRx treated participants moved into the normative range on at least one objective measure of attention across all studies.
STARS-Adjunct Study:
- IRS (Parent/Clinician-rated impairment): Significant improvement for both On Stimulants (-0.7, p1%): decreased frustration tolerance (2.8%), headache (1.7%), emotional reaction (1.1%). Mostly transient and mild (3 moderate). No AEs led to discontinuation.
STARS-Adjunct Study:
- 37 (18%) participants experienced device-related AEs across two treatment months.
- Most common AEs: decreased frustration tolerance (13.1%), headache (1.9%), irritability (1.5%). All mild or moderate. No SAEs.
- 3 participants discontinued due to treatment-related AE (all decreased frustration tolerance).
ADHD POC Study:
- 9 AEs reported over all study phases, none judged related to EndeavorRx.
Overall: Of 538 EndeavorRx participants across all studies, 50 (9.3%) experienced treatment-related AEs. No SAEs. AEs generally transient. Only 3 led to discontinuation. |
| 2. Software Verification, Validation, and Hazard Analysis | FDA review of the software documentation found to be acceptable. (MODERATE level of concern, prior to risk mitigation). |
| 3. Labeling Requirements | Labeling found to be sufficient and consistent with clinical data, covering hazards and clinically relevant information (e.g., instructions for use, OS requirements, not standalone, not a substitute for medication, summary of clinical performance, warnings for photosensitive epilepsy, color blindness, physical limitations, frustration, seizure, potential screen addiction, decreased sleep quality). |
Study Details
Given the multiple studies described, I'll focus primarily on the STARS-ADHD study as it's identified as the "pivotal" study, and briefly mention key aspects of the others where relevant.
1. Sample Sized Used for the Test Set and Data Provenance:
-
Pivotal Study (STARS-ADHD):
- Total Screened: 857 children
- Test Set (Randomized): 348 children
- EndeavorRx group (treatment): 180 children
- Digital Control group: 168 children
- Data Provenance: Multi-center randomized controlled trial conducted at 20 sites in the USA.
- Retrospective/Prospective: Prospective. This was a randomized, double-blind, digital controlled study.
-
STARS-Adjunct Study:
- Total Enrolled: 206 participants (130 "On Stimulants", 76 "No Stimulants")
- Data Provenance: Not explicitly stated, but likely multi-center and prospective, similar to STARS-ADHD.
- Retrospective/Prospective: Prospective.
-
ADHD Proof of Concept (POC):
- Analyzed Sample: 40 children in ADHD group, 40 Healthy Controls.
- Data Provenance: 3 sites in the US.
- Retrospective/Prospective: Prospective (open-label study).
2. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications:
The concept of "ground truth" for a diagnostic accuracy study (e.g., classifying an image) doesn't directly apply here, as EndeavorRx is a therapeutic device, not a diagnostic one. The "test set" in this context refers to the participant cohorts in the clinical trials.
- Diagnosis of ADHD: Inclusion criteria for ADHD diagnoses were based on the DSM V (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition), and confirmed with a MINI-KID (Mini International Neuropsychiatric Interview for Children and Adolescents) and ADHD-RS (ADHD Rating Scale) Total score. TOVA API score was also used to identify a "demonstrated attention issue."
- Experts involved: Clinicians at each of the 20 sites (STARS-ADHD) or 3 sites (ADHD POC) would have made the formal ADHD diagnoses. The document does not specify the number or specific qualifications (e.g., years of experience) of the individual diagnosing clinicians, but presumably, they were qualified medical professionals (e.g., pediatricians, child psychiatrists, clinical psychologists) experienced in diagnosing ADHD.
3. Adjudication Method for the Test Set:
- STARS-ADHD: This was a double-blind study. This means neither the participants nor the investigators/clinicians interacting with them (who would be part of the "adjudication" of symptoms if it were a diagnostic study) knew whether the child was receiving EndeavorRx or the digital control. This inherent blinding serves as a robust form of "adjudication" against bias in assessing primary (TOVA) and secondary (ADHD-RS, IRS, CGI-I, BRIEF) outcomes. The TOVA is a computer-based, objective measure, which reduces subjective interpretation. Secondary measures like ADHD-RS and IRS are parent/clinician-reported, but the blinding helps mitigate bias.
- ADHD POC: This was an open-label study, meaning participants and investigators knew who was receiving EndeavorRx. This is a less rigorous design in terms of blinding.
- STARS-Adjunct: The summary does not specify blinding, but given the nature of the interventions (different game stimuli or no stimulants), full blinding might have been challenging for participants, though clinician assessment could have been blinded.
There is no mention of a formal adjudicated consensus process by an independent panel of experts reviewing cases, as might be done in studies of diagnostic imaging AI, but rather relies on the study design's blinding and validated assessment tools.
4. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and the Effect Size:
- No, an MRMC comparative effectiveness study was not done. MRMC studies are typically used to compare the performance of multiple human readers (e.g., radiologists) with and without AI assistance on a set of cases, primarily in diagnostic imaging. EndeavorRx is a therapeutic device for direct patient use, so this type of study design is not applicable.
- The studies demonstrated the therapeutic effect of the device compared to a control (e.g., digital control or baseline), not a human reader's improvement with AI.
5. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done:
- Yes, the primary evaluation was standalone. EndeavorRx is a "Software-as-Medical Device (SaMD)" that resides on the user's mobile device and is executed at home. The therapeutic effect (improving attention function) is largely driven by the device's adaptive algorithms (Selective Stimulus Management Engine - SSME™) acting directly on the patient. While it's used "as part of a therapeutic program that may include: clinician-directed therapy," the measured improvements in attention function (TOVA API) are attributed directly to the device's use. The studies measured the outcome of the algorithm's therapeutic delivery.
6. The Type of Ground Truth Used:
Again, "ground truth" as a diagnostic reference standard doesn't perfectly fit a therapeutic device. Instead, the "ground truth" for evaluating the effectiveness of EndeavorRx was:
- Objective Computer-Based Testing: The Test of Variables of Attention (TOVA) Attention Performance Index (API) / Attention Composite Score (ACS). This is an FDA-cleared continuous performance test measuring attention. This is considered an objective measure.
- Validated Rating Scales (Parent/Clinician-Reported):
- ADHD-RS (Total, Inattentive, Hyperactive subscales)
- ADHD Impairment Rating Scale (IRS)
- CGI-I (Clinical Global Impression-Improvement)
- Academic Performance Measures: TOSREC (Test of Silent Reading Efficiency and Comprehension) and MFaCTS (Measured Functional Activities of Daily Living - for children).
These measures served as the "truth" against which the device's therapeutic impact was assessed.
7. The Sample Size for the Training Set:
The document does not specify a "training set" in the sense of a dataset used to train the adaptive algorithms (SSME™) of EndeavorRx.
- The device uses "adaptive algorithms (also known as Selective Stimulus Management Engine. SSME™) to deliver stimuli that are designed to engage the patient in a manner that improves their attention function." It "automatically adjust the difficulty level for a personalized treatment experience."
- This implies that the individual patient's performance during gameplay serves as the dynamic "training set" for the adaptive algorithm's real-time adjustments for that specific patient.
- The core algorithms themselves were developed at the University of California, San Francisco (by Adam Gazzaley, M.D.), likely drawing on neuroscience research; the initial development would have involved a form of "training" or optimization, but this wouldn't be referred to as a "training set" in the context of general clinical trial reporting for an AI/ML device like this. The effectiveness studies (STARS-ADHD, STARS-Adjunct, etc.) are evaluating the final, deployed algorithm's therapeutic effect.
8. How the Ground Truth for the Training Set Was Established:
Since there isn't a described "training set" in the conventional sense for the algorithm's initial development, the concept of "ground truth for the training set" as it applies to a supervised machine learning model isn't directly applicable here.
- The SSME™ adapts to individual player performance. Therefore, the "ground truth" for its adaptive mechanism is the real-time performance data of the player (e.g., accuracy, reaction time, consistency) which the algorithm uses to adjust difficulty and optimize the therapeutic challenge. This is an internal, dynamic "ground truth" established during each play session.
- The science behind EndeavorRx was developed academically (UCSF), which suggests a foundation in cognitive neuroscience principles guiding the design of the adaptive stimuli and feedback loops. The "truth" for this underlying scientific design would be established through decades of cognitive psychology and neuroscience research on attention and learning.
Ask a specific question about this device
Page 1 of 1