(161 days)
The Brainsway Deep TMS System is indicated for the treatment of depressive episodes in adult patients suffering from Major Depressive Disorder who failed to achieve satisfactory improvement from previous anti-depressant medication treatment in the current episode.
The Brainsway Deep TMS (DTMS) System is intended for the treatment of depressive episodes in patients suffering from MDD. Transcranial magnetic stimulation (TMS) is a non-invasive technique used to apply brief magnetic pulses to the brain. The pulses are administered by passing high currents through an electromagnetic coil placed adjacent to a patient's scalp. The pulses induce an electric field in the underlying brain tissue. When the induced field is above a certain threshold, and is directed in an appropriate orientation relative the brain's neuronal pathways, localized axonal depolarizations are produced, thus activating neurons in the targeted brain structure. The Brainsway Deep TMS (DTMS) System is composed of the following main components: An Electromagnetic Coil (H1 Coil), A TMS Neurostimulator, A Cooling System, A Positioning Device, A cart.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The provided text does not explicitly state pre-defined acceptance criteria for the clinical study. Instead, it describes observed effectiveness and safety outcomes that were deemed sufficient for demonstrating the device's performance. The "acceptance criteria" are implied by the statistically significant improvements shown.
| Performance Metric (Implied Acceptance Criteria) | Reported Device Performance (Brainsway Deep TMS) vs. Sham | Statistical Significance (p-value) |
|---|---|---|
| Efficacy | ||
| Change from baseline in HDRS-21 scores (5 weeks) | -6.39 points (vs. -3.28 points for sham) | 0.0080 |
| Response Rate (proportion of subjects) | 38.4% (vs. 21.4% for sham) | 0.0138 (PP analysis set) |
| Remission Rate (proportion of subjects) | 32.6% (vs. 14.6% for sham) | 0.0051 (PP analysis set) |
| Maintenance of efficacy at 16 weeks (HDRS-21) | Statistically significant change from baseline | 0.0259 (PP analysis set) |
| Response Rate at 16 weeks | Significantly better vs. sham | 0.0086 (PP analysis set) |
| CGI-S scores (5 weeks and 16 weeks) | Statistically significant improvement | Not explicitly stated, but "demonstrated" |
| CGI-I, PGI, GAF scores (16 weeks) | Statistically significant improvement | Not explicitly stated, but "demonstrated" |
| CGI-I, PGI, GAF scores (5 weeks) | Statistically significant improvement | Not explicitly stated, but "statistically significantly lower" |
| Cognitive Effects (MMSE, BSRT, AMI-SF) | No negative cognitive effect | Not explicitly stated, but "demonstrated" |
| Safety | ||
| Incidence of adverse events | Comparable or lower incidence rates compared to other TMS devices | See Table 1 for specific p-values for anticipated events |
| Serious Adverse Events | One device-related SAE (seizure, with caveat) | N/A |
Non-Clinical Performance Data: The device also met several non-clinical performance criteria, including output waveform and electric field spatial distribution, magnetic field strength gradient, and cooling system functionality (e.g., consistent temperature during continuous operation >180 days).
2. Sample Size Used for the Test Set and Data Provenance
- Test Set (Clinical Study):
- ITT analysis set: n=229 (108 DTMS, 121 Sham)
- mITT analysis set: n=212 (101 DTMS, 111 Sham)
- PP analysis set (most relevant for efficacy): n=181 (89 DTMS, 92 Sham)
- Data Provenance: Prospective, double-blind, randomized, controlled, multi-center trial conducted at 20 study sites in the United States (13 sites), Israel (4 sites), Germany (2 sites), and Canada (1 site).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of those Experts
The document does not specify the number of experts or their qualifications for establishing the "ground truth" for the test set. However, for Major Depressive Disorder (MDD), the "ground truth" for diagnosis and severity is typically established by trained psychiatrists or other mental health professionals using standardized diagnostic criteria (e.g., DSM-IV, DSM-5) and validated rating scales (e.g., HDRS-21, CGI-S, BDI). The study used the HDRS-21 score (Hamilton Depression Rating Scale) as a primary outcome measure, which is administered and scored by trained clinicians.
4. Adjudication Method for the Test Set
The document describes the study as "double blind, randomized, controlled," which implies that clinicians assessing patient outcomes were blinded to the treatment arm (DTMS or Sham). However, it does not explicitly detail any specific adjudication method (e.g., 2+1, 3+1 for resolving discrepancies in assessments).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, this was not an MRMC comparative effectiveness study involving human readers and AI assistance. This study evaluated the effectiveness of a medical device (Deep TMS System) in treating Major Depressive Disorder, comparing it to a sham treatment, not the performance of human readers with or without AI.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
No, this was a clinical study evaluating a medical device treatment, not an AI algorithm. The device itself is a treatment modality, not a diagnostic or interpretive tool that would have a standalone algorithm performance.
7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)
The ground truth for the effectiveness of the Brainsway Deep TMS System was established based on clinical outcomes data from a randomized, controlled trial. Key metrics used included:
- Hamilton Depression Rating Scale (HDRS-21) scores (change from baseline, response rates, remission rates).
- Clinical Global Impression – Severity (CGI-S) and Improvement (CGI-I) scores.
- Patient Global Impression (PGI) scores.
- Global Assessment of Functioning (GAF) scores.
- Cognitive tests (MMSE, BSRT, AMI-SF) to assess cognitive effects.
- Adverse event reporting for safety.
These are all standard outcome measures in psychiatric clinical trials, typically assessed by trained clinicians.
8. The Sample Size for the Training Set
The document does not describe a "training set" in the context of an AI algorithm. This study is a clinical trial evaluating a medical device, not a machine learning model. Therefore, there is no separate training set as understood in AI/ML contexts. The clinical study itself served as the primary validation for the device's efficacy and safety.
9. How the Ground Truth for the Training Set Was Established
As explained above, there was no "training set" for an AI algorithm. The clinical study served as the primary evidence. The ground truth for this study (diagnosis of MDD, severity, response, remission, etc.) was established by clinicians using standard diagnostic criteria and validated rating scales within the context of a prospective clinical trial.
§ 882.5805 Repetitive transcranial magnetic stimulation system.
(a)
Identification. A repetitive transcranial magnetic stimulation system is an external device that delivers transcranial repetitive pulsed magnetic fields of sufficient magnitude to induce neural action potentials in the prefrontal cortex to treat the symptoms of major depressive disorder without inducing seizure in patients who have failed at least one antidepressant medication and are currently not on any antidepressant therapy.(b)
Classification. Class II (special controls). The special control is FDA's “Class II Special Controls Guidance Document: Repetitive Transcranial Magnetic Stimulation System.” See § 882.1(e) for the availability of this guidance document.