(122 days)
The Embrace is a prescription only device that is indicated for use as an adjunct to seizure monitoring of adults in home or healthcare facilities during periods of rest. The device is worn on the wrist, and senses Electrodermal Activity (EDA) and motion data to detect patterns that may be associated with generalized tonic seizures in patients with epilepsy or at risk of having epilepsy. When a seizure event is detected, Embrace sends a command to a paired wireless device that is programmed to initiate an alert to a designated caregiver. The System records and stores data from Accelerometers, EDA, and Temperature for subsequent review by a trained healthcare professional.
The Embrace is a wearable biosensor device that can capture, store, and wirelessly transmit sensor data via Bluetooth to a paired remote device. Embrace runs an on-board algorithm to continuously process sensor data and make a decision about whether the data might indicate a generalized tonic clonic seizure (GTC). The algorithm has been validated on data, labeled using the gold-standard video-Electroencephalogram (EEG) methodology designed by a group of epileptologists at a top level 4 epilepsy center, from epilepsy patients experiencing GTCs in hospital Epilepsy Monitoring Units. When a likely GTC is detected, the Embrace sends a message to the Alert smartphone application, which initiates calls and texts to summon the attention of designated caregivers.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. A table of acceptance criteria and the reported device performance
| Metric | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Positive Percent Agreement (PPA) | Not explicitly stated but inferred to be clinically acceptable and comparable to prior devices, ensuring effective detection of GTC seizures. Given the reported value, it's expected to be high. | 1.0 (corrected PPA = 0.9334) with a 95% CI of [0.9213, 0.9424] |
| False Alarm Rate (FAR) | Not explicitly stated but inferred to be clinically acceptable and comparable to prior devices, ensuring a manageable number of false alerts for caregivers. Given the reported value, it's expected to be low. | 0.4286 false alarms per 24 hours with a 95% CI of [0.3425, 0.7002], corresponding to a mean FAR of 0.5894 |
Note on Acceptance Criteria: The document does not explicitly state numerical acceptance criteria for PPA and FAR. Instead, it reports the device's performance metrics and implicitly suggests that these values are deemed acceptable for marketing the device as "substantially equivalent" to a predicate device. The FDA typically relies on a comparison to predicate devices and clinical justification for novel devices.
2. Sample size used for the test set and the data provenance
- Sample Size for Test Set:
- 135 patients diagnosed with epilepsy were enrolled.
- Out of these, 22 patients experienced a total of 40 generalized tonic clonic seizures (GTCSs).
- 113 patients did not experience any seizures during the study.
- Data Provenance:
- Country of Origin: Not explicitly stated, but the study was conducted within an "Epilepsy Monitoring Unit (EMU)" in a "top level 4 epilepsy center," suggesting a clinical setting in a developed healthcare system (likely the US, given the FDA submission).
- Retrospective or Prospective: The study enrolled patients and observed them for seizure events within the EMU, implying a prospective collection of data for the purpose of validating the device.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- Number of Experts: Three readers.
- Qualifications of Experts: They are referred to as a "panel of three readers." Although their specific qualifications (e.g., neurologists, epileptologists, years of experience) are not explicitly detailed in this document, the context of an "Epilepsy Monitoring Unit" and "gold-standard video-Electroencephalogram (EEG) methodology designed by a group of epileptologists" strongly suggests they were highly qualified medical professionals specializing in epilepsy and EEG interpretation.
4. Adjudication method (e.g., 2+1, 3+1, none) for the test set
The document mentions "relative to a panel of three readers" for PPA calculation. This implies that the ground truth was established by consensus (or agreement) among these three readers. The specific adjudication method (e.g., if at least 2 out of 3 had to agree, or if all 3 had to agree, or if a tie-breaking fourth expert was used) is not explicitly stated. However, the use of a "panel" suggests a structured review process.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- Was an MRMC comparative effectiveness study done? No. The study described focuses on the standalone performance of the Embrace device (i.e., the algorithm's ability to detect seizures) against a human-established ground truth. It does not evaluate how human readers' performance might change with or without the device's assistance.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, a standalone performance study was done. The performance metrics (PPA and FAR) directly reflect the algorithm's ability to detect GTC seizures based on sensor data. The device "runs an on-board algorithm to continuously process sensor data and make a decision about whether the data might indicate a generalized tonic clonic seizure (GTC)."
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
- Ground Truth Type: Expert consensus based on "gold-standard video-Electroencephalogram (EEG) methodology." This is a strong form of ground truth for epilepsy, as video-EEG is the primary method for diagnosing and classifying seizures in clinical practice. The data was "labeled using the gold-standard video-Electroencephalogram (EEG) methodology designed by a group of epileptologists."
8. The sample size for the training set
The document states, "The algorithm has been validated on data, labeled using the gold-standard video-Electroencephalogram (EEG) methodology designed by a group of epileptologists at a top level 4 epilepsy center, from epilepsy patients experiencing GTCs in hospital Epilepsy Monitoring Units."
However, it does not explicitly state the sample size of the training set. It only describes the validation set (the 135 patients). It's common for regulatory submissions to omit detailed training set information, focusing instead on the independent validation (test) set.
9. How the ground truth for the training set was established
- The ground truth for the training set (implied, as the exact set is not detailed) was established using the "gold-standard video-Electroencephalogram (EEG) methodology designed by a group of epileptologists at a top level 4 epilepsy center." This indicates that the training data was meticulously labeled by highly qualified experts using the most reliable method for seizure detection.
§ 882.1580 Non-electroencephalogram (EEG) physiological signal based seizure monitoring system.
(a)
Identification. A non-electroencephalogram (non-EEG) physiological signal based seizure monitoring system is a noninvasive prescription device that collects physiological signals other than EEG to identify physiological signals that may be associated with a seizure.(b)
Classification. Class II (special controls). The special controls for this device are:(1) The technical parameters of the device, hardware and software, must be fully characterized and include the following information:
(i) Hardware specifications must be provided. Appropriate verification, validation, and hazard analysis must be performed.
(ii) Software, including any proprietary algorithm(s) used by the device to achieve its intended use, must be described in detail in the Software Requirements Specification (SRS) and Software Design Specification (SDS). Appropriate software verification, validation, and hazard analysis must be performed.
(2) The patient-contacting components of the device must be demonstrated to be biocompatible.
(3) The device must be designed and tested for electrical, thermal, and mechanical safety and electromagnetic compatibility (EMC).
(4) Clinical performance testing must demonstrate the ability of the device to function as an assessment aid for monitoring for seizure-related activity in the intended population and for the intended use setting. Performance measurements must include positive percent agreement and false alarm rate.
(5) Training must be provided for intended users that includes information regarding the proper use of the device and factors that may affect the collection of the physiologic data.
(6) The labeling must include health care professional labeling and patient-caregiver labeling. The health care professional and the patient-caregiver labeling must include the following information:
(i) A detailed summary of the clinical performance testing, including any adverse events and complications.
(ii) Any instructions technicians and clinicians should convey to patients and caregivers regarding the proper use of the device and factors that may affect the collection of the physiologic data.
(iii) Instructions to technicians and clinicians regarding how to set the device threshold to achieve the intended performance of the device.