(122 days)
The RemLogic software is intended for Polysomnography studies and allows recording, displaying, and storage of physiological signals to assist in the diagnosis of various sleep related respiratory disorders. The RemLogic software allows:
Automated analysis of physiological signals that is intended for use only in adults;
An optional audio/visual alert for user defined threshold on calibrated DC input. These alerts are not intended for use as life support such as vital signs monitoring or continuous medical surveillance in intensive care units.
Sleep report templates which summarize recorded and scored sleep data using simple measures including count, average, maximum and minimum values as well as data ranges for trended values;
The RemLogic software does not provide any diagnostic conclusion about the patient's condition and is intended to be used only by qualified and trained medical practitioners, in research and clinical environments.
The RemLogic Application is a software product that runs on a desktop or laptop computer and requires no specialized hardware. It is a Windows based application used by trained medical professionals to investigate sleep disorders. The RemLogic application collects and digitizes the electrical voltages of patient physiological signals. After collecting and saving the signals, it provides tools and analyzers to analyze the signals, which aid in the interpretation of a sleep study. The software consists of three main functional areas: Acquisition, Scoring & Review, and Reports. It also contains a number of computer-assisted scoring analyzers for various sleep events.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The document does not explicitly state pre-defined acceptance criteria for the PPA and FD/h metrics in a clear pass/fail format with specific numerical thresholds. Instead, it frames the conclusion as the device showing "performance levels comparable to the manual markings of expert reviewers" and being "clinically equivalent to the Reference standard."
However, we can infer the reported device performance from the "PPA and False Detection Rate Per Hour of RemLogic Event Detection Assisted Scoring Analyzers" table:
| Event | Mean PPA | 95% CI PPA | Mean FD/h | 95% CI FD/h |
|---|---|---|---|---|
| Central Apnea | 98% | 96% to 99% | 2.1 | 1.2 to 4.1 |
| Mixed Apnea | 98% | 96% to 99% | 2.4 | 1.3 to 5.1 |
| Obstructive Apnea | 94% | 91% to 95% | 7.8 | 5.6 to 10.9 |
| Hypopnea | 86% | 83% to 87% | 17.4 | 15.0 to 20.0 |
| Arousal | 83% | 81% to 85% | 20.2 | 17.8 to 22.6 |
| Limb Movement | 86% | 84% to 88% | 16.8 | 14.0 to 19.9 |
| Snore | 85% | 83% to 88% | 17.8 | 14.9 to 20.9 |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: 51 diagnostic PSG sleep studies (one study per subject) were used. This means 51 subjects were evaluated.
- Data Provenance: The document states that the data were collected from "diagnostic PSG sleep studies" and that "All subjects involved in this study were adult (>18 years old) subjects with a clinical indication for a sleep study." It does not explicitly state the country of origin, but the company (Embla Systems) is based in Canada. The study design is described as "clinical validation," implying these were existing or collected for the study, and they were de-identified and randomized for the review process. It appears to be retrospective analysis of collected physiological data.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
- Number of Experts: Three experienced and certified PSG specialists.
- Qualifications of Experts: The experts included "one board certified sleep specialist" and two other "experienced and certified PSG specialists" who independently marked events.
4. Adjudication Method for the Test Set
- Adjudication Method: "Reference standard" was defined using "majority rule, that is, at least two out of three expert scorings (medical professionals certified on PSG recording and analysis) agree on the presence of an event within an epoch." This is a form of 2-out-of-3 (2+1) adjudication.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
- No, a multi-reader multi-case (MRMC) comparative effectiveness study evaluating human reader improvement with AI assistance was not performed. The study aimed to establish the standalone performance of the RemLogic automated analysis against a consensus "Reference standard" derived from expert human scoring.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, a standalone performance evaluation of the RemLogic computer-assisted scoring analyzers was done. The results in the table ("RemLogic PPA and FD/h") represent the algorithm's performance compared to the established ground truth.
7. The Type of Ground Truth Used
- The ground truth used was expert consensus. Specifically, a "majority rule" (2 out of 3 experts agreeing) on the presence of an event within an epoch was used to define the "Reference standard." The criteria applied for scoring various events (Apnea, Hypopnea, Limb Movement, Snore, Arousals) were based on established clinical guidelines.
8. The Sample Size for the Training Set
- The document does not provide information regarding the sample size of any training set used for the RemLogic software's automated analysis algorithms. The study described is a validation study, focusing on the performance of the already-developed algorithms.
9. How the Ground Truth for the Training Set Was Established
- Since information on a training set is not provided, details on how its ground truth was established are also not available in this document. The document focuses solely on the clinical validation of the completed software.
§ 882.1400 Electroencephalograph.
(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).