(198 days)
The SBV2 System is an activity monitor designed and intended for documenting physical movements associated with applications in physiological monitoring. The device's intended use is to analyze limb activity associated with movement during sleep and to extract information about certain sleep parameters from these movements. SBV2 can also be used to assess activity in any instance where quantifiable analysis of physical motion is desirable. The use of SBV2 is indicated for adults 22 years of age and over.
The SBV2 System is a device that monitors activity. It relies on the measurement and analysis of wrist movements to detect and characterize sleep/wake periods. The device allows some aspects of sleep derived from the analysis of activity to be reported. The SBV2 System is graphically depicted in Figure 1.
Here's a breakdown of the acceptance criteria and study information for the SBV2 System, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria (Target Performance) | Reported Device Performance (SBV2) |
|---|---|
| Phase 1: Differentiation of Activity States | |
| Identify "Awake, up and about" | 95% accuracy |
| Identify "In bed, awake or asleep" | 95% accuracy |
| Identify "Off wrist" | 87% accuracy |
| Phase 2: Sleep/Wake Classification vs. Polysomnography (in-bed) | |
| Sensitivity (correctly identify sleep) | 88% |
| Specificity (correctly identify wake) | 55% |
| Overall SBV2 vs. Polysomnography Agreement | 93% (weighted average of 95% in-bed state detection and 88% in-bed-asleep detection) |
Detailed Study Information
1. A table of acceptance criteria and the reported device performance
(See table above)
2. Sample sizes used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Phase 1 (Differentiation of Activity States):
- Sample Size: 180 participants
- Data Provenance: Not explicitly stated, but the study was conducted by Archinoetics, LLC and Fatigue Science. Given the later mention of the Kettering Sleep Disorders Center in Ohio for Phase 2, it's plausible the data originates from the US. The data was "de-identified." This phase was retrospective, using existing actigraph data.
- Phase 2 (Sleep/Wake Classification vs. Polysomnography):
- Sample Size: 50 patients
- Data Provenance: Conducted at the Wallace Kettering Health Networks Sleep Disorders Center in Kettering, Ohio, USA. This phase involved simultaneous collection of actigraphy data and polysomnography, suggesting a prospective study design.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
- Phase 1: "Research personnel at Archinoetics, LLC," described as having "substantial expertise in the visual examination and classification of raw actigraphy data." The exact number of experts is not specified.
- Phase 2:
- Dr. Donna Arand, Board Certified Sleep Specialist, Kettering Hospital.
- Dr. John Caldwell, Experimental Psychologist, Fatigue Science.
- Dr. Chris Russell, Senior Scientist, Archinoetics, LLC.
- Total: 3 experts.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The document does not explicitly describe an adjudication method for disagreements among experts. For Phase 1, it implies the "research personnel" collectively established the ground truth. For Phase 2, ground truth was established by polysomnography and compared to expert actigraphy scoring, but the method for reconciling potential disagreements among the three mentioned experts or between their expert actigraphy scoring and the SBV2 algorithm is not detailed. However, the study validated the SBV2 against expert human actigraphy scoring (Phase 1) and against polysomnography (Phase 2), suggesting that polysomnography was the ultimate ground truth for in-bed sleep/wake.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a multi-reader, multi-case (MRMC) comparative effectiveness study evaluating human readers with and without AI assistance was not conducted or reported in this document. The study focused on the standalone performance of the SBV2 system against human expert actigraphy and polysomnography.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, a standalone performance study was conducted. The performance metrics (accuracy, sensitivity, specificity) listed in the table are for the SBV2 algorithm's classification of sleep/wake states based on actigraphy data, without human-in-the-loop intervention for classification.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
- Phase 1: Expert human actigraphy scoring ("visual examination and classification of raw actigraphy data").
- Phase 2: Polysomnography (PSG) is explicitly stated as the "accepted medical standard for conducting sleep evaluations" and was used as the ground truth for sleep/wake classification.
8. The sample size for the training set
The document does not provide information about the sample size used for training the SBV2 algorithm. The reported studies are validation studies demonstrating the performance of an already-developed algorithm.
9. How the ground truth for the training set was established
The document does not provide information on how the ground truth for any potential training set was established.
§ 882.5050 Biofeedback device.
(a)
Identification. A biofeedback device is an instrument that provides a visual or auditory signal corresponding to the status of one or more of a patient's physiological parameters (e.g., brain alpha wave activity, muscle activity, skin temperature, etc.) so that the patient can control voluntarily these physiological parameters.(b)
Classification. Class II (special controls). The device is exempt from the premarket notification procedures in subpart E of part 807 of this chapter when it is a prescription battery powered device that is indicated for relaxation training and muscle reeducation and prescription use, subject to § 882.9.