(145 days)
The device is intended to measure, amplify, and record physiological signals acquired from a patient for archival in a sleep study. The physiological signals and conditioned for analysis and display. The data may be analyzed in real-time or offline on dedicated polysomnography software running on a personal computer by a qualified sleep clinician to aid in the diagnosis of Sleep Disorders.
The device is intended for use by healthcare professionals within a hospital, laboratory, clinic, or nursing home; or outside of a medical facility under direct supervision of a medical professional.
The device is intended for use on both adults and children only under the direction of a physician or qualified sleep technician.
The device or any accessory, does not include alarms, and is not intended to be used as a critical component of an alarm system.
The device or any accessory, is not to be used alone as an apnea monitor or as a component in an apnea monitoring system.
The device or any accessory, is not to be used alone as a life support device or as a critical component of a life support system.
The Polysmith Sleep System, Model NTI6600 is intended to amplify and record physiologic potentials used for Polysomnography (PSG) or Sleep Studies. The device consists of a compatible amplifier, head box, PC, patient sensors, and may include optional external devices, USB DC Box, and audio/video input devices.
Compatible amplifiers may use commercially available sensors and electrodes, an internal SpO2 module, and internal pressure transducers to collect, digitize, and send physiological signals to the host PC.
The Polysmith software may record from video, speaker and microphone equipment. The Polysmith software may also record auxiliary signals from compatible amplifiers or USB DC Box which allow for data inputs from compatible sources.
Polysmith records and displays the data for online or offline review. Qualified practitioners use the information to score Polysomnograms and diagnose Sleep Disorders.
Here's a breakdown of the acceptance criteria and study details for the Polysmith Sleep System, Model NTI6600, based on the provided document:
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" as separate performance thresholds the device must meet. Instead, it presents the "Automated Analysis Agreement" as the device's performance, which is then compared against "Average Human Scoring Agreement." Assuming the implied acceptance is substantial equivalence to human performance for these metrics, the table is structured as follows:
| Metric | Implied Acceptance: Match/Approach Average Human Scoring Agreement | Reported Device Performance (Average Automated Analysis Agreement) | Difference (Human - Automated) | Confidence Interval +/- (95.0%) |
|---|---|---|---|---|
| Sleep Staging | ~82.8% | 71.53% | 11.27% | 3.04% |
| Microarousal | ~88.8% | 80.15% | 8.65% | 3.98% |
| Apnea | ~98.7% | 97.28% | 1.42% | 1.32% |
| Hypopnea | ~96.4% | 95.44% | 0.96% | 2.55% |
| Desaturation | ~97.1% | 95.72% | 1.38% | 1.32% |
| Limb Movement | ~98.9% | 92.57% | 6.33% | 4.35% |
In the "Additional Sleep Staging Analysis," which seems to be a more detailed analysis, the total agreement for automated scoring against consensus human scoring is:
| Metric | Implied Acceptance Criteria | Reported Device Performance |
|---|---|---|
| Total Agreement | Not explicitly stated | 70.35% |
| Kappa | Not explicitly stated | 0.58 |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size (Automated Analysis): 8,053 recorded epochs, of which 7,606 were scored.
- Sample Size (Additional Sleep Staging Analysis): 20 sleep studies.
- Data Provenance: The studies were chosen at random from existing sleep studies of acceptable signal quality from accredited labs. For the automated analysis, studies were chosen at random from a sleep lab independent of Neurotronics. The document does not specify the country of origin, but given the FDA submission, it's likely US-based or compliant with US standards. The data is retrospective.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
- Automated Analysis: Two expert scorers from different organizations. Each is a registered sleep clinician with at least 5 years' experience scoring sleep records.
- Additional Sleep Staging Analysis: Three independent scorers. Each is a registered sleep specialist with at least five years' experience in the field. Each had some prior experience using Polysmith to manually score studies.
4. Adjudication Method for the Test Set
- Automated Analysis: The document states "The human scoring was completed by two expert scorers." It then calculates "Average Human Scoring Agreement," implying an average of their individual scores or comparison. It doesn't explicitly describe an adjudication process for disagreements between the two human scorers to establish a single ground truth for direct comparison with the automated analysis. Instead, it compares the automated system's performance against this average human agreement, and also compares the two human scorers' agreement with each other.
- Additional Sleep Staging Analysis: This section explicitly used a majority method (2/3 majority-rule) to establish the "true sleep staging (consensus)." Epochs where there was no agreement among the human scorers were discarded (non-consensus).
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
- No, a direct MRMC comparative effectiveness study "with AI vs without AI assistance" was not explicitly described in the context of improving human reader performance. The study focused on the standalone performance of the automated analysis system compared to human scoring. While humans scored the data, the goal was to assess the device's accuracy relative to human experts, not to measure the improvement in human performance aided by the device.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
- Yes, a standalone performance analysis was conducted. The "Automated Analysis Agreement" and the "Results" section for "Additional Sleep Staging Analysis" directly report the performance of the algorithm itself against human expert assessments (either average or consensus). The scorers for the "Additional Sleep Staging Analysis" were explicitly provided with a version of Polysmith that "did not have any automated analysis capability," emphasizing that the human scoring served purely as ground truth.
7. The Type of Ground Truth Used
- Expert Consensus / Expert Scoring: For both analyses, the ground truth was established by human expert scorers. In the "Automated Analysis," it's based on two expert scorings. In the "Additional Sleep Staging Analysis," it's based on a 2/3 majority rule consensus of three expert scorers.
8. The Sample Size for the Training Set
- Not provided. The document states, "The studies used for this testing were not used to train the algorithm," but it does not specify the size or characteristics of the training set used for the algorithm's development.
9. How the Ground Truth for the Training Set Was Established
- Not provided. The document focuses solely on the ground truth for the test sets and explicitly states the test sets were not used for training. Information on how the training data's ground truth was established is absent.
§ 882.1400 Electroencephalograph.
(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).