Search Results
Found 15 results
510(k) Data Aggregation
(70 days)
LEL
A wrist-worn activity monitor designed for documenting physical movement associated with applications in physiological monitoring. The device is intended to monitor the activity associated with movement during sleep and make estimates of sleep quantity/quality using accelerometry, based on actigraph algorithms designed specifically for the device's unique signal processing techniques. Can be used to analyze circadian rhythms and assess activity in any instance where quantifiable analysis of physical motion is desirable.
The results of the processed data are graphical and numerical presentations and reports of sleep latency, sleep duration, sleep quality and circadian rhythms for the use by or on the order of physicians, trained technicians, or other healthcare professionals.
The Sleep Watch System is intended for use on a general-purpose computing platform, it does not issue any alarms.
The Sleep Watch System is intended for use in the natural environment for passive, noninvasive, data collection of physiological parameters that will later be transmitted to a SaaS platform for remote review by a clinician. The Sleep Watch device is intended for use in children and older.
The Sleep Watch is a wrist-worn device that monitors activity, temperature, and light exposure, it can be used to analyze sleep quantity and quality, circadian rhythms, automatically collect and store data for sleep parameters, and assess activity, intended for use by or on the order of a Healthcare Professional to aid in the evaluation of sleep disorders based on Actigraphy recordings, typically collected during sleep.
The results of the processed data are graphical and numerical presentations and reports of sleep latency, sleep duration, sleep quality and circadian rhythms for the use by or on the order of physicians, trained technicians, or other healthcare professionals.
The Sleep Watch System is intended for use on a general-purpose computing platform; it does not issue any alarms.
The Sleep Watch system consists of:
- The Sleep Watch built-in with accelerometer, gyroscope, PPG, temperature, and light sensors, as well as BLE and WiFi chips.
- The Sleep Watch collects raw data from each sensor.
- The Sleep Watch processes signals with filters and stores raw data in eMMC storage.
- Psychomotor Vigilance Task (PVT)
- An App manages Sleep Watches
- A web Application Programing Interface (API) to allow authenticated users to upload data collected form Sleep Watch to AMI Cloud Platform
- A database to store the input, intermedium output, final output and associated data.
- A web-based database API to access the database and get outputs.
- A dashboard, a web-based user interface, to display, retrieve, manage, edit, verify, and summarize Sleep Watch outputs.
- Proprietary algorithms to analyze actigraphy.
- A reporting API to generate sleep reports.
The Sleep Watch System is intended for patients in the home environment for passive, noninvasive, data collection of physiological parameters that will later be transmitted to a SaaS platform for remote review by a clinician. The Sleep Watch device is intended for use in children and older.
The Sleep Watch System measures and records:
- PPG (Red, Green, Infrared) raw data
- Accelerometer (X, Y, X) and Gyroscope (Vx, Vy, Vz) raw data
- Light (R, G, B) data
- ZCM (Zero Crossing Mode)
- PIM (Proportional Integrating Measure)
- Estimate Sleep and Wake
- PVT test results
- Skin Temperatures
- MESOR (Midline Estimated Statistic of Rhythm), amplitude, and acrophase
The Sleep Watch allows for on-wrist and/or in-App rating scales (0 to 10), with experimenter selectable initial value (0,5,10) and/or questionnaires (each limited by the constraints of readability). These features should be on-demand, according to an experimenter's selected schedule, or both.
The Sleep Watch device does not provide physiological alarms.
This FDA 510(k) clearance letter and summary for the Sleep Watch device focuses heavily on regulatory compliance, technological comparison, and general software/hardware verification. Crucially, it lacks specific information about clinical performance studies, particularly concerning the quantitative measures of sleep quantity/quality estimates and their accuracy against a gold standard.
Therefore, I cannot fulfill all parts of your request with the provided information. I will construct a response based on the available data, highlighting where information is missing and inferring what would typically be required for such a device clearance.
Here's a breakdown of the acceptance criteria and the study information based on the provided text:
Acceptance Criteria and Device Performance for Sleep Watch
Based on the provided 510(k) summary, the acceptance criteria are not explicitly stated in a quantitative manner (e.g., "accuracy greater than X%"). Instead, the document discusses meeting general design requirements, software verification/validation, and demonstrating substantial equivalence to the predicate device. For a device estimating sleep quantity/quality, performance would typically be assessed by comparing its output to a recognized "gold standard" for sleep measurement, such as Polysomnography (PSG).
Given the absence of specific performance metrics in the provided text, the table below reflects what would typically be expected as acceptance criteria for a device making sleep estimates using actigraphy, and it would normally be accompanied by the device's reported performance against those criteria. As these are not present, I will denote them as "Not Specified in Document."
Acceptance Criteria Category | Typical Metric (Not Specified in Document) | Reported Device Performance (Not Specified in Document) |
---|---|---|
Accuracy of Sleep/Wake Estimation | Sensitivity (true positive rate for sleep) vs. PSG | Not Specified in Document |
Specificity (true negative rate for wake) vs. PSG | Not Specified in Document | |
Overall Agreement/Accuracy vs. PSG | Not Specified in Document | |
Accuracy of Sleep Duration | Mean Absolute Error (MAE) compared to PSG | Not Specified in Document |
Bland-Altman agreement with PSG | Not Specified in Document | |
Accuracy of Sleep Latency | Mean Absolute Error (MAE) compared to PSG | Not Specified in Document |
Reliability/Consistency | Test-retest reliability (e.g., ICC) | Not Specified in Document |
Usability | User satisfaction, ease of use (qualitative) | "Meets its requirements, performs as intended" (general statement) |
Safety | Compliance with electrical, biocompatibility, and cybersecurity standards | Compliant to IEC 60601-1, ISO 10993-1, ANSI/UL 2900-2-1, etc. |
Cybersecurity | Robustness against cyber threats, data integrity | Authentication, authorization, cryptographic controls, etc. |
Study Proving Device Meets Acceptance Criteria
The provided 510(k) summary (Section 7, "Performance Data") describes the testing performed. However, it primarily focuses on non-clinical (software, electrical, and mechanical) testing and verification/validation activities, rather than a clinical performance study demonstrating the accuracy of the sleep estimation algorithms against a gold standard.
Here's the information extracted and inferred from the document:
-
A table of acceptance criteria and the reported device performance:
- As detailed above, specific quantitative acceptance criteria and corresponding reported performance metrics for sleep quantity/quality estimations are not specified in the provided document. The document primarily states that "all pre-defined acceptance criteria for the Sleep Watch were met and all software test cases passed" and that the device "meets its requirements, performs as intended." This refers to internal design and software validation, not clinical performance against a gold standard like PSG.
-
Sample sizes used for the test set and the data provenance:
- Test Set Sample Size: Not Specified. The document refers to "system testing," "verification," and "validation" but does not provide a sample size in terms of patient data or clinical recordings used to validate the accuracy of sleep/wake estimates.
- Data Provenance: Not Specified. There is no mention of the country of origin of any data (clinical or otherwise) or whether it was retrospective or prospective.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Not Applicable/Not Specified. Since a clinical performance study comparing the device's sleep estimations to a ground truth (like PSG scored by experts) is not described in the provided text as part of the "Performance Data," there's no mention of experts establishing ground truth for a test set. This would be a critical component of a clinical validation study for sleep monitoring devices.
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- Not Applicable/Not Specified. As no expert-adjudicated ground truth acquisition process is described for a clinical test set, no adjudication method is mentioned.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, not specified. The document does not describe any MRMC study involving human readers or clinicians using or being aided by the Sleep Watch. This type of study would be more relevant to AI-assisted diagnostic tools where human interpretation is central. The Sleep Watch primarily provides processed data and reports for review by clinicians, it's not described as an AI-assistance tool for human interpretation of raw signals.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Implicitly, yes, for the algorithm's internal function, but not for its clinical accuracy against a gold standard. The document states "Proprietary algorithms to analyze actigraphy" and "Design validation testing which simulated the intended use to confirm that the end-to-end functionality of the Sleep Watch in conjunction with the actigraphy algorithms meets the design requirements." This suggests standalone testing of the algorithms' functionality. However, it does not confirm a standalone clinical performance study where the device's estimated sleep parameters are compared directly to a clinical gold standard (like PSG) without human intervention in the data acquisition/processing chain beyond collecting the actigraphy data.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Not Specified in the context of clinical performance. For the non-clinical testing, requirements were confirmed against "design requirements." For sleep monitoring, the gold standard ground truth would typically be Polysomnography (PSG) data, often scored by certified sleep technologists and overseen by sleep physicians. The document does not state that PSG was used as ground truth for validating the sleep estimation accuracy.
-
The sample size for the training set:
- Not Specified. The document mentions "proprietary algorithms" but does not detail their development, including the size or nature of any training data used for these algorithms.
-
How the ground truth for the training set was established:
- Not Specified. Given the lack of information on training sets, the method for establishing their ground truth is also not mentioned.
Summary of Missing Information Critical for Clinical Performance Evaluation:
The provided 510(k) summary focuses on the technical aspects and regulatory compliance of the Sleep Watch (e.g., software, hardware, safety standards, cybersecurity, and equivalence to a predicate actigraph). It explicitly mentions "Proprietary algorithms to analyze actigraphy" but does not describe the clinical validation study that would typically be performed to demonstrate the accuracy of these algorithms in estimating sleep quantity and quality against a clinical gold standard (like PSG). For a device making sleep estimates, objective clinical performance data (e.g., sensitivity, specificity, accuracy, or agreement metrics against PSG) would be crucial for establishing its effectiveness in its intended use. Without this, the "acceptance criteria" for the clinical performance of its sleep estimation function are not transparent in this document.
Ask a specific question about this device
(155 days)
LEL
DCM is a small worn activity monitor designed for documenting physical movement associated with applications in physiological monitoring.
The device is intended to monitor the activity associated with movement during sleep.
DCM can be used to analyze circadian rhythms and assess activity in any instance where quantifiable analysis of physical motion is desirable.
DCM is indicated for monitoring of adult patients only.
DCM is a wrist-worn wearable device intended to continuously record high resolution digital acceleration data associated with a patient's physical movement.
In practice, a healthcare professional or researcher can prescribe the device to collect physiological data from patients during sleep and in applications where quantifiable analysis of physical motion is desirable.
The device is set up to collect data by the healthcare professional then placed on the subject's wrist. The device is designed to be worn during normal activities and/or during sleep over a period of days to weeks. The patient does not need to interact with the device to control data collection.
The data stored on the device can be transmitted to the cloud for storage, and made accessible to healthcare professionals or researchers for further analysis. Downloaded data can be post-processed based on the timestamp and magnitude of acceleration along each axis.
The DCM system comprises a system of components:
- wearable biosensor (PW010)
- off the shelf mobile device (PW030) running the DCM mobile app (PW400)
- cloud-based data storage and data processing (PW100) (back-end)
- investigator dashboard (PW500) accessed through a web browser (front-end)
The provided FDA 510(k) clearance letter for the DCM (PW-DCM) device does not describe a study involving a test set, ground truth experts, or human readers for assessing device performance related to diagnostic accuracy or interpretation.
Instead, the document focuses on the technical performance of the device as a physical activity monitor, comparing it to a predicate device (Actigraph LEAP) primarily on its physical and operational characteristics. The acceptance criteria and "study" described are more akin to verification and validation (V&V) testing of hardware and software components, rather than a clinical performance study measuring accuracy against a diagnostic gold standard involving human interpretation.
Therefore, many of the requested categories (e.g., number of experts, adjudication method, MRMC study, effect size on human readers, type of ground truth for diagnostic accuracy) are not applicable or cannot be extracted from this document, as the device's function is data collection and not direct diagnostic interpretation.
However, I can extract the information that is present and explain why other information is not available from this document.
Acceptance Criteria and Reported Device Performance
The table below summarizes the technical acceptance criteria for the DCM device and the reported outcomes, as found in the "Summary of Testing" section.
Requirement | Acceptance Criteria / Pass/Fail Criteria | Reported Device Performance (Result) |
---|---|---|
Acceleration Measurement Accuracy | Accuracy of 5% or better (at 1g) in 3 orthogonal directions with sensitivity to at least 0.005g. Accelerometer accuracy to be tested across extended duration data collection runs to confirm no sensor drift. | PASS |
Timing Accuracy (Sensor Data Capture) | Timing accuracy within ±10 seconds per hour. Data is transmitted to the cloud and timestamps are visible and accurate within requirements when viewed in the Investigator Dashboard. | PASS |
Data Storage upon Connectivity Issues | Data is stored on the biosensor when connection to the mobile device is interrupted and transferred when connection is restored. Data is stored on the mobile device when connection to the cloud platform is interrupted and transferred when connection is restored. | PASS |
Usability | Usability activities are conducted according to the IEC 62366-1 process and demonstrates that the usability of the medical device is acceptable with regard to safety. | PASS |
Packaging | Device meets visual inspection criteria and passes functional tests following exposure to typical shipping stresses and rough handling. | PASS |
EMC (Electromagnetic Compatibility) | Device meets requirements for emissions (Class B) and immunity per IEC 60601-1-2 and 47 CFR Part 15 Subpart B. | PASS |
Wireless Coexistence | No interruption to wireless data connections per ANSI C63.27. | PASS |
Radio Frequency (Radiated Spurious Emissions) | Device meets requirements for spurious emissions per 47 CFR 15.247. | PASS |
Electrical Safety | Device meets applicable requirements for electrical, mechanical and thermal safety, for healthcare and home use environments per IEC 60601-1 and IEC 60601-1-11. | PASS |
Software Verification and Validation | Software developed and maintained in accordance with the IEC 62304 lifecycle process, and all verification and validation tests passed. | PASS |
Study Details (based on available information)
-
Sample size used for the test set and the data provenance:
- Test set sample size: Not explicitly stated for each test. The tests described are bench tests ("Bench testing with the biosensor in a range of orientations," "Bench testing with mobile app paired to biosensor," "manual interruption and restoration of connectivity"). This implies testing of device units, not a patient cohort.
- Data provenance: Not explicitly stated. Given the nature of the tests (bench testing, design validation), the "data" being generated is measurement data from the device itself rather than clinical patient data. The document does not refer to geographical origin or patient type for these validation tests.
- Retrospective or Prospective: Not applicable in the context of device design verification and validation testing. These are controlled engineering tests.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Not applicable for these types of tests. The "ground truth" for these engineering and software tests would be established by calibrated measurement equipment (e.g., accelerometers for accuracy, timing devices for accuracy) and adherence to international standards (e.g., IEC 62366-1 for usability, IEC 60601 series for safety, IEC 62304 for software). There is no mention of human experts interpreting data to establish a ground truth for diagnostic purposes because the device's function is data collection, not interpretation.
-
Adjudication method (e.g. 2+1, 3+1, none) for the test set:
- None. This concept is for clinical performance studies where multiple human readers interpret medical images or data. The described tests are technical performance evaluations.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No, an MRMC comparative effectiveness study was not done. The document explicitly states: "DCM did not require clinical studies to support substantial equivalence to the predicate device." The device is a "small worn activity monitor designed for documenting physical movement," not a device that provides AI-assisted interpretations for human clinicians.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, a form of standalone testing was done for the technical performance. The "Summary of Testing" section describes tests where the device's inherent capabilities (e.g., acceleration measurement, timing accuracy, data storage) were evaluated against predetermined engineering criteria. This is performance of the algorithm/device itself, without human interpretation in the loop beyond setting up the test and interpreting the test results (e.g., "PASS").
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Technical/Engineering Standards and Calibrated Equipment: For accuracy measurements, the ground truth would be from highly accurate, calibrated reference instruments. For safety, EMC, and software, the ground truth is adherence to established international standards (e.g., IEC 60601-1, IEC 62304) and internal design specifications. There is no biological or clinical "ground truth" (e.g., pathology, outcomes data, expert consensus on patient diagnosis) applied here.
-
The sample size for the training set:
- Not applicable / Not disclosed. The document does not describe a machine learning algorithm that requires a "training set" in the context of clinical AI. The device collects raw acceleration data. While there might be internal algorithms for processing this data (e.g., activity counts, sleep/wake detection, circadian rhythm analysis from raw data), the document describes validation of the data collection capability, not the performance of an AI model trained on a specific dataset for diagnostic tasks.
-
How the ground truth for the training set was established:
- Not applicable. As no training set for a clinical AI algorithm is described, there's no ground truth establishment for such a set.
Ask a specific question about this device
(182 days)
LEL
The VERABAND™ is a compact, lightweight, body-worn activity monitoring device designed to document physical movement associated with applications in physiological monitoring. The device is intended to monitor limb or body movements during daily living and sleep for a limited time interval (up to 30- days).
The VERABAND™ can be used to assess activity in any instance where quantifiable analysis of physical motion is desired. VERABAND™ is not intended for diagnostic purposes.
The VERABANDTM is a compact, wrist-worn battery-operated wearable device intended for collecting a patient's motion data for assessing patient activity. VERABANDTM is intended to acquire and store data while being worn during normal activities and/or during sleep. The device consists of a wearable band with compact housing for battery-powered on-board electronics with an accelerometer. The recorded activity data is timestamped and stored in non- volatile memory for later retrieval. Downloaded VERABANDTM data can be post-processed based on the timestamp and magnitude of acceleration for reporting.
Here's a breakdown of the acceptance criteria and study information for the Veraband™ device based on the provided FDA 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
Test | Acceptance Criteria | Reported Device Performance |
---|---|---|
Accelerometer Accuracy and Precision | - Accelerometer shall meet full-range accuracy requirement. |
- Accelerometer shall meet static calibration point accuracy requirement.
- Accelerometer shall meet repeatability and reproducibility requirement. | Pass |
| Activation Trigger | - Device shall activate at the required light level. - Device shall not activate while in the packaging. | Pass |
| Device Donning/Doffing | Device shall be able to survive the required donning and doffing. | Pass |
| Device Band Separation and Elongation Forces | - Device shall meet force requirements for elongation. - Device shall meet force requirements for separation. | Pass |
| Battery Life for Duration of Use | Device shall meet the battery life requirements for Expected Device Life. | Pass |
| User Cleaning | Device shall maintain function after a required minimum number of cleanings. | Pass |
| Device Sampling Rate and Full-Scale Dynamic Range | Device shall have a sampling rate and full-scale dynamic range that meets the requirements. | Pass |
| Device Frequency Response | Device shall be within the requirement of the predicate's bandwidth. | Pass |
| VERABAND™ Intended Use | Device shall meet repeatability and reproducibility requirements for activity levels. | Pass |
| VERABAND™ Report Generation Comparison | Device output metrics shall meet the predicate comparison requirements. | Pass |
| Usability | Device shall meet the related usability requirement survey scores. | Pass |
| Packaging Testing | Device shall maintain function per requirements after shipping. | Pass |
| EMC | Device shall meet the related EMC requirements. | Pass |
| Electrical Basic Safety | Device shall meet the related electrical basic safety requirements. | Pass |
| Biocompatibility Testing | Device shall meet the related biocompatibility requirements. | Pass |
| Firmware Verification and Validation Testing | Device shall meet the related Firmware requirements. | Pass |
| Software Verification and Validation Testing | Device shall meet the related Software requirements. | Pass |
2. Sample Size and Data Provenance (for test set, if applicable)
The document primarily describes non-clinical engineering and performance testing. It does not explicitly state a "test set" in the context of clinical data. For the "VERABAND™ intended use" test, it states "Simulated users wearing the device perform activities at different intensities of motion and wear compliance times." While it implies subjects, no specific sample size is provided. The data is non-clinical/simulated.
3. Number of Experts for Ground Truth and Qualifications (for test set, if applicable)
Not applicable, as the provided data focuses on non-clinical and simulated testing for performance validation rather than expert-derived ground truth for a test set of patient data.
4. Adjudication Method (for test set, if applicable)
Not applicable. The reported tests are primarily objective engineering and performance validations against predefined criteria, not subjective human evaluations requiring adjudication.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No. The document explicitly states: "The VERABAND™ did not require clinical studies to support substantial equivalence to the primary predicate device." Therefore, no MRMC study was conducted or reported.
6. Standalone (Algorithm Only) Performance Study
Yes, the majority of the reported testing falls under standalone performance. The "non-clinical testing summary" details various tests (e.g., accelerometer accuracy, battery life, sampling rate, frequency response, report generation comparison) that evaluate the device's technical functionality and performance in isolation or in comparison to a predicate device's output, without human-in-the-loop performance measurement.
7. Type of Ground Truth Used
For the non-clinical tests, the ground truth was based on:
- Established engineering specifications and requirements (e.g., full-range accuracy, required light level, expected device life, minimum number of cleanings, sampling rate, full-scale dynamic range, predicate's bandwidth, repeatability and reproducibility for activity levels, predicate comparison requirements for output metrics).
- Recognized consensus standards (e.g., IEC, ISO, ASTM).
- Simulated motions and user activities.
8. Sample Size for the Training Set
The document does not mention a "training set" in the context of machine learning or AI. The device is described as an activity monitoring device with an accelerometer to acquire and store data, which is then post-processed. It doesn't appear to be an AI/ML-driven diagnostic device that would typically involve a training set for model development.
9. How the Ground Truth for the Training Set Was Established
Not applicable, as there is no mention of a "training set" for an AI/ML model. The device's operation relies on sensor data acquisition and processing based on established algorithms and engineering principles, not on a trained machine learning model.
Ask a specific question about this device
(142 days)
LEL
The Oxevision Sleep Device is an activity monitor designed and intended for documenting physical movements associated with applications in physiological monitoring. The device's intended use is to analyze subject activity, movement and physiological sign data associated with movement during sleep and to extract information about certain sleep parameters from these movements and physiological sign data.
The device provides a timeline of periods when a bed space is occupied, and periods when a subject is asleep when the bed space is occupied.
The Oxevision Sleep Device is software assessing video from a fixed-installation device for use within single occupancy bed spaces within hospitals, general care and secured environments.
The Oxevision Sleep Device is indicated for use on subjects 18 years of age or older.
Oxevision Sleep is a software-only medical device (SaMD) that provides noncontact sleep assessment in the inpatient setting based on the analysis of patient movement, activity and physiological sign data derived from video, without the need for contact devices to be attached to the patient or bed.
The device consists of custom-designed software assessing video footage collected using off-the-shelf cameras installed within single occupancy bed spaces within hospitals, general care and secured environments. Proprietary software-controlled algorithms are used to derive patient movement, activity and physiological sign data and then to obtain information on bed occupancy and sleep state from the analysis of this data.
The device software automates recognition of sleep periods, generation of sleep reports, and their presentation in a graphical display for use by a healthcare professional.
Here's a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:
Acceptance Criteria and Device Performance for Oxevision Sleep Device
1. Table of Acceptance Criteria and Reported Device Performance
Acceptance Criteria | Oxevision Sleep Device Reported Performance | Meets Criteria? |
---|---|---|
Bed Occupancy Detection: Accuracy of periods of bed occupancy not inferior to 95% | 99% (95% CI: 99.0% - 99.7%) | Yes |
Sleep/Wake Classification (Overall Agreement): Not inferior to 82% | 90% (95% CI: 89.0% - 91.8%) | Yes |
Sleep/Wake Classification (Positive Agreement): Not inferior to 88% | 94% (95% CI: 92.3% - 95.6%) | Yes |
Sleep/Wake Classification (Negative Agreement): Not inferior to 55% | 80% (95% CI: 74.3% - 83.5%) | Yes |
2. Sample Size and Data Provenance for Test Set
- Sample Size: 60 individuals, resulting in a total of 772.65 hours of data.
- Data Provenance: The text does not explicitly state the country of origin. It mentions "a sample of 60 individuals" and "validation data collected from the 60 adults." The study appears to be prospective as it involved collecting "Reference measurements (physiological signals and video polysomnography data) ... concurrently from a standard off-the-shelf camera and hardware installed in two rooms."
3. Number of Experts and Qualifications for Ground Truth (Test Set)
- Polysomnography (PSG) Scoring:
- Number of Experts: Three trained sleep physiologists.
- Qualifications: "trained sleep physiologists, blinded to the video data collected by the standard off-the-shelf camera." They scored in accordance with the American Academy of Sleep Medicine Manual for the Scoring of Sleep and Associated Events version 2.6 of January 2020.
- Bed Occupancy Annotation:
- Number of Experts: Two reviewers.
- Qualifications: "blinded to the algorithm development details."
4. Adjudication Method for Test Set
- Sleep State (PSG): The ground truth for sleep state was established using "triple-scored PSG data" with an "epoch-by-epoch majority vote." Epochs where no majority label was available (e.g., due to artifact) were excluded from the analysis.
- Bed Occupancy (Video Annotation): The ground truth for bed occupancy was established by "two reviewers, blinded to the algorithm development details" who "reviewed and annotated" the video data. The specific adjudication method beyond "annotated" by two reviewers is not explicitly detailed (e.g., if discrepancies were resolved by a third reviewer).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- The document does not indicate that an MRMC comparative effectiveness study was done to evaluate the effect size of human readers improving with AI vs. without AI assistance. The study focuses on the standalone performance of the algorithm against reference standards.
6. Standalone Performance
- Yes, a standalone (algorithm only without human-in-the-loop performance) study was done. The entire clinical performance section describes the algorithm's performance against established reference standards for bed occupancy detection and sleep/wake classification.
7. Type of Ground Truth Used
- Bed Occupancy: Expert annotation of video data.
- Sleep/Wake Classification: Expert consensus from "triple-scored PSG data" by trained sleep physiologists, adhering to AASM guidelines. This can be categorized as a type of expert consensus based on a gold-standard diagnostic tool (PSG).
8. Sample Size for Training Set
- The document does not explicitly state the sample size for the training set. The "Clinical Performance" section specifically focuses on the "validation data collected from the 60 adults."
9. How Ground Truth for Training Set Was Established
- The document does not explicitly state how the ground truth for the training set (if distinct from the validation set) was established. It only describes the ground truth establishment for the clinical validation test set.
Ask a specific question about this device
(28 days)
LEL
The ActiGraph LEAP™ is a small worn activity monitor designed for documenting physical movement associated with applications in physiological monitoring. The device is intended to monitor the activity associated with movement during sleep. The ActiGraph LEAP™ can be used to analyze circadian rhythms and assess activity in any instance where quantifiable analysis of physical motion is desirable.
The ActiGraph LEAP™ is a wrist-worn wearable device intended to continuously record high resolution digital acceleration data associated with a patient's physical movement. In practice, a healthcare professional or researcher can prescribe the device to collect physiological data from patients in applications where quantifiable analysis of physical motion is desirable. Having physical characteristics like those of an electronic wristwatch, the device is set to collect data by the healthcare professional then placed on the subject's wrist. The device is designed to be worn during normal activities and/or during sleep over a period of days to weeks. The patient does not need to interact with the device to control the operation or data collection. The data stored on the device can be downloaded via USB or Bluetooth Low Energy and made accessible to healthcare professionals or researchers for further analysis.
The ActiGraph LEAP™ device will be supported by accessories for recharging the battery and transferring data from the device. A USB Charging Dock with a three-foot USB A cable for both charging and data transfer to a PC using the supplied communication software. The USB Charging Dock connects to the recessed electrical contacts on the back of the device. An off-the-shelf international Wall Mount AC Adapter is also supplied for optional wall charging. The USB Charging Dock can be plugged into the Wall Mount AC Adapter's USB A port for charging the device.
The device uses a high-resolution digital accelerometer to accurately measure linear accelerations in 3axes associated with the patient's physical movement. The accelerometer technology is a microelectromechanical system (MEMS) implemented as an integrated circuit. The accelerometer data is converted to a digital representation on the MEMS accelerometer and then recorded, with timestamp, to the device's on-board memory. The memory is an 8 Gb serial NAND flash capable of storing 30 days of accelerometer data under the default operating mode. The sample rate of the accelerometer is configurable at the following rates: 32Hz, 64Hz, 128 Hz and 256Hz.
The LCD display indicates the battery level, current functional state of the device, and date and time. The device has a 30-day battery life under the default operating mode and can be charged using the USB Charging Dock accessory. The display does not provide feedback to the wearer/patient regarding data measures. There is a simple button on the side used to turn on the display so the wearer can read the date/time and button presses are recorded in the log.
The device firmware executes on internal processors to control the device operations, display, and external communication protocols. The accelerometer sensor data can be downloaded from the device either via USB (using the dock) or via Bluetooth Low Energy.
The provided text is a 510(k) summary for the ActiGraph LEAP activity monitor. It details device characteristics, intended use, and comparison to a predicate device. However, it does not contain any information about acceptance criteria or a study that proves the device meets specific performance criteria related to its functionality (e.g., accuracy of movement tracking, sleep monitoring, or circadian rhythm analysis).
The document focuses on demonstrating substantial equivalence to a predicate device based on:
- Same Indications for Use: Both the predicate and subject devices are intended to monitor activity associated with movement during sleep, analyze circadian rhythms, and assess activity where quantifiable analysis of physical motion is desirable.
- Similar Technological Characteristics: Both use MEMS accelerometers, are wrist-worn, have similar displays, power sources, and data transfer methods.
- Biocompatibility Testing: This addresses changes in patient-contacting materials, ensuring they are still safe.
The document explicitly states: "Clinical testing is not applicable to this submission." This means that no clinical study was conducted to establish performance metrics like accuracy or effectiveness against ground truth on human subjects for this 510(k) clearance.
Therefore, I cannot provide the requested information regarding acceptance criteria and a study proving the device meets them, as that information is not present in the provided text. The submission focuses on showing that the new device is substantially equivalent to an already cleared device, rather than proving de novo performance against specific acceptance criteria.
To answer your request, if this were a dataset that did contain a study with acceptance criteria, the information would typically be presented as follows:
Example of how the information would be presented if available in a different document:
1. Table of Acceptance Criteria and Reported Device Performance (Hypothetical):
Metric | Acceptance Criteria | Reported Device Performance (Hypothetical) |
---|---|---|
Sleep/Wake Accuracy | Sensitivity > 90%, Specificity > 85% vs. Polysomnography | Sensitivity: 92.5%, Specificity: 88.0% |
Activity Count Error | Mean Absolute Error 0.8 vs. Actigraphy Reference Device | Pearson's r: 0.85 |
2. Sample Size and Data Provenance (Hypothetical):
- Test Set Sample Size: 150 participants (e.g., 50 healthy adults, 50 insomnia patients, 50 shift workers).
- Data Provenance: Prospective, multi-center study conducted in the USA, UK, and Germany.
3. Number and Qualifications of Experts (Hypothetical):
- Experts: 3 Board-Certified Sleep Physicians (average 12 years of experience in sleep medicine, specializing in polysomnography interpretation).
4. Adjudication Method (Hypothetical):
- Adjudication: 2+1. Initial assessment by two experts; in cases of disagreement, a third senior expert provided a binding decision.
5. MRMC Comparative Effectiveness Study (Hypothetical):
- MRMC Study: Yes, an MRMC study was conducted comparing sleep staging performance of human experts with and without AI assistance from the ActiGraph LEAP data.
- Effect Size: Human readers improved sleep stage classification accuracy by an average of 7% (from 82% to 89%) when assisted by the AI algorithm compared to performing the task unassisted.
6. Standalone Performance (Hypothetical):
- Standalone Performance: Yes, the algorithm achieved 91% accuracy in detecting sleep onset/offset events and 87% accuracy in differentiating wake, NREM, and REM sleep stages when compared to polysomnography.
7. Type of Ground Truth (Hypothetical):
- Ground Truth: Polysomnography (PSG) for sleep parameters, motion capture system for activity counts, and validated actigraphy devices for circadian rhythm analysis.
8. Training Set Sample Size (Hypothetical):
- Training Set Sample Size: 5,000 subjects.
9. How Ground Truth for Training Set was Established (Hypothetical):
- Training Ground Truth: Ground truth for the training set was established through a combination of expert-annotated polysomnography data from a diverse patient population, alongside simultaneously recorded high-resolution motion data from the device and other reference sensors. Annotations were initially made by trained technicians and then reviewed and confirmed by a panel of 5 board-certified sleep specialists using an iterative consensus approach.
Ask a specific question about this device
(30 days)
LEL
The ActiGraph CentrePoint Insight Watch is a small worn activity monitor designed for document associated with applications in physiological monitoring. The device is intended to monitor the activity associated with movement during sleep. The Insight watch can be used to analyze circadian rhythms and assess activity in any instance where quantifiable analysis of physical motion is desirable.
The ActiGraph CentrePoint Insight Watch is a compact, battery-operated wearable activity recording device with physical characteristics similar to those of a small wristwatch. The watch is intended to acquire and store data from an onboard accelerometer while being worn during normal activities and/or during sleep. The data record is timestamped and stored in non-volatile memory for later retrieval. Downloaded data can be post-processed based on the timestamp and magnitude of acceleration along each axis.
The housing is constructed of a combination of opaque and clear copolyesters formulated specifically for medical devices (i.e., tested and determined biocompatible), and the core data collection sensor is a 3-axis microelectromechanical system (MEMS) accelerometer. A charging dock connected to a USB power source is used to charge the device battery and communicate with a PC or peripheral.
The ActiGraph CentrePoint Insight Watch is a wrist-worn activity monitor designed for physiological monitoring, particularly for tracking movement during sleep to analyze circadian rhythms and assess physical motion.
Here's a breakdown of the acceptance criteria and supporting studies:
- Table of acceptance criteria and the reported device performance:
Characteristic | Acceptance Criteria (Predicate Device K080545) | Reported Device Performance (Subject Device K181077) |
---|---|---|
Indications for Use | Rx Only; A small worn activity monitor designed for documenting physical movement associated with applications in physiological monitoring. The device is intended to monitor the activity associated with movement during sleep. Can be used to analyze circadian rhythms and assess activity in any instance where quantifiable analysis of physical motion is desirable. | Same |
Materials of Construction | Polycarbonate (housing); Nylon & Velcro® (wrist band) | Different: Combination Copolymer (housing); Silicon (wrist band), conforms to 10993-1 Fourth edition 2009-10-15 |
Power Source | Lithium Ion Battery Rechargeable via USB | Same |
Accelerometer Type | Microelectromechanical system (MEMS)-based integrated circuit | Same |
Accelerometer Sampling Rate | 30 Hz, Analog method | Different: Digital method, 32 Hz – 256 Hz |
Accelerometer Dynamic Range | +/- 5 g | +/- 8 g |
Firmware | Embedded C | Embedded C (updated version) |
Wireless Communications Interface | Polar® module | Different: Bluetooth® Low Energy; conforms to AAMI / ANSI / IEC 60601-1-2, Medical Electrical Equipment - Part 1-2: General Requirements for Safety - Collateral Standard: Electromagnetic Compatibility - Requirements and Tests and IEC 60601-1-2, Medical Electrical Equipment - Part 1-2: General Requirements For Basic Safety And Essential Performance - Collateral Standard: Electromagnetic Disturbances – Requirements And Tests. |
Memory | 1024kB | 512 MB |
Heart Rate | BPM | Same |
Accelerometer Sensitivity | 4 milli-g per Least Significant Bit | Different: 2.4 milli-g per Least Significant Bit |
Storage Temperature | -10 °C to 50 °C | Same |
Operating Temperature | 0 °C to 40 °C | Different: -10°C to 55°C (discharging); 0°C to 45°C while charging |
Water Resistance | IP21 (condensation) | Minimum IP57 (1m for 30 minutes) |
Weight | 51 grams | 33 grams |
Size | Width: 3.37 in (85.6 mm); Height: 1.5 in (38.1 mm); Thickness: 0.6 in (15.2 mm) | Width: 1.41 in (35.8 mm); Height: 1.97 in (50.1 mm); Thickness: 0.41 in (10.5 mm) |
Recording Time @ 1 min. Epoch | 14 days | 30 days |
-
Sample size used for the test set and the data provenance (e.g., country of origin of the data, retrospective or prospective):
The document states that "This premarket submission did not rely on the assessment of clinical performance data to demonstrate substantial equivalence." Therefore, there is no clinical test set described in this submission. The "test set" in this context refers to devices used for non-clinical bench testing. The sample size for these non-clinical tests is not explicitly mentioned, nor is the provenance of data for these tests beyond being "bench testing." -
Number of experts used to establish the ground truth for the test set and the qualifications of those experts: Not applicable, as no clinical test set for ground truth establishment is described. The acceptance is based on substantial equivalence to a predicate device through non-clinical bench testing.
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set: Not applicable, as no clinical test set requiring adjudication is described.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance: Not applicable. This device is an activity monitor, not an AI-assisted diagnostic tool that would involve human readers or MRMC studies.
-
If a standalone (i.e., algorithm only without human-in-the loop performance) was done: The document describes the device as a standalone activity monitor that records and stores data from an accelerometer. Non-clinical bench testing was performed to demonstrate its performance and reliability in this standalone function. The "study" mentioned is the series of non-clinical bench tests performed to support substantial equivalence.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.): For the non-clinical tests, the "ground truth" would be established engineering specifications and recognized voluntary consensus standards. For example, accelerometer dynamic range, sampling rate, memory capacity, water resistance, weight, size, and recording time were compared against the predicate device's capabilities and against the device's own internal specifications validated through bench testing. The non-clinical bench tests included:
- Performance and reliability testing
- Comparative data analysis
- Basic safety and essential performance in accordance with AAMI ES60601
- Electromagnetic compatibility (EMC) in accordance with IEC 60601
- Biocompatibility and material standards confirms there is no harm to the patient wearing the device.
- System compatibility with ActiGraph software for data download and collection
-
The sample size for the training set: Not applicable. This is not an AI/ML device that requires a training set for model development.
-
How the ground truth for the training set was established: Not applicable, as there is no training set for an AI/ML model for this device.
Ask a specific question about this device
(329 days)
LEL
The ActTrust is an ultra-compact, lightweight, wrist-worn activity, temperature and ambient light monitor that can be used to analyze circadian rhythms, automatically collect and store data for sleep parameters, and assess activity in any instance where quantifiable analysis of physical motion is desirable. The device is intended to monitor limb or body movements during daily living and sleep. ActTrust is indicated for adults of 22 years of age and over.
ActTrust is a compact, ambulatory, battery-operated data recorder with physical characteristics similar to a small wristwatch. The ActTrust is intended for the acquisition and analysis of the physical activity of the body, peripheral temperature and light exposure during daily living and sleep. The ActTrust uses state of the art miniature electronic technology to measure the data and store these data within the device. The ActTrust require operational software to allow configuration, data download, storage and off-line analysis of activity data by a health care provider. The device is connected directly by means of a standard Universal Serial Bus connection for configuration and download.
The ActTrust device is an activity, temperature, and ambient light monitor designed to analyze circadian rhythms, collect sleep parameters, and assess physical movement.
Here's an analysis of the acceptance criteria and study proving the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance:
Requirement Summary | Pass/Fail Criteria | Test Result |
---|---|---|
The device shall measure linear acceleration with an accuracy of +/-5% over the full range. | The recorded acceleration over the test range shall meet the requirement. | PASS |
The device accuracy shall be |
Ask a specific question about this device
(139 days)
LEL
The MotionWatch and PRO-Diary are compact, lightweight, body-worn activity monitoring devices that may be used to document physical movement associated with applications in physiological monitoring. The devices are intended to monitor limb or body movements during daily living and sleep. The MotionWatch and PRO-Diary can be used to assess activity in any instance where quantifiable analysis of physical motion is desired.
Additionally, the PRO-Diary has a built-in score pad that allows the wearer to subjectively assign and enter responses to pre-programmed questions. The score pad can be used as a substitute or in addition to the traditional written patient diary in conjunction with activity monitoring.
MotionWatch and PRO-Diary are compact, ambulatory, battery-operated activity recorders with physical characteristics similar to a small wristwatch.
The MotionWatch and PRO-Diary are intended for the acquisition and analysis of the physical activity of the body during daily living and sleep. The MotionWatch and PRO-Diary use state of the art miniature accelerometer technology to measure movements of the limb or torso and store these data within the devices differ in that the MotionWatch incorporates an ambient light sensor whereby the PRO-Diary incorporates a display and score-pad to allow subjective inputs.
The MotionWatch and PRO-Diary require operational software to allow configuration. data download, storage and off-line analysis of activity data by a health, care provider. The software can be run on an IBM-Compatible PC and the device is connected directly by means of a standard Universal Serial Bus connection for configuration and download.
The MotionWatch and PRO-Diary utilize a motion sensor known as an "accelerometer" to measure the occurrence and degree of motion. The sensor is a solid state device with a digital output directly proportional to physical acceleration in 1, 2 or 3 axes of orientation. The acceleration data are processed into "counts" before being stored in the non-volatile memory of the device.
Here's the breakdown of the acceptance criteria and the study details for the MotionWatch and PRO-Diary devices, based on the provided text:
Acceptance Criteria and Device Performance
The provided document details non-clinical performance testing for both the MotionWatch and PRO-Diary. The acceptance criteria and reported device performance are presented below. It's important to note that these are engineering performance specifications rather than clinical study endpoints.
MotionWatch Non-Clinical Performance Testing:
Requirement Summary | Test/Verification Method | Pass/Fail Criteria | Test Result |
---|---|---|---|
Measure linear acceleration with an accuracy of +/-5% over the full range | Apply a range of simulated reference acceleration and record the results. | The recorded acceleration over the test range shall meet the requirement. | PASS |
Accuracy shall be |
Ask a specific question about this device
(198 days)
LEL
The SBV2 System is an activity monitor designed and intended for documenting physical movements associated with applications in physiological monitoring. The device's intended use is to analyze limb activity associated with movement during sleep and to extract information about certain sleep parameters from these movements. SBV2 can also be used to assess activity in any instance where quantifiable analysis of physical motion is desirable. The use of SBV2 is indicated for adults 22 years of age and over.
The SBV2 System is a device that monitors activity. It relies on the measurement and analysis of wrist movements to detect and characterize sleep/wake periods. The device allows some aspects of sleep derived from the analysis of activity to be reported. The SBV2 System is graphically depicted in Figure 1.
Here's a breakdown of the acceptance criteria and study information for the SBV2 System, based on the provided text:
Acceptance Criteria and Device Performance
Acceptance Criteria (Target Performance) | Reported Device Performance (SBV2) |
---|---|
Phase 1: Differentiation of Activity States | |
Identify "Awake, up and about" | 95% accuracy |
Identify "In bed, awake or asleep" | 95% accuracy |
Identify "Off wrist" | 87% accuracy |
Phase 2: Sleep/Wake Classification vs. Polysomnography (in-bed) | |
Sensitivity (correctly identify sleep) | 88% |
Specificity (correctly identify wake) | 55% |
Overall SBV2 vs. Polysomnography Agreement | 93% (weighted average of 95% in-bed state detection and 88% in-bed-asleep detection) |
Detailed Study Information
1. A table of acceptance criteria and the reported device performance
(See table above)
2. Sample sizes used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)
- Phase 1 (Differentiation of Activity States):
- Sample Size: 180 participants
- Data Provenance: Not explicitly stated, but the study was conducted by Archinoetics, LLC and Fatigue Science. Given the later mention of the Kettering Sleep Disorders Center in Ohio for Phase 2, it's plausible the data originates from the US. The data was "de-identified." This phase was retrospective, using existing actigraph data.
- Phase 2 (Sleep/Wake Classification vs. Polysomnography):
- Sample Size: 50 patients
- Data Provenance: Conducted at the Wallace Kettering Health Networks Sleep Disorders Center in Kettering, Ohio, USA. This phase involved simultaneous collection of actigraphy data and polysomnography, suggesting a prospective study design.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)
- Phase 1: "Research personnel at Archinoetics, LLC," described as having "substantial expertise in the visual examination and classification of raw actigraphy data." The exact number of experts is not specified.
- Phase 2:
- Dr. Donna Arand, Board Certified Sleep Specialist, Kettering Hospital.
- Dr. John Caldwell, Experimental Psychologist, Fatigue Science.
- Dr. Chris Russell, Senior Scientist, Archinoetics, LLC.
- Total: 3 experts.
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The document does not explicitly describe an adjudication method for disagreements among experts. For Phase 1, it implies the "research personnel" collectively established the ground truth. For Phase 2, ground truth was established by polysomnography and compared to expert actigraphy scoring, but the method for reconciling potential disagreements among the three mentioned experts or between their expert actigraphy scoring and the SBV2 algorithm is not detailed. However, the study validated the SBV2 against expert human actigraphy scoring (Phase 1) and against polysomnography (Phase 2), suggesting that polysomnography was the ultimate ground truth for in-bed sleep/wake.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
No, a multi-reader, multi-case (MRMC) comparative effectiveness study evaluating human readers with and without AI assistance was not conducted or reported in this document. The study focused on the standalone performance of the SBV2 system against human expert actigraphy and polysomnography.
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, a standalone performance study was conducted. The performance metrics (accuracy, sensitivity, specificity) listed in the table are for the SBV2 algorithm's classification of sleep/wake states based on actigraphy data, without human-in-the-loop intervention for classification.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)
- Phase 1: Expert human actigraphy scoring ("visual examination and classification of raw actigraphy data").
- Phase 2: Polysomnography (PSG) is explicitly stated as the "accepted medical standard for conducting sleep evaluations" and was used as the ground truth for sleep/wake classification.
8. The sample size for the training set
The document does not provide information about the sample size used for training the SBV2 algorithm. The reported studies are validation studies demonstrating the performance of an already-developed algorithm.
9. How the ground truth for the training set was established
The document does not provide information on how the ground truth for any potential training set was established.
Ask a specific question about this device
(83 days)
LEL
The Cadwell Easy Body Position Module is intended for use to detect body positions during physiological recording. It provides output data that corresponds to five body positions (supine, prone, left side, right side and sitting up). It may be used in a clinical or ambulatory setting for both EEG and sleep disorder studies.
Not Found
The provided text is a 510(k) premarket notification letter from the FDA for a device called "EasyNet Body Position Module". This document primarily focuses on regulatory approval and does not contain the detailed information necessary to answer your request about acceptance criteria, study design, and performance metrics.
The document states the device's indications for use and general regulatory information, but it does not include:
- A table of acceptance criteria and reported device performance.
- Sample sizes for test sets, data provenance, number of experts, adjudication methods, MRMC studies, standalone performance, or training set details.
- Specific ground truth information.
Therefore, based solely on the provided text, I cannot answer the questions about the acceptance criteria and the study that proves the device meets them. This type of information is typically found in the full 510(k) submission, specifically in the performance testing sections, which are not included in this FDA decision letter.
Ask a specific question about this device
Page 1 of 2