Search Results

The Rune Labs Kinematic System is intended to quantify kinematics of movement disorder symptoms including tremor and dyskinesia, in adults (45 years of age or older) with mild to moderate Parkinson's disease.

The Rune Labs Kinematic System collects derived tremor and dyskinesia probability scores using processes running on the Apple Watch, and then processes and uploads this data to Rune's cloud platform where it is available for display for clinicians.

The Rune Labs Kinematic System uses software that runs on the Apple Watch to measure patient wrist movements. These movements are used to determine how likely dyskinesias or tremors are to have occurred. The times with symptoms are then sent to the Rune Labs Cloud Platform using the Apple Watch's internet connection, which is then displayed for clinician use.

The Apple Watch contains accelerometers and gyroscopes which provide measurements of wrist movement. The Motor Fluctuations Monitor for Parkinson's Disease (MM4PD) is a toolkit developed by Apple for the Apple Watch that assesses the likely presence of tremor and dyskinesia as a function of time. Specifically, every minute, the Apple Watch calculates what percentage of the time that tremor and dyskinesia were likely to occur. The movement disorder data that is output from the Apple's MM4PD toolkit have been validated in a clinical study (Powers et al., 20211).

The Rune Labs Kinematic System is software that receives, stores, and transfers the Apple Watch MM4PD classification data to the Rune Labs Cloud Platform where it is available for visualization by clinicians. The device consists of custom software that runs on the users' smart watch and web browsers.

Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:

Acceptance Criteria and Reported Device Performance

The acceptance criteria are implicitly defined by the correlation and differentiation shown by the device's measurements against established clinical ratings and conditions. The study highlights the performance in terms of correlation coefficients and statistical significance.

Study Details (Powers et al., 2021)

1. Sample sizes used for the test set and the data provenance:

   • Motion Measurement Correlation (initial validation step): A single healthy control subject (likely a very small test set to validate the sensor itself, not the clinical algorithm performance).

   • Tremor Validation:

     • Design Set: n = 95 patients (from longitudinal patient study)
     • Hold-out Set: n = 43 patients (from longitudinal patient study)
     • False Positive Testing: 171 elderly, non-PD longitudinal control subjects.

   • Dyskinesia Validation:

     • Choreiform Movement Score (CMS) differentiation:
       • 65 subjects with confirmed absence of in-session dyskinesia (89 tasks)
       • 69 subjects with discordant dyskinesia ratings (109 tasks)
       • 19 subjects with confirmed dyskinesia across all three raters (22 tasks)
     • Longitudinal Dyskinesia Detection:
       • Design Set: 125 patients with no known dyskinesia, 32 patients with chorea.
       • Hold-out Set: 47 subjects with no reported dyskinesia, 10 subjects with chorea.
     • False Positive Testing: 171 elderly, non-PD longitudinal control subjects.

   • Data Provenance: The study was conducted by Apple, implying a global or multi-center approach, but specific country of origin is not mentioned. The studies were likely prospective observational studies where data was collected over time from participants wearing the Apple Watch. Some initial development data may have been retrospective, but the validation steps appear prospective.

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g., radiologist with 10 years of experience):

   • For the Dyskinesia validation (specifically the "Choreiform Movement Score" differentiation), three MDS-certified experts were used to provide dyskinesia ratings during multiple MDS-UPDRS assessments. Their specific experience level (e.g., "10 years of experience") is not detailed, but MDS certification implies a high level of specialized expertise in movement disorders.
   • For the Tremor validation, the "clinician's overall tremor rating" and "MDS-UPDRS tremor constancy score" were used. While it mentions "clinician's," it doesn't specify if this was a consensus or single reading, nor the number of clinicians. Given the use of MDS-UPDRS, it implies assessment by trained medical professionals (neurologists or movement disorder specialists).

3. Adjudication method (e.g., 2+1, 3+1, none) for the test set:

   • For Dyskinesia validation, the ratings from the three MDS-certified experts were categorized as:
     • "confirmed absence" (all three agreed absence)
     • "discordant" (raters disagreed)
     • "confirmed dyskinesia" (all three agreed presence).
       This implicitly suggests a form of consensus-based adjudication (3/3 agreement for "confirmed," disagreement acknowledged for "discordant").
   • For Tremor validation, the adjudication method for the "clinician's overall tremor rating" or "MDS-UPDRS tremor constancy score" is not explicitly stated. It likely refers to standard clinical assessment practices using the UPDRS scale, which can be done by a single trained rater or with multiple raters for research purposes (though not explicitly detailed here as an adjudication).

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No, a multi-reader, multi-case (MRMC) comparative effectiveness study evaluating human readers with vs. without AI assistance was not described. The study focused on validating the device's standalone ability to quantify movements against clinical ground truth (UPDRS scores, expert ratings of dyskinesia). The device is described as quantifying kinematics for clinicians to display, implying it's an assessment tool rather than an AI-assisted diagnostic aid for interpretation by human readers.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, the core validation steps for tremor and dyskinesia detection described in the Powers et al. (2021) paper are standalone algorithm-only performance evaluations. The Apple Watch's MM4PD toolkit calculates the percentage of time tremor and dyskinesia were likely to occur, and this algorithm's output is directly compared to clinical ground truth. The Rune Labs Kinematics System then receives, stores, and transfers this classification data for display.

6. The type of ground truth used (expert consensus, pathology, outcomes data, etc.):

   • Expert Consensus/Clinical Ratings:
     • For Tremor: "clinician's overall tremor rating" and "MDS-UPDRS tremor constancy score" (a widely accepted clinical rating scale for Parkinson's disease).
     • For Dyskinesia: Ratings from "three MDS-certified experts" during MDS-UPDRS assessments, leading to classifications like "confirmed absence," "discordant," and "confirmed dyskinesia." Clinical history (e.g., "known chorea") was also used.
   • Objective Measurement Reference: For the fundamental sensor accuracy, a commercially available motion tracking system (Vicon) was used as a reference to compare against the watch's displacement measurements.

7. The sample size for the training set:

   • The document implies that the MM4PD algorithms were developed using data from various studies.

     • Tremor Algorithm Development:
       • Pilot study: N=69 subjects
       • Longitudinal patient study: first 143 subjects enrolled (used for the "design set" and hold-out set, so the training set would be a subset of these or distinct, but not explicitly broken out).
       • Longitudinal control study: 236 subjects (for false positive rates, likely also contributed to defining normal movement).
     • Dyskinesia Algorithm Development:
       • Pilot study: N=10 subjects (divided evenly between dyskinetic and non-dyskinetic)
       • Longitudinal patient study: N=97 subjects (first 143 enrolled; 22 with choreiform dyskinesia, 75 without)
       • Longitudinal control study: N=171 subjects.

   • The term "design set" is used for both tremor and dyskinesia validation, which often implies the data used for training/tuning the algorithm. So, the explicit "training set" size for each specific algorithm (tremor vs. dyskinesia) isn't given as a distinct number separate from the "design set," but the various datasets described contributed to algorithm development. For tremor, the "design set" was effectively the training/tuning set (n=95), with n=43 being the hold-out test set. For dyskinesia, a "design set" of n=97 (or n=157 total from longitudinal study) was used for development, and subsets of this were then characterized.

8. How the ground truth for the training set was established:

   • The ground truth for the training/design sets mirrored how it was established for the test sets:
     • Clinical Ratings: For tremor, clinicians' overall tremor ratings and MDS-UPDRS tremor constancy scores were collected. For dyskinesia, ratings from MDS-certified experts during MDS-UPDRS assessments were used to label data within the training/design sets.
     • Self-Reported History: "Self-reported history" was also mentioned for certain conditions (e.g., history of tremor, dyskinesia) in the demographics, which likely informed initial subject stratification.
     • Observed Behavior within Tasks: For dyskinesia, observations during specific tasks (e.g., in-clinic cognitive distraction tasks) provided context for the expert ratings.

Ask a specific question about this device

The Fitbit ECG App is a software-only mobile medical application intended for use with Fitbit wrist wearable devices to create, record, store, transfer, and display a single channel electrocardiogram (ECG) qualitatively similar to a Lead I ECG. The Fitbit ECG App determines the presence of atrial fibrillation (AFib) or sinus rhythm on a classifiable waveform. The AFib detection feature is not recommended for users with other known arrhythmias.

The Fitbit ECG App is intended for over-the-counter (OTC) use. The ECG data displayed by the Fitbit ECG App is intended for informational use only. The user is not interpret or take clinical action based on the device output without consultation of a qualified healthcare professional. The ECG waveform is meant to supplement rhythm classification for the purposes of discriminating AFib from normal sinus rhythm and not intended to replace traditional methods of diagnosis or treatment. The Fitbit ECG App is not intended for use by people under 22 years old.

The Fitbit ECG App is a software-only medical device used to create, record, display, store and analyze a single channel ECG. The Fitbit ECG App consists of a Device application ("Device app") on a consumer Fitbit wrist-worn product and a mobile application tile ("mobile app") on Fitbit's consumer mobile application. The Device app uses data from electrical sensors on a consumer Fitbit wrist-worn product to create and record an ECG. The algorithm on the Device app analyzes a 30 second recording of the ECG and provides results to the user. Users are able to view their past results as well as a pdf report of the waveform similar to a Lead I ECG on the mobile app.

Below is the information regarding the Fitbit ECG App's acceptance criteria and the study that proves it, based on the provided document:

1. Table of acceptance criteria and the reported device performance

2. Sample size used for the test set and the data provenance

• Sample Size: 475 subjects.
• Data Provenance: Subjects were recruited across 9 US sites. This indicates prospective data collection from the United States.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

• Number of Experts: For subjects with a known history of AFib, a "single qualified physician" performed the screening and assigned them to the AFib cohort. The document doesn't specify how many experts reviewed the 12-lead ECGs for the ground truth of AFib or Sinus Rhythm (NSR) for all 475 subjects, beyond the single physician for the AFib cohort screening. For the overall study, it implies a 12-lead ECG was the reference, which would typically be interpreted by qualified cardiologists or electrophysiologists.
• Qualifications of Experts: For AFib screening, the expert was referred to as a "single qualified physician." Specific qualifications like "radiologist with 10 years of experience" are not provided.

4. Adjudication method for the test set

The document does not explicitly state an adjudication method (e.g., 2+1, 3+1). It mentions that subjects with a known history of AFib were screened by a "single qualified physician." For the simultaneous 12-lead ECG, it implies a clinical standard interpretation which often involves adjudicated reads, but this is not detailed in the provided text.

5. If a Multi-Reader, Multi-Case (MRMC) comparative effectiveness study was done

No, a Multi-Reader, Multi-Case (MRMC) comparative effectiveness study comparing human readers with and without AI assistance was not reported in this document. The study focuses on evaluating the standalone performance of the Fitbit ECG App against a clinical standard (12-lead ECG).

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance study was done. The document states: "The Fitbit ECG App software algorithm was able to detect AF with the sensitivity and specificity of 98.7% and 100%, respectively." This indicates a direct evaluation of the algorithm's performance.

7. The type of ground truth used

The ground truth was established using a simultaneous 30-second 12-lead ECG. This is a clinical gold standard for rhythm analysis.

8. The sample size for the training set

The document does not provide the sample size for the training set. It only details the clinical testing conducted for validation/evaluation of the device.

9. How the ground truth for the training set was established

The document does not provide information on how the ground truth for the training set was established, as it focuses on the validation study.

Ask a specific question about this device