(33 days)
The Irregular Rhythm Notification Feature is a software-only mobile medical application that is intended to be used with the Apple Watch. The feature analyzes pulse rate data to identify episodes of irregular heart rhythms suggestive of atrial fibrillation (AFib) and provides a notification to the user. The feature is intended for over-the-counter (OTC) use. It is not intended to provide a notification on every episode of irregular rhythm suggestive of AFib and the absence of a notification is not intended to indicate no disease process is present; rather the feature is intended to opportunistically surface a notification of possible AFib when sufficient data are available for analysis. These data are only captured when the user is still. Along with the user's risk factors, the feature can be used to supplement the decision for AFib screening. The feature is not intended to replace traditional methods of diagnosis or treatment.
The feature has not been tested for and is not intended for use in people under 22 years of age. It is also not intended for use in individuals previously diagnosed with AFib.
The Irregular Rhythm Notification Feature comprises a pair of mobile medical apps, one on Apple Watch and the other on the iPhone. The Irregular Rhythm Notification Feature analyzes pulse rate data collected by the Apple Watch photoplethysmograph (PPG) sensor to identify episodes of irregular heart rhythms consistent with atrial fibrillation (referred to in this document as AF or AFib) and provides a notification to the user. It is a background screening tool and there is no way for a user to initiate analysis of pulse rate data. The Irregular Rhythm Notification Feature is part of the Health App, which allows users to store, manage, and share health and fitness data, and comes pre-installed on every iPhone. Users must opt-in and go through onboarding prior to use of the Irregular Rhythm Notification Feature.
The Irregular Rhythm Notification Feature is not intended to diagnose atrial fibrillation, and is not intended to be used to guide clinical treatment or care.
Here's a summary of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance for Irregular Rhythm Notification Feature
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly define a "table of acceptance criteria" with specific numerical targets in the same way an academic paper might. However, by synthesizing the "Clinical Study" section and the "Risks to Health" with their "Mitigation Measures," we can infer the key performance metrics and their achieved results. The pre-specified performance goal was explicitly stated for the primary endpoint.
| Acceptance Criterion (Inferred from study objectives & risks) | Reported Device Performance |
|---|---|
| Primary Endpoint: Positive Predictive Value (PPV) of spot irregular tachograms to detect AF (against ambulatory ECG) | 66.6% (lower 97.5% confidence bound: 63.0%). This failed to meet the pre-specified (0)% performance goal. (Note: The "0%" pre-specified goal seems like a typo in the original document and likely should have been a specific non-zero percentage. Given the context, they were looking for a high PPV.) |
| Secondary Endpoint: Notification-level PPV for AF in enriched population (users who received at least one prior notification) | 78.9% (95% CI: 66.1%, 88.6%). (This metric, while not the primary, was supportive of effectiveness despite the primary endpoint failure.) |
| Probability of being diagnosed with AF on subsequent 7-day patch cardiac monitoring for subjects who received one or more device notifications (Post-hoc analysis) | 41.6% (95% CI: 35.1%, 48.3%). |
| User comprehension of "lack of notification does not affect medical decisions" | 36/37 participants successfully responded indicating this. |
| User comprehension of "would not reduce care if experiencing acute symptoms" upon receiving a notification | 35/35 participants successfully received a notification and indicated this. |
| Software Validation | Moderate Level of Concern (LOC) guidelines met. |
| Functional Performance (Bench Testing) | Acceptable performance demonstrated against commercial, FDA-cleared clinical ECG. |
| Skin Tone Performance | No clinically relevant difference from Fitzpatrick VI to Fitzpatrick I subjects. No algorithm changes needed. |
| Detection of adequate PPG signal quality (Non-clinical performance testing) | Demonstrated. |
2. Sample Size for the Test Set and Data Provenance
-
Clinical Study Test Set:
- Full Analysis Set (FAS): 269 subjects
- Analyzable ECG monitor and tachogram data: 226 subjects (after exclusions)
- Irregular tachograms recorded: 2634 (out of 10432 total tachograms during monitoring)
- Subjects receiving at least one alert: 57 (25.2% of the 226 analyzable subjects)
-
Data Provenance: The data was collected from a large, prospective, single-arm study conducted to investigate PPG data from Apple Watch for AF-related irregular heart rhythm identification. The sub-study enrolled participants who had already received one prior Irregular Rhythm notification. The country of origin is not explicitly stated but implies a broad user base given Apple's global reach, although the study itself was managed by Apple Inc. in Cupertino, CA.
-
Bench Testing Test Set:
- External Aggressor Condition Testing:
- Riding in a car (vibration): 44 subjects, 1434 measurements
- Targeted Hand + finger motions: 20 subjects, 246 measurements
- Low perfusion: 102 subjects, 2461 measurements
- Hand Tremors: 143 subjects, 936 measurements
- Skin Tone Performance: 1124 subjects, 1.3 million measurements
- External Aggressor Condition Testing:
-
Human Factors and Usability Study Test Set: 37 participants (16 "Active Interest", 21 "Passive Interest")
3. Number of Experts and Qualifications for Ground Truth - Clinical Study
- Number of Experts: Unspecified exact number of independent cardiologists. It states "independent cardiologists" (plural).
- Qualifications: "Independent cardiologists" (no further details on experience or specialization, but the title implies appropriate medical expertise).
4. Adjudication Method for the Test Set - Clinical Study
The document states: "For each one-minute irregular rhythm episode (tachogram) identified by the software, the corresponding patch ECG recording was extracted and classified by independent cardiologists as either 'Sinus rhythm', 'AF', 'Unreadable', or 'Other Irregular Rhythm.'" This implies a single expert classification or a pre-defined process without explicitly stating a consensus method like 2+1 or 3+1.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- No, an MRMC comparative effectiveness study was not done or reported comparing human readers with and without AI assistance. The clinical study focused on the standalone performance of the device's notification feature against an ambulatory ECG.
6. Standalone (Algorithm Only) Performance
- Yes, the primary clinical study was a standalone (algorithm only) performance evaluation. The device independently identified irregular rhythm episodes, which were then compared to the ground truth established by ambulatory ECG and cardiologist review. There was no human-in-the-loop component for the detection performance evaluation itself.
7. Type of Ground Truth Used
- Clinical Study:
- For the primary and secondary endpoints related to AF detection, the ground truth was established by 7-day ambulatory patch ECG recordings, which were then "classified by independent cardiologists as either 'Sinus rhythm', 'AF', 'Unreadable', or 'Other Irregular Rhythm.'" This combines outcomes data (ECG) with expert consensus/interpretation.
- Bench Testing:
- Functional performance: Compared to "commercial, FDA-cleared clinical ECG."
- Human Factors Study:
- User comprehension: Observational data and subjective evaluations of user responses to questions.
8. Sample Size for the Training Set
The document does not explicitly state the sample size for the training set used to develop the Irregular Rhythm Notification Feature algorithm. It only describes the evaluation of the algorithm on the described test sets.
9. How the Ground Truth for the Training Set Was Established
The document does not explicitly describe how the ground truth for the training set was established. It focuses only on the validation study. It can be inferred that similar methods (ECG data interpreted by cardiologists) would likely have been used during development, but this is not detailed in the provided text.
§ 870.2790 Photoplethysmograph analysis software for over-the-counter use.
(a)
Identification. A photoplethysmograph analysis software device for over-the-counter use analyzes photoplethysmograph data and provides information for identifying irregular heart rhythms. This device is not intended to provide a diagnosis.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing must demonstrate the performance characteristics of the detection algorithm under anticipated conditions of use.
(2) Software verification, validation, and hazard analysis must be performed. Documentation must include a characterization of the technical specifications of the software, including the detection algorithm and its inputs and outputs.
(3) Non-clinical performance testing must demonstrate the ability of the device to detect adequate photoplethysmograph signal quality.
(4) Human factors and usability testing must demonstrate the following:
(i) The user can correctly use the device based solely on reading the device labeling; and
(ii) The user can correctly interpret the device output and understand when to seek medical care.
(5) Labeling must include:
(i) Hardware platform and operating system requirements;
(ii) Situations in which the device may not operate at an expected performance level;
(iii) A summary of the clinical performance testing conducted with the device;
(iv) A description of what the device measures and outputs to the user; and
(v) Guidance on interpretation of any results.