(28 days)
The ECG app is a software-only mobile medical application intended for use with the Apple Watch to create, record, store, transfer, and display a single channel electrocardiogram (ECG) similar to a Lead I ECG. The ECG app determines the presence of atrial fibrillation (AFib) or sinus rhythm on a classifiable waveform. The ECG app is not recommended for users with other known arrhythmias.
The ECG app is intended for over-the-counter (OTC) use. The ECG data displayed by the ECG app is intended for informational use only. The user is not intended to interpret or take clinical action based on the device output without consultation of a qualified healthcare professional. The ECG waveform is meant to supplement rhythm classification for the purposes of discriminating AFib from normal sinus rhythm and not intended to replace traditional methods of diagnosis or treatment.
The ECG app is not intended for use by people under 22 years old.
The device (ECG App) comprises a pair of mobile medical apps — one on Apple Watch (the Watch App) and the other on the iPhone (iPhone App) - intended to record, store, transfer, and display a single lead ECG signal similar to a lead I. The ECG Watch App is intended to analyze this single lead data and detect the presence of atrial fibrillation (referred into this document as AFib or AF) and sinus rhythm in adults. It is also intended to acquire and analyze the single lead ECG recordings for display on the iPhone. The ECG iPhone App is included in the Health App, which is intended to store, manage, and share health and fitness data, and comes pre-installed on every iPhone.
Here's a breakdown of the acceptance criteria and the study that proves the device (ECG App) meets these criteria, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria (Performance Goals) | Reported Device Performance |
|---|---|
| Primary Endpoint: Sensitivity of the ECG App algorithm in detecting AF compared with physician-adjudicated 12-lead ECG. Goal: ≥ 90% sensitivity. | 98.3% sensitivity (97.5% LCB: 95.8%) for AF detection among classifiable recordings. This meets the 90% goal. |
| Primary Endpoint: Specificity of the ECG App algorithm in detecting AF compared with physician-adjudicated 12-lead ECG. Goal: ≥ 92% specificity. | 99.6% specificity (97.5% LCB: 97.7%) for AF detection among classifiable recordings. This meets the 92% goal. |
| Secondary Endpoint 1: Qualitative assessment: The proportion of paired ECG strips appear to overlay to the unaided eye. Goal: > 0.80 (80%) | 99.2% of subjects (125/126) had an ECG App waveform that was considered clinically equivalent to the gold standard based on qualitative assessment. This meets the 80% goal. |
| Secondary Endpoint 2: Quantitative assessment: The proportion of paired R-wave amplitude measurements within 2 mm of each other. Goal: > 0.80 (80%) | 97.6% of subjects (123/126) had a paired R-Wave amplitude difference ≤ 2 mm. This meets the 80% goal. |
| Special Control 1.a: Ability to obtain an ECG of sufficient quality for display and analysis. | Demonstrated through electromagnetic compatibility, electrical safety, and signal acquisition assessments (IEC standards), and confirmed by the high rates of classifiable recordings in the clinical study. Also evidenced by the waveform assessment results (99.2% qualitative equivalence, 97.6% R-wave agreement). |
| Special Control 1.b: Performance characteristics of the detection algorithm as reported by sensitivity and either specificity or positive predictive value. | Met by the primary endpoint results for sensitivity (98.3%) and specificity (99.6%). |
| Special Control 2: Software verification, validation, and hazard analysis. | Documentation indicates all elements for "Moderate" level of concern software, including V&V testing, hazard analysis, cybersecurity, etc., were performed. |
| Special Control 3: Non-clinical performance testing validated detection algorithm performance using a previously adjudicated data set. | "ECG Database Testing" was conducted using adjudicated AHA and MIT databases, with "The database annotations used as ground truth." Specific results are redacted but the testing itself was performed. |
| Special Control 4.a: Human factors and usability testing: The user can correctly use the device based solely on reading the device labeling. | A Human Factors Validation Study was performed with 50 participants across three user groups to assess usability and critical tasks. The study assessed completion and success criteria. |
| Special Control 4.b: Human factors and usability testing: The user can correctly interpret the device output and understand when to seek medical care. | Assessed during the Human Factors Validation Study, which focused on whether users understood output and limitations, and if they failed to seek care when needed. |
| FDA's Probable Benefits Outweigh Probable Risks Conclusion | Achieved, leading to the De Novo grant. The study demonstrated high accuracy and minimal safety concerns. |
Study Details
-
Sample Size and Data Provenance:
- Clinical Study Test Set Sample Size: 602 total subjects enrolled. After exclusions, 588 eligible subjects were used for analysis (AF Cohort: 301, SR Cohort: 287). For the "Classifiable Analysis Set" (where the algorithm output a diagnosis), 556 subjects were included (488 had a diagnosis).
- Data Provenance: Not explicitly stated regarding country of origin, but it was a "multi-center" study, implying multiple sites within a geographic region (likely the US, given FDA submission). The study was prospective.
- Waveform Assessment Analysis Set Sample Size: 139 subjects initially selected, with 126 subjects remaining for analysis after exclusions (60 AF, 65 SR, 99.2%).
- ECG Database Testing Sample Size: "(b) (4)" records from adjudicated AHA and MIT databases, split into "(b) (4)" 30-second segments. Specific numbers are redacted.
-
Number of Experts and Qualifications for Clinical Study Ground Truth:
- Number of Experts: Three (3) blinded independent board-certified cardiologists.
- Qualifications: "board-certified cardiologists." No specific years of experience are listed.
-
Adjudication Method for Clinical Study Test Set Ground Truth:
- Adjudication Method: "If the readers disagreed on the diagnosis, the final interpretation was determined by the simple majority rule." This is a 3-reader majority rule method.
-
Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- The provided text does not describe an MRMC comparative effectiveness study where human readers' performance with and without AI assistance is compared.
- The study primarily focuses on the standalone performance of the AI algorithm (ECG App) against physician-adjudicated ground truth.
- There was an "Additional Analysis" involving physician interpretation of ECG App strips compared to 12-lead ECGs, but this was to evaluate the quality of the ECG App recording and not a comparative study of human performance with/without AI assistance.
-
Standalone Performance:
- Yes, a standalone (algorithm only without human-in-the loop performance) evaluation was done as the primary endpoint of the clinical study. The ECG App algorithm's classification was directly compared against the physician-adjudicated 12-lead ECG ground truth.
-
Type of Ground Truth Used:
- Clinical Study: Expert Consensus (three blinded independent board-certified cardiologists reviewing 12-lead ECG recordings with majority rule).
- ECG Database Testing: "The database annotations were used as ground truth." These are typically expert-adjudicated public datasets like AHA and MIT databases.
-
Sample Size for Training Set:
- The provided text does not specify the sample size for the training set of the ECG App algorithm. It only details the test sets used for validation.
-
How the Ground Truth for the Training Set was Established:
- The text does not provide information on how the ground truth for the training set was established. It only mentions the ground truth methodology for the test sets (clinical study and database testing).
This detailed analysis covers the requested information based on the provided FDA De Novo submission text.
§ 870.2345 Electrocardiograph software for over-the-counter use.
(a)
Identification. An electrocardiograph software device for over-the-counter use creates, analyzes, and displays electrocardiograph data and can provide information for identifying cardiac arrhythmias. This device is not intended to provide a diagnosis.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Clinical performance testing under anticipated conditions of use must demonstrate the following:
(i) The ability to obtain an electrocardiograph of sufficient quality for display and analysis; and
(ii) The performance characteristics of the detection algorithm as reported by sensitivity and either specificity or positive predictive value.
(2) Software verification, validation, and hazard analysis must be performed. Documentation must include a characterization of the technical specifications of the software, including the detection algorithm and its inputs and outputs.
(3) Non-clinical performance testing must validate detection algorithm performance using a previously adjudicated data set.
(4) Human factors and usability testing must demonstrate the following:
(i) The user can correctly use the device based solely on reading the device labeling; and
(ii) The user can correctly interpret the device output and understand when to seek medical care.
(5) Labeling must include:
(i) Hardware platform and operating system requirements;
(ii) Situations in which the device may not operate at an expected performance level;
(iii) A summary of the clinical performance testing conducted with the device;
(iv) A description of what the device measures and outputs to the user; and
(v) Guidance on interpretation of any results.