(60 days)
DeepRhythmAI is a cloud-based software that utilizes AI algorithms to assess cardiac arrhythmias using a single- or two-lead ECG data from adult patients. It is intended for use by a healthcare solution integrator to build web, mobile or another types of applications to let qualified healthcare professionals review and confirm the analytic result. The product supports downloading and analyzing data recorded in the compatible formats from ECG devices such as Holter, Event recorder, Outpatient Cardiac Telemetry devices or other similar recorders when the assessment of the rhythm is necessary. The product can be electronically interfaced and perform analysis with data transferred from other computer-based ECG systems, such as an ECG management system. DeepRhythmAI can be integrated into medical devices. In this case, the medical device manufacturer will identify the indication for use depending on the application of their device. DeepRhythmAI is not for use in life-supporting or sustaining systems or ECG Alarm devices. Interpretation results are not intended to be the sole means of diagnosis. It is offered to physicians and clinicians on an advisory basis only in conjunction with the physician's knowledge of ECG patterns, patient background, clinical history, symptoms and other diagnostic information.
The DeepRhythmAI is a cloud-based software utilizing CNN and transformer models for automated analysis of ECG data. It uses a scalable Application Programming Interface (API) to enable easy integration with other medical products. The main component of DeepRhythmAI is an automated proprietary deep-learning algorithm, which measures and analyzes ECG data to provide qualified healthcare professional with supportive information for review. DeepRhythmAI can be integrated into medical devices. The product supports downloading and analyzing data recorded in compatible formats from ECG devices such as Holter, Event recorder, Outpatient Cardiac Telemetry devices or other similar recorders used when assessment of the rhythm is necessary. The DRAI can also be electronically interfaced and perform analysis with data transferred from other computer-based ECG systems, such as an ECG management system. DeepRhythmAI doesn't have User Interface therefore it should be integrated with the external visualization software used by the ECG technicians for the ECG visualization and analysis reporting.
The provided FDA 510(k) clearance letter and summary for DeepRhythmAI offer general statements about performance testing but lack the specific details required to fully address all aspects of the request, especially quantifiable acceptance criteria and the results that prove them. The document primarily focuses on the substantial equivalence argument against a predicate device (which is itself DeepRhythmAI).
Based on the provided text, here's an attempt to extract and infer the information:
Acceptance Criteria and Device Performance:
The document mentions that the device was tested "according to the recognized consensus standards, ANSI/AAMI/IEC 60601-2-47:2012/(R)2016 and AAMI/ANSI/EC57:2012." These standards define performance requirements for ECG analysis devices, including aspects like beat detection accuracy, heart rate accuracy, and arrhythmia detection. However, the exact quantifiable acceptance criteria (e.g., "accuracy must be >X%") and the observed numeric device performance (e.g., "accuracy was Y%") are not reported in the provided text.
The closest we get to "reported performance" is the statement: "Overall, the software verification & validation testing was completed successfully and met all requirements. Testing demonstrated that the subject device performance was deemed to be acceptable." This is a qualitative statement, not quantitative performance data.
Table of Acceptance Criteria and Reported Device Performance:
| Acceptance Criteria (Inferred from Standards) | Reported Device Performance (Not Quantified in Doc) |
|---|---|
| QRS detection accuracy (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| Heart rate determination accuracy for non-paced adult (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| R-R interval detection accuracy (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| Non-paced arrhythmias interpretation accuracy (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| Non-paced ventricular arrhythmias calls accuracy (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| Atrial fibrillation detection accuracy (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| Cardiac beats detection accuracy (Ventricular ectopic beats, Supraventricular ectopic beats) (as per ANSI/AAMI standards) | Met all requirements; performance deemed acceptable. |
| Cyber security requirements met | No vulnerabilities identified. |
| Software requirements satisfied | All software requirements satisfied. |
Study Details:
-
Sample size used for the test set and the data provenance:
- Test Set Sample Size: The document states the algorithm was "tested against the proprietary database (MDG validation db) that includes a large number of recordings captured among the intended patient population." The exact number of recordings is not specified, only "a large number."
- Data Provenance: The data comes from a "proprietary database (MDG validation db)." The country of origin is not explicitly stated. The document indicates it includes data for both two-lead and single-lead patch recorders, implying diverse ECG device sources. It is implied to be retrospective data collected for validation purposes.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- This information is not provided in the document. The document states a "proprietary database" was used for validation, but it does not detail how the ground truth within this database was established (e.g., by how many cardiologists or expert technicians, or their qualifications).
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- This information is not provided in the document.
-
If a multi-reader multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- A MRMC comparative effectiveness study involving human readers and AI assistance is not mentioned in the provided text. The study described focuses on the standalone performance of the device against a ground truth. The device "is offered to physicians and clinicians on an advisory basis only" and results are "not intended to be the sole means of diagnosis," indicating a human-in-the-loop context, but no study is presented to quantify this human-AI interaction's effect on reader performance.
-
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance study was done. The document states the algorithm was "tested against the proprietary database (MDG validation db)." The entire summary of performance data refers to evaluation of the "DeepRhythmAI software for arrhythmia detection and automated analysis of ECG data." There is no mention of human interaction during this performance evaluation.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- The document implies the use of an "MDG validation db" but does not specify the type of ground truth used to annotate this database. It's common for such ECG databases to rely on expert adjudicated annotations, but this is not explicitly stated.
-
The sample size for the training set:
- The sample size for the training set is not provided. The document only discusses the "MDG validation db" which is used for testing/validation.
-
How the ground truth for the training set was established:
- As the training set sample size is not provided, neither is information on how its ground truth was established.
Summary of Missing Information:
The provided document, being a 510(k) clearance letter and summary, serves to establish substantial equivalence. It confirms that specific performance testing was conducted according to recognized standards and deemed acceptable, but it does not provide the detailed scientific study results that would include:
- Quantifiable acceptance criteria and the exact numeric performance results for each criterion.
- The raw sample size of the test set.
- Details on the experts involved in ground truth creation for the test set (number, qualifications, adjudication method).
- Information on any MRMC studies or effect sizes of AI assistance on human readers.
- Explicit details about the ground truth methodology for the validation database.
- Any information regarding the training dataset (size, ground truth methodology).
To fully answer the request, one would typically need access to the full 510(k) submission, which contains the detailed V&V (Verification and Validation) reports.
§ 870.1425 Programmable diagnostic computer.
(a)
Identification. A programmable diagnostic computer is a device that can be programmed to compute various physiologic or blood flow parameters based on the output from one or more electrodes, transducers, or measuring devices; this device includes any associated commercially supplied programs.(b)
Classification. Class II (performance standards).