(265 days)
Not Found
Yes
The document explicitly states that the device is an "automated machine learning-based decision support system" and that the output is based on an "artificial intelligence (AI) model".
No
This device is a diagnostic aid, providing information to help detect heart failure, rather than directly treating or preventing a disease.
Yes
The "Intended Use / Indications for Use" section states that the device is "indicated as a diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography" and provides "information that may be useful in detecting heart failure with preserved ejection fraction (HFpEF)." The "Device Description" also clarifies that it is "intended to be used by an interpreting clinician as an aid to diagnosis for HFpEF."
Yes
The device description explicitly states that EchoGo Heart Failure 2.0 is "fully automated and does not comprise a graphical user interface," and its input is a 2D echocardiogram, implying it processes existing image data rather than being part of the hardware that acquires the data. It functions as a decision support system based on machine learning, which is a software function.
Based on the provided information, this device is not an IVD (In Vitro Diagnostic).
Here's why:
- IVD definition: In Vitro Diagnostics are tests performed on samples taken from the human body, such as blood, urine, or tissue, to detect diseases, conditions, or infections.
- Device input: EchoGo Heart Failure 2.0 takes a 2D echocardiogram as input. An echocardiogram is an ultrasound of the heart, which is an imaging technique performed on the body, not on a sample taken from the body.
- Intended Use: The intended use is as a "diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography." This reinforces that the device is analyzing imaging data acquired directly from the patient.
Therefore, because the device analyzes imaging data rather than biological samples, it falls outside the definition of an In Vitro Diagnostic. It is a medical device that utilizes imaging and machine learning for diagnostic support.
No
The input letter does not explicitly state that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device.
Intended Use / Indications for Use
EchoGo Heart Failure 2.0 is an automated machine learning-based decision support system, indicated as a diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography. When utilised by an interpreting clinician, this device provides information that may be useful in detecting heart failure with preserved ejection fraction (HFpEF).
EchoGo Heart Failure 2.0 is indicated in adult populations over 25 years of age. Patient management decisions should not be made solely on the results of the EchoGo Heart Failure 2.0 analysis.
EchoGo Heart Failure 2.0 takes as input an apical 4-chamber view of the heart that has been captured and assessed to have an ejection fraction >=50%.
Product codes (comma separated list FDA assigned to the subject device)
OUO
Device Description
EchoGo Heart Failure 2.0 takes as input a 2D echocardiogram of an apical four chamber tomographic view and reports as output a binary classification suggestive of the presence, or absence of heart failure with preserved ejection fraction (HFpEF). EchoGo Heart Failure 2.0 also provides users with an EchoGo Score ranging from 0 to 100% to support the binary classification. The EchoGo Score informs the binary classification when referenced against the pre-determined decision threshold (50%).
To aid in the interpretation of the EchoGo Score, a comparative visual analysis is provided. A histogram format displays the reported EchoGo Score output against a population of patients with known disease status (Independent Testing Dataset). This allows the user to interpret the EchoGo Score relative to the decision threshold of 50%.
EchoGo Heart Failure 2.0 should receive an input echocardiogram acquired without contrast and contain at least one full cardiac cycle.
EchoGo Heart Failure 2.0 is fully automated and does not comprise a graphical user interface.
EchoGo Heart Failure 2.0 is intended to be used by an interpreting clinician as an aid to diagnosis for HFpEF. The ultimate diagnostic decision remains the responsibility of the interpreting clinician using patient presentation, medical history, and the results of available diagnostic tests, one of which may be EchoGo Heart Failure 2.0.
EchoGo Heart Failure 2.0 is a prescription only device.
Mentions image processing
Not Found
Mentions AI, DNN, or ML
EchoGo Heart Failure 2.0 is an automated machine learning-based decision support system
The output of both devices is based on an artificial intelligence (AI) model developed using a convolutional network that produces a classification result.
Subject device AI model was trained on more data and with additional preprocessing steps and data augmentations.
Machine Learning-Based Algorithm: Yes
Input Imaging Modality
2D echocardiogram
Anatomical Site
Cardiovascular
Indicated Patient Age Range
Adult populations over 25 years of age.
Intended User / Care Setting
Interpreting clinician
Description of the training set, sample size, data source, and annotation protocol
Subject device AI model was trained on more data and with additional preprocessing steps and data augmentations. (No further details on training set)
Description of the test set, sample size, data source, and annotation protocol
An independent clinical validation study was conducted on a clinical data set representative of the intended use population and containing a range of data sources and data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment. The study was used to demonstrate consistency of the device output as well as to assess agreement with reference ground truth.
Device performance was determined according to a retrospective case:control study including multiple sites spanning five states in the final testing data cohort amounted to 1,578 patients, comprising 785 controls and 793 cases.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Device performance was determined according to a retrospective case:control study including multiple sites spanning five states in the final testing data cohort amounted to 1,578 patients, comprising 785 controls and 793 cases.
Key Results:
- To assess sensitivity and specificity, we compared the device output for a single echocardiogram videoclip per patient to the ground truth classifications of cases (HFpEF) or controls. EchoGo Heart Failure 2.0 correctly identified 673 true positives, and 617 true negatives, alongside 100 false positives, 72 false negatives, and 116 no classification outputs (48 cases, 68 controls).
- To determine the accuracy of the EchoGo Score, we compared the EchoGo Score to known and expected proportions of HFpEF. The p value for the Hosmer-Lemeshow Test for goodness-of-fit was not significant (p=0.304), indicating acceptable fit between observed and known probabilities. The area under the receiver operator characteristic curve (AUROC) was 0.947 (95% Cl: 0.934, 0.958) when removing no classification studies, and 0.937 (95% Cl: 0.924, 0.949) when considering all studies and ignoring uncertainty and instability metrics.
- The proportion of non-diagnostic (i.e., "No classification") outputs of the device were within a priori acceptance limits. Of the 1,578 studies analysed by the device, 116 (7.4%) were categorized as "No Classification".
- The device output classification from a single Digital Imaging and Communications in Medicine (DICOM) clip analysed twice (repeatability), and the device output classification from different DICOM clips from the same individual (reproducibility) was assessed for precision. The device demonstrated 100% repeatability in all measures and 82.6% Positive Agreement and 82.4% Negative Agreement for reproducibility.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Sensitivity: 90.3% (95% Cl: 88.5, 92.4%) when removing no classification studies; 84.9% (95% Cl: 83.0, 87.5%) when including no classification studies.
Specificity: 86.1% (95% Cl: 83.4, 88.3%) when removing no classification studies; 78.6% (95% Cl: 75.3, 81.1%) when including no classification studies.
AUROC: 0.947 (95% Cl: 0.934, 0.958) when removing no classification studies; 0.937 (95% Cl: 0.924, 0.949) when considering all studies.
Repeatability: 100%
Reproducibility: 82.6% Positive Agreement and 82.4% Negative Agreement.
Predicate Device(s): If the device was cleared using the 510(k) pathway, identify the Predicate Device(s) K/DEN number used to claim substantial equivalence and list them here in a comma separated list exactly as they appear in the text. List the primary predicate first in the list.
Reference Device(s): Identify the Reference Device(s) K/DEN number and list them here in a comma separated list exactly as they appear in the text.
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information for the subject device only (e.g. presence / absence, what scope was granted / cleared under the PCCP, any restrictions, etc).
Not Found
§ 870.2200 Adjunctive cardiovascular status indicator.
(a)
Identification. The adjunctive cardiovascular status indicator is a prescription device based on sensor technology for the measurement of a physical parameter(s). This device is intended for adjunctive use with other physical vital sign parameters and patient information and is not intended to independently direct therapy.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software description, verification, and validation based on comprehensive hazard analysis must be provided, including:
(i) Full characterization of technical parameters of the software, including any proprietary algorithm(s);
(ii) Description of the expected impact of all applicable sensor acquisition hardware characteristics on performance and any associated hardware specifications;
(iii) Specification of acceptable incoming sensor data quality control measures; and
(iv) Mitigation of impact of user error or failure of any subsystem components (signal detection and analysis, data display, and storage) on accuracy of patient reports.
(2) Scientific justification for the validity of the status indicator algorithm(s) must be provided. Verification of algorithm calculations and validation testing of the algorithm using a data set separate from the training data must demonstrate the validity of modeling.
(3) Usability assessment must be provided to demonstrate that risk of misinterpretation of the status indicator is appropriately mitigated.
(4) Clinical data must be provided in support of the intended use and include the following:
(i) Output measure(s) must be compared to an acceptable reference method to demonstrate that the output measure(s) represent(s) the predictive measure(s) that the device provides in an accurate and reproducible manner;
(ii) The data set must be representative of the intended use population for the device. Any selection criteria or limitations of the samples must be fully described and justified;
(iii) Agreement of the measure(s) with the reference measure(s) must be assessed across the full measurement range; and
(iv) Data must be provided within the clinical validation study or using equivalent datasets to demonstrate the consistency of the output and be representative of the range of data sources and data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment.
(5) Labeling must include the following:
(i) The type of sensor data used, including specification of compatible sensors for data acquisition;
(ii) A description of what the device measures and outputs to the user;
(iii) Warnings identifying sensor reading acquisition factors that may impact measurement results;
(iv) Guidance for interpretation of the measurements, including warning(s) specifying adjunctive use of the measurements;
(v) Key assumptions made in the calculation and determination of measurements;
(vi) The measurement performance of the device for all presented parameters, with appropriate confidence intervals, and the supporting evidence for this performance; and
(vii) A detailed description of the patients studied in the clinical validation (
e.g., age, gender, race/ethnicity, clinical stability) as well as procedural details of the clinical study.
0
Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, with the letters "FDA" in a blue square. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
September 23, 2024
Ultromics Limited Elena Traistaru Head of Quality and Regulatory Affairs 4630 Kingsgate Cascade Way, Oxford Business Park Oxford, OX4 2SU United Kingdom
Re: K240013
Trade/Device Name: EchoGo Heart Failure (2.0) Regulation Number: 21 CFR 870.2200 Regulation Name: Adjunctive Cardiovascular Status Indicator Regulatory Class: Class II Product Code: OUO Dated: August 23, 2024 Received: August 23, 2024
Dear Elena Traistaru:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
1
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30. Design controls; 21 CFR 820.90. Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatory
2
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Robert T. Kazmierski -S
for
LCDR Stephen Browning Assistant Director Division of Cardiac Electrophysiology, Diagnostics, and Monitoring Devices Office of Cardiovascular Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
3
Indications for Use
Submission Number (if known)
Device Name
EchoGo Heart Failure (2.0)
Indications for Use (Describe)
EchoGo Heart Failure 2.0 is an automated machine learning-based decision support system, indicated as a diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography. When utilised by an interpreting clinician, this device provides information that may be useful in detecting heart failure with preserved ejection fraction (HFpEF).
EchoGo Heart Failure 2.0 is indicated in adult populations over 25 years of age. Patient management decisions should not be made solely on the results of the EchoGo Heart Failure 2.0 analysis.
EchoGo Heart Failure 2.0 takes as input an apical 4-chamber view of the heart that has been captured and assessed to have an ejection fraction ≥50%.
Type of Use (Select one or both, as applicable)
Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
4
1 Submitter
| Company | Ultromics Limited
4630 Kingsgate Cascade Way, Oxford Business Park South,
Oxford, Oxfordshire, United Kingdom, OX4 2SU |
|---------|------------------------------------------------------------------------------------------------------------------------------|
| Contact | Dr. Elena Traistaru |
2 Subject Device
| Product Trade Name | EchoGo Heart Failure |
|---|---|
| Model Number | 2.0 |
| 510(k) | K240013 |
| Manufacturer | Ultromics Limited |
| Medical Speciality | Cardiology |
| Regulation | 21 CFR 870.2200 - Cardiovascular Monitoring Devices |
| Product Code | QUO- Adjunctive Heart Failure Status Indicator |
| Regulatory Class | II |
EchoGo Heart Failure is the product trade name and 2.0 is the model number. For the avoidance of doubt, in this submission we combine the product trade name and model number and refer to the subject device as EchoGo Heart Failure 2.0.
3 Predicate Device
| Predicate Device | EchoGo Heart Failure |
|---|---|
| 510(k) | K222463 |
| Manufacturer | Ultromics Limited |
5
4 Device Description
EchoGo Heart Failure 2.0 takes as input a 2D echocardiogram of an apical four chamber tomographic view and reports as output a binary classification suggestive of the presence, or absence of heart failure with preserved ejection fraction (HFpEF). EchoGo Heart Failure 2.0 also provides users with an EchoGo Score ranging from 0 to 100% to support the binary classification. The EchoGo Score informs the binary classification when referenced against the pre-determined decision threshold (50%).
To aid in the interpretation of the EchoGo Score, a comparative visual analysis is provided. A histogram format displays the reported EchoGo Score output against a population of patients with known disease status (Independent Testing Dataset). This allows the user to interpret the EchoGo Score relative to the decision threshold of 50%.
EchoGo Heart Failure 2.0 should receive an input echocardiogram acquired without contrast and contain at least one full cardiac cycle.
EchoGo Heart Failure 2.0 is fully automated and does not comprise a graphical user interface.
EchoGo Heart Failure 2.0 is intended to be used by an interpreting clinician as an aid to diagnosis for HFpEF. The ultimate diagnostic decision remains the responsibility of the interpreting clinician using patient presentation, medical history, and the results of available diagnostic tests, one of which may be EchoGo Heart Failure 2.0.
EchoGo Heart Failure 2.0 is a prescription only device.
5 Context
5.1 Intended Use
Providing adjunctive information on a patient's cardiovascular condition (diagnostic aid for Heart Failure with Preserved Ejection Fraction (HFpEF)).
5.2 Intended User
The clinician interpreting the report produced by EchoGo Heart Failure 2.0 and making a diagnostic decision.
5.3 Indications for Use
EchoGo Heart Failure 2.0 is an automated machine learning-based decision support system, indicated as a diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography. When utilised by an interpreting
6
clinician, this device provides information that may be useful in detecting heart failure with preserved ejection fraction (HFpEF).
EchoGo Heart Failure 2.0 is indicated in adult populations over 25 years of age. Patient management decisions should not be made solely on the results of the EchoGo Heart Failure 2.0 analysis.
EchoGo Heart Failure 2.0 takes as input an apical 4-chamber view of the heart that has been captured and assessed to have an ejection fraction ≥50%.
5.4 Patient Population
Patients undergoing routine functional cardiovascular assessment using diagnostic echocardiography or those suspected of heart failure.
6 Comparison of Intended Use/Indications for Use
The subject and predicate device have identical intended use for use.
Indications for use of predicate and subject device is identical (unchanged).
7 Comparison of Technological Characteristics
7.1 Subject and predicate device (EchoGo Heart Failure (K222463))
At a high level, the subject and primary predicate device is based on the following same technological elements:
- Both device takes as input a DICOM file containing an echocardiogram as input numeric physiological information from medical devices to which it is connected. Both devices therefore receive as input data that is the output of another medical device.
- The output of both devices is based on an artificial intelligence (AI) model developed using a convolutional network that produces a classification result.
- Both device reports a classification decision as suggestive or not suggestive of the presence of heart failure with preserved ejection fraction (HFpEF). Both devices are adjunctive cardiovascular status indicators.
- Subject device includes a class probability score along with comparative analysis to a population of cases with known ground truth (reference dataset) using a histogram display format in the report.
7
- Subject device includes additional application programming interfaces for input and output expanding the methods of interfacing with external applications and medical devices.
- Subject device allows more the deployment options permitting functionality to be distributed and replicated, increasing the scalability, robustness and non-clinical performance of the device.
- Subject device AI model was trained on more data and with additional preprocessing steps and data augmentations.
| Characteristic | Subject Device
EchoGo Heart Failure 2.0 | Predicate Device
EchoGo Heart Failure
(K222463) |
|-------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Regulation | 21 CFR 870.2200 | 21 CFR 870.2200 |
| Generic Device Type | Adjunctive cardiovascular
status indicator | Adjunctive cardiovascular
status indicator |
| SaMD | Yes | Yes |
| Intended Use | Providing adjunctive
information on a patient's
cardiovascular condition
(diagnostic aid for Heart
Failure with Preserved
Ejection Fraction (HFpEF)). | Providing adjunctive
information on a patient's
cardiovascular condition
(diagnostic aid for Heart
Failure with Preserved
Ejection Fraction (HFpEF)). |
| Characteristic | Subject Device
EchoGo Heart Failure 2.0 | Predicate Device
EchoGo Heart Failure
(K222463) |
| Indications for Use | EchoGo Heart Failure 2.0 is
an automated machine
learning-based decision
support system, indicated
as a diagnostic aid for
patients undergoing routine
functional cardiovascular
assessment using
echocardiography. When
utilised by an interpreting
clinician, this device
provides information that
may be useful in detecting
heart failure with preserved
ejection fraction (HFpEF).
EchoGo Heart Failure 2.0 is
indicated in adult
populations over 25 years
of age. Patient
management decisions
should not be made solely
on the results of the
EchoGo Heart Failure 2.0
analysis.
EchoGo Heart Failure 2.0
takes as input an apical 4-
chamber view of the heart
that has been captured and
assessed to have an
ejection fraction ≥50%. | EchoGo Heart Failure 1.0 is
an automated machine
learning-based decision
support system, indicated
as a diagnostic aid for
patients undergoing routine
functional cardiovascular
assessment using
echocardiography. When
utilised by an interpreting
clinician, this device
provides information that
may be useful in detecting
heart failure with preserved
ejection fraction (HFpEF).
EchoGo Heart Failure 1.0 is
indicated in adult
populations over 25 years
of age. Patient
management decisions
should not be made solely
on the results of the
EchoGo Heart Failure 1.0
analysis.
EchoGo Heart Failure 1.0
takes as input an apical 4-
chamber view of the heart
that has been captured and
assessed to have an
ejection fraction ≥50%. |
| Population | Adults over the age of 25 | Adults over the age of 25 |
| Anatomical Site | Cardiovascular | Cardiovascular |
| Users | Interpreting clinician | Interpreting clinician |
| Machine Learning-Based
Algorithm | Yes | Yes |
| Characteristic | Subject Device
EchoGo Heart Failure 2.0 | Predicate Device
EchoGo Heart Failure
(K222463) |
| Operating platform | Hosted on Ultromics'
platform or on third party
infrastructure. | Hosted on Ultromics'
platform or on third party
infrastructure. |
| Interoperability | Interoperability testing
conducted with device
capable of calculating an
ejection fraction on the
apical 4 chamber view. | Interoperability testing
conducted with device
capable of calculating an
ejection fraction on the
apical 4 chamber view. |
| Software | Complies with IEC
62304:2015 and GPSV.
Developed under an FDA
QSR and ISO 13485:2016
compliant QMS
incorporating risk
management per ISO
14971:2019. Software
verification and validation
testing conducted. | Complies with IEC
62304:2015 and GPSV.
Developed under an FDA
QSR and ISO 13485:2016
compliant QMS
incorporating risk
management per ISO
14971:2019. Software
verification and validation
testing conducted. |
| Risk Management | In accordance with ISO
14971:2019 | In accordance with ISO
14971:2019 |
| Cybersecurity | Post-market Management
of Cybersecurity in Medical
Devices.
Content of Premarket
Submissions for
Management of
Cybersecurity in Medical
Devices.
Cybersecurity for
Networked Medical Devices
Containing Off-the-Shelf
(OTS) Software: Guidance
for Industry. | Post-market Management
of Cybersecurity in Medical
Devices.
Content of Premarket
Submissions for
Management of
Cybersecurity in Medical
Devices.
Cybersecurity for
Networked Medical Devices
Containing Off-the-Shelf
(OTS) Software: Guidance
for Industry. |
| Characteristic | Subject Device
EchoGo Heart Failure 2.0 | Predicate Device
EchoGo Heart Failure
(K222463) |
| Usability | Complies with IEC
62366-1:2020 and general
use of FDA guidance
documents on usability
engineering. Formative
evaluations conducted with
accredited cardiac
physiologists (N=2) and
cardiologists (N=5). Formal
summative human factors
testing was conducted with
15 users (Board certified
clinicians with experience
in Echocardiography). | Complies with IEC
62366-1:2020 and general
use of FDA guidance
documents on usability
engineering. Formative and
summative evaluations
conducted with accredited
cardiac physiologists (N=2)
and cardiologists (N=5). |
| Pre-clinical Performance
Testing | No animal studies were
conducted. | No animal studies were
conducted. |
| Bench Performance Testing | Technical validation,
numerical stability, and
regression testing. | Technical validation,
numerical stability, and
regression testing. |
| Clinical Performance
Testing | Validated on a US cohort
population, comprising 8
independent clinical sites
representative of the
intended use population. | Validated on a US cohort
population, comprising 8
independent clinical sites
representative of the
intended use population. |
8
9
10
Any technological differences between the subject and predicate devices raise no new concerns with regards to safety and efficacy. In addition, Ultromics is of the view that general controls alongside special controls introduced under the primary product code of the predicate are sufficient to ensure safety and efficacy of the EchoGo Heart Failure 2.0 device.
8 Special Controls
Special controls for regulation 21 CFR 870.2200 follow. The submission itself contains detailed references to supporting documentation and/or data allowing the verification of the implementation of the associated special controls.
11
| Control | Description | |
|---|---|---|
| 1 | Software description, verification, and validation based on comprehensive | |
| hazard analysis: | ||
| a | Full characterization of technical parameters of | |
| the software, including any proprietary | ||
| algorithm(s) | Control implemented | |
| b | Description of the expected impact of all | |
| applicable acquisition hardware characteristics | ||
| on performance and any associated hardware | ||
| specifications. | Control implemented | |
| C | Specification of acceptable data quality control | |
| measures. | Control implemented | |
| d | Mitigation of impact of user error or failure of | |
| any components (data detection and analysis, | ||
| data display, and storage) on accuracy of | ||
| patient reports. | Control implemented | |
| 2 | Scientific justification for the validity of the | |
| status indicator algorithm(s) must be provided. | ||
| Verification of algorithm calculations and | ||
| validation testing of the algorithm using a data | ||
| set separate from the training data must | ||
| demonstrate the validity of modelling. | Control implemented | |
| 3 | Usability assessment must be provided to | |
| demonstrate that risk of misinterpretation of | ||
| the status indicator is appropriately mitigated. | Control implemented | |
| 4 | Clinical data must be provided in support of the intended use and include the | |
| following: | ||
| a | Output measure(s) must be compared to an | |
| acceptable reference method to demonstrate | ||
| that the output measure(s) represent(s) the | ||
| predictive measure(s) that the device provides | ||
| in an accurate and reproducible manner. | Control implemented | |
| Control | Description | |
| b | The data set must be representative of the | |
| intended use population for the device. Any | ||
| selection criteria or limitations of the samples | ||
| must be fully described and justified. | Control implemented | |
| c | Agreement of the measure(s) with the reference | |
| measure(s) must be assessed across the full | ||
| measurement range. | Control implemented | |
| d | Data must be provided within the clinical | |
| validation study or using equivalent datasets to | ||
| demonstrate the consistency of the output and | ||
| be representative of the range of data sources | ||
| and data quality likely to be encountered in the | ||
| intended use population and relevant use | ||
| conditions in the intended use environment. | Control implemented | |
| 5 | Labelling must include the following: | |
| a | The type of input data used, including | |
| specification of compatible hardware for data | ||
| acquisition. | Control implemented | |
| b | A description of what the device measures and | |
| outputs to the user. | Control implemented | |
| c | Warnings identifying acquisition or other factors | |
| that may impact output measures. | Control implemented | |
| d | Guidance for interpretation of the output | |
| measures, including warning(s) specifying | ||
| adjunctive use of the results. | Control implemented | |
| e | Key assumptions made in the calculation and | |
| determination of results. | Control implemented | |
| f | The measurement performance of the device | |
| for all presented parameters, with appropriate | ||
| confidence intervals, and the supporting | ||
| evidence for this performance. | Control implemented | |
| Control | Description | |
| g | A detailed description of the patients studied in | |
| the clinical validation (e.g., age, gender, race/ | ||
| ethnicity, clinical stability) as well as procedural | ||
| details of the clinical study. | Control implemented |
12
13
9 Consensus Standards
The following consensus standards were used in the design and manufacture of EchoGo Heart Failure 2.0.
| Standard | Recognition Number |
|---|---|
| ISO 14971:2019 - Medical Devices - Application of Risk | |
| Management to Medical Devices | 5-125 |
| IEC 62304:2015 - Medical Device Software - Software Life Cycle | |
| Processes | 13-79 |
| IEC 62366-1:2020 - Medical Devices - Application of Usability | |
| Engineering to Medical Devices | 5-129 |
| NEMA PS 3.1 - 3.20 2021e - Digital Imaging and Communications | |
| in Medicine (DICOM) Set | 12-342 |
| IEC ISO 10918-1:1994 - Digital Compression and Coding of | |
| Continuous-tone Still Images | 12-261 |
| ISO 14155:2020 - Clinical investigation of medical devices for | |
| human subjects - Good clinical practice | 2-282 |
Ultromics Limited cites conformity to the voluntary standards above. In addition, EchoGo Heart Failure 2.0 was designed and manufactured under a QMS that fully conforms to ISO 13485:2016 and FDA 21 CFR Part 820 Compliance.
10 Performance Data
10.1 Software Verification and Validation
EchoGo Heart Failure 2.0 software was developed and tested in accordance with Ultromics' Design Control processes and has been subjected to extensive safety and
14
performance testing. Non-clinical verification and validation test results established that the device meets its design requirements and intended use.
Specifically, software verification was conducted at unit, module, and system integration levels. Risk management analysis generated multiple risk mitigation measures and verification activities. Regression- and numerical stability testing were conducted to ensure the device meets algorithmic specifications. Formative and summative usability assessments were conducted to validate labelling and mitigate against the device outputs being misinterpreted by the clinical user. Cybersecurity and data security testing were conducted to verify that data and patient protected health information security measures are included in the design of the software.
Software verification and validation testing were conducted, and documentation was provided as recommended by FDA's Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices. The software for this device is considered as a moderate level of concern since a failure or latent design flaw could indirectly result in minor injury to the patient through incorrect or delayed information or through the action of a care provider.
EchoGo Heart Failure 2.0 passed all software verification and validation tests.
10.2 Essential Performance
Device performance was validated using bench- and clinical performance testing.
An independent clinical validation study was conducted on a clinical data set representative of the intended use population and containing a range of data sources and data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment. The study was used to demonstrate consistency of the device output as well as to assess agreement with reference ground truth.
Device performance was determined according to a retrospective case:control study including multiple sites spanning five states in the final testing data cohort amounted to 1,578 patients, comprising 785 controls and 793 cases. The final testing data represents a 22.9% increase in data beyond the testing data cohort utilized for the 510k submission of EchoGo Heart Failure 1.0.
-
- To assess sensitivity and specificity, we compared the device output for a single echocardiogram videoclip per patient to the ground truth classifications of cases (HFpEF) or controls. EchoGo Heart Failure 2.0 correctly identified 673 true positives, and 617 true negatives, alongside 100 false positives, 72 false negatives, and 116 no classification outputs (48 cases, 68 controls). This equated to a sensitivity of 90.3% (95% Cl: 88.5, 92.4%) and a specificity of 86.1% (95% Cl: 83.4, 88.3%) when removing the no classification studies from the calculation, as per the intended use of the
15
device. When including the no classification studies in the calculation, sensitivity was 84.9% (95% Cl: 83.0, 87.5%) and specificity was 78.6% (95% Cl: 75.3, 81.1%).
-
- To determine the accuracy of the EchoGo Score, we compared the EchoGo Score to known and expected proportions of HFpEF. The p value for the Hosmer-Lemeshow Test for goodness-of-fit was not significant (p=0.304), indicating acceptable fit between observed and known probabilities. The area under the receiver operator characteristic curve (AUROC) was 0.947 (95% Cl: 0.934, 0.958) when removing no classification studies, and 0.937 (95% Cl: 0.924, 0.949) when considering all studies and ignoring uncertainty and instability metrics. When classification statistics are examined across decile cut-offs for the EchoGo Score (0.1 or 10%, to 0.9 or 90%), instead of the default 0.5 (50%) used to determine classification output, we observe high sensitivity and specificity across most decision thresholds (minus the most extreme cut-offs; 0.9 and 0.1 for sensitivity and specificity, respectively). Similarly, when EchoGo Scores are separated into 5 stratum, post-test risk increases with increasing stratum, with 0% and 100% risk for the lowest, and highest stratum (respectively). Finally, a flexible calibration curve with non-parametric loess smoothing results in an intercept close to 0, slope close to 1, and ECI close to 0.
-
- The proportion of non-diagnostic (i.e., "No classification") outputs of the device were within a priori acceptance limits. Of the 1,578 studies analysed by the device, 116 (7.4%) were categorized as "No Classification".
-
- The device output classification from a single Digital Imaging and Communications in Medicine (DICOM) clip analysed twice (repeatability), and the device output classification from different DICOM clips from the same individual (reproducibility) was assessed for precision. The device demonstrated 100% repeatability in all measures and 82.6% Positive Agreement and 82.4% Negative Agreement for reproducibility.
All measurements produced by EchoGo Heart Failure 2.0 were deemed to be substantively equivalent to the predicate device and met pre-specified levels of performance. We therefore consider EchoGo Heart Failure 2.0 to be substantively equivalent to the predicate device and is therefore deemed to be safe and effective.
11 Conclusions
The subject device, EchoGo Heart Failure 2.0 is as safe and as effective as the predicate device, EchoGo Heart Failure, previously cleared under K222463.
Ultromics concludes that the predicate and subject devices have the same intended use as well as similar technological characteristics. Any minor differences between the subject and the predicate device, as described above, do not alter the intended use of the device, and do not raise new or different questions regarding its safety and effectiveness.
16
Furthermore, Ultromics believe special controls introduced under the 21 CFR 870.2200 regulation are sufficient to ensure safety and effectiveness. These include software verification and validation including a comprehensive hazard analysis; validation testing of the AI algorithm using a data set separate from the training data to demonstrate the validity of the device output; a usability assessment; clinical data in support of the intended use; as well as labelling consistent with the intended use. Performance data is provided as part of this PMN application to demonstrate that EchoGo Heart Failure 2.0 performs as intended in the specified use conditions and that it is as safe and effective as the predicate device and therefore substantially equivalent to K222463.