(110 days)
Libby™ Echo:Prio is software that is used to process previously acquired DICOM-compliant cardiac ultrasound images, and to make measurements on these images in order to provide automated estimation of several cardiac measurements. The data produced by this software is intended to be used to support qualified cardiologists, sonographers, or other licensed professional healthcare practitioners for clinical decision-making. Libby™ Echo:Prio is indicated for use in adult patients.
Echo:Prio is an image post-processing analysis software device used for viewing and quantifying cardiovascular ultrasound images. The device is intended to aid diagnostic review and analysis of echocardiographic data, patient record management and reporting. The software provides an interface for a skilled sonographer to perform the necessary markup on the echocardiographic image prior to review by the prescribing physician. The markup includes: the cardiac segments captured, measurements of distance, time, area, quantitative analysis of cardiac function, and a summary report. The software allows the sonographer to enter their markup manually and/or manually correct automatically generated results. It also provides automated markup and analysis, which the sonographer may choose to accept outright, to accept partially and modify, or to reject and ignore. Machine learning based view classification and border segmentation form the basis for this automated analysis. Additionally, the software has features for organizing, displaying, and comparing to reference guidelines the quantitative data from cardiovascular images acquired from ultrasound scanners.
The provided text describes the Libby™ Echo:Prio software, its intended use, and performance data from its premarket notification. Here's a breakdown of the acceptance criteria and the study proving the device meets them:
1. A table of acceptance criteria and the reported device performance
The document does not explicitly state "acceptance criteria" in a tabular format with defined thresholds. However, it presents performance metrics from the validation study which serve as the evidence of the device's capability. We can infer the implicit "acceptance criteria" from these reported performance metrics, which are presented as achieved targets.
| Metric (Implied Acceptance Criteria) | Reported Device Performance |
|---|---|
| View Classification Accuracy | 97% |
| View Classification F1 Score | > 96.6% (average) |
| View Classification Sensitivity (Sn) | 96.8% (average) |
| View Classification Specificity (Sp) | 98.5% (average) |
| Heart Rate (HR) Estimation Bias (Regression Slope) | 0.98 (95% CI) compared to 12-lead ECG ground truth |
| Ejection Fraction (EF) Prediction (Bivariate Linear Regression Slope) | 0.79 (95% CI: 0.52, 0.98) compared to human expert annotations |
2. Sample size used for the test set and the data provenance
- Test Set Sample Size: The document states that performance testing was done "retrospectively on a diverse clinical dataset." However, the exact sample size for this test set is not specified in the provided text.
- Data Provenance:
- Country of Origin: Not specified.
- Retrospective or Prospective: Retrospective.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
- Number of Experts: For the Ejection Fraction (EF) prediction, ground truth was established by "four human experts."
- Qualifications of Experts: The specific qualifications (e.g., years of experience, subspecialty) of these four human experts are not explicitly stated. They are referred to as "human experts" or "clinicians."
4. Adjudication method (e.g. 2+1, 3+1, none) for the test set
The adjudication method for constructing the ground truth from the four human experts for EF prediction is not explicitly stated. It is only mentioned that EF prediction was compared with "annotations by four human experts."
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
A Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly described in the provided text. The performance data focuses on the software's standalone accuracy in comparison to a ground truth, rather than measuring the improvement of human readers assisted by the AI.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance evaluation of the algorithm's predictions (view classification, HR, EF) against established ground truth was performed. The data presented ("view classification accuracy of 97%", "HR output estimate is with minimal bias", "prediction of the EF output... had a slope of 0.79") reflects the algorithm's direct performance.
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The types of ground truth used are:
- 12-lead ECG: For Heart Rate (HR) estimation.
- Human Expert Annotations/Consensus (implied): For Left Ventricular Ejection Fraction (EF) prediction, derived from the "annotations by four human experts." For view classification, the "accuracy" and "F1 value" imply a comparison to a set ground truth, likely also established by experts.
8. The sample size for the training set
The sample size for the training set used to develop the machine learning model is not specified in the provided text.
9. How the ground truth for the training set was established
The method for establishing ground truth for the training set is not specified in the provided text. It only vaguely mentions "Machine learning based view classification and border segmentation form the basis for this automated analysis."
{0}------------------------------------------------
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: the Department of Health & Human Services seal on the left and the FDA acronym along with the full name of the agency on the right. The FDA part of the logo is in blue, with the acronym in a square and the full name written out to the right of it.
Dyad Medical, Inc % Yervant Chijian Regulatory and Quality Consultant Pharmalex Pty Ltd Suite 10.4, 1 Chandos Street St. Leonards, NSW 2068 AUSTRALIA
Re: K220956
July 20, 2022
Trade/Device Name: Libby Echo:Prio Regulation Number: 21 CFR 892.2050 Regulation Name: Medical image management and processing system Regulatory Class: Class II Product Code: OIH Dated: June 17, 2022 Received: June 21, 2022
Dear Yervant Chijian:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801 and Part 809); medical device reporting of medical device-related adverse events) (21 CFR
{1}------------------------------------------------
- for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Jessica Lamb. Ph.D. Assistant Director Imaging Software Team DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K220956
Device Name Libby™ Echo:Prio
Indications for Use (Describe)
Libby™ Echo:Prio is software that is used to process previously acquired DICOM-compliant cardiac ultrasound images, and to make measurements on these images in order to provide automated estimation of several cardiac measurements.
The data produced by this software is intended to be used to support qualified cardiologists, sonographers, or other licensed professional healthcare practitioners for clinical decision-making.
LibbyTM Echo:Prio is indicated for use in adult patients.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| Prescription Use (Part 21 CFR 801 Subpart D) | Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
510(k) Summary – Libby™ Echo:Prio 5
General Information 5.1
| 510(k) Sponsor | Dyad Medical, Inc |
|---|---|
| Address | 215 Brighton Avenue, Suite 203Boston, MA 02134 |
| Correspondence Person | Yervant ChijianQuality and Regulatory ConsultantPharmalex Pty Ltd |
| Contact Information | Email: Yervant.Chijian@pharmalex.comPhone: +61 (0)2 9906 2984 |
| Date Prepared | 31st March 2022 |
5.2 Subject Device
| Proprietary Name | Libby™ Echo:Prio |
|---|---|
| Common Name | Echo:Prio |
| Classification Name | System, Image Processing, Radiological |
| Regulation Number | 21 CFR 892.2050 |
| Regulation Name | Automated Radiological Image Processing Software |
| Product Code | QIH |
| Regulatory Class | II |
5.3 Predicate Device
| Proprietary Name | EchoMD Automated Ejection Fraction Software |
|---|---|
| Premarket Notification | K173780 |
| Classification Name | System, Image Processing, Radiological |
| Regulation Number | 21 CFR 892.2050 |
| Regulation Name | Picture archiving and communications system |
| Product Code | LLZ |
| Regulatory Class | II |
5.4 Device Description
Echo:Prio is an image post-processing analysis software device used for viewing and quantifying cardiovascular ultrasound images. The device is intended to aid diagnostic review and analysis of echocardiographic data, patient record management and reporting.
The software provides an interface for a skilled sonographer to perform the necessary markup on the echocardiographic image prior to review by the prescribing physician. The markup includes: the cardiac segments captured, measurements of distance, time, area, quantitative analysis of cardiac function, and a summary report.
{4}------------------------------------------------
Image /page/4/Picture/1 description: The image shows the logo for DYAD Medical. On the left is a blue graphic that looks like three connected water droplets. To the right of the graphic is the text "DYAD" in large, bold, dark blue letters, with the word "MEDICAL" underneath in smaller, lighter blue letters.
The software allows the sonographer to enter their markup manually and/or manually correct automatically generated results. It also provides automated markup and analysis, which the sonographer may choose to accept outright, to accept partially and modify, or to reject and ignore. Machine learning based view classification and border segmentation form the basis for this automated analysis. Additionally, the software has features for organizing, displaying, and comparing to reference guidelines the quantitative data from cardiovascular images acquired from ultrasound scanners.
The following visualization, quantification and data-reporting functionalities are provided by the software:
5.4.1 Visualization:
- . 2D image review
- Cine loop review
- Secondary captures review
5.4.2 Quantification of classification, segmentation and index calculations:
- Echocardiographic View classification
- . End diastole (ED) and End systole (ES) frame identification enabling Heartbeat rate (HR) estimates.
- Ejection fraction (EF)
Notes: This is achieved via automatic left ventricle (LV) chamber endocardium segmentation, left atrium (LA) segmentation, LV myocardium (LVMC) segmentation allowing muscle thickness calculations.
5.4.3 Data reporting
All the above values are reported via a report generation initiated by the investigator.
5.5 Indications for Use
Libby Echo:Prio is software that is used to process previously acquired DICOM-compliant cardiac ultrasound images, and to make measurements on these images in order to provide automated estimation of several cardiac measurements. The data produced by this software is intended to be used to support qualified cardiologists, sonographers, or other licensed professional healthcare practitioners for clinical decision-making.
Libby Echo:Prio is indicated for use in adult patients.
ટ.6 Substantial Equivalence
The following table demonstrates the similarities and differences between the technological characteristics of the three products. Testing demonstrates that the differences do not raise new questions of safety or effectiveness.
| Topic | Subject DeviceLibby™TM Echo:Prio | Predicate DeviceEchoMD Automated Ejection FractionSoftware | SubstantialEquivalence |
|---|---|---|---|
| Intended Use/Indications forUse | Libby™TM Echo:Prio is software thatis used to process previouslyacquired DICOM-compliant | The Bay Labs, Inc. EchoMDAutomated Ejection Fractionsoftware is used to process | Same |
| Topic | Subject DeviceLibby™ Echo:Prio | Predicate DeviceEchoMD Automated Ejection FractionSoftware | SubstantialEquivalence |
| cardiac ultrasound images, and tomake measurements on theseimages in order to provideautomated estimation of severalcardiac measurements. The dataproduced by this software isintended to be used to supportqualified cardiologists,sonographers, or other licensedprofessional healthcarepractitioners for clinical decision-making.Libby™ Echo:Prio is indicated foruse in adult patients. | previously acquired transthoraciccardiac ultrasound images, to storeimages, and to manipulate andmake measurements on imagesusing a personal computer or acompatible DICOM-compliantPACS system in order to provideautomated estimation of leftventricular ejection fraction. Thismeasurement can be used to assistthe clinician in a cardiacevaluation. The EchoMDAutomated EjectionFractionSoftware is indicatedfor use inadult patients. | ||
| Intended User | Cardiologists and sonographers | Cardiologists and sonographers | Same |
| Rx or OTC | Rx | Rx | Same |
| Intended Location | Medical facility | Medical facility | Same |
| High Level DeviceDescription | The Libby™ Echo:Prio is an imagepost-processing analysis softwaredevice used for viewing andquantifying cardiovascularultrasound images. | EchoMD software is processacquired transthoracic cardiacultrasound images, to analyze andmake measurements on images inorder to provide automatedestimation of left ventricularejection fraction | Same |
| Automated Chamberanalysis Features & Analysis | Yes | Yes | Same |
| Automated measurements | LV Ejection fraction (EF) | Left ventricular ejection fraction | Same |
| Machine Learning BasedAlgorithm | Yes | Yes | Same |
| Operate on DICOM clips | Yes | Yes | Same |
| Automated ViewClassification including:1. long axis view (PLAX)2. short axis view (PSAX)3. four-chamber view (A4C)4. five-chamber view (A5C)5. two-chamber view (A2C)6. long axis view (A3C) | Yes | Yes | Same |
| Automation Level | Fully automated, including clipselection | Fully automated, including clipselection | Same |
| Algorithm Confidence | Qualitative user feedback ontransthoracic cardiacultrasound image quality | Qualitative and quantitative userfeedback on transthoracic cardiacultrasound image quality | Same |
| EF Method | Single plane & biplane (withsegmentation and endocardialtrace) | Biplane (non-segmentation/ non-endocardial trace) | Same |
| Topic | Subject DeviceLibby™ Echo:Prio | Predicate DeviceEchoMD Automated Ejection FractionSoftware | SubstantialEquivalence |
| Offline EF evaluation usingclips from multipleultrasound scanners | Yes | Yes | Same |
| Automated Ejection FractionCalculation | Yes | Yes | Same |
| Ejection Fraction reported | Whole number estimate(percentage) | Whole number estimate(percentage) | Same |
| User Confirmation/rejectionof result | Yes | Yes | Same |
| Manual editing ofautomated result by user | Yes | Yes | Same |
| Physical Characteristics | Software package that operates onoff-the-shelf hardware | Software package that operates onoff-the-shelf hardware | Not equivalentbut no issueswith safety andefficacy. |
| DICOM Standard Compliance | The software processes DICOMcompliant image data | The software processes DICOMcompliant image data | Same |
| Modalities | Ultrasound | Ultrasound | Same |
| User Interface | The software is designed for usewithin a web browser on a personalcomputer. | The software is designed for useon personal computer or acompatible DICOM-compliantPACS system. | Not equivalentbut no issueswith safety andefficacy. |
Table 1: Libby™ Echo:Prio Device Comparison.
{5}------------------------------------------------
Image /page/5/Picture/0 description: The image shows the logo for DYAD MEDICAL. The logo consists of a blue, stylized image of three interconnected circles on the left, and the words "DYAD MEDICAL" in blue on the right. The word "DYAD" is in a larger font than the word "MEDICAL."
{6}------------------------------------------------
Image /page/6/Picture/0 description: The image contains the logo for DYAD Medical. The logo consists of two parts: a blue, abstract, interconnected shape on the left, and the text "DYAD MEDICAL" on the right. The word "DYAD" is in a larger, bolder font, while "MEDICAL" is in a smaller, thinner font.
5.7 Performance Data
Safety and performance of the Libby™ Echo:Prio has been evaluated and verified in accordance with software specifications and applicable performance standards through software verification and validation testing. Additionally, the software validation activities were performed in accordance with IEC 62304:2006/AC: 2008- Medical device software – Software life cycle processes, in addition to the FDA Guidance documents, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" and "Content of Premarket Submission for Management of Cybersecurity in Medical Devices."
Performance Testing demonstrated robustness and accuracy retrospectively on a diverse clinical dataset. This demonstrates consistent analysis and low inter/intra-analyst variability of the automated procedures.
The testing demonstrated view classification accuracy of 97% with an average F1 value of >96.6%, average sensitivity (Sn) of 96.8% and average Specificity (Sp) of 98.5%. The testing also demonstrated that the HR output estimate is with minimal bias (slope of 0.98 in linear regression with confidence interval of 95%) compared to the ground truth (12-lead ECG) also showing exceptional accuracy in ED/ES identification.
Finally, the prediction of the EF output using the Libby™ Echo:Prio software had a slope of 0.79 for Bivariate Linear Regression (BLS) with the 95% confidence interval (Cl) of (0.52, 0.98) compared with the annotations by four human experts. This is well within the range of typical measurement variation between different clinicians, which is usually described as inter-observer variation and can be as low as (0.37, 0.52) of the 95% Cl.
{7}------------------------------------------------
Image /page/7/Picture/1 description: The image contains the logo for DYAD MEDICAL. The logo consists of a blue graphic on the left and the text "DYAD MEDICAL" on the right. The graphic appears to be a stylized representation of molecules or cells, with interconnected circles in varying shades of blue. The text "DYAD" is in a bold, dark blue font, while "MEDICAL" is in a thinner, lighter blue font and is positioned below "DYAD".
Standards Applied 5.8
The standards applied for the development of the software is listed below:
- NEMA PS 3.1 3.20 2021e Digital Imaging and Communications in Medicine (DICOM) Set .
- . IEC 62304:2006/A1:2016 Medical device software - Software life cycle processes
- ISO 14971:2019 Medical Devices -- Application of Risk Management to Medical Devices ●
- IEC 62366-1 Edition 1.1 2020-06 Medical Devices -- Part 1: Application of Usability Engineering to Medical Devices
- . 21 CFR 820 Quality System Regulations
- ISO 15223-1 Medical devices — Symbols to be used with medical device labels, labelling and information to be supplied — Part 1: General requirements
Conclusion 5.9
Based on the information submitted in this premarket notification, and based on the indications for use, technological characteristics and performance testing, the Libby™ Echo:Prio raises no new questions of safety and effectiveness and is substantially equivalent to the predicate device in terms of safety, efficacy, and performance.
{8}------------------------------------------------
Image /page/8/Picture/0 description: The image shows the logo for DYAD Medical. On the left is a blue graphic that looks like three connected water droplets. To the right of the graphic is the text "DYAD" in large, bold, blue letters, with the word "MEDICAL" underneath in smaller, thinner, blue letters.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).