Search Filters

Search Results

Found 4 results

510(k) Data Aggregation

    K Number
    K240860
    Manufacturer
    Date Cleared
    2024-11-15

    (232 days)

    Product Code
    Regulation Number
    870.2200
    Reference & Predicate Devices
    Why did this record match?
    Applicant Name (Manufacturer) :

    Ultromics Limited

    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    EchoGo Amyloidosis 1.0 is an automated machine learning-based decision support system, indicated as a screening tool for adult patients aged 65 years and over with heart failure undergoing cardiovascular assessment using echocardiography.

    When utilised by an interpreting physician, this device provides information alerting the physician for referral to confirmatory investigations.

    EchoGo Amyloidosis 1.0 is indicated in adult patients aged 65 years and over with heart failure. Patient management decisions should not be made solely on the results of the EchoGo Amyloidosis 1.0 analysis.

    Device Description

    EchoGo Amyloidosis 1.0 takes a 2D echocardiogram of an apical four chamber (A4C) as its input and reports as an output a binary classification decision suggestive of the presence of Cardiac Amyloidosis (CA).

    The binary classification decision is derived from an AI algorithm developed using a convolutional neural network that was pre-trained on a large dataset of cases and controls.

    The A4C echocardiogram should be acquired without contrast and contain at least one full cardiac cycle. Independent training, tune and test datasets were used for training and performance assessment of the device.

    EchoGo Amyloidosis 1.0 is fully automated without a graphical user interface.

    The ultimate diagnostic decision remains the responsibility of the interpreting clinician using patient presentation, medical history, and the results of available diagnostic tests, one of which may be EchoGo Amyloidosis 1.0.

    EchoGo Amyloidosis 1.0 is a prescription only device.

    AI/ML Overview

    The provided text describes the acceptance criteria and a study proving that the EchoGo Amyloidosis 1.0 device meets these criteria.

    Here's a breakdown of the requested information:

    1. Table of Acceptance Criteria and Reported Device Performance

    The acceptance criteria are not explicitly stated as clear, quantitative thresholds in a "table" format within the provided text. Instead, the document describes the study that was conducted to demonstrate performance against generally accepted metrics for such devices (e.g., sensitivity, specificity, PPP, NPV, repeatability, reproducibility).

    However, based on the results presented in the "10.2 Essential Performance" and "10.4 Precision" sections, we can infer the achieved performance metrics. The text states: "All measurements produced by EchoGo Amyloidosis 1.0 were deemed to be substantially equivalent to the predicate device and met pre-specified levels of performance." It does not, however, explicitly list those "pre-specified levels."

    Here's a table summarizing the reported device performance:

    MetricReported Device Performance (95% CI)Notes
    Essential Performance
    Sensitivity84.5% (80.3%, 88.5%)Based on native disease proportion (36.7% prevalence)
    Specificity89.7% (87.0%, 92.4%)Based on native disease proportion (36.7% prevalence)
    Positive Predictive Value (PPV)82.7% (78.8%, 86.5%)At 36.7% prevalence
    Negative Predictive Value (NPV)90.9% (88.8%, 93.2%)At 36.7% prevalence
    PPV (Inferred)15.6% (11.0%, 20.8%)At 2.2% prevalence
    NPV (Inferred)99.6% (99.5%, 99.7%)At 2.2% prevalence
    No-classifications Rate14.0%Proportion of data for which the device returns "no classification"
    Precision
    Repeatability (Positive Agreement)100%Single DICOM clip analyzed multiple times
    Repeatability (Negative Agreement)100%Single DICOM clip analyzed multiple times
    Reproducibility (Positive Agreement)85.5% (82.4%, 88.2%)Different DICOM clips from the same individual
    Reproducibility (Negative Agreement)79.9% (76.5%, 83.2%)Different DICOM clips from the same individual

    2. Sample Size Used for the Test Set and Data Provenance

    • Test Set Sample Size: 1,164 patients
      • 749 controls
      • 415 cases
    • Data Provenance: Retrospective case:control study, collected from multiple sites spanning nine states in the USA. The data also included some "Non-USA" origin (as seen in the subgroup analysis table, but the overall testing data seems to be primarily US-based based on the description).

    3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

    The document does not explicitly state the number of experts or their specific qualifications (e.g., radiologists with X years of experience) used to establish the ground truth for the test set. It mentions that clinical validation was conducted to "assess agreement with reference ground truth" but does not detail how this ground truth was derived or by whom.

    4. Adjudication Method for the Test Set

    The document does not specify an adjudication method (e.g., 2+1, 3+1, none) used for the test set's ground truth establishment.

    5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

    No, the document does not describe an MRMC comparative effectiveness study where human readers improve with AI vs. without AI assistance. The study described is a standalone performance validation of the algorithm against a defined ground truth.

    6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done

    Yes, a standalone performance study was done. The results presented (sensitivity, specificity, PPV, NPV) are for the algorithm's performance without a human-in-the-loop. The device is described as "fully automated without a graphical user interface" and is a "decision support system" that "provides information alerting the physician for referral." The performance metrics provided are directly from the algorithm's output compared to ground truth.

    7. The Type of Ground Truth Used

    The document states: "The clinical validation study was used to demonstrate consistency of the device output as well as to assess agreement with reference ground truth." However, it does not specify the nature of this "reference ground truth" (e.g., expert consensus, pathology, outcomes data).

    8. The Sample Size for the Training Set

    The training data characteristics table shows the following sample sizes:

    • Controls: 1,262 (sum of age categories: 118+197+337+388+222)
    • Cases: 1,302 (sum of age categories: 122+206+356+389+229)
    • Total Training Set Sample Size: 2,564 patients

    9. How the Ground Truth for the Training Set Was Established

    The document states: "The binary classification decision is derived from an AI algorithm developed using a convolutional neural network that was pre-trained on a large dataset of cases and controls." It mentions that "Algorithm training data was collected from collaborating centres." However, it does not explicitly describe how the ground truth labels (cases/controls) for the training set were established. It is implied that these were clinically confirmed diagnoses of cardiac amyloidosis (cases) and non-amyloidosis (controls), but the method (e.g., biopsy, clinical diagnosis based on multiple tests, expert review) is not detailed.

    Ask a Question

    Ask a specific question about this device

    K Number
    K240013
    Manufacturer
    Date Cleared
    2024-09-23

    (265 days)

    Product Code
    Regulation Number
    870.2200
    Reference & Predicate Devices
    Why did this record match?
    Applicant Name (Manufacturer) :

    Ultromics Limited

    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    EchoGo Heart Failure 2.0 is an automated machine learning-based decision support system, indicated as a diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography. When utilised by an interpreting clinician, this device provides information that may be useful in detecting heart failure with preserved ejection fraction (HFpEF).

    EchoGo Heart Failure 2.0 is indicated in adult populations over 25 years of age. Patient management decisions should not be made solely on the results of the EchoGo Heart Failure 2.0 analysis.

    EchoGo Heart Failure 2.0 takes as input an apical 4-chamber view of the heart that has been captured and assessed to have an ejection fraction ≥50%.

    Device Description

    EchoGo Heart Failure 2.0 takes as input a 2D echocardiogram of an apical four chamber tomographic view and reports as output a binary classification suggestive of the presence, or absence of heart failure with preserved ejection fraction (HFpEF). EchoGo Heart Failure 2.0 also provides users with an EchoGo Score ranging from 0 to 100% to support the binary classification. The EchoGo Score informs the binary classification when referenced against the pre-determined decision threshold (50%).

    To aid in the interpretation of the EchoGo Score, a comparative visual analysis is provided. A histogram format displays the reported EchoGo Score output against a population of patients with known disease status (Independent Testing Dataset). This allows the user to interpret the EchoGo Score relative to the decision threshold of 50%.

    EchoGo Heart Failure 2.0 should receive an input echocardiogram acquired without contrast and contain at least one full cardiac cycle.

    EchoGo Heart Failure 2.0 is fully automated and does not comprise a graphical user interface.

    EchoGo Heart Failure 2.0 is intended to be used by an interpreting clinician as an aid to diagnosis for HFpEF. The ultimate diagnostic decision remains the responsibility of the interpreting clinician using patient presentation, medical history, and the results of available diagnostic tests, one of which may be EchoGo Heart Failure 2.0.

    EchoGo Heart Failure 2.0 is a prescription only device.

    AI/ML Overview

    Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:

    Acceptance Criteria and Reported Device Performance

    CriteriaAcceptance LimitReported Device Performance
    I. Device Performance (Sensitivity & Specificity)Implicit within reporting of performance: The device must demonstrate sufficient sensitivity and specificity for detecting HFpEF as a diagnostic aid. The specific acceptance limits are not explicitly stated as numerical thresholds but are demonstrated by the reported performance being "substantively equivalent to the predicate device and met pre-specified levels of performance."Sensitivity: 90.3% (95% CI: 88.5, 92.4%) when removing "no classification" studies. 84.9% (95% CI: 83.0, 87.5%) when including "no classification" studies. Specificity: 86.1% (95% CI: 83.4, 88.3%) when removing "no classification" studies. 78.6% (95% CI: 75.3, 81.1%) when including "no classification" studies.
    II. Accuracy of EchoGo Score (AUROC & Goodness-of-Fit)Implicit within reporting of performance: The EchoGo Score must be accurate and align with known and expected proportions of HFpEF. Statistical significance (p-value > 0.05) for the Hosmer-Lemeshow Test and a sufficiently high AUROC are expected.Area Under the Receiver Operating Characteristic Curve (AUROC): 0.947 (95% CI: 0.934, 0.958) when removing "no classification" studies. 0.937 (95% CI: 0.924, 0.949) when considering all studies. Hosmer-Lemeshow Test for goodness-of-fit: p=0.304 (not significant, indicating acceptable fit).
    III. Proportion of Non-Diagnostic OutputsA priori acceptance limits: The proportion of "no classification" outputs must be within pre-specified limits (the exact numerical limit is not provided, but the text states it was "within a priori acceptance limits").7.4% (116 out of 1,578 studies) were categorized as "No Classification."
    IV. Precision (Repeatability and Reproducibility)Implicit within reporting of performance: The device must demonstrate high repeatability and acceptable reproducibility in its classification output.Repeatability: 100% in all measures. Reproducibility: 82.6% Positive Agreement and 82.4% Negative Agreement.

    Study Details

    1. Sample size used for the test set and the data provenance:

      • Test Set Sample Size: 1,578 patients (785 controls and 793 cases).
      • Data Provenance: Retrospective case:control study. The data was collected from multiple independent clinical sites spanning five states in the US.
    2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

      • The document states that the ground truth was established by "ground truth classifications of cases (HFpEF) or controls," but it does not specify the number or qualifications of experts who established this ground truth for the test set.
    3. Adjudication method for the test set:

      • The document does not explicitly describe an adjudication method (e.g., 2+1, 3+1). It only refers to "ground truth classifications," implying these were already established.
    4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

      • No, a multi-reader multi-case (MRMC) comparative effectiveness study evaluating human readers with and without AI assistance was not done. The study focuses on the standalone performance of the device. The device is intended as a "diagnostic aid" for use "by an interpreting clinician," but its performance evaluation presented here is not an MRMC study.
    5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

      • Yes, a standalone performance study was done. The reported sensitivity, specificity, AUROC, and precision values are for the device (algorithm) itself without human intervention in the classification output for the test set. The device provides a "binary classification suggestive of the presence, or absence of heart failure with preserved ejection fraction (HFpEF)" and an "EchoGo Score."
    6. The type of ground truth used (expert consensus, pathology, outcomes data, etc):

      • The ground truth was based on "ground truth classifications of cases (HFpEF) or controls," and "known and expected proportions of HFpEF." While not explicitly stated as "expert consensus," this terminology strongly implies clinical diagnoses were used to establish the HFpEF status for each patient in the dataset. It does not mention pathology or outcomes data specifically for ground truth.
    7. The sample size for the training set:

      • The sample size for the training set is not explicitly stated in the provided text. It mentions that the "Subject device AI model was trained on more data and with additional preprocessing steps and data augmentations" compared to the predicate device, and the testing data cohort was a "22.9% increase in data beyond the testing data cohort utilized for the 510k submission of EchoGo Heart Failure 1.0." However, the exact size of the training set is not provided.
    8. How the ground truth for the training set was established:

      • The document does not explicitly describe how the ground truth for the training set was established. It only states that the AI model was "trained on more data" with "additional preprocessing steps and data augmentations." It is highly probable it was established similarly to the test set ground truth (i.e., using clinical diagnoses or expert classifications), given the nature of the diagnostic task.
    Ask a Question

    Ask a specific question about this device

    K Number
    K222463
    Manufacturer
    Date Cleared
    2022-11-23

    (100 days)

    Product Code
    Regulation Number
    870.2200
    Reference & Predicate Devices
    Why did this record match?
    Applicant Name (Manufacturer) :

    Ultromics Limited

    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    EchoGo Heart Failure 1.0 is an automated machine learning-based decision support system, indicated as a diagnostic aid for patients undergoing routine functional cardiovascular assessment using echocardiography. When utilised by an interpreting clinician, this device provides information that may be useful in detecting heart failure with preserved ejection fraction (HFpEF).

    EchoGo Heart Failure 1.0 is indicated in adult populations over 25 years of age. Patient management decisions should not be made solely on the results of the EchoGo Heart Failure 1.0 analysis.

    EchoGo Heart Failure 1.0 takes as input an apical 4-chamber view of the heart that has been captured and assessed to have an ejection fraction ≥50%.

    Device Description

    EchoGo Heart Failure 1.0 is a software-only medical device manufactured by Ultromics Limited and granted breakthrough status by the FDA under Q212613.

    EchoGo Heart Failure 1.0 takes as input a DICOM file containing an echocardiogram and reports a classification decision suggestive of the presence or absence of heart failure with preserved ejection fraction (HFpEF). The output of this device is based on an artificial intelligence (Al) model developed using a convolutional neural network that produces the classification result. The model takes as input a 2D echocardiogram in which an apical 4-chamber view of the heart has been captured and assessed to have an ejection fraction ≥50% (this would normally be computed using a medical device for the assessment of cardiac function of the left ventricle, for example K213275). The echocardiogram should be acquired without contrast and contain at least one full cardiac cycle.

    Independent training, validation and test datasets were used for training and performance assessment of the device. EchoGo Heart Failure 1.0 is fully automated and does not comprise a user interface.

    EchoGo Heart Failure 1.0 produces a report containing the result of the classification, and this report is intended to be used by an interpreting clinician as an aid to diagnosis for HFpEF. The results are intended as an additional input to standard diagnostic pathways and should only be used by an interpreting clinician. The device is a diagnostic aid and thus according to common medical sense and the principles of differential diagnosis any diagnostic finding derived from usage of this product must be confirmed by additional diagnostic investigations, if in doubt. The ultimate diagnostic decision remains the responsibility of the interpreting clinician using patient presentation, medical history, and the results of available diagnostic tests, one of which may be EchoGo Heart Failure 1.0.

    EchoGo Heart Failure 1.0 is a prescription only device.

    AI/ML Overview

    Here's a breakdown of the acceptance criteria and the study proving the device meets those criteria, based on the provided text:

    1. Table of Acceptance Criteria and Reported Device Performance

    The provided text only explicitly states the reported device performance and the p-values for one-sided binomial exact tests against a priori acceptance criteria. The specific numerical acceptance criteria (e.g., minimum sensitivity and specificity thresholds) are not directly stated in the document. However, since the p-values are reported as p

    Ask a Question

    Ask a specific question about this device

    K Number
    K213275
    Manufacturer
    Date Cleared
    2021-12-20

    (81 days)

    Product Code
    Regulation Number
    892.2050
    Reference & Predicate Devices
    Why did this record match?
    Applicant Name (Manufacturer) :

    Ultromics Limited

    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    EchoGo Core is intended to be used for quantification and reporting of results of cardiovascular function to support physician diagnosis. EchoGo Core is indicated for use in adult populations.

    Device Description

    EchoGo Core 2.0 is a software application manufactured by Ultromics to provide a report of left ventricular cardiac function, in the form of secondary capture DICOM files and/or as a structured DICOM report, to aid interpreting physicians with diagnostic decision-making process. EchoGo Core 2.0 applies to ultrasound images of the heart (echocardiograms).

    EchoGo Core 2.0 utilizes artificial intelligence (AI) for the operator-assisted automatic quantification of commonly measured echocardiographic metrics. Independent training, test and validation datasets were used for training and performance assessment of the device.

    AI/ML Overview

    Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:

    1. Table of Acceptance Criteria and Reported Device Performance

    The acceptance criteria were formalized in terms of Root Mean Square (RMS) error against reference values generated using the comparator device, TomTec Arena TTA2. The text states that "the acceptance criteria were formalized such that EchoGo Core 2.0 would produce measures of left ventricular (LV) length, volume at end diastole (ED), and end systole (ES), ejection fraction (EF), stroke volume, cardiac output, global longitudinal strain (GLS) and segmental longitudinal strain (SLS) with an RMS error below a set, pre-determined threshold." While the exact pre-determined thresholds for acceptance are not explicitly listed in numerical values, the "Performance against the comparator device is summarised as follows," implying these are the reported performance values that met the (unspecified) acceptance thresholds.

    Left Ventricular MetricRoot Mean Square Error (% RMS)
    Length3.06 - 4.59
    Volume at End Diastole and End Systole8.57 - 16.59
    Ejection Fraction6.69 - 8.50
    Stroke Volume10.57 - 13.68
    Global Longitudinal Strain3.36 - 4.79
    Systolic Segmental Longitudinal Strain5.51 - 9.98

    2. Sample Size Used for the Test Set and Data Provenance

    • Sample Size: 214 previously unseen studies.
    • Data Provenance:
      • Country of Origin: Not explicitly stated, but the submission is from Ultromics Limited in the United Kingdom.
      • Retrospective or Prospective: Retrospective. The study was described as a "formal retrospective, non-interventional validation study."

    3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

    The ground truth for the test set was established using a comparator device, TomTec Arena TTA2 (K150122), not human experts providing a direct ground truth. The acceptance criteria were formalized in terms of Root Mean Square (RMS) error against reference values generated using the comparator device. There is no mention of experts establishing ground truth for the test set; instead, the comparator device served as the reference.

    4. Adjudication Method for the Test Set

    Not applicable, as the ground truth was established by a comparator device rather than human interpretation requiring adjudication.

    5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, and the Effect Size

    No, an MRMC comparative effectiveness study where human readers improve with AI vs. without AI assistance was not conducted or reported. The study focused on the device's performance against a comparator device.

    6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) was Done

    The primary performance study described seems to align with a standalone assessment against a comparator device. The text states, "EchoGo Core 2.0 would produce measures... with an RMS error below a set, pre-determined threshold" and "Performance against the comparator device is summarised as follows." This suggests the algorithm's output was directly compared to the output of the TomTec Arena TTA2.

    However, it's crucial to note the device description also states: "EchoGo Core 2.0 requires an operator at key steps to confirm or relabel automatically labeled acquisition views (if required) and approve the left ventricle segmentations (contours) proposed by the AI." And "The operator will review the report produced and may be asked to approve cautions that are added to the report." This indicates that while the core performance metrics were evaluated in an algorithm-centric manner against a comparator, the device's intended use involves a "human-in-the-loop" for confirmation/approval steps. The reported performance metrics (RMS error) are likely for the algorithm's core measurements before potential human override, as the study compared them to an automated comparator device.

    7. The Type of Ground Truth Used

    The ground truth was established by comparison to a legally marketed predicate device (TomTec Arena TTA2, K150122). Values generated by the TomTec Arena TTA2 served as the reference values for calculating RMS error.

    8. The Sample Size for the Training Set

    The sample size for the training set is not specified in the provided text. The text only mentions that "Independent training, test and validation datasets were used for training and performance assessment of the device" and that "Test datasets were strictly segregated from algorithm training datasets."

    9. How the Ground Truth for the Training Set was Established

    The method for establishing ground truth for the training set is not explicitly detailed in the provided text. It only states that "Independent training, test and validation datasets were used for training and performance assessment of the device."

    Ask a Question

    Ask a specific question about this device

    Page 1 of 1