(232 days)
EchoGo Amyloidosis 1.0 is an automated machine learning-based decision support system, indicated as a screening tool for adult patients aged 65 years and over with heart failure undergoing cardiovascular assessment using echocardiography.
When utilised by an interpreting physician, this device provides information alerting the physician for referral to confirmatory investigations.
EchoGo Amyloidosis 1.0 is indicated in adult patients aged 65 years and over with heart failure. Patient management decisions should not be made solely on the results of the EchoGo Amyloidosis 1.0 analysis.
EchoGo Amyloidosis 1.0 takes a 2D echocardiogram of an apical four chamber (A4C) as its input and reports as an output a binary classification decision suggestive of the presence of Cardiac Amyloidosis (CA).
The binary classification decision is derived from an AI algorithm developed using a convolutional neural network that was pre-trained on a large dataset of cases and controls.
The A4C echocardiogram should be acquired without contrast and contain at least one full cardiac cycle. Independent training, tune and test datasets were used for training and performance assessment of the device.
EchoGo Amyloidosis 1.0 is fully automated without a graphical user interface.
The ultimate diagnostic decision remains the responsibility of the interpreting clinician using patient presentation, medical history, and the results of available diagnostic tests, one of which may be EchoGo Amyloidosis 1.0.
EchoGo Amyloidosis 1.0 is a prescription only device.
The provided text describes the acceptance criteria and a study proving that the EchoGo Amyloidosis 1.0 device meets these criteria.
Here's a breakdown of the requested information:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are not explicitly stated as clear, quantitative thresholds in a "table" format within the provided text. Instead, the document describes the study that was conducted to demonstrate performance against generally accepted metrics for such devices (e.g., sensitivity, specificity, PPP, NPV, repeatability, reproducibility).
However, based on the results presented in the "10.2 Essential Performance" and "10.4 Precision" sections, we can infer the achieved performance metrics. The text states: "All measurements produced by EchoGo Amyloidosis 1.0 were deemed to be substantially equivalent to the predicate device and met pre-specified levels of performance." It does not, however, explicitly list those "pre-specified levels."
Here's a table summarizing the reported device performance:
| Metric | Reported Device Performance (95% CI) | Notes |
|---|---|---|
| Essential Performance | ||
| Sensitivity | 84.5% (80.3%, 88.5%) | Based on native disease proportion (36.7% prevalence) |
| Specificity | 89.7% (87.0%, 92.4%) | Based on native disease proportion (36.7% prevalence) |
| Positive Predictive Value (PPV) | 82.7% (78.8%, 86.5%) | At 36.7% prevalence |
| Negative Predictive Value (NPV) | 90.9% (88.8%, 93.2%) | At 36.7% prevalence |
| PPV (Inferred) | 15.6% (11.0%, 20.8%) | At 2.2% prevalence |
| NPV (Inferred) | 99.6% (99.5%, 99.7%) | At 2.2% prevalence |
| No-classifications Rate | 14.0% | Proportion of data for which the device returns "no classification" |
| Precision | ||
| Repeatability (Positive Agreement) | 100% | Single DICOM clip analyzed multiple times |
| Repeatability (Negative Agreement) | 100% | Single DICOM clip analyzed multiple times |
| Reproducibility (Positive Agreement) | 85.5% (82.4%, 88.2%) | Different DICOM clips from the same individual |
| Reproducibility (Negative Agreement) | 79.9% (76.5%, 83.2%) | Different DICOM clips from the same individual |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: 1,164 patients
- 749 controls
- 415 cases
- Data Provenance: Retrospective case:control study, collected from multiple sites spanning nine states in the USA. The data also included some "Non-USA" origin (as seen in the subgroup analysis table, but the overall testing data seems to be primarily US-based based on the description).
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts
The document does not explicitly state the number of experts or their specific qualifications (e.g., radiologists with X years of experience) used to establish the ground truth for the test set. It mentions that clinical validation was conducted to "assess agreement with reference ground truth" but does not detail how this ground truth was derived or by whom.
4. Adjudication Method for the Test Set
The document does not specify an adjudication method (e.g., 2+1, 3+1, none) used for the test set's ground truth establishment.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, the document does not describe an MRMC comparative effectiveness study where human readers improve with AI vs. without AI assistance. The study described is a standalone performance validation of the algorithm against a defined ground truth.
6. If a Standalone (i.e. algorithm only without human-in-the-loop performance) Was Done
Yes, a standalone performance study was done. The results presented (sensitivity, specificity, PPV, NPV) are for the algorithm's performance without a human-in-the-loop. The device is described as "fully automated without a graphical user interface" and is a "decision support system" that "provides information alerting the physician for referral." The performance metrics provided are directly from the algorithm's output compared to ground truth.
7. The Type of Ground Truth Used
The document states: "The clinical validation study was used to demonstrate consistency of the device output as well as to assess agreement with reference ground truth." However, it does not specify the nature of this "reference ground truth" (e.g., expert consensus, pathology, outcomes data).
8. The Sample Size for the Training Set
The training data characteristics table shows the following sample sizes:
- Controls: 1,262 (sum of age categories: 118+197+337+388+222)
- Cases: 1,302 (sum of age categories: 122+206+356+389+229)
- Total Training Set Sample Size: 2,564 patients
9. How the Ground Truth for the Training Set Was Established
The document states: "The binary classification decision is derived from an AI algorithm developed using a convolutional neural network that was pre-trained on a large dataset of cases and controls." It mentions that "Algorithm training data was collected from collaborating centres." However, it does not explicitly describe how the ground truth labels (cases/controls) for the training set were established. It is implied that these were clinically confirmed diagnoses of cardiac amyloidosis (cases) and non-amyloidosis (controls), but the method (e.g., biopsy, clinical diagnosis based on multiple tests, expert review) is not detailed.
§ 870.2200 Adjunctive cardiovascular status indicator.
(a)
Identification. The adjunctive cardiovascular status indicator is a prescription device based on sensor technology for the measurement of a physical parameter(s). This device is intended for adjunctive use with other physical vital sign parameters and patient information and is not intended to independently direct therapy.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software description, verification, and validation based on comprehensive hazard analysis must be provided, including:
(i) Full characterization of technical parameters of the software, including any proprietary algorithm(s);
(ii) Description of the expected impact of all applicable sensor acquisition hardware characteristics on performance and any associated hardware specifications;
(iii) Specification of acceptable incoming sensor data quality control measures; and
(iv) Mitigation of impact of user error or failure of any subsystem components (signal detection and analysis, data display, and storage) on accuracy of patient reports.
(2) Scientific justification for the validity of the status indicator algorithm(s) must be provided. Verification of algorithm calculations and validation testing of the algorithm using a data set separate from the training data must demonstrate the validity of modeling.
(3) Usability assessment must be provided to demonstrate that risk of misinterpretation of the status indicator is appropriately mitigated.
(4) Clinical data must be provided in support of the intended use and include the following:
(i) Output measure(s) must be compared to an acceptable reference method to demonstrate that the output measure(s) represent(s) the predictive measure(s) that the device provides in an accurate and reproducible manner;
(ii) The data set must be representative of the intended use population for the device. Any selection criteria or limitations of the samples must be fully described and justified;
(iii) Agreement of the measure(s) with the reference measure(s) must be assessed across the full measurement range; and
(iv) Data must be provided within the clinical validation study or using equivalent datasets to demonstrate the consistency of the output and be representative of the range of data sources and data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment.
(5) Labeling must include the following:
(i) The type of sensor data used, including specification of compatible sensors for data acquisition;
(ii) A description of what the device measures and outputs to the user;
(iii) Warnings identifying sensor reading acquisition factors that may impact measurement results;
(iv) Guidance for interpretation of the measurements, including warning(s) specifying adjunctive use of the measurements;
(v) Key assumptions made in the calculation and determination of measurements;
(vi) The measurement performance of the device for all presented parameters, with appropriate confidence intervals, and the supporting evidence for this performance; and
(vii) A detailed description of the patients studied in the clinical validation (
e.g., age, gender, race/ethnicity, clinical stability) as well as procedural details of the clinical study.