(308 days)
The HeartFlow Analysis is an AI-based medical device software for the clinical quantitative and qualitative analysis of previously acquired Computed Tomography DICOM data for patients with suspected coronary artery disease. It provides anatomic data, plaque identification and characterization, as well as the calculations of FFRCT, a coronary physiological simulation, computed from simulated pressure, velocity and blood flow information obtained from a 3D computer model generated from static coronary CT images. The HeartFlow Analysis is intended to support the risk assessment and functional evaluation of coronary artery disease.
The HeartFlow Analysis is provided to support qualified clinicians to aid in the evaluation and risk assessment of coronary artery disease. The HeartFlow Analysis is intended to be used by qualified clinicians in conjunction with the patient's clinical history, symptoms, and other diagnostic tests, as well as the clinician's professional judgment.
The HeartFlow Analysis is an Al-based medical device software developed for the clinical quantitative and qualitative analysis of CT DICOM data. It is a tool for the analysis of CT DICOM-compliant cardiac images and data, to assess the anatomy and function of the coronary arteries in the risk stratification and evaluation of coronary artery disease.
The software displays coronary and functional information using graphics and text, including computed and derived quantities of percent stenosis, plaque volumes, blood flow, pressure and velocity, to aid the clinician in the assessment and treatment planning of coronary artery disease.
The HeartFlow Analysis is performed on previously physician-acquired image data and is unrelated to acquisition equipment and clinical workstations.
Here's a breakdown of the acceptance criteria and study information based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The provided text does not explicitly state specific acceptance criteria with numerical targets or thresholds. It generally discusses "validation studies including stress testing, and repeatability testing to ensure the safety and effectiveness of the device" and that "results concluded the device was acceptable for use."
However, based on the context of the device and its intended use, we can infer general performance areas. Since no specific acceptance criteria are given, the "Reported Device Performance" column will reflect the general conclusion from the document.
| Criteria Area (Inferred) | Acceptance Criteria (Not Explicitly Stated) | Reported Device Performance |
|---|---|---|
| Safety and Effectiveness | Device is safe and effective for its intended use. | Validation studies included stress testing and repeatability testing. Medical device design validation has been completed, encompassing testing and evaluation using previously acquired diagnostic images from HeartFlow-sponsored clinical trials. Results concluded the device was acceptable for use. |
| Accuracy of FFRct Calculation | FFRct calculations are accurate. | The device computes FFRct, a coronary physiological simulation, from simulated pressure, velocity, and blood flow information obtained from a 3D computer model generated from static coronary CT images. No specific accuracy metrics are provided in this document, but implies accuracy through its intended use and general validation. |
| Plaque Identification/Characterization | Plaque identification and characterization is accurate. | The device provides plaque identification and characterization, as well as anatomic data. Supported by comparison to the Autoplaque predicate device. Implies accuracy through intended use and general validation. |
| Anatomic Data Extraction | Accurate extraction of anatomic data. | The device extracts anatomic data to aid in risk assessment and functional evaluation. Implies accuracy through intended use and general validation. |
2. Sample Size for Test Set and Data Provenance
- Sample Size for Test Set: The document states that testing and evaluation used "previously acquired diagnostic images received through HeartFlow sponsored clinical trials." However, a specific number for the test set sample size is not provided.
- Data Provenance: The data was "previously acquired diagnostic images received through HeartFlow sponsored clinical trials." This suggests the data is retrospective (already acquired) and likely originates from various clinical trial sites, but specific countries are not mentioned.
3. Number of Experts Used to Establish Ground Truth and Qualifications
The document does not specify the number of experts used or their qualifications for establishing ground truth for the test set.
4. Adjudication Method for the Test Set
The document does not describe an adjudication method (e.g., 2+1, 3+1).
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
The provided text does not mention if an MRMC comparative effectiveness study was done, nor does it specify an effect size of human readers' improvement with AI vs. without AI assistance. The focus is on the device's standalone validation.
6. Standalone (Algorithm Only) Performance
Yes, a standalone (algorithm only) performance assessment was done. The document states:
- "The software was designed, developed, tested and validated according to written procedures."
- "Validation studies included stress testing, and repeatability testing to ensure the safety and effectiveness of the device."
- "Medical device design validation has been completed. Medical device design included testing and evaluation using previously acquired diagnostic images received through HeartFlow sponsored clinical trials."
- "Summaries of pre-clinical studies were reviewed as part of a prior predicate review (K161772, the original predicate of K182035/K190925/K203329). The results concluded the device was acceptable for use."
This indicates that the device's performance was evaluated independently without human intervention during the "testing and evaluation" phase described, proving its standalone capabilities.
7. Type of Ground Truth Used
The document implies that the ground truth was established through clinical diagnosis and evaluation of the "previously acquired diagnostic images" from HeartFlow-sponsored clinical trials. While it doesn't explicitly state "expert consensus," this is the most likely method for establishing ground truth in clinical trials concerning coronary artery disease diagnoses from images. No mention of pathology or outcomes data as direct ground truth is made in this specific excerpt for the validation studies mentioned.
8. Sample Size for the Training Set
The document states: "The core technology remains unchanged from the primary predicate and continues to be trained using deep learning (AI and machine learning) since 2015, to incorporate learnings from the volumes of CT data and studies."
However, a specific sample size for the training set is not provided. It only mentions "volumes of CT data and studies."
9. How the Ground Truth for the Training Set Was Established
The document implies that the ground truth for the training set was established through "learnings from the volumes of CT data and studies." Similar to the test set, this would likely involve expert interpretation and analysis of the CT data used for training the deep learning algorithms, reflecting accepted clinical diagnoses and findings within those studies. No further details are given.
§ 870.1415 Coronary vascular physiologic simulation software device.
(a)
Identification. A coronary vascular physiologic simulation software device is a prescription device that provides simulated functional assessment of blood flow in the coronary vascular system using data extracted from medical device imaging to solve algorithms and yield simulated metrics of physiologic information (e.g., blood flow, coronary flow reserve, fractional flow reserve, myocardial perfusion). A coronary vascular physiologic simulation software device is intended to generate results for use and review by a qualified clinician.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Adequate software verification and validation based on comprehensive hazard analysis, with identification of appropriate mitigations, must be performed, including:
(i) Full characterization of the technical parameters of the software, including:
(A) Any proprietary algorithm(s) used to model the vascular anatomy; and
(B) Adequate description of the expected impact of all applicable image acquisition hardware features and characteristics on performance and any associated minimum specifications;
(ii) Adequate consideration of privacy and security issues in the system design; and
(iii) Adequate mitigation of the impact of failure of any subsystem components (
e.g., signal detection and analysis, data storage, system communications and cybersecurity) with respect to incorrect patient reports and operator failures.(2) Adequate non-clinical performance testing must be provided to demonstrate the validity of computational modeling methods for flow measurement; and
(3) Clinical data supporting the proposed intended use must be provided, including the following:
(i) Output measure(s) must be compared to a clinically acceptable method and must adequately represent the simulated measure(s) the device provides in an accurate and reproducible manner;
(ii) Clinical utility of the device measurement accuracy must be demonstrated by comparison to that of other available diagnostic tests (
e.g., from literature analysis);(iii) Statistical performance of the device within clinical risk strata (
e.g., age, relevant comorbidities, disease stability) must be reported;(iv) The dataset must be adequately representative of the intended use population for the device (
e.g., patients, range of vessel sizes, imaging device models). Any selection criteria or limitations of the samples must be fully described and justified;(v) Statistical methods must consider the predefined endpoints:
(A) Estimates of probabilities of incorrect results must be provided for each endpoint,
(B) Where multiple samples from the same patient are used, statistical analysis must not assume statistical independence without adequate justification, and
(C) The report must provide appropriate confidence intervals for each performance metric;
(vi) Sensitivity and specificity must be characterized across the range of available measurements;
(vii) Agreement of the simulated measure(s) with clinically acceptable measure(s) must be assessed across the full range of measurements;
(viii) Comparison of the measurement performance must be provided across the range of intended image acquisition hardware; and
(ix) If the device uses a cutoff threshold or operates across a spectrum of disease, it must be established prior to validation, and it must be justified as to how it was determined and clinically validated;
(4) Adequate validation must be performed and controls implemented to characterize and ensure consistency (
i.e., repeatability and reproducibility) of measurement outputs:(i) Acceptable incoming image quality control measures and the resulting image rejection rate for the clinical data must be specified, and
(ii) Data must be provided within the clinical validation study or using equivalent datasets demonstrating the consistency (
i.e., repeatability and reproducibility) of the output that is representative of the range of data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment;(A) Testing must be performed using multiple operators meeting planned qualification criteria and using the procedure that will be implemented in the production use of the device, and
(B) The factors (
e.g., medical imaging dataset, operator) must be identified regarding which were held constant and which were varied during the evaluation, and a description must be provided for the computations and statistical analyses used to evaluate the data;(5) Human factors evaluation and validation must be provided to demonstrate adequate performance of the user interface to allow for users to accurately measure intended parameters, particularly where parameter settings that have impact on measurements require significant user intervention; and
(6) Device labeling must be provided that adequately describes the following:
(i) The device's intended use, including the type of imaging data used, what the device measures and outputs to the user, whether the measure is qualitative or quantitative, the clinical indications for which it is to be used, and the specific population for which the device use is intended;
(ii) Appropriate warnings specifying the intended patient population, identifying anatomy and image acquisition factors that may impact measurement results, and providing cautionary guidance for interpretation of the provided measurements;
(iii) Key assumptions made in the calculation and determination of simulated measurements;
(iv) The measurement performance of the device for all presented parameters, with appropriate confidence intervals, and the supporting evidence for this performance. Per-vessel clinical performance, including where applicable localized performance according to vessel and segment, must be included as well as a characterization of the measurement error across the expected range of measurement for key parameters based on the clinical data;
(v) A detailed description of the patients studied in the clinical validation (
e.g., age, gender, race or ethnicity, clinical stability, current treatment regimen) as well as procedural details of the clinical study (e.g., scanner representation, calcium scores, use of beta-blockers or nitrates); and(vi) Where significant human interface is necessary for accurate analysis, adequately detailed description of the analysis procedure using the device and any data features that could affect accuracy of results.