(57 days)
HeartFlow FFRCT is a coronary physiologic simulation software for the clinical quantitative and qualitative analysis of previously acquired Computed Tomography DICOM data for clinically stable symptomatic patients with coronary artery disease. It provides FFRCT, a mathematically derived quantity, computed from simulated pressure, velocity and blood flow information obtained from a 3D computer model generated from static coronary CT images. FFRCT analysis is intended to support the functional evaluation of coronary artery disease.
The results of this analysis are provided to support qualified clinicians to aid in the evaluation and assessment of coronary arteries. The results of HeartFlow FFRCT are intended to be used by qualified clinicians in conjunction with the patient's clinical history, symptoms, and other diagnostic tests, as well as the clinician's professional judgment.
FFRct v2.0 is coronary physiologic simulation software developed for the clinical quantitative and qualitative analysis of CT DICOM data. It is a tool for the analysis of CT DICOM-compliant cardiac images and data, to assess the anatomy and function of the coronary arteries.
The software displays the anatomy combined with function using graphics and text, including computed and derived quantities of blood flow, pressure and velocity, to aid the clinician in the assessment of coronary artery disease.
Here is a detailed breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria are presented as target rates for sensitivity and specificity, with achievement determined if the lower one-sided 95% confidence bound (LCL) exceeded the target rate.
| Metric | Acceptance Criteria (Target Rate) | Reported Device Performance (Estimate %) | Lower One-Sided 95% Confidence Bound | Result (Met/Not Met) |
|---|---|---|---|---|
| Sensitivity | 65% | 84.2% | 75.8% | MET |
| Specificity | 55% | 84.9% | 80.4% | MET |
Note: The definition for "Diseased" was: FFRCT ≤ 0.80 and FFR (reference standard) ≤ 0.80.
Additionally, a per-subject diagnostic performance analysis against the invasive FFR reference standard was provided, showing:
| Metric | FFRCT ≤ 0.80 (Estimate % (95% Wilson CI)) |
|---|---|
| Diagnostic Accuracy | 80.0% (74.4%-84.6%) |
| Sensitivity | 87.8% (78.5%-93.5%) |
| Specificity | 76.4% (69.3%-82.3%) |
| PPV | 63.1% (53.5%-71.8%) |
| NPV | 93.2% (87.5%-96.4%) |
2. Sample Size Used for the Test Set and the Data Provenance
- Test Set Sample Size: The document implies the use of the "sequestered HeartFlowNXT dataset" for clinical validation of FFRct v2.0. While the exact number of patients or vessels in this specific sequestered dataset is not explicitly stated in this document, the original HEARTFLOW NXT study recruited 633 patients undergoing standard clinical care across 24 sites in the US, Europe, and Asia. It's reasonable to infer that the sequestered dataset for v2.0 validation was derived from this larger study.
- Data Provenance: The data was collected as part of the HeartFlowNXT study, which was a prospective, multicenter, non-randomized study. The participating sites included locations in the US, Europe, and Asia.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and the Qualifications of Those Experts
The ground truth for the test set was established using invasive Fractional Flow Reserve (FFR). Invasive FFR is a direct physiological measurement, and therefore, it does not typically involve interpretation by a panel of experts in the same way imaging ground truths sometimes do. The interpretation of FFR values (e.g., ≤ 0.80 indicating disease) is based on established clinical guidelines and not subjective expert consensus. The document does not specify the qualifications of the individuals who performed the invasive FFR procedures.
4. Adjudication Method for the Test Set
The ground truth was invasive FFR, which is an objective measurement. Therefore, no adjudication method (like 2+1 or 3+1 consensus) was needed for the ground truth itself.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
Yes, a form of comparative effectiveness study was done. The document states:
"Per-subject FFRct specificity compared to site-read cCTA demonstrated superior diagnostic ability (p 50% stenosis severity for site-read cCTA."
-
Effect Size (AI vs. without AI assistance):
For "FFRCT ≤ 0.80":- Diagnostic Accuracy: 80.0% (74.4%-84.6%)
- Sensitivity: 87.8% (78.5%-93.5%)
- Specificity: 76.4% (69.3%-82.3%)
For "SITE-READ CCTA > 50%":
- Diagnostic Accuracy: 51.9% (45.5%-58.2%)
- Sensitivity: 93.2% (85.1%-97.1%)
- Specificity: 32.9% (26.1%-40.5%)
Compared to site-read cCTA, FFRct demonstrates:
- Significantly higher diagnostic accuracy (80.0% vs. 51.9%).
- Much higher specificity (76.4% vs. 32.9%), indicating a substantial reduction in false positives.
- Slightly lower sensitivity (87.8% vs. 93.2%), but the overall diagnostic ability is noted as superior for FFRct (p
§ 870.1415 Coronary vascular physiologic simulation software device.
(a)
Identification. A coronary vascular physiologic simulation software device is a prescription device that provides simulated functional assessment of blood flow in the coronary vascular system using data extracted from medical device imaging to solve algorithms and yield simulated metrics of physiologic information (e.g., blood flow, coronary flow reserve, fractional flow reserve, myocardial perfusion). A coronary vascular physiologic simulation software device is intended to generate results for use and review by a qualified clinician.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Adequate software verification and validation based on comprehensive hazard analysis, with identification of appropriate mitigations, must be performed, including:
(i) Full characterization of the technical parameters of the software, including:
(A) Any proprietary algorithm(s) used to model the vascular anatomy; and
(B) Adequate description of the expected impact of all applicable image acquisition hardware features and characteristics on performance and any associated minimum specifications;
(ii) Adequate consideration of privacy and security issues in the system design; and
(iii) Adequate mitigation of the impact of failure of any subsystem components (
e.g., signal detection and analysis, data storage, system communications and cybersecurity) with respect to incorrect patient reports and operator failures.(2) Adequate non-clinical performance testing must be provided to demonstrate the validity of computational modeling methods for flow measurement; and
(3) Clinical data supporting the proposed intended use must be provided, including the following:
(i) Output measure(s) must be compared to a clinically acceptable method and must adequately represent the simulated measure(s) the device provides in an accurate and reproducible manner;
(ii) Clinical utility of the device measurement accuracy must be demonstrated by comparison to that of other available diagnostic tests (
e.g., from literature analysis);(iii) Statistical performance of the device within clinical risk strata (
e.g., age, relevant comorbidities, disease stability) must be reported;(iv) The dataset must be adequately representative of the intended use population for the device (
e.g., patients, range of vessel sizes, imaging device models). Any selection criteria or limitations of the samples must be fully described and justified;(v) Statistical methods must consider the predefined endpoints:
(A) Estimates of probabilities of incorrect results must be provided for each endpoint,
(B) Where multiple samples from the same patient are used, statistical analysis must not assume statistical independence without adequate justification, and
(C) The report must provide appropriate confidence intervals for each performance metric;
(vi) Sensitivity and specificity must be characterized across the range of available measurements;
(vii) Agreement of the simulated measure(s) with clinically acceptable measure(s) must be assessed across the full range of measurements;
(viii) Comparison of the measurement performance must be provided across the range of intended image acquisition hardware; and
(ix) If the device uses a cutoff threshold or operates across a spectrum of disease, it must be established prior to validation, and it must be justified as to how it was determined and clinically validated;
(4) Adequate validation must be performed and controls implemented to characterize and ensure consistency (
i.e., repeatability and reproducibility) of measurement outputs:(i) Acceptable incoming image quality control measures and the resulting image rejection rate for the clinical data must be specified, and
(ii) Data must be provided within the clinical validation study or using equivalent datasets demonstrating the consistency (
i.e., repeatability and reproducibility) of the output that is representative of the range of data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment;(A) Testing must be performed using multiple operators meeting planned qualification criteria and using the procedure that will be implemented in the production use of the device, and
(B) The factors (
e.g., medical imaging dataset, operator) must be identified regarding which were held constant and which were varied during the evaluation, and a description must be provided for the computations and statistical analyses used to evaluate the data;(5) Human factors evaluation and validation must be provided to demonstrate adequate performance of the user interface to allow for users to accurately measure intended parameters, particularly where parameter settings that have impact on measurements require significant user intervention; and
(6) Device labeling must be provided that adequately describes the following:
(i) The device's intended use, including the type of imaging data used, what the device measures and outputs to the user, whether the measure is qualitative or quantitative, the clinical indications for which it is to be used, and the specific population for which the device use is intended;
(ii) Appropriate warnings specifying the intended patient population, identifying anatomy and image acquisition factors that may impact measurement results, and providing cautionary guidance for interpretation of the provided measurements;
(iii) Key assumptions made in the calculation and determination of simulated measurements;
(iv) The measurement performance of the device for all presented parameters, with appropriate confidence intervals, and the supporting evidence for this performance. Per-vessel clinical performance, including where applicable localized performance according to vessel and segment, must be included as well as a characterization of the measurement error across the expected range of measurement for key parameters based on the clinical data;
(v) A detailed description of the patients studied in the clinical validation (
e.g., age, gender, race or ethnicity, clinical stability, current treatment regimen) as well as procedural details of the clinical study (e.g., scanner representation, calcium scores, use of beta-blockers or nitrates); and(vi) Where significant human interface is necessary for accurate analysis, adequately detailed description of the analysis procedure using the device and any data features that could affect accuracy of results.