(133 days)
DEEPVESSEL FFR is a coronary physiological simulation software for the clinical quantitative analysis of previously acquired Computed Tomography DICOM data for clinically stable symptomatic patients with coronary artery disease. It provides DVFFR (a CT-derived FFR measurement) computed from static coronary CTA images using deep learning neural networks that encode imaging, structural, and functional characteristics of coronary arteries through learning.
DEEPVESSEL FFR analysis is intended to support the functional evaluation of coronary artery disease.
The results of the analysis are provided to support qualified clinicians to aid in the evaluation and assessment of coronary arteries. DEEPVESSEL FFR results are intended to be used by qualified clinicians in conunction with the patient's clinical history, symptoms, and other diagnostic tests, as well as the clinician's professional judgment.
DEEPVESSEL FFR is a coronary physiological simulation software for the clinical quantitative and qualitative analysis of previously acquired Computed Tomography DICOM data for clinically stable symptomatic patients with coronary artery disease. It estimates FFR values from static coronary CTA images with extracted coronary tree structures using deep learning neural networks. DEEPVESSEL FFR analysis is intended to support the functional evaluation of CAD.
The software processes these images semi-automatically, and it generates a 3D model of the coronay artery tree and computes DVFFR (CT-derived FFR) values. Qualified image analysts interact with the software by providing manual edits to the 3D coronary artery tree segmentations when needed, and oversees outputs along the processing steps. DVFFR analysis results are sent electronically to the physicians via a third-party service portal application.
DVFFR software is independent of imaging equipment, imaging protocols and equipment vendors; the clinical validation study report includes the specific imaging scanner types and imaging acquisition parameters used in the clinical validation of the product.
Here's a summary of the acceptance criteria and study details for the DEEPVESSEL FFR device, based on the provided FDA 510(k) summary:
1. Table of Acceptance Criteria and Reported Device Performance
| Metric (Per-vessel) | Acceptance Crit. (Target Rate) | Reported Performance (Estimate) | Lower Bound 95% CI | Met/Not Met |
|---|---|---|---|---|
| Sensitivity | 75% | 86.9% | 80.6% | Met |
| Specificity | 70% | 86.7% | 82.0% | Met |
Additional Reported Performance (Per-vessel):
- Accuracy: 86.8% (95% CI: 83.0%-90.4%)
- PPV: 79.4% (95% CI: 71.8%-86.2%)
- NPV: 91.9% (95% CI: 87.7%–95.6%)
Patient-level Diagnostic Performance:
- Sensitivity: 87.4% (95% CI: 79.4%-93.1%)
- Specificity: 83.7% (95% CI: 76.5%-89.4%)
- Accuracy: 85.2% (95% CI: 80.2%-89.4%)
- PPV: 79.6% (95% CI: 71.0%-86.6%)
- NPV: 90.1% (95% CI: 83.6%-94.6%)
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size: 244 patients with 311 target vessels.
- Data Provenance: Multi-national (US and EU), multi-center clinical validation study, conducted at 8 clinical sites (4 from EU and 4 from US). The study was prospective in its design to evaluate the device against an invasive reference standard, suggesting the data used for validation was gathered for this specific purpose.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications
The document does not explicitly state the number of experts or their specific qualifications for establishing the ground truth. However, the ground truth was "invasive FFR measurement." Invasive FFR procedures are typically performed and interpreted by interventional cardiologists.
4. Adjudication Method for the Test Set
The document does not explicitly state an adjudication method (e.g., 2+1, 3+1). The ground truth was established by "invasive FFR measurement," which is a direct physiological measurement, lessening the need for expert adjudication of the ground truth itself, though interpretation of the invasive FFR values might have involved standard clinical protocols.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done
No, an MRMC comparative effectiveness study that demonstrates how much human readers improve with AI vs. without AI assistance was not reported in this summary. The clinical study focused on the standalone performance of the DEEPVESSEL FFR device compared to invasive FFR. The device is described as supporting qualified clinicians and being used in conjunction with their judgment and other tests, but no study on human reader improvement with assistance was included.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
Yes, a standalone performance study was done. The reported sensitivity, specificity, accuracy, PPV, and NPV values reflect the performance of the DEEPVESSEL FFR software (DVFFR) itself in detecting ischemic conditions compared to invasive FFR. While image analysts perform semi-automatic processing and manual edits to 3D coronary artery tree segmentations, the diagnostic performance metrics provided represent the device's output.
7. The Type of Ground Truth Used
The ground truth used was invasive FFR measurement. This is considered a direct physiological measurement and a gold standard for assessing the hemodynamic significance of coronary artery stenoses.
8. The Sample Size for the Training Set
The document does not specify the sample size for the training set. It mentions that the device uses "deep learning neural networks that encode imaging, structural, and functional characteristics of coronary arteries through learning," but provides no details on the data used for this learning phase.
9. How the Ground Truth for the Training Set was Established
The document does not specify how the ground truth for the training set was established. It only mentions the use of deep learning neural networks. Typically, for such devices, the training data's ground truth would also be established by invasive FFR measurements or a similar reference standard.
§ 870.1415 Coronary vascular physiologic simulation software device.
(a)
Identification. A coronary vascular physiologic simulation software device is a prescription device that provides simulated functional assessment of blood flow in the coronary vascular system using data extracted from medical device imaging to solve algorithms and yield simulated metrics of physiologic information (e.g., blood flow, coronary flow reserve, fractional flow reserve, myocardial perfusion). A coronary vascular physiologic simulation software device is intended to generate results for use and review by a qualified clinician.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Adequate software verification and validation based on comprehensive hazard analysis, with identification of appropriate mitigations, must be performed, including:
(i) Full characterization of the technical parameters of the software, including:
(A) Any proprietary algorithm(s) used to model the vascular anatomy; and
(B) Adequate description of the expected impact of all applicable image acquisition hardware features and characteristics on performance and any associated minimum specifications;
(ii) Adequate consideration of privacy and security issues in the system design; and
(iii) Adequate mitigation of the impact of failure of any subsystem components (
e.g., signal detection and analysis, data storage, system communications and cybersecurity) with respect to incorrect patient reports and operator failures.(2) Adequate non-clinical performance testing must be provided to demonstrate the validity of computational modeling methods for flow measurement; and
(3) Clinical data supporting the proposed intended use must be provided, including the following:
(i) Output measure(s) must be compared to a clinically acceptable method and must adequately represent the simulated measure(s) the device provides in an accurate and reproducible manner;
(ii) Clinical utility of the device measurement accuracy must be demonstrated by comparison to that of other available diagnostic tests (
e.g., from literature analysis);(iii) Statistical performance of the device within clinical risk strata (
e.g., age, relevant comorbidities, disease stability) must be reported;(iv) The dataset must be adequately representative of the intended use population for the device (
e.g., patients, range of vessel sizes, imaging device models). Any selection criteria or limitations of the samples must be fully described and justified;(v) Statistical methods must consider the predefined endpoints:
(A) Estimates of probabilities of incorrect results must be provided for each endpoint,
(B) Where multiple samples from the same patient are used, statistical analysis must not assume statistical independence without adequate justification, and
(C) The report must provide appropriate confidence intervals for each performance metric;
(vi) Sensitivity and specificity must be characterized across the range of available measurements;
(vii) Agreement of the simulated measure(s) with clinically acceptable measure(s) must be assessed across the full range of measurements;
(viii) Comparison of the measurement performance must be provided across the range of intended image acquisition hardware; and
(ix) If the device uses a cutoff threshold or operates across a spectrum of disease, it must be established prior to validation, and it must be justified as to how it was determined and clinically validated;
(4) Adequate validation must be performed and controls implemented to characterize and ensure consistency (
i.e., repeatability and reproducibility) of measurement outputs:(i) Acceptable incoming image quality control measures and the resulting image rejection rate for the clinical data must be specified, and
(ii) Data must be provided within the clinical validation study or using equivalent datasets demonstrating the consistency (
i.e., repeatability and reproducibility) of the output that is representative of the range of data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment;(A) Testing must be performed using multiple operators meeting planned qualification criteria and using the procedure that will be implemented in the production use of the device, and
(B) The factors (
e.g., medical imaging dataset, operator) must be identified regarding which were held constant and which were varied during the evaluation, and a description must be provided for the computations and statistical analyses used to evaluate the data;(5) Human factors evaluation and validation must be provided to demonstrate adequate performance of the user interface to allow for users to accurately measure intended parameters, particularly where parameter settings that have impact on measurements require significant user intervention; and
(6) Device labeling must be provided that adequately describes the following:
(i) The device's intended use, including the type of imaging data used, what the device measures and outputs to the user, whether the measure is qualitative or quantitative, the clinical indications for which it is to be used, and the specific population for which the device use is intended;
(ii) Appropriate warnings specifying the intended patient population, identifying anatomy and image acquisition factors that may impact measurement results, and providing cautionary guidance for interpretation of the provided measurements;
(iii) Key assumptions made in the calculation and determination of simulated measurements;
(iv) The measurement performance of the device for all presented parameters, with appropriate confidence intervals, and the supporting evidence for this performance. Per-vessel clinical performance, including where applicable localized performance according to vessel and segment, must be included as well as a characterization of the measurement error across the expected range of measurement for key parameters based on the clinical data;
(v) A detailed description of the patients studied in the clinical validation (
e.g., age, gender, race or ethnicity, clinical stability, current treatment regimen) as well as procedural details of the clinical study (e.g., scanner representation, calcium scores, use of beta-blockers or nitrates); and(vi) Where significant human interface is necessary for accurate analysis, adequately detailed description of the analysis procedure using the device and any data features that could affect accuracy of results.