(133 days)
CathWorks FFRangio™ is a software device for the clinical quantitative and qualitative analysis of previously acquired angiography DICOM data for patients with coronary artery disease. It provides FFRangio™, a mathematically derived quantity, computed from simulated blood flow information obtained from a 3D computer model, generated from coronary angiography images. FFRangio™ analysis is intended to support the functional evaluation of coronary artery disease. The results of this analysis are provided as a supportive aid for qualified clinicians in the evaluation and assessment of coronary arteries physiology. The results of CathWorks FFRangio™ are intended to be used by qualified clinicians in conjunction with the patient's clinical history, symptoms, and other diagnostic tests, as well as the clinician's professional evaluation.
The FFRangio™ system is a computer system installed on a mobile cart that is to be located inside the catheterization room or in an adjacent technical/viewing area. The cart holds the computer processing unit, user interface control station (LCD screen and keyboard/mouse), medical isolation transformer, and network isolator. Operation requires only connections to mains and a DICOM communication port. The system supports optional visual media output to the Cath Lab main displays, so the system GUI may be observed on both the system's LCD display and on the Cath Lab's main display (boom monitor).
FFRancio™ uses standard angiographic images (angiograms) that are retrieved from the X-ray Imaging System (C-arm) in DICOM format. The user selects the images and, following the system prompts, marks key features on the images including the target lesion, ostium location, main vessel, target vessel, and its side branches. The system then matches the corresponding vessels among the projections and generates a 3D computer model of the vessels. The 3D model is used for blood flow analysis and determination of the FFRangio.
Here's a breakdown of the acceptance criteria and study details for the CathWorks FFRangio™ system, based on the provided text:
Acceptance Criteria and Device Performance
| Criteria | Performance Goal / Criteria | Reported Device Performance (Lower 95% CI) |
|---|---|---|
| Sensitivity | 70% | 93.5% (87.8%) |
| Specificity | 75% | 91.2% (86.0%) |
| Correlation Coefficient (R) | 0.65 | 0.80 |
| Intercept | -0.20 - 0.20 | 0.04 (0.03 - 0.05) |
| Slope | 0.80 - 1.20 | 0.93 (0.92 - 0.95) |
Notes:
- The reported performance for Sensitivity and Specificity includes the lower one-sided 95% confidence interval, which was used to determine if the device met the pre-specified target goals. Both were significantly above the targets.
- The correlation coefficient, intercept, and slope were secondary endpoints, and their direct "performance goals" are stated along with the achieved values.
Study Information
The primary study referenced is the FAST-FFR Trial.
2. Sample Size and Data Provenance:
- Test Set (Efficacy Analysis): 301 adult subjects and 319 coronary lesions.
- Study Enrollment: 382 subjects (30 roll-in, 352 study subjects).
- Data Provenance: Prospective, multicenter, international trial conducted at 10 sites in 5 countries: United States, Europe, and Israel.
- 124 subjects (32.5%) were enrolled at sites in the United States.
3. Number and Qualifications of Experts for Test Set Ground Truth:
The document does not explicitly state the number of experts or their specific qualifications (e.g., years of experience) for establishing the ground truth of the invasive FFR. However, it does state:
- "All invasive FFR data were reviewed post-hoc by an independent FFR physiology core laboratory."
4. Adjudication Method for Test Set:
The document does not explicitly describe an adjudication method like 2+1 or 3+1 for the test set's ground truth by the independent FFR physiology core laboratory. It simply states that the data was "reviewed."
Furthermore, it states for the FFRangio™ data:
- "all FFRangio™ data were reviewed post-hoc by a core laboratory at CathWorks."
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:
- No, a multi-reader multi-case (MRMC) comparative effectiveness study evaluating human readers with and without AI assistance was not explicitly mentioned or presented in the provided text. The study primarily focused on the standalone diagnostic accuracy of the FFRangio™ system compared to invasive FFR.
6. Standalone (Algorithm Only) Performance:
- Yes, a standalone performance study was conducted. The FAST-FFR trial specifically evaluated the FFRangio™ system's accuracy as compared to invasive wire FFR, with on-site hospital users blinded to the invasive FFR and the FFRangio™ not being used for diagnostic or clinical decisions. This demonstrates the algorithm's performance independent of real-time human-in-the-loop influence on clinical decisions.
7. Type of Ground Truth Used:
- Invasive Wire FFR: The ground truth for the test set was obtained from "invasive wire FFR" measured using a coronary pressure wire and hyperemic stimulus. This is considered the gold standard for FFR measurement.
8. Sample Size for the Training Set:
- The document does not explicitly state the sample size for the training set used to develop the FFRangio™ algorithm. It mentions two clinical studies:
- A two-phase validation study using an "earlier version of the operator interface but the same image processing and computation algorithms as the final device." This study analyzed "203 coronary lesions" and demonstrated sensitivity, specificity, and diagnostic accuracy. While this study helped validate the algorithms, it's not explicitly labeled as the training set.
- The FAST-FFR trial (the pivotal study), which serves as the test set.
9. How the Ground Truth for the Training Set Was Established:
- Similar to the training set size, the document does not explicitly detail how the ground truth for an independent training set was established.
- The "two-phase validation study" mentioned in point 8, which used an earlier version of the algorithm, compared FFRangio™ to "invasive FFR obtained from patients already scheduled for coronary assessment in the cath lab." This suggests invasive FFR was used as ground truth for that prior validation, which likely informed the development and refinement of the final algorithm's "image processing and computation algorithms." However, a specific training set and its ground truth acquisition are not explicitly isolated in the provided text.
§ 870.1415 Coronary vascular physiologic simulation software device.
(a)
Identification. A coronary vascular physiologic simulation software device is a prescription device that provides simulated functional assessment of blood flow in the coronary vascular system using data extracted from medical device imaging to solve algorithms and yield simulated metrics of physiologic information (e.g., blood flow, coronary flow reserve, fractional flow reserve, myocardial perfusion). A coronary vascular physiologic simulation software device is intended to generate results for use and review by a qualified clinician.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Adequate software verification and validation based on comprehensive hazard analysis, with identification of appropriate mitigations, must be performed, including:
(i) Full characterization of the technical parameters of the software, including:
(A) Any proprietary algorithm(s) used to model the vascular anatomy; and
(B) Adequate description of the expected impact of all applicable image acquisition hardware features and characteristics on performance and any associated minimum specifications;
(ii) Adequate consideration of privacy and security issues in the system design; and
(iii) Adequate mitigation of the impact of failure of any subsystem components (
e.g., signal detection and analysis, data storage, system communications and cybersecurity) with respect to incorrect patient reports and operator failures.(2) Adequate non-clinical performance testing must be provided to demonstrate the validity of computational modeling methods for flow measurement; and
(3) Clinical data supporting the proposed intended use must be provided, including the following:
(i) Output measure(s) must be compared to a clinically acceptable method and must adequately represent the simulated measure(s) the device provides in an accurate and reproducible manner;
(ii) Clinical utility of the device measurement accuracy must be demonstrated by comparison to that of other available diagnostic tests (
e.g., from literature analysis);(iii) Statistical performance of the device within clinical risk strata (
e.g., age, relevant comorbidities, disease stability) must be reported;(iv) The dataset must be adequately representative of the intended use population for the device (
e.g., patients, range of vessel sizes, imaging device models). Any selection criteria or limitations of the samples must be fully described and justified;(v) Statistical methods must consider the predefined endpoints:
(A) Estimates of probabilities of incorrect results must be provided for each endpoint,
(B) Where multiple samples from the same patient are used, statistical analysis must not assume statistical independence without adequate justification, and
(C) The report must provide appropriate confidence intervals for each performance metric;
(vi) Sensitivity and specificity must be characterized across the range of available measurements;
(vii) Agreement of the simulated measure(s) with clinically acceptable measure(s) must be assessed across the full range of measurements;
(viii) Comparison of the measurement performance must be provided across the range of intended image acquisition hardware; and
(ix) If the device uses a cutoff threshold or operates across a spectrum of disease, it must be established prior to validation, and it must be justified as to how it was determined and clinically validated;
(4) Adequate validation must be performed and controls implemented to characterize and ensure consistency (
i.e., repeatability and reproducibility) of measurement outputs:(i) Acceptable incoming image quality control measures and the resulting image rejection rate for the clinical data must be specified, and
(ii) Data must be provided within the clinical validation study or using equivalent datasets demonstrating the consistency (
i.e., repeatability and reproducibility) of the output that is representative of the range of data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment;(A) Testing must be performed using multiple operators meeting planned qualification criteria and using the procedure that will be implemented in the production use of the device, and
(B) The factors (
e.g., medical imaging dataset, operator) must be identified regarding which were held constant and which were varied during the evaluation, and a description must be provided for the computations and statistical analyses used to evaluate the data;(5) Human factors evaluation and validation must be provided to demonstrate adequate performance of the user interface to allow for users to accurately measure intended parameters, particularly where parameter settings that have impact on measurements require significant user intervention; and
(6) Device labeling must be provided that adequately describes the following:
(i) The device's intended use, including the type of imaging data used, what the device measures and outputs to the user, whether the measure is qualitative or quantitative, the clinical indications for which it is to be used, and the specific population for which the device use is intended;
(ii) Appropriate warnings specifying the intended patient population, identifying anatomy and image acquisition factors that may impact measurement results, and providing cautionary guidance for interpretation of the provided measurements;
(iii) Key assumptions made in the calculation and determination of simulated measurements;
(iv) The measurement performance of the device for all presented parameters, with appropriate confidence intervals, and the supporting evidence for this performance. Per-vessel clinical performance, including where applicable localized performance according to vessel and segment, must be included as well as a characterization of the measurement error across the expected range of measurement for key parameters based on the clinical data;
(v) A detailed description of the patients studied in the clinical validation (
e.g., age, gender, race or ethnicity, clinical stability, current treatment regimen) as well as procedural details of the clinical study (e.g., scanner representation, calcium scores, use of beta-blockers or nitrates); and(vi) Where significant human interface is necessary for accurate analysis, adequately detailed description of the analysis procedure using the device and any data features that could affect accuracy of results.