(489 days)
FEops HEARTguide is indicated for patient-specific simulation of transcatheter left atrial appendage occlusion (LAAO) device implantation during procedural planning.
The software performs computer simulation to predict implant frame deformation to support the evaluation for LAAO device size and placement.
FEops HEARTguide is intended to be used by qualified clinicians in conjunction with the simulated device instructions-for-use, the patient's clinical history, symptoms, and other preprocedural evaluations, as well as the clinician's professional judgment.
FEops HEARTguide is not intended to replace the simulated device's instructions for use for final LAAO device selection and placement.
FEops HEARTguide is prescription use only.
FEops HEARTguide is a computer simulation device which provides a prediction of implant frame deformation (device-tissue interaction) post transcatheter LAAO device implantation. The device performs simulation by combining a predefined device model with a patient-specific model of the patient anatomy (Figure 1). The simulation results are intended to be used by qualified clinicians as a pre-procedural planning adjunct for LAAO implantation.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Quantitative Evaluation: Maximum allowed difference in percentage ((predicted Dmax - observed Dmax)/observed Dmax) must be less than the predetermined performance goal of ±15%. | Met: The mean difference was -1.9%, and the limits of agreement (95% CI) were 7.4% and -11.2%. Since the 95% CI of the agreement limits were within ±15%, the quantitative endpoint was met. |
| Qualitative Evaluation: More than 75% of the verdicts should be "similar" or "acceptable" when 3 cardiology experts rated the similarity between the visualization of the simulated deployed device in the anatomy versus the geometry reconstructed from the postoperative CT images. | Met: Overall, 90.6% (169/180) of the grades from the experts were "acceptable" or better, thus meeting the qualitative performance goal. |
Study Details for Clinical Accuracy Validation
1. Sample Size and Data Provenance
- Test Set Sample Size: 60 retrospective Watchman left atrial appendage occlusion (LAAO) cases.
- Data Provenance: Retrospective, from 5 centers. The specific country of origin is not explicitly stated, but the sponsor is based in Belgium.
2. Number of Experts and Qualifications for Ground Truth
- Number of Experts: 3 cardiology experts.
- Qualifications: Referred to as "cardiology experts." No further specific details on years of experience or board certification are provided in the text.
3. Adjudication Method
- Adjudication Method: Not explicitly detailed as 2+1 or 3+1. The text states that "3 cardiology experts rated the similarity." It implies a consensus or majority rule for the qualitative endpoint, as 90.6% of the grades (plural) were "acceptable" or better. It doesn't specify if discrepancies were resolved.
4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- MRMC Study Done? No, a traditional MRMC comparative effectiveness study where human readers' performance with and without AI assistance is compared was not reported. The study focused on the accuracy of the device simulation's output compared to observed data and expert qualitative assessment of that output.
- Effect Size of Human Improvement (if applicable): Not applicable, as this type of MRMC study was not performed.
5. Standalone (Algorithm Only) Performance
- Standalone Performance: Yes, the study evaluates the performance of the FEops HEARTguide software in predicting device deformation without direct human-in-the-loop assistance during the simulation process. The experts evaluate the output of the simulation.
6. Type of Ground Truth Used
- Quantitative Ground Truth: Actual Watchman deformation observed on post-operative cardiac CT, measured by the maximum device diameter (Dmax).
- Qualitative Ground Truth: Geometry reconstructed from post-operative CT images, against which the simulated deployed device visualization was compared by cardiology experts.
7. Training Set Sample Size
- The text does not explicitly state the sample size for the training set. It mentions training, but not specific numbers for a "training set" in the context of a machine learning model. The "Model Development" section describes the creation of the computational models and material validation, but does not detail a separate training set for an AI/ML algorithm in the typical sense. It implies the model was developed based on CAD files and expansion tests from the manufacturer, and pre/post-operative CT images for material validation, but these "datasets" were not re-used in the clinical validation, suggesting they might have been part of development/training.
8. How Ground Truth for Training Set was Established
- As the training set size is not explicitly stated in the context of an AI/ML algorithm, the method for establishing its ground truth is also not detailed.
- For material validation (which could be considered analogous to part of model development/training), the text states: "Material properties validation of the LAA soft tissue used in the patient-specific simulations. This was achieved using datasets consisting of pre- and post-operative CT images." This implies the "ground truth" for material properties came from the observed changes between pre- and post-operative CT images. The "Implant Device Model" validation used "expansion tests received from the manufacturer" to validate the material model based on CAD files.
{0}------------------------------------------------
DE NOVO CLASSIFICATION REQUEST FOR
FEops HEARTguide
REGULATORY INFORMATION
FDA identifies this generic type of device as:
Interventional cardiovascular implant simulation software device. An interventional cardiovascular implant simulation software device is a prescription device that provides a computer simulation of an interventional cardiovascular implant device inside a patient's cardiovascular anatomy. It performs computational modeling to predict the interaction of the interventional cardiovascular implant device with the patient-specific anatomical environment.
NEW REGULATION NUMBER: 21 CFR 870.1405
CLASSIFICATION: Class II
PRODUCT CODE: QQI
BACKGROUND
DEVICE NAME: FEops HEARTguide
SUBMISSION NUMBER: DEN200030
DATE DE NOVO RECEIVED: May 7, 2020
SPONSOR INFORMATION:
FEops NV Technologiepark 122 B-9052 Gent Belgium
INDICATIONS FOR USE
FEops HEARTguide is indicated for patient-specific simulation of transcatheter left atrial appendage occlusion (LAAO) device implantation during procedural planning.
The software performs computer simulation to predict implant frame deformation to support the evaluation for LAAO device size and placement.
FEops HEARTguide is intended to be used by qualified clinicians in conjunction with the simulated device instructions-for-use, the patient's clinical history, symptoms, and other preprocedural evaluations, as well as the clinician's professional judgment.
{1}------------------------------------------------
FEops HEARTguide is not intended to replace the simulated device's instructions for use for final LAAO device selection and placement.
FEops HEARTguide is prescription use only.
LIMITATIONS
The sale, distribution, and use of the FEops HEARTguide are restricted to prescription use in accordance with 21 CFR 801.109.
The device is not intended to be used as a stand-alone diagnostic device.
PLEASE REFER TO THE LABELING FOR A COMPLETE LIST OF WARNINGS, PRECAUTIONS AND CONTRAINDICATIONS.
DEVICE DESCRIPTION
FEops HEARTguide is a computer simulation device which provides a prediction of implant frame deformation (device-tissue interaction) post transcatheter LAAO device implantation. The device performs simulation by combining a predefined device model with a patient-specific model of the patient anatomy (Figure 1). The simulation results are intended to be used by qualified clinicians as a pre-procedural planning adjunct for LAAO implantation.
Image /page/1/Figure/8 description: The image shows a diagram of a computer simulation process for predicting the morphology of a device post-LAAO (Left Atrial Appendage Occlusion). The process starts with a patient-specific computer model of the left atrium based on routine pre-operative CT scans, incorporating the mechanical behavior of different tissue regions. This is combined with a 3D computer model of the LAA occluder device, taking into account its mechanical behavior. The final step is the prediction of device morphology post-LAAO using computer simulations that account for tissue-device interaction.
Figure 1: Illustration of FEops HEARTguide Working Principle
MODEL DEVELOPMENT
FEops HEARTguide conducts LAAO device implantation via Finite Element Analysis (FEA). The following steps were performed to develop the device and the patient models:
{2}------------------------------------------------
- . Computational Model Design - Implant Device Model: Generation of a finite element model of the Boston Scientific Watchman LAAO device. After the geometry was established using CAD files received from the manufacturer, the material model was validated based on expansion tests received from the manufacturer.
- . Computational Model Design - LAAO FEA - LAA Material Validation: Material properties validation of the LAA soft tissue used in the patient-specific simulations. This was achieved using datasets consisting of pre- and post-operative CT images. The datasets used for the validation were not re-used in subsequent clinical validation.
- . Computational Model Design - LAAO FEA Patient Model: Generation of the patientspecific finite element models, these models are used to simulate the deployment of the LAAO device in patient-specific anatomic models.
SUMMARY OF NONCLINICAL/BENCH STUDIES
SOFTWARE
Software documentation was provided in accordance with the FDA Guidance Document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" (issued May 11, 2005) for a Moderate Level of Concern. A Moderate Level of Concern is deemed appropriate as prior to mitigation of hazards, a malfunction of, or a latent design flaw in, the Software Device could lead to erroneous treatment planning by selecting inappropriate planning parameters (e.g., device size, position). This could result in additional peri-procedural manipulations or require additional treatment postintervention. Both can be considered to result in a minor injury.
Software verification and validation activities were conducted to confirm that the device software met the defined software specifications.
Cybersecurity information on both the web-based components and the Simulation Application were provided in accordance with the FDA Guidance Document "Content of Premarket Submissions for Management of Cybersecurity in Medical Devices" (issued October 2, 2014).
Clinical Accuracy Validation Study was also conducted to demonstrate that the device software met the software specifications. This study was performed using retrospective Watchman left atrial appendage occlusion (LAAO) cases. Additional details are provided in the "Summary of Clinical Information" Section.
SIMULATION MODEL CREDIBILITY ASSESSMENT
The sponsor conducted a simulation model credibility assessment following recognized ASME V&V 40-2018: Assessing Credibility of Computational Modeling Through Verification and Validation: Application to Medical Devices.
{3}------------------------------------------------
Based on the assessment, all conducted activities have been considered adequate and sufficient compared to the associated Context of Use (COU) and model risk. Table 1 below summarizes the rigor selected for each credibility activity and the level achieved during the analysis. All selected credibility levels are at least MEDIUM, which is consistent with the Medium model risk of this simulation model.
| Activity | Credibility factor | Level of Rigor | Achievedcredibility level | ||
|---|---|---|---|---|---|
| Selected | Maximum | ||||
| Code | SQA (6.1.1) | (b) (4) | |||
| NCV (6.1.2) | |||||
| Verification | Calculation | Discretization error (6.2.1) | |||
| Numerical solver error (6.2.2) | |||||
| Use error (6.2.3) | |||||
| Computationalmodel | Model form (7.1.1) | ||||
| Model inputs (7.1.2) | |||||
| Validation | Comparator | Test samples (7.2.1) | |||
| Test conditions (7.2.2) | |||||
| Assessment | Equivalency of input parameters (7.3.1) | ||||
| Output comparison (7.3.2) | |||||
| Applicability | Relevance of the quantities of interest (8.1) | ||||
| Relevance of the validation activities to theCOU (8.2) |
Table 1: Summary of the targeted and achieved credibility level for V&V activities
REPEATABILITY AND REPRODUCIBILITY
A repeatability and reproducibility study was performed to examine the repeatability and reproducibility of the generation of simulation results by qualified sponsor analysts. In this study, both experienced and minimally experienced analysts processed a selected set of images that included a range of device sizes and patient anatomy. The study ran all the steps involved in the FEops HEARTguide simulation process. The acceptance criteria was defined as "good agreement", measured by an intra-class coefficient (ICC) of being ≥ 0.75. The obtained results showed acceptable agreement among the as-generated simulation results. The repeatability and reproducibility of the process has been demonstrated.
SUMMARY OF CLINICAL INFORMATION
The sponsor provided a clinical accuracy validation study protocol and results to support the safety and effectiveness of the device. The validation study included 60 retrospective Watchman left atrial appendage occlusion (LAAO) cases from 5 centers to encompass the full range of left atrial appendage morphologies and simulated device sizes. Blinded HEART guide simulation was performed based on pre-operative CT images and a description of the device implantation
{4}------------------------------------------------
location. The simulation results were compared to the actual Watchman deformation observed on the post-operative cardiac CT quantitatively and qualitatively.
For quantitative evaluation, agreement between the predicted and the actual Watchman deformation as measured by the maximum device diameter (Dmax) was determined. To meet this endpoint, the maximum allowed difference in percentage ((predicted Dmax - observed Dmax)/observed Dmax) must be less than the predetermined performance goal of ±15%.
For qualitative evaluation, 3 cardiology experts rated the similarity between the visualization of the simulated deployed device in the anatomy versus the geometry reconstructed from the postoperative CT images. For this endpoint to be met, more than 75% of the verdicts should be "similar" or "acceptable."
Figure 2 presents the Bland-Altman plot that describes the agreement between the predicted and the observed measurements. The mean difference was -1.9%, and the limits of agreement were 7.4% (95% CI: 80%, "11%) and -11.2% (95% CI: (6) (41%). Since the 95% CI of the agreement limits were within ± 15%, the quantitative endpoint was met.
Image /page/4/Figure/4 description: This image is a Bland-Altman plot comparing two methods of measuring Dmax, SIM_Dmax and MSCT_Dmax. The y-axis represents the difference between the two methods as a percentage of the mean, while the x-axis represents the mean of the two methods. The plot shows the mean difference (-1.9), as well as the upper (+1.96 SD = 7.4) and lower (-1.96 SD = -11.2) limits of agreement.
Figure 2: Bland-Altman Plot for Predicted Dmax vs. Observed Dmax
Independent cardiology experts rated the paired images (simulated output and the corresponding post-operative CT geometry) as "similar" in most cases. Overall, 90.6% (169/180) of the grades were "acceptable" or better, and the qualitative performance goal was also met.
The results of the validation study support that the device's accuracy is clinically acceptable for its intended use.
Pediatric Extrapolation
In this De Novo request, existing clinical data were not leveraged to support the use of the device in a pediatric patient population.
{5}------------------------------------------------
SUMMARY OF HUMAN FACTORS/USABILITY
Usability testing scope and sample size were agreed upon between the FDA and FEops.
- . Case Analyst Summative Evaluation: Seven (7) qualified case analysts participated in the summative evaluation. The summative evaluation concluded the design of the Simulation Application (production version), in combination with the current training and qualification program, process setup and case file, is free from unacceptable use errors.
- User Interface Summative Evaluation: No summative evaluation was needed on the User . Interface intended for the professional users (clinicians) due to the simplicity of the User Interface, the limited criticality of the tasks, and the extensive education level of the endusers.
- . Labeling Formative Evaluation: A formative evaluation of the labeling was conducted with four (4) representative users. The formative evaluation concluded the current labeling is adequate and is an effective risk control measure as planned in the risk analysis.
LABELING
The labeling of the device satisfies the labeling special controls listed below:
- . Warnings that identify anatomy and image acquisition factors that may impact simulation results and provide cautionary guidance for interpretation of the provided simulation results:
- . Device simulation inputs and outputs, and key assumptions made in the simulation and determination of simulated outputs; and
- The computational modeling performance of the device for presented simulation outputs, ● and the supporting evidence for this performance.
RISKS TO HEALTH
The table below identifies the risks to health that may be associated with use of an interventional cardiovascular implant simulation software device and the measures necessary to mitigate these risks.
{6}------------------------------------------------
| Identified Risks to Health | Mitigation Measures |
|---|---|
| Inaccurate simulation results leading toselection of suboptimal treatment plan,leading to prolonged procedure time and/orpatient injury | Software verification, validation, and hazardanalysisComputational modeling verification andvalidationPerformance validation with clinical dataHuman factors testingLabeling |
| Delayed delivery of results due to softwarefailure or use error, leading to delay oftreatment | Software verification, validation and hazardanalysisHuman factors testingLabeling |
| Failure to properly interpret device resultsleading to selection of suboptimal treatmentplan, leading to prolonged procedure timeand/or patient injury | Human factors testingLabeling |
SPECIAL CONTROLS
In combination with the general controls of the FD&C Act, the interventional cardiovascular implant simulation software device is subject to the following special controls:
-
- Software verification, validation, and hazard analysis, with identification of appropriate mitigations, must be performed, including a full verification and validation of the software according to the pre-defined software specifications.
-
- Computational modeling verification and validation activities must be performed to establish the predictive capability of the device for its indications for use
-
- Performance validation testing must be provided to demonstrate accuracy and clinical relevance of the modeling methods for the intended implantations, including the following:
- i. Computational modeling results must be compared to clinical data supporting the indications for use to demonstrate accuracy and clinical meaningfulness of the simulations;
- ii. Agreement between computational modeling results and clinical data must be assessed and demonstrated across the full intended operating range (e.g., full range of patient population, implant device sizes and patient anatomic morphologies). Any selection criteria or limitations of the samples must be described and justified; and
- iii. Endpoints (e.g., performance goals) and sample sizes established must be justified as to how they were determined and why they are clinically meaningful;
{7}------------------------------------------------
- iv. Validation must be performed and controls implemented to characterize and ensure consistency (i.e., repeatability and reproducibility) of modeling outputs:
- a. Testing must be performed using multiple qualified operators and using the procedure that will be implemented under anticipated conditions of use; and
- b. The factors (e.g., medical imaging dataset, operator) must be identified regarding which were held constant and which were varied during the evaluation, and a description must be provided for the computations and statistical analyses used to evaluate the data.
-
- Human factors evaluation must be performed to evaluate the ability of the user interface and labeling to allow for intended users to correctly use the device and interpret the provided information.
-
- Device labeling must be provided that describes the following:
- i. Warnings that identify anatomy and image acquisition factors that may impact simulation results and provide cautionary guidance for interpretation of the provided simulation results;
- ii. Device simulation inputs and outputs, and key assumptions made in the simulation and determination of simulated outputs; and
- iii. The computational modeling performance of the device for presented simulation outputs, and the supporting evidence for this performance.
BENEFIT-RISK DETERMINATION
The subject device is intended as an adjunct for interventional LAAO implantation preprocedural planning. As stated in the proposed IFU, the final selection of device size and position still requires confirmation by intraoperative imaging as per the simulated LAAO device labeling. Inaccurate simulation results provided by the subject device may lead to selection of suboptimal treatment plan and subsequently procedure prolongation. However, the risk of harm is minimal since the LAAO device would still be implanted per its approved instructions for use (IFU). Furthermore, the device has been validated to have clinically acceptable accuracy. Another risk can be delayed delivery of simulation results leading to delay of treatment. However, the risk of harm is still minimal as the clinician would not rely solely on the simulation results of the subject device to conduct LAAO implantation procedural planning. .
The subject device provides simulation results to better inform the Watchman LAAO device sizing and implant positioning during procedural planning. For the proposed intended use measurements, the provided clinical data supports that the simulation produces accurate expected device behavior for a given device size and position. While there is no objective evidence that having such information available will lead to improved clinical outcomes, users (i.e., clinician implanters) could gain better understanding about device-LAA interaction during preprocedural
{8}------------------------------------------------
planning. It is reasonable to expect that the additional insights gained from the simulation results will improve procedural efficiency. Optimal implant device sizing and positioning continue to be challenging for transcatheter LAA closure. There is a probable benefit in having the simulation results to map out the implant approach prior to the procedure.
Patient Perspectives
This submission did not include specific information on patient perspectives for this device.
Benefit/Risk Conclusion
In conclusion, given the available information above, for the following indication statement:
FEops HEARTguide is indicated for patient-specific simulation of transcatheter left atrial appendage occlusion (LAAO) device implantation during procedural planning.
The software performs computer simulation to predict implant frame deformation to support the evaluation for LAAO device size and placement. FEops HEARTguide is intended to be used by qualified clinicians in conjunction with the simulated device instructions-for-use, the patient's clinical history, symptoms, and other preprocedural evaluations, as well as the clinician's professional judgment. FEops HEARTguide is not intended to replace the simulated device's instructions for use for final LAAO device selection and placement.
FEops HEARTguide is prescription use only.
The probable benefits outweigh the probable risks for the FEops HEARTguide. The device provides benefits, and the risks can be mitigated by the use of general controls and the identified special controls.
CONCLUSION
The De Novo request for the FEops HEARTguide is granted and the device is classified as follows:
Product Code: OOI Device Type: Interventional cardiovascular implant simulation software device Regulation Number: 21 CFR 870.1405 Class: II
§ 870.1405 Interventional cardiovascular implant simulation software device.
(a)
Identification. An interventional cardiovascular implant simulation software device is a prescription device that provides a computer simulation of an interventional cardiovascular implant device inside a patient's cardiovascular anatomy. It performs computational modeling to predict the interaction of the interventional cardiovascular implant device with the patient-specific anatomical environment.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software verification, validation, and hazard analysis, with identification of appropriate mitigations, must be performed, including a full verification and validation of the software according to the predefined software specifications.
(2) Computational modeling verification and validation activities must be performed to establish the predictive capability of the device for its indications for use.
(3) Performance validation testing must be provided to demonstrate the accuracy and clinical relevance of the modeling methods for the intended implantation simulations, including the following:
(i) Computational modeling results must be compared to clinical data supporting the indications for use to demonstrate accuracy and clinical meaningfulness of the simulations;
(ii) Agreement between computational modeling results and clinical data must be assessed and demonstrated across the full intended operating range (
e.g., full range of patient population, implant device sizes and patient anatomic morphologies). Any selection criteria or limitations of the samples must be described and justified;(iii) Endpoints (
e.g., performance goals) and sample sizes established must be justified as to how they were determined and why they are clinically meaningful; and(iv) Validation must be performed and controls implemented to characterize and ensure consistency (
i.e., repeatability and reproducibility) of modeling outputs:(A) Testing must be performed using multiple qualified operators and using the procedure that will be implemented under anticipated conditions of use; and
(B) The factors (
e.g., medical imaging dataset, operator) must be identified regarding which were held constant and which were varied during the evaluation, and a description must be provided for the computations and statistical analyses used to evaluate the data.(4) Human factors evaluation must be performed to evaluate the ability of the user interface and labeling to allow for intended users to correctly use the device and interpret the provided information.
(5) Device labeling must be provided that describes the following:
(i) Warnings that identify anatomy and image acquisition factors that may impact simulation results and provide cautionary guidance for interpretation of the provided simulation results;
(ii) Device simulation inputs and outputs, and key assumptions made in the simulation and determination of simulated outputs; and
(iii) The computational modeling performance of the device for presented simulation outputs, and the supporting evidence for this performance.