(145 days)
TruPlan enables visualization and measurement of structures of the heart and vessels for:
- · Pre-procedural planning and sizing for the left atrial appendage closure (LAAC) procedure
- · Post-procedural evaluation for the LAAC procedure
To facilitate the above, TruPlan provides general functionality such as:
- · Segmentation of cardiovascular structures
- · Visualization and image reconstruction techniques: 2D review, Volume Rendering, MPR
- · Simulation of TEE views, ICE views, and fluoroscopic rendering
- · Measurement and annotation tools
- · Reporting tools
TruPlan's intended patient population is comprised of adult patients.
The TruPlan Computed Tomography (CT) Imaging Software application ("TruPlan") is a software as a medical device that helps qualified users with image-based pre-procedural planning and post-procedural follow-up of the Left Atrial Appendage Closure (LAAC) procedure using CT data. TruPlan is designed to support the anatomical assessment of the Left Atrial Appendage (LAA) prior to and following the LAAC procedure. This includes the assessment of the LAA size, shape, and relationships with adjacent cardiac and extracardiac structures. This assessment helps the physician determine the size of a closure device needed for the LAAC procedure and evaluate LAAC device placement in a follow-up CT study. The TruPlan application is a visualization software and has basic measurement tools. The device is intended to be used as an aid to the existing standard of care and does not replace existing software applications physicians use for planning or follow-up for a LAAC procedure.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided FDA 510(k) submission for TruPlan Computed Tomography (CT) Imaging Software:
Acceptance Criteria and Device Performance Study for TruPlan CT Imaging Software
The TruPlan Computed Tomography (CT) Imaging Software by Circle Cardiovascular Imaging, Inc. underwent validation of its machine learning (ML) derived outputs to demonstrate its performance relative to pre-defined acceptance criteria. The device contains two primary ML algorithms: Left Heart Segmentation and Landing Zone Detection.
1. Table of Acceptance Criteria and Reported Device Performance
| Feature / Metric | Acceptance Criteria (Pre-defined) | Reported Device Performance |
|---|---|---|
| Left Heart Segmentation Algorithm | ||
| Probability of Bone Removal (Segmentation Accuracy) | Not explicitly stated as a numerical threshold, but implied to be high for correct segmentation. | 532/533 cases (99.81%) for bone removal |
| Probability of LAA Visualization (Segmentation Accuracy) | Not explicitly stated as a numerical threshold, but implied to be high for correct visualization. | 519/533 cases (97.37%) for LAA visualization |
| Landing Zone Detection Algorithm | ||
| Landing Zone Plane Distance Metric | Within 10 mm | 97/100 cases (97%) within 10 mm (mean distance: 3.87 mm) |
| Landing Zone Contour Center Distance Metric | Within 12 mm | 99/100 cases (99%) within 12 mm (mean distance: 2.92 mm) |
Note: The document states that "All performance testing results met Circle's pre-defined acceptance criteria," indicating that the reported performance metrics met or exceeded the internal thresholds established by the manufacturer, even if the exact numerical acceptance percentages for segmentation accuracy were not explicitly listed as criteria in the provided text.
2. Sample Sizes Used for the Test Set and Data Provenance
- Test Set Sample Size:
- Left Heart Segmentation: 533 anonymized patient images
- Landing Zone Detection: 100 anonymized patient images
- Data Provenance:
- Country of Origin: The validation data was sourced from multiple sites across the U.S. and other urban regions. Specifically:
- Left Heart Segmentation: U.S., Canada, South America, Europe, and Asia.
- Landing Zone Detection: Various sites across the U.S.
- Retrospective/Prospective: The data used for validation were pre-existing CT images, common for retrospective studies. The document states "All data used for validation were not used during the development of the training algorithms," ensuring independence.
- Country of Origin: The validation data was sourced from multiple sites across the U.S. and other urban regions. Specifically:
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
The document mentions that for the Landing Zone Detection algorithm, "The landing zone was manually contoured by multiple expert readers for evaluation."
For the training data of the Left Heart Segmentation algorithm, it states "the left heart structures were manually annotated by multiple expert readers." While this refers to training, it implies a similar process and expert qualification for testing.
The specific number of experts and their explicit qualifications (e.g., "radiologist with 10 years of experience") are not specified in the provided text for either training or validation ground truth establishment. It only states "expert readers."
4. Adjudication Method for the Test Set
The document indicates that for the Landing Zone Detection ground truth, the "landing zone was manually contoured by multiple expert readers." For the Left Heart Segmentation training data, "manually annotated by multiple expert readers." This implies a consensus or majority vote approach might have been used, but the specific adjudication method (e.g., 2+1, 3+1, none) is not explicitly detailed in the provided text.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done
No, a MRMC comparative effectiveness study was not done. The document explicitly states: "No clinical studies were necessary to support substantial equivalence." The performance data presented is that of the algorithm's standalone performance against expert-defined ground truth, rather than a comparison of human readers with and without AI assistance. Therefore, an effect size of human reader improvement with AI vs. without AI assistance is not provided and was not part of this submission.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes, standalone performance was evaluated. The metrics reported (Probability of Bone Removal, Probability of LAA Visualization, Landing Zone Plane Distance, Landing Zone Contour Center Distance) are direct measurements of how accurately the ML algorithms perform their specific tasks when processing the CT images. The "Validation of Machine Learning Derived Outputs" section focuses purely on the algorithm's performance against ground truth.
7. The Type of Ground Truth Used
The ground truth used was expert consensus/manual contouring/annotation.
- For Left Heart Segmentation: "left heart structures were manually annotated by multiple expert readers."
- For Landing Zone Detection: "the landing zone was manually contoured by multiple expert readers."
This is observational data interpreted by human experts, not pathology or outcomes data.
8. The Sample Size for the Training Set
- Left Heart Segmentation: 113 cases
- Landing Zone Detection: 273 cases
9. How the Ground Truth for the Training Set Was Established
- Left Heart Segmentation: "the left heart structures were manually annotated by multiple expert readers."
- Landing Zone Detection: "the landing zone was manually contoured by expert readers."
Similar to the test set, the ground truth for training was established through manual annotation and contouring by expert readers. The document emphasizes that "the separation into training versus validation datasets is made on the study level to ensure no overlap between the two sets."
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).