(149 days)
Yes
The device description explicitly states that the software "applies machine learning algorithms" and uses an "algorithm that was derived through use of deep learning". It also mentions a "convolutional network" and a "machine learning model trained on apical and parasternal long-axis views".
No.
The software processes previously acquired images to provide automated estimation of left ventricular ejection fraction, which is used to assist clinicians in cardiac evaluation. It does not directly provide therapy or affect the structure or function of the body.
Yes
The device provides an "automated estimation of left ventricular ejection fraction," which is a clinical measurement used to "assist the clinician in a cardiac evaluation." This function directly contributes to identifying a medical condition or its severity, qualifying it as a diagnostic device.
Yes
The device is described as "Caption Interpretation Automated Ejection Fraction Software" and its description focuses entirely on the software's functionality, algorithms, and integration with existing hardware (ultrasound devices, PCs, PACS). There is no mention of any proprietary hardware components included with the device.
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs are used to examine specimens derived from the human body. The intended use and device description clearly state that this software processes previously acquired transthoracic cardiac ultrasound images. Ultrasound images are generated in vivo (within the living body) and are not specimens derived from the body in the way that blood, urine, or tissue samples are.
- The device analyzes images, not biological samples. The core function is image processing and measurement on those images.
Therefore, while this device is a medical device used for diagnostic purposes, it falls under the category of imaging software or analysis tools, not In Vitro Diagnostics.
Yes
The "Device Description" section states, "The current implementation of the device adds a predetermined change control plan (PCCP) to the device cleared under K210747, which allows future modifications to the device." Additionally, an entire section titled "Predetermined Change Control Plan (PCCP) - All Relevant Information" details the plan.
Intended Use / Indications for Use
The Caption Interpretation Automated Ejection Fraction software is used to process previously acquired transthoracic cardiac ultrasound images, to store images, and to manipulate and make measurements on images using an ultrasound device, personal computer, or a compatible DICOM-compliant PACS system in order to provide automated estimation of left ventricular ejection fraction. This measurement can be used to assist the clinician in a cardiac evaluation.
The Caption Interpretation Automated Ejection Fraction Software is indicated for use in adult patients.
Product codes
QVD
Device Description
The Caption Interpretation Automated Ejection Fraction Software ("AutoEF") applies machine learning algorithms to process two-dimensional transthoracic echocardiography images for calculating left ventricular ejection fraction.
The current implementation of the device adds a predetermined change control plan (PCCP) to the device cleared under K210747, which allows future modifications to the device.
The version of Caption Interpretation AutoEF cleared under K210747 performs left ventricular ejection fraction estimation using apical four chamber (A4C), apical two chamber (A2C) and the parasternal long-axis (PLAX) cardiac ultrasound images.
The software uses an algorithm that was derived through use of deep learning and locked prior to validation. The product operates as an add-in to a DICOM PACS system, ultrasound device, or personal computer. Caption Interpretation receives imaging data either directly from an ultrasound system or from a module in a PACS system.
The device includes the following main components:
-
- Clip Annotation and Selection: The AutoEF software includes a function that processes video clips in a study to automatically classify clips that are PLAX. AP4, and AP2 views. This view selection is based on a convolutional network. It also includes a function, Image Quality Score (IQS), that allows selection of best available PLAX, AP4, and AP2 clips within the study or provide an indication to the user if there are no clips with sufficient quality to estimate ejection fraction, based on prespecified IQS thresholds.
-
- Eiection Fraction Estimator and Confidence Metric: The automated eiection fraction estimation is performed using a machine learning model trained on apical and parasternal long-axis views. The model is trained with a dataset from a large number of patients. representative of the intended patient population and variety of contemporary cardiac ultrasound scanners. The EF calculation can be performed on a 3-view combination (PLAX, AP4 and AP2), 2-view combinations (AP4 and AP2, AP2 and PLAX, AP4 and PLAX), or single views (AP4, AP2). The confidence metric provides expected error in left-ventricle ejection fraction estimation and is based on IQS.
The outputs of the software are presented to the clinician. The outputs are as follows:
- Eiection Fraction Estimator and Confidence Metric: The automated eiection fraction estimation is performed using a machine learning model trained on apical and parasternal long-axis views. The model is trained with a dataset from a large number of patients. representative of the intended patient population and variety of contemporary cardiac ultrasound scanners. The EF calculation can be performed on a 3-view combination (PLAX, AP4 and AP2), 2-view combinations (AP4 and AP2, AP2 and PLAX, AP4 and PLAX), or single views (AP4, AP2). The confidence metric provides expected error in left-ventricle ejection fraction estimation and is based on IQS.
- EF calculation stated as a percentage (e.g., "59%"); .
- Predicted standard error based on image quality (e.g., "+6%"); .
- . Qualitative characterization of the EF estimate mapped to guidelines from the American Society of Echocardiography (e.g., "Normal"); Predicted likelihood of the qualitative bin based on the standard error listed above (e.g., "80% likelihood"):
- . Information about which clip(s) and view(s) were used as the basis of the EF calculation;
- . Single-view EF calculations of views incorporated into overall EF calculation (e.g., "PLAX: 55%; AP4: 63%; AP2: 58%").
The clinician can edit the estimation, if desired, and must decide whether to accept or reject it for use in the final report.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Yes
Input Imaging Modality
Transthoracic cardiac ultrasound images
Anatomical Site
Left Ventricle
Indicated Patient Age Range
Adult patients
Intended User / Care Setting
Clinician; Operated by or under the direction and supervision of a licensed physician.
Description of the training set, sample size, data source, and annotation protocol
The automated ejection fraction estimation is performed using a machine learning model trained on apical and parasternal long-axis views. The model is trained with a dataset from a large number of patients, representative of the intended patient population and variety of contemporary cardiac ultrasound scanners.
Description of the test set, sample size, data source, and annotation protocol
Caption Health AutoEF performance, including the integrated operation of the Clip Selector, was validated on a dataset of 186 patient studies in a retrospective multicenter study. These studies were representative of patient conditions, including range of ejection fraction values, and a significant portion of high body mass index patients as is typically associated with technically difficult studies.
A Clip Annotator study verified the ability of the software to receive, annotate and select clips for Auto EF calculation. Results of the Clip Annotator were compared to evaluation by a panel of expert readers. The study-level positive value (PPV) and sensitivity were computed, and each was tested for whether it exceeded the pre-specified performance goals of 80% PPV and 80% sensitivity.
The primary endpoint was the root mean square deviation (RMSD) between AutoEF derived values based on the best available view and the reference standard. The acceptance criterion was defined as RMSD superior to 9.2% for view combinations. Likewise, the acceptance criteria for the combination of views were also defined as RMSD superior to 9.2%. The performance criterion for single views was defined as RMSD superior to 11.024%.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Algorithm Performance Testing
The only component of this device which is new compared to the previously cleared device in K210747 is the addition of a PCCP. No new performance testing was provided for the current device and performance testing used for clearance of K210747 also applies to this device and is summarized below.
Caption Health AutoEF performance, including the integrated operation of the Clip Selector, was validated on a dataset of 186 patient studies in a retrospective multicenter study. These studies were representative of patient conditions, including range of ejection fraction values, and a significant portion of high body mass index patients as is typically associated with technically difficult studies.
Clip Annotation Performance
A Clip Annotator study verified the ability of the software to receive, annotate and select clips for Auto EF calculation. Results of the Clip Annotator were compared to evaluation by a panel of expert readers. The study-level positive value (PPV) and sensitivity were computed, and each was tested for whether it exceeded the pre-specified performance goals of 80% PPV and 80% sensitivity. That study met the pre-defined acceptance criteria and found that the observed PPV point estimates for the Clip Annotator were greater than 97% for identification of the imaging mode and the view. Similarly, observed sensitivity point estimates were greater than 96% across views and imaging mode.
Clinical Validation of AutoEF
Caption Interpretation AutoEF was also the subject of a pivotal clinical investigation to validate successful performance of the EF calculation in comparison to conventional EF calculation methods. The study comprised a dataset of 186 patient studies acquired from three sites across the US. This study was conducted to support the clearance of K210747.
Dataset Demographics (n=186):
- Sex: Male 104 (56%), Female 82 (44%)
- BMI: 97% for identification of the imaging mode and the view.
- Clip Annotator Sensitivity: > 96% across views and imaging mode.
- Root Mean Square Deviation (RMSD):
- Best available view: 7.21% [95% CI: 6.62%, 7.74%]
- AP4 and AP2: 7.27% [95% CI: 6.55%, 7.92%]
- AP4 and PLAX: 7.50% [95% CI: 6.85%, 8.09%]
- AP4, AP2, and PLAX: 7.24% [95% CI: 6.64%, 7.80%]
- AP2 and PLAX: 8.04% [95% CI: 7.32%, 8.7%]
- AP4 only: 7.76% [95% CI: 7.01%, 8.45%]
- AP2 only: 8.27% [95% CI: 7.44%, 9.03%]
Predicate Device(s)
Reference Device(s)
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information
Caption Interpretation Auto Ejection Fraction Software includes an FDA-authorized predetermined change control plan (PCCP). The authorized PCCP includes detailed description of planned device modifications, and the associated methodology to develop validate and implement those modifications in a manner that ensures the continued safety and effectiveness of the Caption Interpretation Automated Ejection Fraction Software. The plan includes the following four modification categories:
- Modification #1-Training on Additional Data .
- Modification #2-Incorporation of additional 2D TTE views .
- Modification #3-Optimization of implementation of core algorithm(s) in software .
- . Modification #4-Improving Algorithm Operating Speed
Verification and Validation Activities
PCCP describes specifications and methods to achieve appropriate risk control for anticipated changes. Risk Analysis will be performed to identify any new hazards created by algorithm changes, and to trace mitigation activity and completeness.
The evaluation of the modification will be performed with verification at the algorithm. subsystem, and end-to-end system level. Functional and end-to-end verification tests confirm proper operation of revised algorithms in the product. Validation will be performed at the end-to-end system level with expert-labeled ground truth data. Training and test datasets will be independent. The performance targets must meet the prespecified criteria as detailed in the PCCP. The specific verification and validation details for each individual modification are described in the PCCP.
The PCCP specifies a minimum percentage of new data for the test datasets, and that test datasets may not be used more than a specified number of times for the same purpose.
The key acceptance criteria included, but were not limited to, the following:
-
- 80% positive predictive value and 80% sensitivity for correct mode, view, and minimum number of frames.
-
- 80% of clips meet expert criteria for suitability for EF estimation.
-
- Overall (all views) and all combined views Auto EF is within 9.2% RMSD of expert EF.
-
- New views superior to 11.024% RMSD individually.
-
- For each standard view, the Confidence Metric must meet the equivalence to expert EF criteria as defined in PCCP.
Implementation of Modifications and Control
The machine learning algorithms used in Caption Interpretation Auto Ejection Fraction Software are trained, tuned, and "locked" prior to release i.e., they do not continuously learn in the field. Caption Health follows its design controls as well as the test methods and acceptance criteria defined in the PCCP to verify and validate new release versions of Caption Interpretation AutoEF. Caption Health intends to deploy a single version of the software within the United States, e.g., there will not be site specific versions of the software.
Communication and transparency to users
If a new release version of the software is available, due to implementation of modification made in accordance with the PCCP, Caption Health notifies all existing customers that an update is available and requests that they upgrade to the latest version of the software. Caption Health continues to follow-up with each customer to ensure that they are on the latest version of the software. Version numbering of the Caption Interpretation software will be updated in the software interface and in the User Manual. User manual will be updated to reflect the associated performance, associated inputs, supporting evidence, version history and other necessary information about the modification(s) made pursuant to the PCCP.
The probable benefits of machine learning-based devices include the capability to improve their performance through iterative modifications, including learning from real-world data. Caption interpretation automated ejection fraction software is a machine learning based device. It includes a predetermined change control plan (PCCP) which supports performance improvement through specific device modifications. The authorized PCCP includes detailed description of planned device modifications, and the associated methodology to develop validate and implement those modifications in a manner that ensures the continued safety and effectiveness of the Caption interpretation automated ejection fraction software.
The implementation plan for the modifications authorized in the PCCP would allow enhancing the generalizability of the device and improving the performance for quantitative imaging applications while ensuring continued safety and effectiveness. The implementation of modifications in the authorized PCCP, made according to a verification and validation plan developed in conjunction with FDA input and agreed to by FDA. would allow health care professionals quicker access to improved tools for patient management, without necessitating additional marketing submissions.
N/A
0
DE NOVO CLASSIFICATION REQUEST FOR CAPTION INTERPRETATION AUTOMATED EJECTION FRACTION SOFTWARE
REGULATORY INFORMATION
FDA identifies this generic type of device as:
Radiological machine learning-based quantitative imaging software with predetermined change control plan. A radiological machine learning based quantitative imaging software with predetermined change control plan is a software-only device which employs machine learning algorithms on radiological images to provide quantitative imaging outputs. The device includes functions to support outputs such as view selection, segmentation and landmarking. The design specifications include planned modifications that may be made to the device consistent with an established predetermined change control plan.
NEW REGULATION NUMBER: 21 CFR 892.2055
CLASSIFICATION: Class II
PRODUCT CODE: QVD
BACKGROUND
DEVICE NAME: Caption Interpretation Automated Ejection Fraction Software
SUBMISSION NUMBER: DEN220063
DATE DE NOVO RECEIVED: September 28, 2022
SPONSOR INFORMATION:
Caption Health, Inc. 2000 Sierra Point Pkwy, 8th Floor Brisbane, CA 94005
INDICATIONS FOR USE
The Caption Interpretation Automated Ejection Fraction software is used to process previously acquired transthoracic cardiac ultrasound images, to store images, and to manipulate and make measurements on images using an ultrasound device, personal computer, or a compatible DICOM-compliant PACS system in order to provide automated estimation of left ventricular ejection fraction. This measurement can be used to assist the clinician in a cardiac evaluation.
The Caption Interpretation Automated Ejection Fraction Software is indicated for use in adult patients.
1
LIMITATIONS
Prescription Use only: The sale, distribution, and use of this device are restricted to prescription use in accordance with 21 CFR 801.109.
Warnings:
- . Product users are responsible for image quality and diagnosis. A qualified medical professional must inspect the data being used for analysis and diagnosis, and ensure that the data is sufficient and appropriate in anatomical correctness and both spatial and temporal resolution for the measurement being employed.
- . The Caption Interpretation Automated Ejection Fraction Software is not intended for transesophageal echocardiograms (TEE).
- . This device must be operated by or under the direction and supervision of a licensed physician.
PLEASE REFER TO THE LABELING FOR A COMPLETE LIST OF WARNINGS, PRECAUTIONS AND CONTRAINDICATIONS.
DEVICE DESCRIPTION
The Caption Interpretation Automated Ejection Fraction Software ("AutoEF") applies machine learning algorithms to process two-dimensional transthoracic echocardiography images for calculating left ventricular ejection fraction.
The current implementation of the device adds a predetermined change control plan (PCCP) to the device cleared under K210747, which allows future modifications to the device.
The version of Caption Interpretation AutoEF cleared under K210747 performs left ventricular ejection fraction estimation using apical four chamber (A4C), apical two chamber (A2C) and the parasternal long-axis (PLAX) cardiac ultrasound images.
The software uses an algorithm that was derived through use of deep learning and locked prior to validation. The product operates as an add-in to a DICOM PACS system, ultrasound device, or personal computer. Caption Interpretation receives imaging data either directly from an ultrasound system or from a module in a PACS system.
The device includes the following main components:
-
- Clip Annotation and Selection: The AutoEF software includes a function that processes video clips in a study to automatically classify clips that are PLAX. AP4, and AP2 views. This view selection is based on a convolutional network. It also includes a function, Image Quality Score (IQS), that allows selection of best available PLAX, AP4, and AP2 clips within the study or provide an indication to the user if there are no clips with sufficient quality to estimate ejection fraction, based on prespecified IQS thresholds.
2
-
- Eiection Fraction Estimator and Confidence Metric: The automated eiection fraction estimation is performed using a machine learning model trained on apical and parasternal long-axis views. The model is trained with a dataset from a large number of patients. representative of the intended patient population and variety of contemporary cardiac ultrasound scanners. The EF calculation can be performed on a 3-view combination (PLAX, AP4 and AP2), 2-view combinations (AP4 and AP2, AP2 and PLAX, AP4 and PLAX), or single views (AP4, AP2). The confidence metric provides expected error in left-ventricle ejection fraction estimation and is based on IQS.
The outputs of the software are presented to the clinician. The outputs are as follows:
- Eiection Fraction Estimator and Confidence Metric: The automated eiection fraction estimation is performed using a machine learning model trained on apical and parasternal long-axis views. The model is trained with a dataset from a large number of patients. representative of the intended patient population and variety of contemporary cardiac ultrasound scanners. The EF calculation can be performed on a 3-view combination (PLAX, AP4 and AP2), 2-view combinations (AP4 and AP2, AP2 and PLAX, AP4 and PLAX), or single views (AP4, AP2). The confidence metric provides expected error in left-ventricle ejection fraction estimation and is based on IQS.
-
EF calculation stated as a percentage (e.g., "59%"); .
-
Predicted standard error based on image quality (e.g., "+6%"); .
-
. Qualitative characterization of the EF estimate mapped to guidelines from the American Society of Echocardiography (e.g., "Normal"); Predicted likelihood of the qualitative bin based on the standard error listed above (e.g., "80% likelihood"):
-
. Information about which clip(s) and view(s) were used as the basis of the EF calculation;
-
. Single-view EF calculations of views incorporated into overall EF calculation (e.g., "PLAX: 55%; AP4: 63%; AP2: 58%").
The clinician can edit the estimation, if desired, and must decide whether to accept or reject it for use in the final report.
PREDETERMINED CHANGE CONTROL PLAN (PCCP)
Caption Interpretation Auto Ejection Fraction Software includes an FDA-authorized predetermined change control plan (PCCP). The authorized PCCP includes detailed description of planned device modifications, and the associated methodology to develop validate and implement those modifications in a manner that ensures the continued safety and effectiveness of the Caption Interpretation Automated Ejection Fraction Software. The plan includes the following four modification categories:
- Modification #1-Training on Additional Data .
- Modification #2-Incorporation of additional 2D TTE views .
- Modification #3-Optimization of implementation of core algorithm(s) in software .
- . Modification #4-Improving Algorithm Operating Speed
Verification and Validation Activities
PCCP describes specifications and methods to achieve appropriate risk control for anticipated changes. Risk Analysis will be performed to identify any new hazards created by algorithm changes, and to trace mitigation activity and completeness.
The evaluation of the modification will be performed with verification at the algorithm. subsystem, and end-to-end system level. Functional and end-to-end verification tests confirm proper operation of revised algorithms in the product. Validation will be performed at the end-to-end system level with expert-labeled ground truth data. Training and test datasets will be independent. The performance targets must meet the prespecified criteria as detailed in the
3
PCCP. The specific verification and validation details for each individual modification are described in the PCCP.
The PCCP specifies a minimum percentage of new data for the test datasets, and that test datasets may not be used more than a specified number of times for the same purpose.
The key acceptance criteria included, but were not limited to, the following:
-
- 80% positive predictive value and 80% sensitivity for correct mode, view, and minimum number of frames.
-
- 80% of clips meet expert criteria for suitability for EF estimation.
-
- Overall (all views) and all combined views Auto EF is within 9.2% RMSD of expert EF.
-
- New views superior to 11.024% RMSD individually.
-
- For each standard view, the Confidence Metric must meet the equivalence to expert EF criteria as defined in PCCP.
Implementation of Modifications and Control
The machine learning algorithms used in Caption Interpretation Auto Ejection Fraction Software are trained, tuned, and "locked" prior to release i.e., they do not continuously learn in the field. Caption Health follows its design controls as well as the test methods and acceptance criteria defined in the PCCP to verify and validate new release versions of Caption Interpretation AutoEF. Caption Health intends to deploy a single version of the software within the United States, e.g., there will not be site specific versions of the software.
Communication and transparency to users
If a new release version of the software is available, due to implementation of modification made in accordance with the PCCP, Caption Health notifies all existing customers that an update is available and requests that they upgrade to the latest version of the software. Caption Health continues to follow-up with each customer to ensure that they are on the latest version of the software. Version numbering of the Caption Interpretation software will be updated in the software interface and in the User Manual. User manual will be updated to reflect the associated performance, associated inputs, supporting evidence, version history and other necessary information about the modification(s) made pursuant to the PCCP.
SUMMARY OF NON-CLINICAL/BENCH STUDIES
Software
The only change in this device compared to the Caption Interpretation Auto Ejection Fraction Software cleared in K210747 was the addition of PCCP. The software documentation was, therefore, referenced from K210747. The device was identified as having a moderate level of concern as defined in the FDA guidance document "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices." The software documentation included:
-
- Software/Firmware Description
-
- Device Hazard Analysis
-
- Software Requirement Specifications
4
-
- Architecture Design Chart
-
- Software Design Specifications
-
- Traceability
-
- Software Development Environment Description
-
- Verification and Validation Documentation
-
- Revision Level History
-
- Unresolved Anomalies
-
- Cybersecurity
Software documentation and testing provided demonstrate that the device meets all the recommendations in the guidance document.
SUMMARY OF CLINICAL INFORMATION
Algorithm Performance Testing
The only component of this device which is new compared to the previously cleared device in K210747 is the addition of a PCCP. No new performance testing was provided for the current device and performance testing used for clearance of K210747 also applies to this device and is summarized below.
Caption Health AutoEF performance, including the integrated operation of the Clip Selector, was validated on a dataset of 186 patient studies in a retrospective multicenter study. These studies were representative of patient conditions, including range of ejection fraction values, and a significant portion of high body mass index patients as is typically associated with technically difficult studies.
Clip Annotation Performance
A Clip Annotator study verified the ability of the software to receive, annotate and select clips for Auto EF calculation. Results of the Clip Annotator were compared to evaluation by a panel of expert readers. The study-level positive value (PPV) and sensitivity were computed, and each was tested for whether it exceeded the pre-specified performance goals of 80% PPV and 80% sensitivity. That study met the pre-defined acceptance criteria and found that the observed PPV point estimates for the Clip Annotator were greater than 97% for identification of the imaging mode and the view. Similarly, observed sensitivity point estimates were greater than 96% across views and imaging mode.
Clinical Validation of AutoEF
Caption Interpretation AutoEF was also the subject of a pivotal clinical investigation to validate successful performance of the EF calculation in comparison to conventional EF calculation methods. The study comprised a dataset of 186 patient studies acquired from three sites across the US. This study was conducted to support the clearance of K210747.
The dataset demographics and distribution of ultrasound scanners are tabulated below:
5
| Sex | Studies n, % |
|---|---|
| Male | 104 (56%) |
| Female | 82 (44%) |
| Total | 186 (100%) |
Table 1a: Dataset Demographics - Sex
Table 1b: Dataset Demographics - BMI
| BMI | Studies n, % |
|---|---|
| BMI |