(216 days)
AISAP CARDIO V1.0 is a software platform that automatically processes and analyzes acquired cardiac POCUS images, producing a report with diagnostic assessment and measurements of several key cardiac structural and functional parameters, including: presence of valvular pathology (regurgitations of the mitral, tricuspid, aortic valves and aortic stenosis), and measurements of the Left Ventricle Ejection Fraction (LVEF), right and left ventricular dimensions, right ventricular fractional area change (RV FAC), atrial areas, ascending aorta diameter, and inferior vena cava (IVC) diameter.
The device outputs are provided in a report that is intended to support qualified physicians in their analysis and interpretation of adult cardiac POCUS images, using FDA-cleared ultrasound devices. Physicians should be trained and privileged by their organization following education processes and should perform cardiac POCUS according to their specialty professional society clinical guidelines.
AISAP CARDIO V1.0 has not been validated for the assessment of congenital heart disease, and/or intra-cardiac lesions (e.g., tumors, thrombi, vegetations), prosthetic valves, and in the presence of ventricular assist devices.
AISAP CARDIO V1.0 is indicated for use in adult patients only.
AISAP CARDIO V1.0 is a machine learning-based decision support software device, indicated as an adjunct to diagnostic Cardiac point of care ultrasound (C-POCUS) for adult patients undergoing assessment for cardiac disease. This device performs automated analysis of ultrasound images and generates valvular assessments and measurements of standard cardiac structural and functional parameters.
-
- Inform the user of a suspected cardiac valvular regurgitation (mitral, tricuspid, or aortic), and/or aortic stenosis is: either greater than mild severity or none to mild severity.
-
- Inform the user of the 4 class American Society of Echocardiography (ASE) recommended category for valvular regurgitation (mitral, tricuspid, or aortic), and or aortic stenosis. Each finding categorizes according to none, mild, moderate, or severe.
-
- Measurements of the following standard cardiac structural or functional parameters:
- Left Ventricular Ejection Fraction (LVEF) (percent) a.
- Left ventricular end diastolic diameter (cm) b.
- Right ventricular area change (RV FAC [ratio]) C.
- Inferior vena cava (IVC) maximal diameter (mm) d.
- e. Aortic root diameter (cm)
- ਿ Right atrium (RA) area (cm2)
- Left atrium (LA) area (cm²) g.
AISAP CARDIO V1.0 assists the physician in assessing 4 major valvular findings in adults, along with providing information on several correlated cardiac ultrasound measurements frequently found to be abnormal in association with valvular heart disease. Used together and interpreted by the physician, the device provides information that may assist in rendering an accurate diagnosis of selected cardiac findings. AISAP CARDIO V1.0 is adjunctive to cardiac POCUS (C-POCUS) use by privileged physicians in use scenarios supported by clinical guidelines. Specifically, patient management decisions are not intended to be and should not be made solely on the results of the software analysis of the proposed device. When significant valve pathology is suspected comprehensive echocardiography should be considered in accordance with the relevant professional guidelines.
AISAP CARDIO V1.0 uses machine learning NN (neural network) models trained to recognize patterns and make decisions. AISAP CARDIO V1.0 contains classification models which identify categories within data, regression models which predict numerical values, and instance segmentation models that detect and segment objects within images.
Here's a breakdown of the acceptance criteria and supporting studies for the AISAP Cardio V1.0 device, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Feature/Measurement | Acceptance Criteria | Reported Device Performance | Study Type |
|---|---|---|---|
| Structural & Functional Measurements | Standalone Model Performance | ||
| LVEF | RMSE < 7% | 6.20% (95% CI: 5.57-6.83) | Standalone Model Performance |
| IVC maximal diameter | RMSE < 0.3 cm | 0.25 cm (95% CI: 0.20-0.29) | Standalone Model Performance |
| Left atrium (LA) area | RMSE < 4.28 cm² | 2.39 cm² (95% CI: 1.96-2.82) | Standalone Model Performance |
| Right atrium (RA) area | RMSE < 3.36 cm² | 2.11 cm² (95% CI: 1.75-2.47) | Standalone Model Performance |
| LV end diastolic diameter | RMSE < 9.15 mm | 5.06 mm (95% CI: 4.58-5.55) | Standalone Model Performance |
| Right ventricle (RV) fractional area change (FAC) | RMSE < 9.12% | 10.17% (95% CI: 9.01-11.33) | Standalone Model Performance |
| Aortic root diameter | RMSE < 0.62 cm | 0.19 cm (95% CI: 0.16-0.21) | Standalone Model Performance |
| Valvular Pathology Detection | Standalone Model Performance | ||
| All 4 valves (MR, AS, AR, TR) | AUC > 0.80 | MR: 0.975; AS: 0.969; AR: 0.993; TR: 0.973 | Standalone Model Performance |
| Clinical Reader Performance | Multi-Reader Study | ||
| MR (Aided vs. Un-aided) | Lower bound of 95% CI for (AUC_aided - AUC_unaided) > 0 | AUC_aided (0.963) > AUC_unaided (0.870) | Clinical Reader Performance (MRMC) |
| TR (Aided vs. Un-aided) | Lower bound of 95% CI for (AUC_aided - AUC_unaided) > 0 | AUC_aided (0.937) > AUC_unaided (0.851) | Clinical Reader Performance (MRMC) |
| AR (Aided vs. Un-aided) | Lower bound of 95% CI for (AUC_aided - AUC_unaided) > 0 | AUC_aided (0.947) > AUC_unaided (0.868) | Clinical Reader Performance (MRMC) |
| AS (Aided vs. Un-aided) | Lower bound of 95% CI for (AUC_aided - AUC_unaided) > 0 | AUC_aided (0.925) > AUC_unaided (0.897) | Clinical Reader Performance (MRMC) |
| View Classification | Accuracy > 95% | PLAX: 100%; PSAX: 99.2%; A4C: Not reported; SC IVC: 98.8% | View Classification Validation Study |
2. Sample Size for Test Set and Data Provenance
- Standalone Structural and Functional Measurements Study: 200 cases
- Standalone Valvular Pathology Study: 329 cases
- Clinical Reader Performance (MRMC) Study: 260 cases
- View Classification Validation Study: 500 sampled loops per cardiac view (from the clinical study dataset)
Data Provenance:
The test data was collected prospectively at 4 clinical reader sites located in the United States (51% of cases) and Israel (49% of cases). Images were acquired with different US device vendors (Philips, GE, Wisonic, EchoNous) from both in-patient and out-patient settings. Both physicians and sonographers performed the POCUS exams.
3. Number of Experts and Qualifications for Test Set Ground Truth
- Structural and Functional Measurements Study: 3 US board-certified cardiologists with a minimum of 5 years of experience.
- Valvular Pathology Study: Cardiologist interpretations (number not specified, but the context implies multiple experts as ground truth for other studies).
- Clinical Reader Performance (MRMC) Study: 3 US Board Certified cardiologists (for the severity grade of valvular pathologies).
- View Classification Validation Study: 2 certified experienced echo technicians, with over-read by a lead technician and a senior cardiologist.
4. Adjudication Method for Test Set
- Structural and Functional Measurements Study: Ground truth was established by the mean value determined by the 3 cardiologists' measurements following ASE guidelines.
- Valvular Pathology Study: Not explicitly stated, but implies expert interpretation as the ground truth.
- Clinical Reader Performance (MRMC) Study: "2+1" annotation strategy. The 260 cases were interpreted independently by two U.S. Board Certified ground truth cardiologists. Any discrepancies were interpreted by a third ground truth cardiologist. Any persistent disagreements were decided at a meeting of the three ground truth cardiologists.
- View Classification Validation Study: View verification by 2 certified experienced echo technicians, with an over-read of 30% of cases by a lead technician and an additional 10% over-read by a senior cardiologist.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
Yes, a MRMC comparative effectiveness study was done. It was called the "Clinical Reader Performance" study.
Effect Size (Improvement with AI vs. without AI assistance):
The study demonstrated an improvement in AUC, Kappa, and Accuracy when readers were aided by the device.
-
AUC Improvement (AI-aided vs. Unaided):
- MR: 0.963 vs. 0.870 (Improvement: 0.093)
- TR: 0.937 vs. 0.851 (Improvement: 0.086)
- AR: 0.947 vs. 0.868 (Improvement: 0.079)
- AS: 0.925 vs. 0.897 (Improvement: 0.028)
(The passing criteria states the lower bound of the 95% CI for this difference lay entirely above zero, indicating a statistically significant positive effect.)
-
Kappa Improvement (AI-aided vs. Unaided):
- MR: 0.881 vs. 0.756 (Improvement: 0.125)
- TR: 0.881 vs. 0.765 (Improvement: 0.116)
- AR: 0.913 vs. 0.815 (Improvement: 0.098)
- AS: 0.850 vs. 0.792 (Improvement: 0.058)
-
Accuracy Improvement (AI-aided vs. Unaided):
- MR: 73.6% vs. 61.6% (Improvement: 12.0%)
- TR: 75.3% vs. 64.1% (Improvement: 11.2%)
- AR: 80.6% vs. 71.7% (Improvement: 8.9%)
- AS: 74.7% vs. 69.8% (Improvement: 4.9%)
6. Standalone (Algorithm Only) Performance Study
Yes, standalone performance studies were done for both:
- "Standalone Model Performance for Structural and Functional Measurements"
- "Standalone Model Performance for Valvular Pathology"
7. Type of Ground Truth Used
- Structural and Functional Measurements: Expert consensus (mean value of 3 cardiologists' measurements) following American Society of Echocardiography (ASE) guidelines.
- Valvular Pathology: Cardiologist interpretations (implied expert consensus).
- Clinical Reader Performance (MRMC): Expert consensus of 3 US Board Certified cardiologists, established via an adjudication process ("2+1" strategy and consensus meeting).
- View Classification: Expert consensus of 2 certified experienced echo technicians, with over-read by a lead technician and a senior cardiologist.
8. Sample Size for Training Set
Over 140,000 individual exams were used for training the machine learning models, representing > 1 billion frames.
9. How Ground Truth for Training Set Was Established
The AISAP CARDIO V1.0 algorithms were trained at 2 academic institutions that perform cardiac ultrasound examinations and interpretations according to ASE guidelines. This implies that the ground truth for the training data was established by expert interpretation and measurements conforming to these professional guidelines at the academic institutions.
{0}------------------------------------------------
August 1, 2024
Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.
AISAP % John Smith Partner Hogan Lovells 555 Thirteenth Street, NW Washington, District of Columbia 20004
Re: K234141
Trade/Device Name: AISAP Cardio V1.0 Regulation Number: 21 CFR 892.2060 Regulation Name: Radiological computer-assisted diagnostic software for lesions suspicious of cancer Regulatory Class: Class II Product Code: POK, QIH Dated: July 1, 2024 Received: July 1, 2024
Dear John Smith:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
{1}------------------------------------------------
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Samal for
Jessica Lamb, Ph.D. Assistant Director Imaging Software Team DHT8B: Division of Radiological Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known) K234141
Device Name AISAP Cardio V1.0
Indications for Use (Describe)
AISAP CARDIO V1.0 is a software platform that automatically processes and analyzes acquired cardiac POCUS images. producing a report with diagnostic assessment and measurements of several key cardiac structural and functional parameters, including: presence of valvular pathology (regurgitations of the mitral, tricuspid, aortic stenosis), and measurements of the Left Ventricle Ejection Fraction (LVEF), right and left ventricular dimensions, right ventricular fractional area change (RV FAC), atrial areas, ascending aorta diameter, and inferior vena cava (IVC) diameter.
The device outputs are provided in a report that is intended to support qualified physicians in their and interpretation of adult cardiac POCUS images, using FDA-cleared ultrasound devices. Physicians should be trained and privileged by their organization following education processes and should perform cardiac POCUS according to their specialty professional society clinical guidelines.
AISAP CARDIO V1.0 has not been validated for the assessment of congenital heart disease, and/or intra-cardiac lesions (e.g., tumors, thrombi, vegetations), prosthetic valves, and in the presence of ventricular assist devices.
AISAP CARDIO V1.0 is indicated for use in adult patients only.
| Type of Use ( Select one or both, as applicable ) |
|---|
| ☑ Prescription Use (Part 21 CFR 801 Subpart D) ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW!
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments reqarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
510(k) SUMMARY
AISAP's CARDIO V1.0 Device
K234141
Submitter:
AISAP LTD
Sheba Heart Center Tel Hashomer, Sheba road 2 Ramat Gan , Israel 52621 Phone: +97235309360 Facsimile: +97235342392 Contact Person: Roni Attali, CEO Date Prepared: July 29, 2024
Name of Device: AISAP CARDIO V1.0
Common or Usual Name: Computer-Assisted Diagnostic Software (CADx)
Classification Name: Computer-Assisted Diagnostic Software (CADx) for lesions suspicious for cancer
Regulation: 21 CFR §892.2060
Regulatory Class: Class II
Product Code: POK
Secondary Product Code: QIH
Predicate Devices:
Primary predicate - Ultromics Echo Go Pro v1.0.2; K201555
Secondary predicate - Ultromics Echo Go Core 2.0; K213275
Reference Devices - None
{4}------------------------------------------------
1. Device Description
AISAP CARDIO V1.0 is a machine learning-based decision support software device, indicated as an adjunct to diagnostic Cardiac point of care ultrasound (C-POCUS) for adult patients undergoing assessment for cardiac disease. This device performs automated analysis of ultrasound images and generates valvular assessments and measurements of standard cardiac structural and functional parameters.
-
- Inform the user of a suspected cardiac valvular regurgitation (mitral, tricuspid, or aortic), and/or aortic stenosis is: either greater than mild severity or none to mild severity.
-
- Inform the user of the 4 class American Society of Echocardiography (ASE) recommended category for valvular regurgitation (mitral, tricuspid, or aortic), and or aortic stenosis. Each finding categorizes according to none, mild, moderate, or severe.
-
- Measurements of the following standard cardiac structural or functional parameters:
- Left Ventricular Ejection Fraction (LVEF) (percent) a.
- Left ventricular end diastolic diameter (cm) b.
- Right ventricular area change (RV FAC [ratio]) C.
- Inferior vena cava (IVC) maximal diameter (mm) d.
- e. Aortic root diameter (cm)
- ਿ Right atrium (RA) area (cm2)
- Left atrium (LA) area (cm²) g.
AISAP CARDIO V1.0 assists the physician in assessing 4 major valvular findings in adults, along with providing information on several correlated cardiac ultrasound measurements frequently found to be abnormal in association with valvular heart disease. Used together and interpreted by the physician, the device provides information that may assist in rendering an accurate diagnosis of selected cardiac findings. AISAP CARDIO V1.0 is adjunctive to cardiac POCUS (C-POCUS) use by privileged physicians in use scenarios supported by clinical guidelines. Specifically, patient management decisions are not intended to be and should not be made solely on the results of the software analysis of the proposed device. When significant valve pathology is suspected comprehensive echocardiography should be considered in accordance with the relevant professional guidelines.
AISAP CARDIO V1.0 uses machine learning NN (neural network) models trained to recognize patterns and make decisions. AISAP CARDIO V1.0 contains classification models which identify categories within data, regression models which predict numerical values, and instance segmentation models that detect and segment objects within images.
The AISAP CARDIO V1.0 algorithms were trained at 2 academic institutions that perform cardiac ultrasound examinations and interpretations according to ASE guidelines. Over 140,000 individual exams were used for training the machine learning models representing > 1 billion frames. Cohort characteristics are summarized in the table below.
{5}------------------------------------------------
| Characteristic | Percent positiveoraverage andstandard deviation |
|---|---|
| Age (years) | $67\pm16.73$ |
| Sex (Female) | 39.6% |
| Body mass index (BMI) | 15 to 49 (mean $6\pm18$ ) |
| POCUS device (not high-end cardiac ultrasound system) | 25.9% |
| Bedside ultrasound exam (outside of the echo lab) | 23.8% |
| Dyslipidemia (lipid disorder) | 6.7% |
| Diabetes Mellitus (DM) | 15.2% |
| Obstructive lung disease | 2.9% |
| Myocardial infarction (MI past recent) | 7.4% |
| Heart failure (HF) or cardiomyopathy (CMP) | 5.8% |
| Renal dysfunction including chronic kidney disease (CKD) | 10.0% |
| Hypertension | 41.9% |
| Ischemic heart disease (IHD including past angioplasty or cardiacsurgery) | 17.5% |
| CVA or TIA | 8% |
| Cardiac valve disease (any valve and of any severity) | 40.7% |
Clinical characteristics of the AISAP CARIDO V1.0 training data set
{6}------------------------------------------------
2. Intended Use / Indications for Use
AISAP CARDIO V1.0 is a software platform that automatically processes and analyzes acquired cardiac POCUS images, producing a report with diagnostic assessment and measurements of several key cardiac structural parameters, including: presence of valvular pathology (regurgitations of the mitral, tricuspid, aortic valves and aortic stenosis), and measurements of the Left Ventricle Ejection Fraction (LVEF), right and left ventricular dimensions, right ventricular fractional area change (RV FAC), atrial areas, ascending aorta diameter, and inferior vena cava (IVC) diameter.
The device outputs are provided in a report that is intended to support qualified physicians in their analysis and interpretation of adult cardiac POCUS images, using FDA-cleared ultrasound devices. Physicians should be trained and privileged by their organization following education processes and should perform cardiac POCUS according to their specialty professional society clinical guidelines.
AISAP CARDIO V1.0 has not been validated for the assessment of congenital heart disease, and/or intra-cardiac lesions (e.g., tumors, thrombi, vegetations), prosthetic valves, and in the presence of ventricular assist devices.
AISAP CARDIO V1.0 is indicated for use in adult patients only.
The proposed device and both the Echo Go Pro v1.0.2 (primary predicate, K201555), and Echo Go Core 2.0 (secondary predicate. K213275) share the same general intended use, as adjunctive aids in the interpretation of diagnostic ultrasound images. In other words, AISAP CARDIO V1.0 has the same intended use its predicate devices. The differences in indications are for clarity regarding the specific technological characteristics of the proposed device as well as to reflect the combination of functionality of both the predicate devices. The differences in indications do not affect the diagnostic use of the device as an adjunctive aid in the interpretation of diagnostic ultrasound images.
3. Summary of Technological Characteristics
Both the subject and predicate devices use Machine Learning technology to provide decision support for assessment of cardiac pathologies and are intended for the automatic assessment of cardiac ultrasound images for diagnostic purposes. Both devices incorporate machine learning algorithms for the analysis of DICOM cardiac ultrasound images. At a high level, the subject and predicate devices are based on the following same technological elements:
- Generation of a categorical and numeric output following analysis imaging data, ● specifically ultrasound images
- Software as a medical device (SaMD)
- Use of artificial intelligence \ Machine Learning-Based Algorithm ●
- Characterization of previously acquired movies features to generate output
- Cardiac ultrasound modality for analysis based on assessing cardiac structures ●
- Input is images provided in a DICOM format ●
- Original images can be viewed by the user ●
- Multiple echo views for the assessment of one anatomical feature
- View classification and ROI are selected automatically ●
- Operates on off-the-shelf hardware
{7}------------------------------------------------
- Primary analysis output is categorical with 2 levels ●
- Output is a report which is returned to the physician for review ●
- . Output of the system is to be used as a diagnostic aid
The following technological differences exist between the subject and predicate devices:
- The output of the proposed device's is categorical assessment as to whether the data are ● suggestive of more than mild valvular pathology (MR. AS, AR, TR), whereas the predicate device outputs an indication of whether the data are suggestive of higher possibility of significant (> 70% proximal or mid artery stenosis) coronary artery disease or lower possibility.
- The subject device outputs a secondary categorical classification for each valvular finding ● of none, mild, moderate, or severe grade.
- Though the subject device provides some different measurements than the secondary ● predicate (K213275) including IVC diameter, and RV FAC.
The technological differences do not raise different questions of safety and effectiveness. Furthermore, the outputs and measurements of the proposed device are supported by performance testing which demonstrate the safety and effectiveness of the proposed device.
4. Performance Data
4.1 Non-Clinical Testing
Risk assessment, performance, and cybersecurity of AISAP CARDIO V1.0 have been evaluated and verified in accordance with pre-defined software specifications and applicable performance standards through software verification testing.
During the development, potential hazards were evaluated and controlled through risk management activities. The performance testing demonstrates that the device meets all its specifications.
4.2 Performance Testing - Clinical
The AISAP CARDIO V1.0 clinical performance testing encompasses multiple assessments, including 2 standalone model performance assessment studies (Structural and Functional Measurements, and Valvular Pathology) and one randomized multi-reader study. An additional evaluation of the devices view classification performance was conducted using the study dataset.
The study cohort consisted of cardiac POCUS exams that have been collected prospectively at 4 clinical reader sites in the United States (51% of cases) and Israel. Images were acquired with different US device vendors from in-patient as well as out-patient settings. Both physicians and sonographers performed the POCUS exam.
260 de-identified cases were used for the reader study, 329 for the stand-alone valve performance study, and 200 cases for the stand-alone measurement performance study. An average of 23.7 video clips (loops) were provided per each case representing the different cardiac views.
The cohort included US (51%) and OUS (49%) cases comprising 13% Black or African, 5.5% Asian and 4% Hispanic or Latino.
{8}------------------------------------------------
Images were obtained from different ultrasound device vendors including Philips (41%), GE (28%), Wisonic (12.5%) and EchoNous (18.5%).
Additional cohort characteristics are summarized in the table below.
Clinical characteristics of the clinical study data set
| Characteristic | Percent positiveoraverage andstandard deviation |
|---|---|
| Age (years) | 68±16.39 |
| Sex (Female) | 36.8% |
| Body mass index (BMI) | 26.69±4.97 |
| Dyslipidemia (lipid disorder) | 59% |
| Diabetes Mellitus (DM) | 29.3% |
| Obstructive lung disease | 5.6% |
| Myocardial infarction (MI past \ recent) | 27% |
| Heart failure (HF) or cardiomyopathy (CMP) | 37% |
| Renal dysfunction including chronic kidney disease (CKD) | 21.2% |
| Hypertension | 62% |
| CVA or TIA | 12.7% |
| Cardiac valve disease (any valve and of any severity) | 55.9% |
A. Standalone Model Performance for Structural and Functional Measurements
Standalone model performance assessment based on the AISAP CARDIO V1.0 for structural and functional measurements on 200 cases where device outputs were compared to truthing. Truthing was performed by 3 US board certified cardiologists with minimum 5 years of experience. The ground truth was established by the mean value determined by 3 cardiologists measurements following the American Society of Echocardiography (ASE) guidelines.
Left Ventricular Eiection Fraction (LVEF): RMSE of 6,20% (Cl 5,57-6,83), indicating high accuracy, Inferior Vena Cava (IVC) Maximal Diameter: RMSE of 0.25 cm (Cl 0.20-0.29), well below the success threshold and below ground truth cardiologist RMSE.
Consistently, the device demonstrated excellent accuracy in the additional measurements: Left atrial area (RMSE 2.39), right atrial area (RMSE 2.11), LV end diastolic diameter (RMSE 5.06), and aortic root diameter (RMSE 0.19), with RMSE values consistently lower than those of ground truth cardiologists.
{9}------------------------------------------------
The table below summarizes the results.
| Structural and functionalmeasurement | Endpointtype | N | SuccessCriteria | RMSE(95% CI) |
|---|---|---|---|---|
| LVEF | Primary | 197 | < 7% | 6.20% (5.57-6.83) |
| IVC maximal diameter | Primary | 175 | < 0.3 | 0.25cm (0.20,0.29) |
| Left atrium (LA) area | Secondary | 188 | < 4.28 | 2.39cm² (1.96,2.82) |
| Right atrium (RA) area | Secondary | 178 | < 3.36 | 2.11cm² (1.75,2.47) |
| LV end diastolic diameter | Secondary | 173 | < 9.15 | 5.06mm (4.58,5.55) |
| Right ventricle (RV) fractionalarea change (FAC) | Secondary | 161 | < 9.12 | 10.17% (9.01,11.33) |
| Aortic root diameter | Secondary | 197 | < 0.62 | 0.19cm (0.16,0.21) |
Subgroup analyses was performed across the following groups: patient demographics (sex, age >65, body mass index - WHO categories, and LVEF categories), technical parameters (device manufacturer, image quality score), data origin (US/OUS cases), and training of person performing the scan (physician vs. sonographer). Across the above subgroups the RMSE results demonstrated acceptable performance and was mostly lower (or similar for LVEDD) to the mean RMSE of the 3 cardiologists that provided the ground truth. Detailed subgroup analysis were reported in the labeling.
B. Standalone Model Performance for Valvular Pathology
Standalone model performance assessment for valvular pathology on 329 cases with device outputs compared to cardiologist interpretations for cardiac valvular findings. Prespecified acceptance criteria was an AUC >0.80 for all the 4 valves.
The AISAP CARDIO device showed high AUC values, sensitivity, specificity and kappa results for detecting more than mild severity valvular pathologies as summarized in the table below.
| Pathology | N | AUC (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | Kappa (95% CI) |
|---|---|---|---|---|---|
| MR | 310 | 0.975 (0.960,0.987) | 95.3% (90.5,98.9) | 90.2% (86.3,93.9) | 0.879 (0.852,0.906) |
| AS | 272 | 0.969 (0.950,0.984) | 86.5% (76.2,95.7) | 94.5% (91.3,97.3) | 0.865 (0.825,0.901) |
| AR | 323 | 0.993 (0.986,0.999) | 96.5% (91.3,100.0) | 97.0% (94.9,98.9) | 0.913 (0.892,0.932) |
| TR | 295 | 0.973 (0.955,0.987) | 93.5% (86.9,98.5) | 89.3% (85.2,93.1) | 0.879 (0.854,0.905) |
Subgroup analyses for each of the primary and secondary endpoints by the following prespecified groups (detailed below) was performed and reported: patient demographics (sex, age >65, body mass index (BMI WHO groups), LVEF categories), technical parameters (device manufacturer, image quality score), data origin (US/OUS cases), training of person performing the scan (physician vs. sonographer) and the presence of concomitant significant valve disease. Across all the subgroups
{10}------------------------------------------------
detailed above, the AUC attained was well above the prespecified success criteria of AUC > 0.80. Detailed subgroup analysis were reported in the labeling.
C. Clinical Reader Performance
Clinical reader performance study that compared clinical reader aided interpretations of 260 cases. The readers participated in a fully crossed over randomized prospective study assessing their diagnostic performance compared to interpretation of board-certified cardiologists.
The 260 cases were interpreted independently by two U.S. Board Certified ground truth cardiologists for the severity grade (normal, mild, moderate, or severe) of each of the 4 valvular pathologies (mitral regurgitation (MR), tricuspid requrgitation (TR), aortic requrgitation (AR) and aortic stenosis (AS)). Any discrepancies were interpreted by a third ground truth cardiologist ("2+1" annotation strategy). Any persistent disagreements were decided at a meeting of the three ground truth cardiologists. The device performed inference on the same cases.
Readers assessed each case twice - with and without device output aid, with a memory washout period in between reads.
The passing criteria were differences in AUCunaided for each of the valvular pathologies where the lower bound of the 95% CI for this difference lay entirely above zero.
| Pathology | N | AUCaided | AUCunaided |
|---|---|---|---|
| MR | 259 | 0.963 (0.943,0.984) | 0.870 (0.807,0.934) |
| TR | 256 | 0.937 (0.903,0.971) | 0.851 (0.794,0.907) |
| AR | 260 | 0.947 (0.899,0.995) | 0.868 (0.789,0.947) |
| AS | 257 | 0.925 (0.895,0.954) | 0.897 (0.851,0.944) |
The aided and unaided AUC analasis results are summerized below.
Accuracy and Kappa agreement also improved when read was aided as summarized in the table below.
| Pathology | Metric | Measurementaided | Measurementunaided | Difference |
|---|---|---|---|---|
| MR | Kappa | 0.881 (0.872,0.890) | 0.756 (0.737,0.774) | 0.125 |
| Accuracy | 73.6% (71.9%,75.2%) | 61.6% (59.8%,63.4%) | 12.0% | |
| TR | Kappa | 0.881 (0.871,0.892) | 0.765 (0.747,0.783) | 0.116 |
| Accuracy | 75.3% (73.6%,77.0%) | 64.1% (62.2%,65.9%) | 11.2% | |
| AR | Kappa | 0.913 (0.905,0.921) | 0.815 (0.798,0.831) | 0.098 |
| Accuracy | 80.6% (79.1%,82.1%) | 71.7% (70.0%,73.3%) | 8.9% | |
| AS | Kappa | 0.850 (0.834,0.864) | 0.792 (0.773,0.809) | 0.058 |
| Accuracy | 74.7% (73.0%,76.3%) | 69.8% (68.1%,71.6%) | 4.9% |
Subgroup analysis was performed by reader specialty and reader country of origin. An increase in the AUC value of the aided read was observed across the above pre-specified subgroup analysis. A higher Kappa value was consistently seen when the read was aided.
{11}------------------------------------------------
This supports robust generalizability of the device across different clinical settings and patient populations. Detailed subgroup analysis were reported in the labeling.
Overall, these study results support that AISAP CARDIO device may improve physician evaluation of cardiac ultrasound images and demonstrate that the proposed device is substantially equivalent to the predicates.
D. View classification validation study
The aim of this study was to assess the performance of the AISAP CARDIO device view classification performance. The device classified different cardiac ultrasound views (images of the heart from different location and different angles with and without color Doppler). The results were compared to view classification by experienced sonographers. This step is required to initiate the algorithmic pipeline. Acceptance criteria was set at > 95%.
Cohort - We performed validation on 500 sampled loops per each cardiac view from the clinical study dataset, classified by the view classification phase as a valid view.
- a. View verification was performed by 2 certified experienced echo technicians.
- Over-read of 30% of cases by lead technician with additional 10% over-read by senior b. cardiologist.
- The assessment measured accuracy of view classification. C.
- Results Correct identification of the video clip view classification: ರ.
- PLAX 500/500 i.
- PSAX 496/500 ii.
- iii. A4C -
- SC IVC 494/500 İV.
- Conclusion Accuracy greater than 98% for all the relevant echocardiographic views. e.
Conclusions
AISAP CARDIO V1.0 is as safe and effective as the Ultromics Go Pro 1.0.2 and Go Core 2.0. The AISAP CARDIO V1.0 has the same intended uses and similar indications, technological characteristics, and principles of operation as its predicate device. The minor differences in indications do not alter the intended use of the device and do not affect its safety and effectiveness when used as labeled. In addition, the minor technological differences between the AISAP CARDIO V1.0 and its predicate devices raise no new issues of safety or effectiveness. Performance data demonstrate that the AISAP CARDIO V1.0 is substantially equivalent the Ultromics EchoGo Pro 1.0.2 and EchoGo Core 2.0.
§ 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.
(a)
Identification. A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will improve reader performance as intended.
(iii) Results from performance testing protocols that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain sufficient numbers of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Standalone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; and description of verification and validation activities including system level test protocol, pass/fail criteria, results, and cybersecurity).(2) Labeling must include:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and recommended user training.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations, including situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Detailed instructions for use.
(viii) A detailed summary of the performance testing, including: Test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders (
e.g., lesion and organ characteristics, disease stages, and imaging equipment).