(131 days)
Not Found
Yes
The document explicitly states that the software was developed using machine learning.
No.
The device is image processing software designed to aid physicians in assessing coronary artery calcification, calculating scores, and providing illustrative segmentations. It provides adjunctive information and is not intended to be used without clinical expert review. Its purpose is diagnostic aid, not therapy.
Yes
The device is image processing software designed to aid physicians in assessing coronary artery calcification, which is a diagnostic purpose. While it states that provided segmentations are for illustrative purposes only and not intended for diagnostic use, and that the software provides adjunctive information not intended to be used without clinical expert review, its primary function is to quantify and categorize coronary artery calcification as an aid to diagnosis.
Yes
The device is described as "image processing software" and a "computer aided quantification system" that receives and generates data in DICOM format. There is no mention of any accompanying hardware components or hardware-specific validation.
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs are used to examine specimens derived from the human body. The intended use and device description clearly state that ClearRead CT CAC processes images (CT scans) of the chest. It does not analyze blood, tissue, urine, or any other biological specimen.
- The output is an image processing result and a score based on image analysis. The primary output is the Agatston score derived from the analysis of the CT images, not from a biological sample.
- The device aids in assessing a condition based on imaging findings. It helps physicians evaluate coronary artery calcification by analyzing the visual information in the CT scans.
While the device is a medical device and uses machine learning, its function is based on image processing and analysis, not on the examination of in vitro specimens.
No
The letter does not state that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device.
Intended Use / Indications for Use
ClearRead CT CAC is image processing software designed to aid physicians in assessing coronary artery calcification on non-gated, non-contrast, standard, or low-dose chest CT scans of adult patients 30 years of age or older. The primary output of ClearRead CT CAC is the Agatston score. ClearRead CT CAC locates calcified coronary lesions and assigns them to one of several coronary arteries: the right coronary artery (RCA), the left main artery (LM), the left anterior descending artery (LAD), and the left circumflex artery (LCX). The software calculates the Agatston score as the sum for each artery while also providing the total across all coronary arteries. The total Agatston score is calibrated to 3.0mm slice thickness and is used to derive a CAC category and the arterial age. The provided segmentations are for illustrative purposes only and are not intended for diagnostic use.
ClearRead CT CAC provides adjunctive information and is not intended to be used without clinical expert review.
Product codes (comma separated list FDA assigned to the subject device)
JAK
Device Description
ClearRead CT CAC is a computer aided quantification system intended to aid in the assessment of coronary artery calcification. The system receives chest Computed Tomography (CT) studies as input, in DICOM® format, and generates output in DICOM (or other) format.
Mentions image processing
ClearRead CT CAC is image processing software designed to aid physicians in assessing coronary artery calcification on non-gated, non-contrast, standard, or low-dose chest CT scans of adult patients 30 years of age or older.
Mentions AI, DNN, or ML
This software, as a medical device product, was developed using machine learning to measure the amount of arterial calcification in non-contrast chest CT cases.
Input Imaging Modality
non-gated, non-contrast, standard, or low-dose chest CT scans
Anatomical Site
Chest
Indicated Patient Age Range
adult patients 30 years of age or older
Intended User / Care Setting
Intended User: Radiologists
Intended location: Medical Facility
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
Approximately 400 cases were collected and held out from development of any kind to estimate the test performance of the CAC scoring system. Regions associated with coronary artery calcification were manually outlined, including the particular artery, from which the ground truth Agatston score was obtained.
In total, 491 cases were used in the clinical assessment of the device: 143 cases with no sign of coronary artery calcification and 348 with evidence of CAC, as determined by a consensus ground truth review by three radiologists. These nearly 500 cases included samples from four CT manufacturers across five clinical sites. In the CAC positive cases, selection was performed to ensure the entire range of scores was spanned.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Non-clinical Testing:
Study type: Software verification and validation testing.
Sample size: Approximately 400 cases.
Key results: A weighted kappa score of 0.958 was achieved, demonstrating good agreement between the automatic method and manual processing. Bland-Altman analysis plot showed strong agreement with the true score compared to the estimated score, with a mean of -19.64 and limits of agreement (LoA) of 163.1 and -202.3.
Clinical Performance Testing:
Study type: Standalone model performance assessment.
Sample size: 491 cases (143 CAC negative, 348 CAC positive).
Key results:
Primary endpoint: kappa of 0.959 (95% CI: 0.943-0.975).
Secondary endpoint: Bland-Altman analysis yielded limits of agreement (LoA) of -12.20 ± 437.84.
Secondary endpoint: Dice score comparison between annotator ground truth and automatic outputs gave average scores of 0.910 (0.893,0.927) for GT versus GT and 0.892 (0.871,0.911) for Device versus GT.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Clinical Performance Testing (Percentage Agreement Scores for CAC Categories):
PPA Category 0: 88.1% (95% CI: 82.5, 93.2)
NPA Category 0: 99.1% (95% CI: 98.0, 100.0)
PPA Category 1-99: 92.8% (95% CI: 87.7, 96.9)
NPA Category 1-99: 95.4% (95% CI: 93.0, 97.4)
PPA Category 100-399: 88.1% (95% CI: 80.5, 94.4)
NPA Category 100-399: 97.1% (95% CI: 95.2, 98.5)
PPA Category ≥ 400: 98.6%
NPA Category ≥ 400: 98.3%
Predicate Device(s): If the device was cleared using the 510(k) pathway, identify the Predicate Device(s) K/DEN number used to claim substantial equivalence and list them here in a comma separated list exactly as they appear in the text. List the primary predicate first in the list.
Reference Device(s): Identify the Reference Device(s) K/DEN number and list them here in a comma separated list exactly as they appear in the text.
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information for the subject device only (e.g. presence / absence, what scope was granted / cleared under the PCCP, any restrictions, etc).
Not Found
§ 892.1750 Computed tomography x-ray system.
(a)
Identification. A computed tomography x-ray system is a diagnostic x-ray system intended to produce cross-sectional images of the body by computer reconstruction of x-ray transmission data from the same axial plane taken at different angles. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.
0
Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which consists of the letters "FDA" in a blue square, followed by the words "U.S. FOOD & DRUG" in blue, with the word "ADMINISTRATION" underneath in a smaller font.
December 3, 2024
Riverain Technologies, Inc. % Jonathan Jackson Director of RAOA 3130 South Tech Boulevard MIAMISBURG, OH 45342
Re: K242188
Trade/Device Name: ClearRead CT CAC Regulation Number: 21 CFR 892.1750 Regulation Name: Computed Tomography X-Ray System Regulatory Class: Class II Product Code: JAK Dated: November 13, 2024 Received: November 14, 2024
Dear Jonathan Jackson:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
1
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30. Design controls; 21 CFR 820.90. Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatory
2
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Lu Jiang
Lu Jiang, Ph.D. Assistant Director Diagnostic X-Ray Systems Team DHT8B: Division of Radiologic Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
3
Indications for Use
510(k) Number (if known) K242188
Device Name ClearRead CT CAC
Indications for Use (Describe)
ClearRead CT CAC is image processing software designed to aid physicians in assessing coronary artery calcification on non-gated, non-contrast, standard, or low-dose chest CT scans of adult patients 30 years of age or older. The primary output of ClearRead CT CAC is the Agatston score. ClearRead CT CAC locates calcified coronary lesions and assigns them to one of several coronary arteries: the right coronary artery (RCA), the left main artery (LM), the left anterior descending artery (LAD), and the left circumflex artery (LCX). The software calculates the Agatston score as the sum for each artery while also providing the total across all coronary arteries. The total Agatston score is calibrated to 3.0mm slice thickness and is used to derive a CAC category and the arterial age. The provided segmentations are for illustrative purposes only and are not intended for diagnostic use.
ClearRead CT CAC provides adjunctive information and is not intended to be used without clinical expert review.
| Type of Use (Select one or both, as applicable) | |
|---|---|
| ☑ Prescription Use (Part 21 CFR 801 Subpart D) | ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
4
K242188
510(k) Summary
Submitter Information:
| Company Name: | Riverain Technologies, Inc. |
|---|---|
| Company Address: | 3130 South Tech Blvd. |
| Miamisburg, OH 45342-4860 | |
| Contact Person: | Jonathan Jackson |
| Director of RAQA | |
| Riverain Technologies, Inc. | |
| 937.531.5092 |
Device Information:
| Trade Name: | ClearRead CT CAC |
|---|---|
| Regulation Number: | 21 CFR 892.1750 |
| Regulation Name: | Computed tomography x-ray system |
| Regulatory Class: | Class II |
| Product Code: | JAK |
| Device Description: | ClearRead CT CAC is a computer aided quantification system |
| intended to aid in the assessment of coronary artery calcification. | |
| The system receives chest Computed Tomography (CT) studies as | |
| input, in DICOM® format, and generates output in DICOM (or | |
| other) format. | |
| Intended Purpose: | ClearRead CT CAC is intended to aid physicians in assessing |
| Coronary Artery Calcification while reviewing non-gated, non- | |
| contrast, standard, and low-dose chest CT exams. | |
| Indications for Use: | ClearRead CT CAC is image processing software designed to aid |
| physicians in assessing coronary artery calcification on non-gated, | |
| non-contrast, standard, or low-dose chest CT scans of adult | |
| patients 30 years of age or older. | |
| The primary output of ClearRead CT CAC is the Agatston score. | |
| ClearRead CT CAC locates calcified coronary lesions and | |
| assigns them to one of several coronary arteries: the right coronary | |
| artery (RCA), the left main artery (LM), the left anterior | |
| descending artery (LAD), and the left circumflex artery (LCX). | |
| The software calculates the Agatston score as the sum for each |
5
artery while also providing the total across all coronary arteries. The total Agatston score is calibrated to 3.0mm slice thickness and is used to derive a CAC category and the arterial age. The provided segmentations are for illustrative purposes only and are not intended for diagnostic use.
ClearRead CT CAC provides adjunctive information and is not intended to be used without clinical expert review.
Bunkerhill, Inc. Predicate Device: (K230223) iCAC Device
Class II
Comparison to Predicate Device Technical Characteristics
Riverain Technologies, Inc. is of the opinion that ClearRead CT CAC is substantially equivalent, both in intended use as well as to the technical characteristics of the listed predicate device. Differences in the design and performance of the cited predicate device do not affect either the safety or the effectiveness of ClearRead CT CAC for its intended use. Table 1 shows the predicate devices listed along with the subject device, including the Product Code as well as the Indications for Use for each device.
| Predicate: | Subject Device: | |
|---|---|---|
| iCAC Device | ||
| (Bunkerhill, Inc.) | ClearRead CT CAC | |
| (Riverain Technologies, Inc.) | ||
| K230223 | ||
| Product | ||
| Code | JAK | JAK |
| Indications for | ||
| Use | iCAC is a software device | |
| intended for use in estimating | ||
| presence and quantity of | ||
| coronary artery calcium for | ||
| patients aged 30 years and above | ||
| during routine care. The device | ||
| automatically analyzes non- | ||
| gated, non-contrast chest | ||
| computed tomography (CT) | ||
| images collected during routine | ||
| care and outputs a visual | ||
| representation of estimated | ||
| coronary artery calcium | ClearRead CT CAC is image | |
| processing software designed to | ||
| aid physicians in assessing | ||
| coronary artery calcification on | ||
| non-gated, non-contrast, standard, | ||
| or low-dose chest CT scans of | ||
| adult patients 30 years of age or | ||
| older. | ||
| The primary output of ClearRead | ||
| CT CAC is the Agatston score. | ||
| ClearRead CT CAC locates | ||
| calcified coronary lesions and | ||
| assigns them to one of several | ||
| informational purposes only) and | ||
| both exact and four-category | ||
| quantitative estimates of the | ||
| patient's coronary artery calcium | ||
| burden in Agatston units. | ||
| The output of the subject device | ||
| is made available to the | ||
| physician on-demand as part of | ||
| his or her standard workflow. | ||
| The device generated calcium | ||
| score or score group can be | ||
| viewed in the patient report at | ||
| the discretion of the physician, | ||
| and the physician also has the | ||
| option of viewing the device- | ||
| generated calcium segmentation | ||
| in a diagnostic image viewer. | ||
| The subject device output in no | ||
| way replaces the original patient | ||
| report or the original chest CT | ||
| scan; both are still available to be | ||
| viewed and used at the discretion | ||
| of the physician. | ||
| The device is intended to provide | ||
| information to the physician to | ||
| provide assistance during review | ||
| of the patient's case. Results of | ||
| the subject device are not | ||
| intended to be used on a stand- | ||
| alone basis and are solely | ||
| intended to aid and provide | ||
| information to the physician. In | ||
| all cases, further action taken on | ||
| a patient should only come at the | ||
| recommendation of the physician | ||
| after further reviewing the | ||
| patient's results. | coronary arteries: the right | |
| coronary artery (RCA), the left | ||
| main artery (LM), the left anterior | ||
| descending artery (LAD), and the | ||
| left circumflex artery (LCX). The | ||
| software calculates the Agatston | ||
| score as the sum for each artery | ||
| while also providing the total | ||
| across all coronary arteries. The | ||
| total Agatston score is calibrated | ||
| to 3.0mm slice thickness and is | ||
| used to derive a CAC category and | ||
| the arterial age. The provided | ||
| segmentations are for illustrative | ||
| purposes only and are not intended | ||
| for diagnostic use. | ||
| ClearRead CT CAC provides | ||
| adjunctive information and is not | ||
| intended to be used without | ||
| clinical expert review. | ||
| Intended User | Radiologists | Radiologists |
| Modality | Routine, non-contrast, non-gated | |
| chest CT Series | Non-contrast, non-gated, standard, | |
| or low-dose chest CT scans | ||
| Anatomical | ||
| Region | Chest | Chest |
| Clinical | ||
| Condition | Coronary Artery Calcification | Coronary Artery Calcification |
| Quantification | ||
| of Calcium | ||
| Burden | Yes | Yes |
| Segmentation | ||
| of Calcium | Yes | Yes |
| Number of | ||
| Detection | ||
| Categories | 4 | 4 |
| Calculation of | ||
| Exact Calcium | ||
| Score | Yes | Yes |
| Type of | ||
| Information | Adjunctive | Adjunctive |
| Intended | ||
| location | Medical Facility | Medical Facility |
| Prescriptive | ||
| Use | Yes | Yes |
| Measurement | ||
| Scale | Agatston Units | Agatston Units |
| Image Format | DICOM | DICOM |
| Slice | ||
| Thickness | Up to 5mm | Up to 3mm |
| Calcification | ||
| Detection | Automatic | Automatic |
| Default | ||
| Threshold of | ||
| Calcium | 130 HU (Hounsfield Units) | 130 HU (Hounsfield Units) |
| Coronary | ||
| Artery | ||
| Calcification | ||
| Quantification | ||
| Method | CAC detection category (based | |
| on Agatston score), exact | ||
| Agatston score. | CAC detection category and | |
| arterial age based on the measured | ||
| Agatston score. | ||
| Annotation of | ||
| Detected | ||
| Calcium | Yes | Yes |
| Generate | ||
| Patient Report | Optional to copy result to | |
| clipboard, insert in report, | ||
| DICOM Secondary Capture | Optional to copy result to | |
| clipboard, insert in report, DICOM | ||
| Secondary Capture | ||
| Report of the | ||
| Calcium Score | Yes, Coronary Calcium | |
| Detection Category and exact | ||
| Agatston score 4 categories (for | ||
| detection category): | ||
| • 0 | ||
| • 1-99 | ||
| • 100-399 | ||
| • 400 + | Yes, Coronary Calcium Detection | |
| Category and measured Agatston | ||
| score 4 categories (for detection | ||
| category): | ||
| • 0 | ||
| • 1-99 | ||
| • 100-399 | ||
| • 400 + |
6
7
8
Traditional 510(k) Premarket Notification ClearRead™ CT CAC
Table 1: Predicate Devices vs. Subject Device
Key features identified that differ from the predicate are discussed below.
Slice Thickness
ClearRead CT CAC operates slice thickness up to 3mm, whereas the predicate device indicates use up to 5mm slice thickness. A limit of 3mm is in keeping with the current standards used for gated protocols, to which the Agatston score is calibrated to. For values exceeding 3mm, it is unclear how the performance of the measurement or detection of CAC scores degrades and, therefore, is excluded from consideration.
Testing Summary
Non-clinical Testing
Non-clinical tests were conducted during the development process in accordance with the Riverain Technologies Design Control Process, which is compliant with the FDA Quality System Regulations, ISO 13485:2016 with MDSAP, and the following standards.
- IEC 62304:2006/AMD1:2016, Medical devices - Software life cycle processes
- . IEC62366-1:2015, Medical device - Part1: Application of usability engineering to medical devices
- ISO14971:2019, Medical devices Application of Risk Assessment to Medical Devices ●
- NEMA PS 3.1-3-20, Digital Imaging and Communications in Medicine (DICOM) Set ● 2016
- Guidance for Industry and Food and Drug Administration Staff - Cybersecurity in Medical Devices: Ouality System Considerations and Content of Premarket Submissions. Issues September 27, 2023
Testing verified the requirements according to the ClearRead CT CAC device specifications.
The ClearRead CT CAC Risk Assessment was created for this device, with risk control measures implemented to mitigate identified hazards. Documentation required for software with
9
Traditional 510(k) Premarket Notification ClearRead™ CT CAC
a Moderate Level of Concern is included as part of this submission. Device labeling, together with the results from verification and validation testing, demonstrate that the device is safe and effective.
Approximately 400 cases were collected and held out from development of any kind to estimate the test performance of the CAC scoring system. Regions associated with coronary artery calcification were manually outlined, including the particular artery, from which the ground truth Agatston score was obtained.
To assess the measurement of the Agatston score, we used weighted kappa scores (weighted, owing to the ordinal nature of the categories) and Bland-Altman analysis. Pre-test acceptance criteria were defined based on existing literature and internal investigation from visually inspected validation performance. If kappa scores were greater than 0.85, these would be considered excellent. Actual CAC scores are those derived from the use of the ground truth mask, while predicted scores come from the ClearRead CT CAC system. Figure 1 below shows a plot of the predicted versus true CAC categories. A weighted kappa score of 0.958 was achieved, demonstrating good agreement between the automatic method and manual processing.
| Predicted | 65 | 5 | 0 | 0 |
|---|---|---|---|---|
| 0 | 115 | 2 | 0 | |
| 0 | 18 | 91 | 1 | |
| 0 | 0 | 10 | 95 |
Ground Truth
Figure 1: Confusion table of predicted versus ground truth CAC categories
A Bland-Altman plot, as shown in Figure 2, demonstrates strong agreement with the true score compared to the estimated score.
10
Image /page/10/Figure/2 description: This image is a scatter plot with the x-axis ranging from 0 to 4000 and the y-axis ranging from -1000 to 200. The plot includes a horizontal line indicating the mean at -19.64, with a confidence interval (CI) of -31.54 to -7.734. Additionally, there are two dashed lines representing the limits of agreement (LoA) at 163.1 (CI: 142.4-183.7) and -202.3 (CI: -223--181.7).
Figure 2: Bland-Altman analysis plot for the Agatston scores within the internal test set
Clinical Performance Testing
A clinical study was performed to validate the clinical efficacy of ClearRead CT CAC.
This study was a standalone model performance assessment of ClearRead CT CAC. This software, as a medical device product, was developed using machine learning to measure the amount of arterial calcification in non-contrast chest CT cases. This assessment involved comparing the algorithm outputs to ground-truth measurements on non-contrast, non-gated chest CT cases to determine the algorithm accuracy in automated measurements of the Agatston score (scaled to 3mm slice thickness). The Agatston score was measured overall and for the four component arteries of the right coronary artery, left anterior descending artery, left circumflex artery, and left main coronary artery.
In total, 491 cases were used in the clinical assessment of the device: 143 cases with no sign of coronary artery calcification and 348 with evidence of CAC, as determined by a consensus ground truth review by three radiologists. These nearly 500 cases included samples from four CT manufacturers across five clinical sites. In the CAC positive cases, selection was performed to ensure the entire range of scores was spanned.
For clinical assessment, the following predefined endpoints were used:
- . Primary endpoint: the accuracy of ClearRead CT CAC in calculating the Agatston score category (0, 1-99, 100-399, ≥400) as defined by the quadratic weighted Kappa coefficient when compared with ground truth radiologist Agatston score category.
- . Secondary endpoint: the accuracy of ClearRead CT CAC in calculating the overall continuous Agatston score as assessed by Bland-Altman analysis.
- Secondary endpoint: the accuracy of ClearRead CT CAC in segmenting the overall coronary artery calcification as assessed by Dice score comparison.
The results of the primary and secondary endpoints are as follows:
- . Primary endpoint: kappa of 0.959 (95% CI: 0.943-0.975). In addition to kappa scoring, percentage agreement scores for CAC positive (PPA) and negative (NPA) cases were
11
assessed. These calculations assessed the proportion of cases correctly identified in the relevant category (for PPA) or in all other categories (for NPA). Figure 3 summarizes all percentage agreement scores as stratified by CAC category.
- Secondary endpoint: Bland-Altman analysis (see Figure 4 for a data plot) yielding limits ● of agreement (LoA) of -12.20 ± 437.84.
- . Secondary endpoint: Dice score comparison (see Table 2) between the annotator ground truth and automatic outputs, giving average scores of 0.910 (0.893,0.927) and 0.892 (0.871,0.911), respectively.
| Ground truth radiologist category | |||
|---|---|---|---|
| Category 0 | Not category 0 | ||
| Device category | Category 0 | 126 | 3 |
| Not category 0 | 17 | 345 | |
| PPA 88.1 | |||
| 95% CI (82.5,93.2) | NPA 99.1 | ||
| 95% CI (98.0,100.0) | |||
| Ground truth radiologist category | |||
| Category 1-99 | Not category 1-99 | ||
| Device category | Category 1-99 | 116 | 17 |
| Not category 1-99 | 9 | 349 | |
| PPA 92.8 | |||
| 95% CI (87.7,96.9) | NPA 95.4 | ||
| 95% CI (93.0,97.4) | |||
| Ground truth radiologist category | |||
| Category 100-399 | Not category 100-399 | ||
| Device category | Category 100-399 | 74 | 12 |
| Not category 100-399 | 10 | 395 | |
| PPA 88.1 | |||
| 95% CI (80.5,94.4) | NPA 97.1 | ||
| 95% CI (95.2,98.5) | |||
| Ground truth radiologist category | |||
| Category ≥ 400 | Not category ≥ 400 | ||
| Device category | Category ≥ 400 | 137 | 6 |
| Not category ≥ 400 | 2 | 346 | |
| PPA 98.6 | NPA 98.3 |
Figure 3: Tabularized percentage agreement scores stratified by CAC category
12
Traditional 510(k) Premarket Notification ClearRead™ CT CAC
Image /page/12/Figure/2 description: The image is a Bland-Altman plot for the Agatston score. The x-axis represents the average of the ground truth interpretation and ClearRead CT CAC Agatston score, while the y-axis represents the difference between the ground truth interpretation and ClearRead CT CAC Agatston score. The plot includes a mean line at -12.20, an upper limit of agreement (LoA) at 425.65, and a lower LoA at -450.04.
Figure 4: Bland-Altman plot from clinical testing
| Average Dice score (95% CI) | Wilcoxon test | ||
|---|---|---|---|
| Device versus GT | GT versus GT | Difference | p-value |
| 0.892 (0.871,0.911) | 0.910 (0.893,0.927) | -0.008 (-0.024,0.007) | 0.121 |
Image /page/12/Figure/5 description: The image shows the title of a table. The title is "Table 2: Dice score comparison to the consensus ground truth". The title indicates that the table will compare dice scores to a consensus ground truth.
Conclusion
In preparing this 510(k) submission, Riverain Technologies has carefully considered the relevant statutory and regulatory requirements and believes that the information contained within satisfies the requirements for demonstrating substantial equivalence in terms of design features, fundamental technology, indications for use, and the safety and effectiveness of the device. Additionally, verification and validation testing demonstrate the safety and efficacy of the device to meet its intended use and specifications when operated as intended, with no detrimental impact on the benefit/risk ratio of the device.