(259 days)
The HealthCCS Device is intended for use as a non-invasive post-processing software that can be used to evaluate calcified plaques in the coronary arteries, which may be a risk factor for coronary artery disease. The software can be used to generate reports of the total risk category of coronary calcium. This information can then be used by a physician for further analysis and treatment. The HealthCCS Device analyses pre-existing heart or chest ECG-Gated Triggered CT scans. The Device is indicated for use only on patients whose age at the CT scan was taken, was above 20 years old. This device generates a 4-category Agatston-equivalent risk score, and the patient management, especially for the patient with the score from 0-10, will depend on the physician's own judgment. It may require further testing to evaluate the appropriate clinical management.
The HealthCCS Device is an automatic non-invasive post processing tool that uses cardiac CT images to identify and quantify calcification in the coronary arteries, known to be a risk factor for coronary disease. HealthCCS Device quantifies calcification on non-contrast cardiac computed tomography (CT) scans. HealthCCS Device calculates the amount of identified calcification and reports the risk category of coronary calcium. This information can then be used by a physician for further analysis and treatment.
The provided document describes the HealthCCS device, a non-invasive post-processing software for evaluating calcified plaques in coronary arteries. Here's a breakdown of the acceptance criteria and the study that proves the device meets those criteria:
1. Table of Acceptance Criteria and Reported Device Performance
The document explicitly states the overall acceptance criterion as "adequate overall agreement" and "adequate agreement per category." It then reports the achieved performance against these criteria.
| Acceptance Criteria | Reported Device Performance | Comments |
|---|---|---|
| Overall Agreement | 0.89 (95% CI: [0.85, 0.92]) | Achieved an "adequate overall agreement." |
| Agreement per Category | Adequate agreement per category | Implied to be met, but no specific values for each category are provided. |
| Reproducibility (Agatston equivalent scores) | Identical over all three readings | Assessed on 150 studies read three times. |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 249 studies.
- Data Provenance: The document states a "retrospective performance study." No specific country of origin is mentioned for the data, but the applicant's address is Israel.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications
- Number of Experts: 3 radiologists.
- Qualifications of Experts: Not specified in the document beyond "radiologists." No information on their years of experience or subspecialty focus is provided.
4. Adjudication Method for the Test Set
The ground truth was established by "3 radiologists using the Kodak Carestream PACS device (K053347)." This implies a consensus or majority rule approach, but the specific adjudication method (e.g., 2+1, 3+1) is not explicitly stated. It seems to be a form of expert consensus derived from their use of a reference device.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size
- A formal MRMC comparative effectiveness study comparing human readers with and without AI assistance was not described. The study described focused on the agreement between the HealthCCS device's categorization and the ground truth established by radiologists using a reference PACS device. It was a comparison of the AI's output against human-derived ground truth, not an assessment of human performance improvement with AI assistance.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Was Done
- Yes, the performance study described is essentially a standalone (algorithm only) performance study. The HealthCCS device's output (4-level risk categorization) was directly compared to the ground truth established by the radiologists. There was no human-in-the-loop component described for this performance evaluation.
7. The Type of Ground Truth Used
- Expert Consensus (using a reference device): The ground truth was established by "3 radiologists using the Kodak Carestream PACS device (K053347)." This indicates that the radiologists performed the calcification scoring using a legally marketed device, and their determinations served as the "ground truth" for comparison.
8. The Sample Size for the Training Set
- The document does not specify the sample size used for the training set. It only details the performance validation study (test set).
9. How the Ground Truth for the Training Set Was Established
- The document does not provide information on how the ground truth for the training set was established. The focus is solely on the validation of the device's performance using a test set. However, a CNN-based probability threshold is mentioned, implying that the model was trained on data with ground truth about what constitutes coronary arterial calcification.
§ 892.1750 Computed tomography x-ray system.
(a)
Identification. A computed tomography x-ray system is a diagnostic x-ray system intended to produce cross-sectional images of the body by computer reconstruction of x-ray transmission data from the same axial plane taken at different angles. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.