(111 days)
Bunkerhill AVC is a software device intended for use in detecting presence and estimating quantity of aortic valve calcification for adult patients aged 40 years and above. The device automatically analyzes non-gated, non-contrast chest computed tomography (CT) images collected during clinical care and outputs the region of interest (intended for informational purposes only) and quantification of detected calcium.
The output of the subject device is made available to the physician on-demand as part of his or her standard workflow. The device-generated quantification can be viewed in the patient report at the discretion of the physician, and the physician also has the option of viewing the device-generated calcium region of interest in a diagnostic image viewer. The subject device output in no way replaces the original patient report or the original non-gated, non-contrast CT scan; both are still available to be viewed and used at the discretion of the physician.
The device is intended to provide information to the physician to provide assistance during review of the patient's case. Results of the subject device are not intended to be used on a stand-alone basis and are solely intended to aid and provide information to the physician. In all cases, further action taken on a patient should only come at the recommendation of the physician after further reviewing the patient's results.
Bunkerhill AVC is a software as a medical device (SaMD) product that interfaces with compatible and commercially available computed tomography (CT) systems. Bunkerhill AVC detects, localizes, and quantifies aortic valve calcification in non-gated, non-contrast chest CT studies. The core features of the product are:
- Detection of aortic valve calcification at an Agatston-equivalent score threshold of 0 AU. •
- . Estimation of the overall aortic valve calcification burden in the form of an estimated Agatston-equivalent Score.
- Localization of estimated calcium burden in the form of AVC region of interest applied . to a copy of the original CT scan.
Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:
1. A table of acceptance criteria and the reported device performance
The provided text describes specific performance metrics that were evaluated, although it doesn't explicitly present a formal "acceptance criteria table" with target values. Instead, it describes how the device's observed performance met the acceptance criteria.
| Metric (Performance Type) | Acceptance Criteria (Implied) | Reported Device Performance |
|---|---|---|
| Primary Endpoint: | ||
| Bias (Bland Altman Agreement) | Low magnitude bias, similar to predicate device performance and clinical literature inter-reader agreement. | -5.15 AU |
| Lower Limit of Agreement (Bland Altman Agreement) | Within acceptable clinical limits, similar to predicate device performance and clinical literature inter-reader agreement. | -200.96 AU |
| Upper Limit of Agreement (Bland Altman Agreement) | Within acceptable clinical limits, similar to predicate device performance and clinical literature inter-reader agreement. | 190.65 AU |
| Secondary Endpoints: | ||
| Precision (Circular ROI) | Met acceptance criteria. | 0.826 (95% CI: 0.784, 0.863) |
| Recall (Circular ROI) | Met acceptance criteria. | 0.855 (95% CI: 0.818, 0.890) |
Notes on Acceptance Criteria: The document states that "The acceptance criteria were derived from the performance of the predicate device and clinical literature in high impact journals that inter-reader agreement of manual segmentation." This indicates a benchmark against established clinical practice and a comparable device.
2. Sample sized used for the test set and the data provenance
- Sample Size for Test Set: Not explicitly stated as a numerical count of patients or cases. However, the data for the pivotal study was "curated from thirty-three (33) sites."
- Data Provenance:
- Country of Origin: United States ("thirty-three (33) sites across three geographical regions in the United States").
- Retrospective or Prospective: Retrospective ("standalone retrospective study").
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
The document does not provide information on the number or qualifications of experts used to establish the ground truth for the test set. It only mentions that the ground truth was "established."
4. Adjudication method for the test set
The document does not specify any adjudication method (e.g., 2+1, 3+1, none) used for the test set.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
A multi-reader multi-case (MRMC) comparative effectiveness study assessing human reader improvement with AI assistance was not conducted or reported. The study described is a "standalone retrospective study for localization and agreement of the device output compared to the established ground truth." The device is intended as an "adjunctive information" tool, not a human-in-the-loop performance enhancer for diagnostic accuracy per se, but rather an aid for quantifying calcification.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done
Yes, a standalone study was done. The text explicitly states: "The Bunkerhill AVC performance was validated in a stand-alone retrospective study for localization and agreement of the device output compared to the established ground truth."
7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)
The type of ground truth used was "established ground truth." While this term is somewhat generic, given the context of Agatston-equivalent scores and "inter-reader agreement of manual segmentation" mentioned for acceptance criteria, it strongly implies ground truth established by expert (likely radiologist or cardiologist) review and manual measurement/segmentation. It is not stated to be pathology or outcomes data.
8. The sample size for the training set
The document does not provide the sample size for the training set. It only discusses the pivotal test set.
9. How the ground truth for the training set was established
The document does not provide information on how the ground truth for the training set was established.
§ 892.1750 Computed tomography x-ray system.
(a)
Identification. A computed tomography x-ray system is a diagnostic x-ray system intended to produce cross-sectional images of the body by computer reconstruction of x-ray transmission data from the same axial plane taken at different angles. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.