(176 days)
BriefCase-Quantification is a software intended for use in the analysis of non-cardiac-gated non-contrast CT (NCCT) images that include the heart in adult patients aged 30 and older.
The device is intended to assist physicians by providing the user with a four-category Coronary Artery Calcification (CAC) of plaques, which present a risk for coronary artery disease, together with preview axial images of the detected calcium meant for informational purposes only.
The BriefCase-Quantification results are not intended to be used on a stand-alone basis for clinical decision-making or otherwise preclude clinical assessment of cases. Clinicians are responsible for viewing full images per the standard of care.
BriefCase-Quantification is a standalone software as a medical device intended for use in the analysis of non-cardiac-gated non-contrast CT (NCCT) images that include the heart to assist hospital networks and appropriately trained medical specialists. The software consists of a single module based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.
The BriefCase-Quantification receives filtered routine, non-gated computed tomography (CT) scans, and processes them chronologically by running the algorithm on relevant series to evaluate calcified plaques in the coronary arteries. Following the AI processing, the output of the algorithm analysis is transferred to an image review software (the PACS or a desktop application).
The device generates a four-category output corresponding with the estimated quantity of calcium detected: very low, low, medium, and high. The categories composing the output of the device correspond with a validated visual assessment categorization of none, mild, moderate, and severe [1] in agreement with categorized Agatston scores indicated in the literature (very low: 0; low: 1-100; medium: 101-400; high: ≥400). In addition, the categories accord with the 2016 SCCT/STR guidelines for coronary artery calcium scoring of non-contrast non-cardiac chest CT scans and are used as standard of care in clinical practice during CAC assessment in NCCT scans.
The BriefCase-Quantification software generates a preliminary summary report that is provided in the desktop application that includes applicable user warnings, the CAC detection category and number of slices that include CAC. The report presents preliminary results only and instructs the user to review the full image and any other clinical information before making a clinical decision. For all analyzed scans, the user will be presented in the PACS with all the slices containing the measured coronary calcifications. On these images, the calcified areas will be represented to provide the user visibility on the areas which supported the category output. These slices will be presented along with the original slices. Preview images of the represented calcium are non-diagnostic and are available in the PACS for informational purposes only.
Here's an analysis of the acceptance criteria and the study for the BriefCase-Quantification device, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Category | Acceptance Criteria (Performance Goal) | Reported Device Performance |
|---|---|---|
| Overall Agreement (Primary Endpoint) | "up to the prespecified performance goal" | 87.1% |
| Agreement for 'Very Low' CAC (Secondary Endpoint) | "up to the prespecified performance goal" | 95.1% |
| Agreement for 'Low' CAC (Secondary Endpoint) | "up to the prespecified performance goal" | 81.3% |
| Agreement for 'Medium' CAC (Secondary Endpoint) | "up to the prespecified performance goal" | 81.5% |
| Agreement for 'High' CAC (Secondary Endpoint) | "up to the prespecified performance goal" | 89.3% |
Note: The document states that the primary and secondary endpoints were achieved because the reported performance was "up to the prespecified performance goal," but it does not explicitly state the numerical value of the prespecified performance goal for each criterion.
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 433 cases
- Data Provenance: Retrospective, multicenter study from 6 US-based clinical sites (both academic and community centers). The cases were distinct in time or center from the cases used to train the algorithm.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications
- Number of Experts: At least one, potentially more. The text states "In cases where the reviewers disagree on the level of CAC, the senior US board-certified radiologist provided a final opinion which has established the ground truth." This implies primary reviewers and a senior radiologist for adjudication.
- Qualifications of Experts: Senior US board-certified radiologist (for final opinion/adjudication). The qualifications of initial reviewers are not specified.
4. Adjudication Method for the Test Set
- Adjudication Method: 2+1 (or similar) where a "senior US board-certified radiologist provided a final opinion" in cases of disagreement among initial reviewers.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, and its Effect Size
- A standalone performance study of the algorithm against ground truth was performed.
- The document does not mention a multi-reader multi-case (MRMC) comparative effectiveness study to assess how much human readers improve with AI vs without AI assistance. The study focuses on the algorithm's agreement with expert-established ground truth.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done
- Yes, a standalone study was done. The pivotal study evaluated "the software's performance in providing estimated coronary artery calcification detection category ... compared to the ground truth." The performance metrics (overall agreement and agreement per category) are for the algorithm's output directly compared to ground truth.
7. The Type of Ground Truth Used
- Type of Ground Truth: Expert consensus/adjudication. The text states, "In cases where the reviewers disagree on the level of CAC, the senior US board-certified radiologist provided a final opinion which has established the ground truth." The ground truth categories ([very low, low, medium, high] corresponding to Agatston scores) are also aligned with "validated visual assessment categorization of none, mild, moderate, and severe" and "2016 SCCT/STR guidelines."
8. The Sample Size for the Training Set
- The sample size for the training set is not provided in the document. The text only mentions that "The cases collected for the pivotal dataset were all distinct in time or center from the cases used to train the algorithm."
9. How the Ground Truth for the Training Set was Established
- The document does not explicitly describe how the ground truth for the training set was established. It only ensures that the training and test sets were distinct. While it's implied that similar expert-based ground truth would be used, the specifics are not detailed.
§ 892.1750 Computed tomography x-ray system.
(a)
Identification. A computed tomography x-ray system is a diagnostic x-ray system intended to produce cross-sectional images of the body by computer reconstruction of x-ray transmission data from the same axial plane taken at different angles. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.