Search Results

Manufacturer

Date Cleared

2025-02-14

(18 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K213886

Intended Use

BriefCase-Triage is a radiological computer aided triage and notification software indicated for use in the analysis of contrast-enhanced images that include the lungs in adults or transitional adolescents age 18 and older. The device is intended to assist hospital networks and appropriately trained medical specialists in workflow triage by flagging and communication of suspect cases of incidental Pulmonary Embolism (iPE) pathologies.

BriefCase-Triage uses an artificial intelligence algorithm to analyze images and highlight cases with detected findings in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for suspect cases. Notifications include compressed preview images that are meant for informational purposes only and not intended for diagnostic use beyond notification. The device does not alter the original medical image and is not intended to be used as a diagnostic device.

The results of BriefCase-Triage are intended to be used in conjunction with other patient information and based on their professional judgment, to assist with triage/prioritization of medical images. Notified clinicians are responsible for viewing full images per the standard of care.

Device Description

Briefcase-Triage is a radiological computer-assisted triage and notification software device. The software is based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.

The Briefcase-Triage receives filtered DICOM Images, and processes them chronologically by running the algorithms on each series to detect suspected cases. Following the AI processing, the output of the algorithm analysis is transferred to an image review software (desktop application). When a suspected case is detected, the user receives a pop-up notification and is presented with a compressed, low-quality, grayscale image that is captioned "not for diagnostic use, for prioritization only" which is displayed as a preview function. This preview is meant for informational purposes only, does not contain any marking of the findings, and is not intended for primary diagnosis beyond notification.

Presenting the users with worklist prioritization facilitates efficient triage by prompting the user to assess the relevant original images in the PACS. Thus, the suspect case receives attention earlier than would have been the case in the standard of care practice alone.

AI/ML Overview

Here's a breakdown of the acceptance criteria and the study details for the BriefCase-Triage device, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

Metric	Acceptance Criteria (Performance Goal)	Reported Device Performance (Default Operating Point)	95% Confidence Interval
Sensitivity	≥ 80%	91.7%	[87.5%, 94.9%]
Specificity	≥ 80%	91.4%	[87.3%, 94.6%]
NPV	Not specified	99.8%	[99.6%, 99.8%]
PPV	Not specified	22.2%	[16.1%, 29.9%]
PLR	Not specified	10.7	[7.2, 16.0]
NLR	Not specified	0.09	[0.06, 0.14]
Time-to-Notification	Comparability to predicate (282.63s)	40.2 seconds	[36.9, 43.5] (for subject device)

Note on Additional Operating Points (AOPs): The document also describes four additional operating points (AOPs 1-4) designed to maximize specificity while maintaining a lower bound 95% CI of 80% for sensitivity. These AOPs achieved similar performance metrics, further demonstrating the device's flexibility.

2. Sample Size Used for the Test Set and Data Provenance

Sample Size: 498 cases
Data Provenance: Retrospective, blinded, multicenter study. Cases were collected from 6 US-based clinical sites. The test data was distinct in time or center from the cases used for algorithm training and was sequestered from algorithm development activities.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications

Number of Experts: Three (3)
Qualifications of Experts: Senior board-certified radiologists.

4. Adjudication Method for the Test Set

The specific adjudication method (e.g., 2+1, 3+1) is not explicitly stated in the provided text. It only mentions that the ground truth was "determined by three senior board-certified radiologists."

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done

A Multi-Reader Multi-Case (MRMC) comparative effectiveness study comparing AI assistance versus without AI assistance for human readers was not described for this device. The study primarily evaluated the standalone performance of the AI algorithm and its time-to-notification compared to the predicate device, not the improvement in human reader performance with AI assistance.

6. If a Standalone (Algorithm Only Without Human-in-the-loop Performance) was done

Yes, a standalone performance study was done. The "Pivotal Study Summary" sections describe the device's sensitivity, specificity, and other metrics based on its algorithm's performance in identifying iPE cases compared to the ground truth established by radiologists. The device is intended to provide notifications and not for diagnostic use, suggesting its performance is evaluated in its autonomous triage function.

7. The Type of Ground Truth Used

The ground truth used was expert consensus, specifically "determined by three senior board-certified radiologists."

8. The Sample Size for the Training Set

The sample size for the training set is not explicitly stated in the provided text. It only mentions that the algorithm was "trained during software development on images of the pathology."

9. How the Ground Truth for the Training Set Was Established

The ground truth for the training set was established through manual labeling ("tagging") by experts. The text states: "deep learning algorithm development consisted of training on manually labeled ("tagged") images. In that process, critical findings were tagged in all CTs in the training data set."

Ask a Question

Ask a specific question about this device

K Number

K243548

Device Name

Manufacturer

Date Cleared

2024-12-11

(26 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K230020

Intended Use

BriefCase-Triage is a radiological computer aided triage and notification software indicated for use in the analysis of CT images with or without contrast that include the ribs, in adults or transitional adolescents aged 18 and older. The device is intended to assist hospital networks and appropriately trained medical specialists in workflow triage by flagging and communicating suspect cases of three or more acute Rib fracture (RibFx) pathologies.

BriefCase-Triage uses an artificial intelligence algorithm to analyze images and highlight cases with detected findings in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for cases with suspected RibFx findings. Notifications include compressed preview images that are meant for informational purposes only, and not intended for diagnostic use beyond notification. The device does not alter the original medical image and is not intended to be used as a diagnostic device.

The results of BriefCase-Triage are intended to be used in conjunction with other patient information and based on their professional judgment, to assist with triage/prioritization of medical images. Notified clinicians are responsible for viewing full images per the standard of care.

Device Description

BriefCase-Triage is a radiological computer-assisted triage and notification software device. The software is based on an algorithm programmed component and is intended to run on a linuxbased server in a cloud environment.

The BriefCase-Triage receives filtered DICOM Images, and processes them chronologically by running the algorithms on each series to detect suspected cases. Following the Al processing, the output of the algorithm analysis is transferred to an image review software (desktop application). When a suspected case is detected, the user receives a pop-up notification and is presented with a compressed, low-quality, grayscale image that is captioned "not for diagnostic use, for prioritization only" which is displayed as a preview function. This preview is meant for informational purposes only, does not contain any marking of the findings, and is not intended for primary diagnosis beyond notification.

Presenting the users with worklist prioritization facilitates efficient triage by prompting the user to assess the relevant original images in the PACS. Thus, the suspect case receives attention earlier than would have been the case in the standard of care practice alone.

AI/ML Overview

Acceptance Criteria and Device Performance for BriefCase-Triage

1. Table of Acceptance Criteria and Reported Device Performance

Metric	Acceptance Criteria (Performance Goal)	Reported Device Performance
AUC	> 0.95	97.2% (95% Cl: 95.5%-99.0%)
Sensitivity	> 80%	95.2% (95% Cl: 89.1%-98.4%)
Specificity	> 80%	95.1% (95% Cl: 91.2%-97.6%)
Time-to-notification (Mean)	Comparability with predicate (70.1 seconds)	41.4 seconds (95% Cl: 40.4-42.5)

Note: The acceptance criteria for sensitivity and specificity are extrapolated from the statement "As the AUC exceeded 0.95 and sensitivity and specificity both exceeded 80%, the study's primary endpoints were met."

2. Sample Size and Data Provenance for Test Set

Sample Size for Test Set: 308 cases
Data Provenance: Retrospective, multicenter study from 5 US-based clinical sites. The cases collected for the pivotal dataset were distinct in time or center from the cases used to train the algorithm.

3. Number and Qualifications of Experts for Ground Truth

Number of Experts: Three senior board-certified radiologists.
Qualifications: Senior board-certified radiologists. (Specific years of experience are not provided in the document).

4. Adjudication Method for Test Set

The adjudication method is not explicitly stated. The document mentions "ground truth, as determined by three senior board-certified radiologists," but does not detail how disagreements among these radiologists were resolved (e.g., 2+1, 3+1, or simple consensus).

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No MRMC comparative effectiveness study was done to assess the effect of AI assistance on human readers' performance. The study focused on the standalone performance of the AI algorithm and its time-to-notification compared to a predicate device.

6. Standalone Performance Study

Yes, a standalone performance study was done. The "Pivotal Study Summary" section explicitly details the evaluation of the software's performance (AUC, sensitivity, specificity, PPV, NPV, PLR, NLR) in identifying RibFx without human intervention, comparing it to the established ground truth.

7. Type of Ground Truth Used

The ground truth used was expert consensus, determined by three senior board-certified radiologists.

8. Sample Size for Training Set

The sample size for the training set is not explicitly provided in the document. The document states, "The algorithm was trained during software development on images of the pathology. As is customary in the field of machine learning, deep learning algorithm development consisted of training on labeled ("tagged") images." It also mentions, "The cases collected for the pivotal dataset were all distinct in time or center from the cases used to train the algorithm, as was used for the most recent clearance (K230020)."

9. How Ground Truth for Training Set Was Established

The ground truth for the training set was established by labeling ("tagging") images based on the presence of the critical finding (three or more acute Rib fractures). This process is described as "each image in the training dataset was tagged based on the presence of the critical finding." The document does not specify who performed this tagging or the exact methodology for establishing the ground truth for the training set (e.g., expert consensus, pathology).

Ask a Question

Ask a specific question about this device

K Number

K242203

Device Name

Manufacturer

Date Cleared

2024-11-22

(119 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K241112

Intended Use

BriefCase-Quantification is a radiological image management and processing system software indicated for use in the analysis of contrast-enhanced CT exams that include the aorta in adults or transitional adolescents aged 18 and older.

BriefCase-Quantification of Aortic Measurement (M-Aorta) is intended to assist hospital networks and appropriately trained medical specialists by providing the user with aortic diameter measurements across the aorta. BriefCase-Quantification is indicated to evaluate normal and aneurysmal aortas and is not intended to evaluate post-operative aortas.

The device provides the following assessments:

Aortic measurements at 10 anatomical landmarks;
Maximum aortic diameter of the abdominal aorta, descending aorta and ascending aorta.

The BriefCase-Quantification results are not intended to be used on a stand-alone basis for clinical decision-making or otherwise preclude clinical assessment of cases. These measurements are unofficial, are not final, and are subject to change after review by a radiologist. For final clinically approved measurements, please refer to the official radiology report. Clinicians are responsible for viewing full images per the standard of care.

Device Description

BriefCase-Quantification is a radiological medical image management and processing device. The software consists of a single module based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.

The BriefCase-Quantification receives filtered DICOM Images, and processes them chronologically by running the algorithm on relevant series to measure the maximum abdominal aortic diameter. Following the AI processing, the output of the algorithm analysis is transferred to an image review software (desktop application), and forwarded to user review in the PACS.

The BriefCase-Quantification produces images and tabular views of the landmarks and segments selected per installation to be displayed in the image review software, provided by the Image Communication Platform integrated with Briefcase-Quantification. The diameter marking is not intended to be a final output, but serves the purpose of visualization and measurement. The original, unmarked series remains available in the PACS as well. The preview image presents an unofficial and not final measurement, and the user is instructed to review the full image and any other clinical information before making a clinical decision. The image includes a disclaimer: "Not for diagnostic use. The measurement is unofficial, not final, and must be reviewed by a radiologist."

BriefCase-Quantification is not intended to evaluate post-operative aortas.

AI/ML Overview

Here's an analysis of the acceptance criteria and study proving device performance, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

Metric / Acceptance Criteria	Reported Device Performance
Primary Endpoint: Mean Absolute Error (MAE) between ground truth measurement and algorithm across all landmarks and studies.	1.88 mm (95% CI: 1.78 mm, 1.99 mm).
(This was "below the prespecified performance goal," thus achieving the primary endpoint. The reported MAE of the subject device (1.88 mm) was comparable to the predicate device [1.95 mm (95% CI: 1.59 mm, 2.32 mm)].)
Secondary Endpoint: Bias between ground truth and algorithm output (Mean Difference in Bland-Altman analysis).	0.1 mm (This indicates "little to no bias between the two measurements," demonstrating the study's secondary endpoint was achieved.)

2. Sample Size and Data Provenance

Test Set Sample Size: 212 cases
Data Provenance: Retrospective, multicenter study. Cases were collected from 6 US-based clinical sites, including both academic and community centers. The cases were distinct in time and/or center from those used to train the algorithm.

3. Number of Experts and Qualifications for Ground Truth

Number of Experts: Three (3)
Qualifications: US board-certified radiologists.

4. Adjudication Method for the Test Set

The document states the ground truth was "determined by three US board-certified radiologists." While it doesn't explicitly detail a 2+1 or 3+1 method, the implication of "determined by" three experts suggests a consensus-based approach, likely one where all three agreed or a majority agreement was required for the final ground truth. It does not mention any specific adjudication process explicitly (e.g., if disagreements were resolved by a fourth reader).

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

Was an MRMC study done? No. The study described is a standalone (algorithm-only) performance evaluation against expert-established ground truth. It does not compare human readers with and without AI assistance.
Effect size of improvement: Not applicable, as no MRMC study was performed.

6. Standalone (Algorithm-Only) Performance

Was a standalone performance study done? Yes. The "Pivotal Study Summary" describes the evaluation of "the software's performance ... compared to the ground truth," focusing on the algorithm's accuracy in measurement against expert consensus.

7. Type of Ground Truth Used

Type of Ground Truth: Expert consensus. Specifically, the ground truth was "determined by three US board-certified radiologists."

8. Sample Size for the Training Set

The document explicitly states: "The cases collected for the pivotal dataset were all distinct in time and/or center from the cases used to train the algorithm." However, the exact sample size for the training set is not provided in the given text.

9. How the Ground Truth for the Training Set was Established

The document does not detail how the ground truth for the training set was established. It only mentions that the training cases were distinct from the test set cases.

Ask a Question

Ask a specific question about this device

K Number

K242837

Device Name

Manufacturer

Date Cleared

2024-10-18

(29 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

N/A

Intended Use

BriefCase-Triage is a radiological computer aided triage and notification software indicated for use in the analysis of CT scans that include the cervical spine, in adults or transitional adolescents aged 18 and older. The device is intended to assist hospital networks and appropriately trained medical specialists in workflow triage by flagging and communication of linear lucencies in the cervical spine bone in patterns compatible with fractures.

BriefCase-Triage uses an artificial intelligence algorithm to analyze images and highlight cases with detected findings in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for cases with suspected findings. Notifications include compressed preview images that are meant for informational purposes only and not intended for diagnostic use beyond notification. The device does not alter the original medical image and is not intended to be used as a diagnostic device.

The results of BriefCase-Triage are intended to be used in conjunction with other patient information and based on their professional judgment, to assist with triage/prioritization of medical images. Notified clinicians are responsible for viewing full images per the standard of care.

Device Description

BriefCase-Triage is a radiological computer-assisted triage and notification software device.

The software is based on an algorithm programmed component and is intended to run on a linuxbased server in a cloud environment.

The BriefCase-Triage receives filtered DICOM Images, and processes them chronologically by running the algorithms on each series to detect suspected cases. Following the Al processing, the output of the algorithm analysis is transferred to an image review software (desktop application). When a suspected case is detected, the user receives a pop-up notification and is presented with a compressed, low-quality, grayscale image that is captioned "not for diagnostic use, for prioritization only" which is displayed as a preview function. This preview is meant for informational purposes only, does not contain any marking of the findings, and is not intended for primary diagnosis beyond notification.

Presenting the users with worklist prioritization facilitates efficient triage by prompting the user to assess the relevant original images in the PACS. Thus, the suspect case receives attention earlier than would have been the case in the standard of care practice alone.

The algorithm was trained during software development on images of the pathology. As is customary in the field of machine learning algorithm development consisted of training on manually labeled ("tagged") images. In that process, critical findings were tagged in all CTs in the training data set.

AI/ML Overview

The Aidoc BriefCase-Triage device, intended for triaging cervical spine CT scans for fractures, underwent a retrospective, blinded, multicenter study to evaluate its performance.

Here's a breakdown of the acceptance criteria and study details:

1. A table of acceptance criteria and the reported device performance

Acceptance Criteria	Reported Device Performance
Sensitivity (performance goal ≥ 80%)	92.1% (95% CI: 87.5%, 95.4%)
Specificity (performance goal ≥ 80%)	92.6% (95% CI: 89.0%, 95.4%)
Time-to-Notification (comparable to predicate)	15.1 seconds (95% CI: 14.1-16.2)

2. Sample size used for the test set and the data provenance

Sample Size: 487 cases
Data Provenance: Retrospective, multicenter study from 6 US-based clinical sites. The cases collected for the pivotal dataset were all distinct in time or center from the cases used to train the algorithm.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

Number of Experts: Three
Qualifications: Senior board-certified radiologists

4. Adjudication method for the test set

The document does not explicitly state the adjudication method (e.g., 2+1, 3+1). However, it implies that the ground truth was "as determined by three senior board-certified radiologists," suggesting a consensus-based approach among these experts.

5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

There is no MRMC comparative effectiveness study presented in this document that evaluates human readers' improvement with AI assistance versus without AI assistance. The study focuses on the standalone performance of the AI device and its time-to-notification compared to a predicate device.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone algorithm-only performance study was done. The primary endpoints (sensitivity and specificity) and secondary endpoints (time-to-notification, PPV, NPV, PLR, NLR) directly measure the performance of the BriefCase-Triage software itself.

7. The type of ground truth used

The ground truth was established by expert consensus (determined by three senior board-certified radiologists).

8. The sample size for the training set

The document states, "The algorithm was trained during software development on images of the pathology." However, it does not specify the sample size for the training set. It only mentions that the test pivotal study data was sequestered from algorithm development activities.

9. How the ground truth for the training set was established

The ground truth for the training set was established by manually labeled ("tagged") images. The document states, "critical findings were tagged in all CTs in the training data set." This implies expert annotation of the training data.

Ask a Question

Ask a specific question about this device

K Number

K241727

Device Name

Manufacturer

Date Cleared

2024-07-12

(28 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K222277

Intended Use

Briefcase-Triage is a radiological computer- aided triage and notification software indicated for use in the analysis of CTPA images, in adults or transitional adolescents aged 18 and older. The device is intended to assist hospital networks and appropriately trained medical specialists in workflow triage by flagging and communicating suspected positive cases of Pulmonary Embolism (PE) pathologies.

Briefcase-Triage uses an artificial intelligence algorithm to analyze images and highlight cases with detected findings in parallel to the ongoing standard of care image interpretation. The user is presented with notifications for cases with suspected PE findings. Notifications include compressed preview images that are meant for informational purposes only and not intended for diagnostic use beyond notification. The device does not alter the original medical image and is not intended to be used as a diagnostic device.

The results of Briefcase-Triage are intended to be used in conjunction with other patient information and based on their professional judgment, to assist with triage/prioritization of medical images. Notified clinicians are responsible for viewing full images per the standard of care.

Device Description

Briefcase-Triage is a radiological computer-assisted triage and notification software device. The software is based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.

The Briefcase-Triage receives filtered DICOM Images, and processes them chronologically by running the algorithms on each series to detect suspected cases. Following the Al processing, the output of the algorithm analysis is transferred to an image review software (desktop application). When a suspected case is detected, the user receives a pop-up notification and is presented with a compressed, low-quality, grayscale image that is captioned "not for diagnostic use, for prioritization only" which is displayed as a preview function. This preview is meant for informational purposes only, does not contain any marking of the findings, and is not intended for primary diagnosis beyond notification.

Presenting the users with worklist prioritization facilitates efficient triage by prompting the user to assess the relevant original images in the PACS. Thus, the suspect case receives attention earlier than would have been the case in the standard of care practice alone.

AI/ML Overview

Here's a breakdown of the acceptance criteria and study details for the Aidoc BriefCase-Triage device, based on the provided document:

1. Table of Acceptance Criteria and Reported Device Performance

Device Name: BriefCase-Triage (for Pulmonary Embolism - PE)

Acceptance Criteria	Performance Goal	Reported Device Performance (Default Operating Point)
Sensitivity	≥ 80%	94.39% (95% CI: 90.41%, 97.07%)
Specificity	≥ 80%	94.39% (95% CI: 91.04%, 96.67%)
Time-to-notification (compared to predicate)	Comparable benefit in time saving	Mean 26.42 seconds (95% CI: 25.3-27.54) vs Predicate's 78.0 seconds (95% CI: 73.6-82.3)

Additional Operating Points (AOPs) Performance:

Operating Point	Sensitivity (95% CI)	Specificity (95% CI)
AOP1	99.53% (97.42%-99.99%)	86.67% (82.16%-90.39%)
AOP2	97.66% (94.63%-99.24%)	91.93% (88.14%-94.82%)
AOP3	91.59% (87.03%-94.94%)	96.49% (93.64%-98.3%)
AOP4	85.98% (80.6%-90.34%)	98.25% (95.95%-99.43%)

Other reported secondary endpoints for the default operating point:

NPV: 98.96% (95% CI: 98.21%- 99.4%)
PPV: 74.79% (95% CI: 64.80%- 82.7%)
PLR: 16.81 (95% CI: 10.43- 27.09)
NLR: 0.059 (95% CI: 0.034- 0.103)

2. Sample Size and Data Provenance for the Test Set

Sample Size: 499 cases
Data Provenance: Retrospective, multicenter study from 6 US-based clinical sites. The cases were distinct in time or center from the cases used to train the algorithm.

3. Number of Experts and Qualifications for Ground Truth

Number of Experts: Three (3)
Qualifications: Senior board-certified radiologists.

4. Adjudication Method

The document states "the ground truth as determined by three senior board-certified radiologists." It does not explicitly state the adjudication method (e.g., 2+1, consensus, majority vote). However, "determined by" implies that their collective judgment established the ground truth for each case.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly described. The study focuses on the standalone performance of the AI algorithm against a ground truth established by experts, and a comparison of its notification time with a predicate device. It does not evaluate human reader performance with and without AI assistance.

6. Standalone Performance Study

Yes, a standalone study (algorithm only without human-in-the-loop performance) was conducted. The primary endpoints (sensitivity and specificity) evaluated the device's ability to identify PE cases independently.

7. Type of Ground Truth Used

Expert Consensus: The ground truth was "determined by three senior board-certified radiologists"

8. Sample Size for the Training Set

The document states that the algorithm was "trained during software development on images of the pathology" and that the subject device was trained on a "larger data set" compared to the predicate. However, it does not specify the exact sample size of the training set.

9. How Ground Truth for the Training Set was Established

"critical findings were tagged in all CTs in the training data set." This implies manual labeling (annotation) of findings by experts on the training images. While not explicitly stated, it is common practice that these tags are done by medical professionals.

Ask a Question

Ask a specific question about this device

K Number

K241112

Device Name

Manufacturer

Date Cleared

2024-05-15

(23 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K230534

Intended Use

BriefCase-Quantification is a radiological image management and processing system software indicated for use in the analysis of CT exams with contrast, that include the abdominal aorta, in adults or transitional adolescents aged 18 and older.

The device is intended to assist appropriately trained medical specialists by providing the user with the maximum abdominal aortic diameter measurement of cases that include the abdominal aorta (M-AbdAo). BriefCase-Quantification is indicated to evaluate normal and aneurysmal abdominal aortas and is not intended to evaluate post-operative aortas.

The BriefCase-Quantification results are not intended to be used on a stand-alone basis for clinical decision-making or otherwise preclude clinical assessment of cases. These measurements are unofficial, are not final, and are subject to change after review by a radiologist. For final clinically approved measurements, please refer to the official radiology report. Clinicians are responsible for viewing full images per the standard of care.

Device Description

BriefCase-Quantification is a radiological medical image management and processing device. The software consists of a single module based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.

The BriefCase-Quantification receives filtered DICOM Images, and processes them chronologically by running the algorithm on relevant series to measure the maximum abdominal aortic diameter. Following the AI processing, the output of the algorithm analysis is transferred to an image review software (desktop application), and forwarded to user review in the PACS.

BriefCase-Quantification produces a preview image annotated with the maximum diameter measurement. The diameter marking is not intended to be a final output, but serves the purpose of visualization and measurement. The original, unmarked series remains available in the PACS as well. The preview image presents an unofficial measurement which is not final, and the user is instructed to review the full image and any other clinical information before making a clinical decision. The image includes a disclaimer: "Not for diagnostic use. The measurement is unofficial, not final, and must be reviewed by a radiologist."

BriefCase-Quantification is not intended to evaluate post-operative aortas.

AI/ML Overview

Here's a breakdown of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Device Performance

Acceptance Criteria (Performance Goal)	Reported Device Performance
Mean Absolute Error (MAE) between ground truth and algorithm measurement below a prespecified goal.	1.52 mm (95% Cl: 1.20 mm, 1.83 mm)
Little to no bias between ground truth and algorithm output.	Mean difference of 0.58 mm

Study Details

1. A table of acceptance criteria and the reported device performance:
See table above.

2. Sample size used for the test set and the data provenance:

Sample Size (Test Set): 162 cases
Data Provenance: Retrospective, from 6 US-based clinical sites (both academic and community centers). The cases were distinct in time and/or center from the training data.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

Number of Experts: Three (3)
Qualifications: US board-certified radiologists

4. Adjudication method for the test set:
The text states the ground truth was "as determined by three US board-certified radiologists." It doesn't explicitly specify an adjudication method like 2+1 or 3+1, but implies a consensus or agreement among the three experts formed the ground truth.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
No, a multi-reader multi-case (MRMC) comparative effectiveness study of human readers with and without AI assistance was not conducted. The study focused on the standalone performance of the algorithm against a human-established ground truth.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
Yes, a standalone study of the algorithm's performance was done. The study evaluated the BriefCase-Quantification software's performance in providing maximum diameter measurements compared to a ground truth established by radiologists.

7. The type of ground truth used:
Expert consensus among three US board-certified radiologists.

8. The sample size for the training set:
The sample size for the training set is not explicitly stated. The text only mentions that the subject device's improved performance is "due to its training on a larger data set" compared to the predicate device, but does not provide a specific number for this larger data set. It also states the test set cases were "distinct in time and/or center from the cases used to train the algorithm."

9. How the ground truth for the training set was established:
The text does not explicitly describe how the ground truth for the training set was established. It only refers to a "larger data set" used for training.

Ask a Question

Ask a specific question about this device

K Number

K231631

Device Name

Manufacturer

Date Cleared

2023-11-28

(176 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K230223

Intended Use

BriefCase-Quantification is a software intended for use in the analysis of non-cardiac-gated non-contrast CT (NCCT) images that include the heart in adult patients aged 30 and older.

The device is intended to assist physicians by providing the user with a four-category Coronary Artery Calcification (CAC) of plaques, which present a risk for coronary artery disease, together with preview axial images of the detected calcium meant for informational purposes only.

The BriefCase-Quantification results are not intended to be used on a stand-alone basis for clinical decision-making or otherwise preclude clinical assessment of cases. Clinicians are responsible for viewing full images per the standard of care.

Device Description

BriefCase-Quantification is a standalone software as a medical device intended for use in the analysis of non-cardiac-gated non-contrast CT (NCCT) images that include the heart to assist hospital networks and appropriately trained medical specialists. The software consists of a single module based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.

The BriefCase-Quantification receives filtered routine, non-gated computed tomography (CT) scans, and processes them chronologically by running the algorithm on relevant series to evaluate calcified plaques in the coronary arteries. Following the AI processing, the output of the algorithm analysis is transferred to an image review software (the PACS or a desktop application).

The device generates a four-category output corresponding with the estimated quantity of calcium detected: very low, low, medium, and high. The categories composing the output of the device correspond with a validated visual assessment categorization of none, mild, moderate, and severe [1] in agreement with categorized Agatston scores indicated in the literature (very low: 0; low: 1-100; medium: 101-400; high: ≥400). In addition, the categories accord with the 2016 SCCT/STR guidelines for coronary artery calcium scoring of non-contrast non-cardiac chest CT scans and are used as standard of care in clinical practice during CAC assessment in NCCT scans.

The BriefCase-Quantification software generates a preliminary summary report that is provided in the desktop application that includes applicable user warnings, the CAC detection category and number of slices that include CAC. The report presents preliminary results only and instructs the user to review the full image and any other clinical information before making a clinical decision. For all analyzed scans, the user will be presented in the PACS with all the slices containing the measured coronary calcifications. On these images, the calcified areas will be represented to provide the user visibility on the areas which supported the category output. These slices will be presented along with the original slices. Preview images of the represented calcium are non-diagnostic and are available in the PACS for informational purposes only.

AI/ML Overview

Here's an analysis of the acceptance criteria and the study for the BriefCase-Quantification device, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

Category	Acceptance Criteria (Performance Goal)	Reported Device Performance
Overall Agreement (Primary Endpoint)	"up to the prespecified performance goal"	87.1%
Agreement for 'Very Low' CAC (Secondary Endpoint)	"up to the prespecified performance goal"	95.1%
Agreement for 'Low' CAC (Secondary Endpoint)	"up to the prespecified performance goal"	81.3%
Agreement for 'Medium' CAC (Secondary Endpoint)	"up to the prespecified performance goal"	81.5%
Agreement for 'High' CAC (Secondary Endpoint)	"up to the prespecified performance goal"	89.3%

Note: The document states that the primary and secondary endpoints were achieved because the reported performance was "up to the prespecified performance goal," but it does not explicitly state the numerical value of the prespecified performance goal for each criterion.

2. Sample Size Used for the Test Set and Data Provenance

Sample Size for Test Set: 433 cases
Data Provenance: Retrospective, multicenter study from 6 US-based clinical sites (both academic and community centers). The cases were distinct in time or center from the cases used to train the algorithm.

3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications

Number of Experts: At least one, potentially more. The text states "In cases where the reviewers disagree on the level of CAC, the senior US board-certified radiologist provided a final opinion which has established the ground truth." This implies primary reviewers and a senior radiologist for adjudication.
Qualifications of Experts: Senior US board-certified radiologist (for final opinion/adjudication). The qualifications of initial reviewers are not specified.

4. Adjudication Method for the Test Set

Adjudication Method: 2+1 (or similar) where a "senior US board-certified radiologist provided a final opinion" in cases of disagreement among initial reviewers.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done, and its Effect Size

A standalone performance study of the algorithm against ground truth was performed.
The document does not mention a multi-reader multi-case (MRMC) comparative effectiveness study to assess how much human readers improve with AI vs without AI assistance. The study focuses on the algorithm's agreement with expert-established ground truth.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was done

Yes, a standalone study was done. The pivotal study evaluated "the software's performance in providing estimated coronary artery calcification detection category ... compared to the ground truth." The performance metrics (overall agreement and agreement per category) are for the algorithm's output directly compared to ground truth.

7. The Type of Ground Truth Used

Type of Ground Truth: Expert consensus/adjudication. The text states, "In cases where the reviewers disagree on the level of CAC, the senior US board-certified radiologist provided a final opinion which has established the ground truth." The ground truth categories ([very low, low, medium, high] corresponding to Agatston scores) are also aligned with "validated visual assessment categorization of none, mild, moderate, and severe" and "2016 SCCT/STR guidelines."

8. The Sample Size for the Training Set

The sample size for the training set is not provided in the document. The text only mentions that "The cases collected for the pivotal dataset were all distinct in time or center from the cases used to train the algorithm."

9. How the Ground Truth for the Training Set was Established

The document does not explicitly describe how the ground truth for the training set was established. It only ensures that the training and test sets were distinct. While it's implied that similar expert-based ground truth would be used, the specifics are not detailed.

Ask a Question

Ask a specific question about this device

K Number

K232083

Device Name

Manufacturer

Date Cleared

2023-11-13

(123 days)

Product Code

Regulation Number

Type

Panel

Reference & Predicate Devices

K211222

Intended Use

BriefCase-Quantification of Midline Shift (MLS) is a radiological image management and processing system software intended for automatic measurement of brain midline shift in non-contrast head CT (NCCT) images, in adults or transitional adolescents aged 18 years and older.

The device is intended to assist appropriately trained medical specialists by providing the user with an automated current manual process of measuring midline shift.

The device provides midline shift measurement from NCCT images acquired at a single time point, and can additionally provide an output with comparative analysis of two or more images that were acquired in the same individual at multiple time points.

The device does not alter the original medical image and is not intended to be used as a diagnostic device. The BriefCase-Quantification results are not intended to be used on a stand-alone basis for clinical decision-making or otherwise preclude clinical assessment of cases. Clinicians are responsible for viewing full images per the standard of care.

Device Description

BriefCase-Quantification is a radiological image management and processing device. The software consists of a single module based on an algorithm programmed component and is intended to run on a linux-based server in a cloud environment.

The BriefCase-Quantification receives filtered DICOM Images, and processes them chronologically by running the algorithm on relevant series to quantify the extent of midline shift. Following the Al processing, the output of the algorithm analysis is transferred to an image review software (the PACS or a desktop application).

The device generates a summary report that includes a preview image of the slice with the largest midline shift. The preview image includes the measured shift, the annotation of the midline, and the annotation of the largest perpendicular distance between the midline and septum pellucidum. Also, the summary report includes a table and a graph showing the measured midline shift over time for patients with multiple scans.

AI/ML Overview

Here's a detailed breakdown of the acceptance criteria and study information for the BriefCase-Quantification device, based on the provided document:

Acceptance Criteria and Device Performance

Criteria	Acceptance Criteria	Reported Device Performance
Primary Endpoint: Mean Absolute Error (MAE)	Mean absolute error estimate must be lower than prespecified performance goal.	0.94 mm (95% CI: 0.74 mm, 1.14 mm) (Lower than prespecified goal)
Secondary Endpoint: Bias (Bland-Altman plot)	Little to no bias between ground truth and algorithm output.	Mean difference of -0.15 mm (Little to no bias)
Secondary Endpoint: MAE for multiple time points (First Case)	Mean absolute error estimate must be lower than prespecified performance goal.	1.16 mm (95% CI: 0.61 mm, 1.71 mm) (Lower than prespecified goal)
Secondary Endpoint: MAE for multiple time points (Follow-up Cases)	Mean absolute error estimate must be lower than prespecified performance goal.	1.28 mm (95% CI: 0.68 mm, 1.88 mm) (Lower than prespecified goal)

Study Details

Sample size for the test set and data provenance:
- Sample Size: 284 cases from 228 unique patients.
- Data Provenance: Retrospective, multi-center study from 6 US-based clinical sites (both academic and community centers). The cases were distinct in time or center from the cases used to train the algorithm, indicating independent test data.
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Three neuroradiologists.
- Qualifications: The document states "appropriately trained medical specialists" and specifically "three neuroradiologists," implying they are qualified experts in the field. Specific experience (e.g., "10 years of experience") is not provided.
Adjudication method for the test set:
- Method: The reference standard (ground truth) was created as the mean of all three independent measurements by the neuroradiologists. This implies a "consensus by average" approach rather than a specific 2+1 or 3+1 voting method.
If a multi-reader, multi-case (MRMC) comparative effectiveness study was done, if so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- MRMC Study: No, an MRMC comparative effectiveness study was not explicitly stated as performed with human readers and AI assistance. The study described focuses on the standalone performance of the AI algorithm against a neuroradiologist-established ground truth.
- Effect Size of Human Improvement with AI: This information is not provided because an MRMC study comparing human readers with and without AI assistance was not detailed.
If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Standalone Performance: Yes, a standalone performance study was done. The reported performance metrics (MAE, Bland-Altman) directly compare the algorithm's output to the ground truth established by experts, without human intervention in the device's measurement process. The device is intended to assist specialists by providing an automated process, but its performance evaluation here is purely algorithmic.
The type of ground truth used:
- Ground Truth Type: Expert consensus. Specifically, the "mean of all three [neuroradiologist] measurements."
The sample size for the training set:
- Training Set Sample Size: Not explicitly stated. The document only mentions that the "cases collected for the pivotal dataset were all distinct in time or center from the cases used to train the algorithm."
How the ground truth for the training set was established:
- Training Set Ground Truth: Not explicitly stated. Given the nature of a supervised learning algorithm, it is implied that the training data also had established ground truth measurements, likely derived through expert review, but the specific method or number of experts for the training data is not detailed in this summary.

Ask a Question

Ask a specific question about this device

K Number

K230534

Device Name

Manufacturer