Search Results

Chest-CAD is a computer-assisted detection (CADe) software device that analyzes chest radiograph studies using machine learning techniques to identify, categorize, and highlight suspicious regions of interest (ROI). Any suspicious ROI identified by Chest-CAD is assigned to one of the following categories: Cardiac, Mediastinum/Hila, Lungs, Pleura, Bones, Soft Tissues, Hardware, or Other. The device is intended for use as a concurrent reading aid for physicians. Chest-CAD is indicated for adults only.

Chest-CAD is a computer-assisted detection (CADe) software device designed to assist physicians in identifying suspicious regions of interest (ROIs) in adult chest X-rays. Suspicious ROIs identified by Chest-CAD are assigned to one of the following categories: Cardiac, Mediastinum/Hila, Lungs, Pleura, Bones, Soft Tissues, Hardware, or Other. Chest-CAD detects suspicious ROIs by analyzing radiographs using deep learning algorithms for computer vision and provides relevant annotations to assist physicians with their interpretations.

For each image within a study, Chest-CAD generates a DICOM Presentation State file (output overlay). If any suspicious ROI is detected by Chest-CAD in the study, the output overlay for all images includes the text "ROI(s) Detected:" followed by a list of the category/categories for which suspicious ROI(s) were found, such as "Lungs, Bones". In addition, if suspicious ROI(s) are detected in the image, bounding boxes surrounding each detected suspicious ROI are included in the output overlay. If no suspicious ROI is detected by Chest-CAD in the study, the output overlay for each image will include the text "No ROI(s) Detected" and no bounding boxes will be included. Regardless of whether a suspicious ROI is detected, the overlay includes text identifying the X-ray study as analyzed by Chest-CAD and a customer configurable message containing a link to or instructions for users to access labeling. The Chest-CAD overlay can be toggled on or off by the physician within their Picture Archiving and Communication System (PACS) viewer, allowing for concurrent review of the X-ray study.

Acceptance Criteria and Device Performance for Chest-CAD (K210666)

The Chest-CAD device is a computer-assisted detection (CADe) software that analyzes chest radiographs to identify, categorize, and highlight suspicious regions of interest (ROI). It is intended as a concurrent reading aid for physicians and is indicated for adults only.

1. Acceptance Criteria and Reported Device Performance

The acceptance criteria for Chest-CAD were established through both a standalone performance assessment and a multi-reader, multi-case (MRMC) comparative effectiveness study. No explicit numerical acceptance criteria were stated as targets for the standalone performance metrics (sensitivity, specificity, AUC). Instead, the performance demonstrated in the standalone test formed a basis for comparison and validation. For the MRMC study, the primary objective was to demonstrate superiority of aided reading over unaided reading in terms of AUC.

Table 1: Acceptance Criteria and Reported Device Performance

2. Sample Sizes and Data Provenance

Standalone Test Set:

• Sample Size: 20,000 chest radiograph cases.
• Data Provenance: From 12 hospitals, outpatient centers, and specialty centers in the United States. The study was retrospective.

MRMC Study Test Set:

• Sample Size: 238 cases.
• Data Provenance: From 9 hospitals, outpatient centers, and specialty centers in the United States. The study was retrospective.

3. Number and Qualifications of Experts for Ground Truth (Test Set)

• Standalone Test Set: Not explicitly stated for the standalone test set. The document indicates "Suspicious ROIs were manually annotated and categorized by board-certified radiologists before the images were used for benchmarking." This implies that the ground truth for standalone testing was established by expert radiologists, but the number and specific qualifications (years of experience) are not detailed.
• MRMC Study Test Set: "Each case was previously evaluated by a panel of U.S. board-certified radiologists who assigned a ground truth binary label indicating the presence or absence of a suspicious ROI for each Chest-CAD category." The exact number of experts in the panel is not specified, nor is their specific level of experience (e.g., 10 years).

4. Adjudication Method (Test Set)

• The document implies a consensus-based adjudication method for the ground truth of the MRMC study cases, stating that "a panel of U.S. board-certified radiologists who assigned a ground truth binary label..." This strongly suggests that multiple radiologists reviewed the cases and reached a consensus for the ground truth. It does not explicitly state a 2+1 or 3+1 method but refers to a "panel" establishing the ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• Yes, a fully-crossed MRMC retrospective reader study was conducted.
• Effect Size of Human Reader Improvement with AI vs. Without AI Assistance:
  • Reader AUC: Improved from 0.836 (Unaided) to 0.894 (Aided). Effect size = 0.058 increase.
  • Reader Sensitivity: Improved from 0.757 (Unaided) to 0.856 (Aided). Effect size = 0.099 increase.
  • Reader Specificity: Improved from 0.843 (Unaided) to 0.870 (Aided). Effect size = 0.027 increase.
    The study concluded that the accuracy of readers was superior when aided by Chest-CAD.

6. Standalone (Algorithm Only) Performance Study

• Yes, a standalone performance assessment was conducted.
• Reported Performance:
  • Overall Sensitivity: 0.908 (95% CI: 0.905, 0.911)
  • Overall Specificity: 0.887 (95% CI: 0.885, 0.889)
  • Overall AUC: 0.976 (95% CI: 0.975, 0.976)
  • AUC by Category: Ranged from 0.921 (Mediastinum/Hila) to 0.994 (Hardware).
  • Sensitivity by Category: Ranged from 0.854 (Bones) to 0.967 (Hardware).
  • Specificity by Category: Ranged from 0.830 (Mediastinum/Hila) to 0.960 (Hardware).

7. Type of Ground Truth Used (Test Set)

• Expert Consensus: For both the standalone testing and the MRMC study, the ground truth was established by board-certified radiologists who manually annotated and assigned binary labels for suspicious ROIs.

8. Sample Size for the Training Set

• The document does not explicitly state the sample size for the training set. It refers to "deep learning algorithms for computer vision" but does not detail the dataset used for training or validation of these algorithms.

9. How the Ground Truth for the Training Set was Established

• The document does not explicitly state how the ground truth for the training set was established. It only mentions that the device uses "machine learning techniques" and "deep learning algorithms." Typically, for such devices, the training set ground truth would also be established by expert radiologists, usually through a process similar to or more extensive than that used for the test sets. However, this is not detailed in the provided text.

Ask a specific question about this device