LogoFDA 510(k) Search
Search Filters

Search Results

Found 1 results

510(k) Data Aggregation

    K Number
    K221305

    Validate with FDA (Live)

    Device Name
    AI-Rad Companion Organs RT
    Manufacturer
    Siemens Medical Solutions USA, Inc
    Date Cleared
    2022-10-14

    (162 days)

    Product Code
    QKB,REC
    Regulation Number
    892.2050
    Type
    Traditional
    Panel
    Radiology
    Reference & Predicate Devices
    K213976,K193562
    Predicate For
    K232928
    AI/MLSaMDIVD (In Vitro Diagnostic)TherapeuticDiagnosticis PCCP AuthorizedThirdpartyExpeditedreview
    Intended Use

    AI-Rad Companion Organs RT is a post-processing software intended to automatically contour DICOM CT imaging data using deep-learning-based algorithms.

    Contours that are generated by AI-Rad Companion Organs RT may be used as input for clinical workflows including external beam radiation therapy treatment planning. AI-Rad Companion Organs RT must be used in conjunction with appropriate software such as Treatment Planning Systems and Interactive Contouring applications, to review, edit, and accept contours generated by AI-Rad Companion Organs RT.

    The output of AI-Rad Companion Organs RT in the format of RTSTRUCT objects are intended to be used by trained medical professionals.

    The software is not intended to automatically detect or contour lesions. Only DICOM images of adult patients are considered to be valid input.

    Device Description

    AI-Rad Companion Organs RT is a post-processing software used to automatically contour DICOM CT imaging data using deep-learning-based algorithms. AI-Rad Companion Organs RT contouring workflow supports CT input data and produces RTSTRUCT outputs. The configuration of the organ database and organ templates defining the organs and structures to be contoured based on the input DICOM data is managed via a configuration interface. Contours that are generated by AI-Rad Companion Organs RT may be used as input for clinical workflows including external beam radiation therapy treatment planning.

    The output of AI-Rad Companion Organs RT, in the form of RTSTRUCT objects, are intended to be used by trained medical professionals. The output of AI-Rad Companion Organs RT must be used in conjunction with appropriate software such as Treatment Planning Systems and Interactive Contouring applications, to review, edit, and accept contours generated by AI-Rad Companion Organs RT application.

    At a high-level, AI-Rad Companion Organs RT includes the following functionality:

      1. Automated contouring of Organs at Risk (OAR) workflow
      • a. Input -DICOM CT
      • b. Output DICOM RTSTRUCT
      1. Organ Templates configuration (incl. Organ Database)
      1. Web-based preview of contouring results to accept or reject the generated contours
    AI/ML Overview

    Here's a breakdown of the acceptance criteria and study details for the AI-Rad Companion Organs RT device, based on the provided text:

    1. Table of Acceptance Criteria & Reported Device Performance:

    Validation Testing SubjectAcceptance CriteriaReported Device Performance (Median)
    Organs in Predicate Device1. All organs segmented in the predicate device are also segmented in the subject device.Met (all predicate organs are segmented in the subject device, implied by comparison tables).
    2. The lower bound of the 95th percentile CI of the segmentation (subject device) is greater than 0.1 Dice lower than the mean of the predicate device segmentation.DICE: Subject: 0.85 (CI: [80.23, 84.61]) vs. Predicate: 0.85 (implied CI close to median). The statement "performance of the subject device and predicate device are comparable in DICE and ASSD" implies this criterion was met.
    ASSD: Subject: 0.93 (CI: [0.86, 1.14]) vs. Predicate: 0.94 (implied CI close to median). The statement "performance of the subject device and predicate device are comparable in DICE and ASSD" implies this criterion was met.
    Head & Neck Lymph Nodes1. The overall fail rate of each organ/anatomical structure is smaller than 15%.Not explicitly stated for each organ/anatomical structure, but generally implied by acceptable DICE and ASSD.
    2. The lower bound of the 95th percentile CI of the segmentation (subject device) is greater than 0.1 Dice lower than the mean of the reference device segmentation.DICE: Subject (Head and Neck lymph node class): Avg 81.32 (CI: [80.32, 82.12]) vs. Reference (Pelvic lymph node class): Avg 80. The statement "performance of the subject device for non-overlapping organs is comparable in DICE to the reference device" and the specific values show that 80.32 is not more than 0.1 lower than 80 (it's higher by 0.32), so this criterion appears met.
    ASSD: Subject (Head and Neck lymph node class): Avg 1.06 (CI: [0.99, 1.19]) vs. Reference: N.A. (No direct comparison for ASSD).

    Note: The text did not explicitly state the "fail rate" for the Head & Neck Lymph Nodes, only that it should be "smaller than 15%". The conclusion implies all acceptance criteria were met. The confidence intervals for the predicate device's DICE and ASSD are missing in Table 4, but the statement "performance of the subject device and predicate device are comparable" suggests the criteria were acceptable.

    2. Sample Size Used for the Test Set and Data Provenance:

    • Sample Size: N = 113 retrospective performance study on CT data.
      • This N=113 is composed of:
        • Cohort A: 73 subjects (14 from Germany, 59 from Brazil)
        • Cohort B: 40 subjects (Canada: 40)
    • Data Provenance: Multiple clinical sites across North America (Canada) and Europe (Germany, Brazil – often considered part of South America, but grouped with "Europe" in the text for data collection context). The study used previously acquired CT data (retrospective).

    3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:

    • Number of Experts: Not explicitly stated as a specific number. The text mentions "a team of experienced annotators" and "a board-certified radiation oncologist".
    • Qualifications:
      • Annotators: "experienced annotators mentored by radiologists or radiation oncologists".
      • Review/Correction: "board-certified radiation oncologist".

    4. Adjudication Method for the Test Set:

    • The ground truth annotations were drawn manually by a team of experienced annotators and then underwent a "quality assessment including review and correction of each annotation was done by a board-certified radiation oncologist". This suggests a method where initial annotations are created by multiple individuals and then reviewed/corrected by a single, highly qualified expert. This could be interpreted as a form of expert review/adjudication.

    5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was done:

    • No, a MRMC comparative effectiveness study was not explicitly stated as having been done. The performance evaluation focused on comparing the AI algorithm's output to expert-generated ground truth and comparing the device's performance to predicate/reference devices, not on how human readers improve with or without AI assistance.

    6. If a Standalone (i.e. algorithm only without human-in-the loop performance) was done:

    • Yes, a standalone performance study was done. The study "validated the AI-Rad Companion Organs RT software from clinical perspective" by evaluating its auto-contouring algorithm, and calculating metrics like DICE coefficients and ASSD against ground truth annotations. The device's output "must be used in conjunction with appropriate software... to review, edit, and accept contours", indicating its standalone output is then reviewed by a human, but the validation of its generation of contours is standalone.

    7. The Type of Ground Truth Used:

    • Expert Consensus/Manual Annotation with Expert Review (following guidelines): "Ground truth annotations were established following RTOG and clinical guidelines using manual annotation. The ground truth annotations were drawn manually by a team of experienced annotators mentored by radiologists or radiation oncologists using an internal annotation tool. Additionally, a quality assessment including review and correction of each annotation was done by a board-certified radiation oncologist..." This indicates a robust expert-derived ground truth.

    8. The Sample Size for the Training Set:

    • 160 datasets (for Head & Neck specifically, other organs might have different training data, but this is the only training set sample size provided).

    9. How the Ground Truth for the Training Set was Established:

    • "In both the annotation process for the training and validation testing data, the annotation protocols for the OAR were defined following the NRG/RTOG guidelines. The ground truth annotations were drawn manually by a team of experienced annotators mentored by radiologists or radiation oncologists using an internal annotation tool. Additionally, a quality assessment including review and correction of each annotation was done by a board-certified radiation oncologist using validated medical image annotation tools."
      • This is the same process as for the test set, ensuring consistency in ground truth establishment.
    Ask a Question

    Ask a specific question about this device

    Page 1 of 1