Search Results

AutoContour is intended to assist radiation treatment planners in contouring and reviewing structures within medical images in preparation for radiation therapy treatment planning.

As with AutoContour Model RADAC V4, the AutoContour Model RADAC V5 device is software that uses DICOM-compliant image data (CT or MR) as input to: (1) automatically contour various structures of interest for radiation therapy treatment planning using machine learning based contouring. The deep-learning-based structure models are trained using imaging datasets consisting of anatomical organs of the head and neck, thorax, abdomen, and pelvis for adult male and female patients, (2) allow the user to review and modify the resulting contours, and (3) generate DICOM-compliant structure set data that can be imported into a radiation therapy treatment planning system.

AutoContour Model RADAC V5 consists of 3 main components:

1. A .NET client application designed to run on the Windows Operating System, allowing the user to load image and structure sets for upload to the cloud-based server for automatic contouring, perform registration with other image sets, as well as review, edit, and export the structure set.
2. A local "agent" service designed to run on the Windows Operating System that is configured by the user to monitor a network storage location for new CT and MR datasets that are to be automatically contoured.
3. A cloud-based automatic contouring service that produces initial contours based on image sets sent by the user from the .NET client application.

Here's a structured summary of the acceptance criteria and study details for the AutoContour Model RADAC V5, based on the provided FDA 510(k) clearance letter:

Acceptance Criteria and Device Performance Study for AutoContour Model RADAC V5

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria for each structure model varied based on its size (Large, Medium, Small) and whether it was a new model, an updated model, or an unchanged existing model. The performance was primarily evaluated through Dice Similarity Coefficient (DSC) and Likert Qualitative Review for new/updated models, and DSC and Hausdorff Distance for existing models.

Note: The document provides specific DSC values for many individual structures. The table above shows aggregated or illustrative examples from the tables provided.

2. Sample Size for Test Set and Data Provenance

• CT Test Sets: An average of 49 testing image sets per CT structure model (approximately 10% of training data). Specific examples include:
  • A_Aorta_Asc (Update): 60 testing sets
  • A_Carotid_L/R (Update): 83 testing sets
  • A_Celiac: 44 testing sets
• MR Test Sets:
  • Brain models: 58 testing image sets (e.g., Amygdala_L/R: 133, CorpusCallosum: 15)
  • Pelvis models: 50 testing image sets (e.g., Rectal_Spacer: 26)
• External Clinical Test Sets:
  • CT: 20 (A_Aorta), 37 (A_Carotid_L/R), 24 (A_Celiac), etc.
  • MR: 20 (Amygdala_L), 45 (Bladder_Trigone), 7 (HDR_Bowel), etc.
• Data Provenance (Training and Testing): Data was gathered from several institutions in several different countries (not specifically enumerated but mentioned for CT and MR). Specific external clinical datasets for CT included TCIA - Pelvic-Ref, TCIA - Head-Neck-PET-CT, TCIA - Pancreas-CT-CB, TCIA - NSCLC data. MR external datasets included "MR - Renown," "Gold Atlas Pelvis," "SynthRad," "MRLinac Pelvis," "Female HDR MR Pelvis," and "MR Pelvis Barrigel," some of which were open-source or shared by clinical partners/institutions in Canada, Spain, Australia, and the United States. The images used for testing were sequestered from the original training and validation data population and removed from the training dataset pool before model training began.

3. Number of Experts and Qualifications for Ground Truth

• Number of Experts: Six (6) clinically experienced experts.
• Qualifications: 2 radiation therapy physicists, 1 radiation dosimetrist, and 3 radiation therapists with specialized training in radiation therapy contouring.

4. Adjudication Method for the Test Set

The ground truthing of each test dataset was generated manually using consensus (NRG/RTOG/ESTRO) guidelines as appropriate. While a specific (e.g., 2+1, 3+1) adjudication method for individual cases or disagreements is not explicitly stated, the use of "consensus" guidelines by multiple experts implies a form of adjudicated agreement for final ground truth.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• The document does not explicitly describe a conventional MRMC comparative effectiveness study comparing human readers with AI assistance versus without AI assistance.
• Instead, it measures the AI's standalone performance against expert-generated ground truth and uses a qualitative review by external experts (average rating 1-5 where >3 means beneficial, 5 means no edits needed) to assess the clinical appropriateness and required modifications for the AI-generated contours. This qualitative review serves as an indirect assessment of human interaction with AI output, but not a formal MRMC study as typically defined for reader performance improvement with assistance.

6. Standalone Performance (Algorithm Only without Human-in-the-loop)

• Yes, standalone performance was done. The primary performance metrics (Dice Similarity Coefficient - DSC and Hausdorff Distance) directly evaluate the algorithm's output against the expert-generated ground truth without human intervention in the contour generation process. The "Training DSC Evaluation" and "External Dataset DSC Evaluation" explicitly refer to the model's direct output.
• The qualitative review by external experts, while involving human assessment, is done after the algorithm has generated its standalone contours, effectively evaluating the standalone output's clinical utility.

7. Type of Ground Truth Used

• Expert Consensus: Ground truth for both training and test sets was established manually by six clinically experienced experts following consensus guidelines (NRG/RTOG/ESTRO).

8. Sample Size for the Training Set

• CT Training Sets: An average of 459 training image sets per CT structure model. Specific examples:
  • A_Aorta_Asc (Update): 240
  • A_Carotid_L/R (Update): 328
  • A_Celiac: 435
• MR Training Sets:
  • Brain models: An average of 259 training image sets.
  • Pelvis models: An average of 243 training image sets.
  • Specific examples: Amygdala_L/R: 493, CorpusCallosum: 56, Rectal_Spacer: 233.

9. How Ground Truth for Training Set was Established

• The ground truth for the training set was established manually by the same group of six clinically experienced experts (2 radiation therapy physicists, 1 radiation dosimetrist, and 3 radiation therapists with specialized training in radiation therapy contouring) using consensus guidelines (NRG/RTOG/ESTRO).

Ask a specific question about this device