Search Results
Found 1 results
510(k) Data Aggregation
(113 days)
Auto Segmentation
Auto Segmentation generates a Radiotherapy Structure Set (RTSS) DICOM with segmented organs at risk which can be used by dosimetrists, medical physicists, and radiation oncologists as initial contours to accelerate workflow for radiation therapy planning. It is the responsibility of the user to verify the processed output contours and user-defined labels for each organ at risk and correct the contours/labels as needed. Auto Segmentation may be used with images acquired on CT scanners, in adult patients.
Auto Segmentation is a post-processing software designed to automatically generate contours of organ(s) at risk (OARs) from Computed Tomography (CT) images in the form of a DICOM Radiotherapy Structure Set (RTSS) series. The application is intended as a workflow tool for initial segmentation of OARs to streamline the process of organ at risk delineation. The Auto Segmentation is intended to be used by radiotherapy (RT) practitioners after review and editing, if necessary, and confirming the accuracy of the contours for use in radiation therapy planning.
Auto Segmentation uses deep learning algorithms to generate organ at risk contours for the head and neck, thorax, abdomen and pelvis regions from CT images across 40 organ subregion(s). The automatically generated organ at risk contours are networked to predefined DICOM destination(s), such as review workstations supporting RTSS format, for review and editing, as needed.
The organ at risk contours generated with the Auto Segmentation are designed to improve the contouring workflow by automatically creating contours for review by the intended users. The application is compatible with CT DICOM images with single energy acquisition modes and may be used with both GE and non-GE CT scanner acquired images (contrast), in adult patients.
Here's an analysis of the acceptance criteria and study detailed in the provided document for the GE Medical Systems Auto Segmentation device:
1. Table of Acceptance Criteria and Reported Device Performance
| OAR | Auto Segmentation (subject device) Dice Mean | Lower CI95 | Acceptance Criteria Type | Acceptance Criteria Dice Mean |
|---|---|---|---|---|
| Adrenal Left | 78.68% | 76.63% | Estimated | 68.0% |
| Adrenal Right | 72.48% | 69.78% | Estimated | 68.0% |
| Bladder | 81.50% | 78.33% | Deep learning | 80.0% |
| Body | 99.50% | 99.38% | Atlas-based | 98.1% |
| Brainstem | 87.69% | 87.15% | Deep learning | 88.4% |
| Chiasma | 43.81% | 41.03% | Atlas-based | 11.7% |
| Esophagus | 81.69% | 80.38% | Atlas-based | 45.8% |
| Eye Left | 91.32% | 89.77% | Deep learning | 90.1% |
| Eye Right | 90.25% | 88.23% | Deep learning | 89.9% |
| Femur Left | 97.65% | 97.18% | Atlas-based | 71.6% |
| Femur Right | 97.92% | 97.78% | Atlas-based | 70.8% |
| Kidney Left | 92.53% | 90.30% | Deep learning | 86.8% |
| Kidney Right | 94.82% | 93.48% | Deep learning | 85.6% |
| Lacrimal Gland Left | 59.79% | 57.65% | Deep learning | 50.0% |
| Lacrimal Gland Right | 58.09% | 55.81% | Deep learning | 50.0% |
| Lens Left | 76.86% | 74.80% | Deep learning | 73.3% |
| Lens Right | 79.09% | 77.40% | Deep learning | 75.6% |
| Liver | 94.28% | 92.27% | Deep learning | 91.1% |
| Lung Left | 97.70% | 97.38% | Deep learning | 97.4% |
| Lung Right | 97.99% | 97.81% | Deep learning | 97.8% |
| Mandible | 92.70% | 92.36% | Deep learning | 94.0% |
| Optic Nerve Left | 79.22% | 77.99% | Deep learning | 71.1% |
| Optic Nerve Right | 80.20% | 78.94% | Deep learning | 71.2% |
| Oral Cavity | 87.43% | 86.20% | Deep learning | 91.0% |
| Pancreas | 80.34% | 78.50% | Estimated | 73.0% |
| Parotid Left | 84.35% | 83.27% | Deep learning | 65.0% |
| Parotid Right | 85.55% | 84.48% | Deep learning | 65.0% |
| Proximal Bronchial Tree (PBtree) | 84.94% | 83.71% | Atlas-based | 54.8% |
| Inferior PCM (Pharyngeal Constrictor Muscle) | 70.51% | 68.72% | Estimated | 68.0% |
| Middle PCM | 67.09% | 65.21% | Estimated | 68.0% |
| Superior PCM | 59.57% | 57.85% | Estimated | 50.0% |
| Pericardium | 93.58% | 92.00% | Atlas-based | 84.4% |
| Pituitary | 75.62% | 74.12% | Deep learning | 78.0% |
| Prostate | 79.67% | 77.60% | Atlas-based | 52.1% |
| Spinal Cord | 88.55% | 87.43% | Deep learning | 87.0% |
| Submandibular Left | 86.85% | 85.95% | Deep learning | 77.0% |
| Submandibular Right | 85.70% | 84.79% | Deep learning | 78.0% |
| Thyroid | 85.37% | 84.27% | Deep learning | 83.0% |
| Trachea | 91.02% | 90.47% | Atlas-based | 69.2% |
| Whole Brain | 98.53% | 98.46% | Estimated | 93.0% |
Note: The reported device performance (Dice Mean and Lower CI95) for almost all organs meets or exceeds the specified acceptance criteria. The only exception where the device's Dice Mean is slightly below the acceptance criteria is for Mandible (92.70% vs 94.0%) and Oral Cavity (87.43% vs 91.0%) and Pituitary (75.62% vs 78.0%), however there is no further discussion or justification provided in the text for these specific instances. The document does state that "The evaluation of the Dice mean for the Auto Segmentation algorithms demonstrates that the algorithm performance is in line with the performance of the predicate, as well as state of the art, recently cleared similar automated contouring devices."
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size for Test Set: 302 retrospective CT radiation therapy planning exams (generating 2552 contours).
- Data Provenance: Multiple clinical sites in North America, Asia, and Europe. The demographic distribution includes adults (18-89 years old) of various genders and ethnicities from 9 global sources (USA, EU, Asia). The data was acquired using a variety of CT scanners and scanner protocols from different manufacturers.
- Retrospective/Prospective: Retrospective.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Their Qualifications
- Number of Experts: Three (3).
- Qualifications of Experts: Independent, qualified radiotherapy practitioners.
- Comment: The document states that the ground truth annotations were established following RTOG and DAHANCA clinical guidelines.
4. Adjudication Method for the Test Set
- The document implies a consensus-based approach guided by clinical guidelines, as "ground truth annotations were established (...) manually by three independent, qualified radiotherapy practitioners," but it does not specify an explicit adjudication method like "2+1" or "3+1" for resolving disagreements between the three experts. The phrase "established following RTOG and DAHANCA clinical guidelines" suggests that these guidelines were used to define the correct contours.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
- The document describes a qualitative preference study that involved three qualified radiotherapy practitioners reviewing the contours generated by the Auto Segmentation application. They assessed the adequacy of the generated contours for radiotherapy planning using a Likert scale.
- However, this was NOT a comparative effectiveness study of human readers with and without AI assistance. It was a study to determine the adequacy of the AI-generated contours themselves for initial use. Therefore, no effect size of human readers improving with AI vs. without AI assistance can be reported from this document.
6. Standalone Performance Study (Algorithm Only)
- Yes, a standalone performance study was conducted. The "Performance testing to evaluate the device's performance in segmenting organs-at-risk was performed using a database of 302 retrospective CT radiation therapy planning exams." The Dice Similarity Coefficient (DSC) was used as the primary metric to compare the Auto Segmentation generated contours to ground truth contours. The reported Dice Mean values and their 95% confidence intervals are direct metrics of the algorithm's standalone performance.
7. Type of Ground Truth Used
- Expert Consensus/Manual Annotation: Ground truth annotations were "established following RTOG and DAHANCA clinical guidelines manually by three independent, qualified radiotherapy practitioners."
8. Sample Size for the Training Set
- 911 different CT exams.
9. How the Ground Truth for the Training Set Was Established
- The document states that "The Auto Segmentation algorithms were developed and trained using a dataset of 911 different CT exams from several clinical sites from multiple countries. The original development and training data was used for radiotherapy planning..."
- It does not explicitly detail the process for establishing ground truth for the training set, but given the context of the test set ground truth and the overall development, it is highly probable it involved manual annotation by experts for radiotherapy planning purposes.
Ask a specific question about this device
Page 1 of 1