Search Results

aprevo® Digital Segmentation software is intended to be used by trained, medically knowledgeable design personnel to perform digital image segmentation of the spine, primarily lumbar anatomy. The device inputs DICOM images and outputs a 3-D model of the spine.

The device is a software medical device that will use DICOM images as input and provide 3D model of the spine structure. Pre-processing will be performed on the uploaded DICOM files to filter soft tissue and identifying spine. Upon removal of soft tissue and identification of spine structure, the software will utilize an AI-based algorithm to segment the structure and render a 3D model as an output.

The provided text describes the acceptance criteria and the study that proves the device meets those criteria for the "aprevo® Digital Segmentation" software.

Here's a breakdown of the requested information:

Acceptance Criteria and Device Performance

Study Details:

1. Sample Size Used for the Test Set and Data Provenance:

   • Test Set Sample Size: Not explicitly stated in the provided text, but it mentions that "Independent training and validation datasets were selected to ensure model performance would reflect real clinical performance" and "Validation datasets represented diversity in populations and equipment."
   • Data Provenance: Not explicitly stated, however, the phrase "diversity in populations and equipment" suggests data from various sources but does not specify countries of origin. The study was a "Non-Clinical Testing" which implies retrospective data.

2. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications:

   • Not specified. The document states that the ground truth for the algorithm was used for model performance evaluation, but does not detail how this ground truth was established, or the number/qualifications of experts involved.

3. Adjudication Method for the Test Set:

   • Not specified.

4. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

   • If done: No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not explicitly mentioned or indicated. The document states "CLINICAL TESTING: Not applicable." The study solely focuses on the standalone performance of the software.
   • Effect size of human readers improvement: Not applicable, as no MRMC study was conducted.

5. Standalone Performance (Algorithm only without human-in-the-loop):

   • If done: Yes, a standalone performance evaluation was done. The "NON-CLINICAL TESTING" section describes the evaluation of the "software performance" using IOU and accuracy metrics for segmentation and labeling, without human intervention in the reported performance metrics.

6. Type of Ground Truth Used:

   • The type of ground truth used is not explicitly stated as expert consensus, pathology, or outcomes data. However, for "segmentation" and "vertebral body labeling," the ground truth would typically be established by expert annotation or a similar gold standard, refined through a consensus process, but this is not detailed.

7. Sample Size for the Training Set:

   • Not explicitly stated. It only mentions that "Independent training and validation datasets were selected to ensure model performance would reflect real clinical performance."

8. How the Ground Truth for the Training Set Was Established:

   • Not explicitly stated. The document mentions that "Independent training and validation datasets were selected," but does not elaborate on the method used to establish the ground truth for the training data (e.g., expert annotations, manual segmentation).

Ask a specific question about this device

MRI Planner is a software-only medical device intended for use by trained radiation oncologists, dosimetrists and physicists to process images from MRI systems to

1. provide the operator with information of tissue properties for radiation attenuation estimation purposes in photon external beam radiotherapy treatment planning, and to
2. derive contours for input to radiation treatment planning by assisting in localization and definition of healthy anatomical structures.
   MRI Planner is not intended to automatically contour tumor clinical target volumes.
   MRI Planner is indicated for radiotherapy planning of adult patients for primary and metastatic cancers in the brain and head-neck regions, as well as soft tissue cancers in the pelvic region.
   MRI Planner generates synthetic CT images for radiation attenuation estimation purposes for the pelvis, brain and head-neck regions only. MRI Planner generates automatically derived contours of the bladder, colon and femoral heads, for prostate cancer patients only.

The product MRI Planner is a stand-alone software providing information to the treatment planning process prior to radiotherapy. Based on a DICOM MR image stack, the software generates synthetic CT images that can be used for attenuation calculations in radiotherapy treatment planning for the pelvis, brain and head-neck regions. In addition, the software also generates contours of anatomical structures in the MR image stack, to be used as a starting point for the manual delineation work required in radiotherapy treatment planning. Contours are generated for prostate cancer patients only (bladder, colon and femoral heads).
MRI Planner utilizes pre-trained machine learning models to perform both the conversion to synthetic CT and the automated structure contouring. The models for synthetic CT generation was trained using a dataset comprising MR and CT images for 244 patients acquired in the treatment position at four hospitals. The model for prostate cancer patient auto contouring was trained using a dataset comprising MR images for 175 patients acquired in the treatment position at four hospitals, together with in-house generated expert manual contours. MRI Planner does not display or store DICOM images. The user is advised to use existing softwares for radiotherapy treatment planning to display and modify generated images and contours.
MRI Planner runs on a standard x86-64 compatible system with a CUDA capable NVIDIA GPU and requires Ubuntu Linux 18.04 operating system.

This document describes the acceptance criteria and the studies conducted to prove that the MRI Planner device meets these criteria. The device is a software-only medical device intended for use by trained radiation oncologists, dosimetrists, and physicists for radiation attenuation estimation and contouring of healthy anatomical structures in radiotherapy treatment planning.

1. Table of Acceptance Criteria and Reported Device Performance

The performance of MRI Planner was evaluated through two main bench tests: a Dose Accuracy Bench Test for synthetic CT (sCT) generation and an Auto Contouring Bench Test for anatomical structure delineation.

2. Sample Sizes and Data Provenance

For Dose Accuracy Bench Test (Synthetic CT)

• Pelvis:
  • Test Set Sample Size: 58 unique pelvis cancer patients.
  • Data Provenance: MR (T2w) and CT images acquired in the treatment position at six different hospitals.
  • Geographic Distribution: 41% of patient images from the US, 59% from outside the US.
  • Demographics: 16% female, 84% male, age range 51-88 years.
  • MRI Scanner Data: Acquired at six different MRI scanner models from two different vendors (1.5T and 3T field strengths).
• Head-Neck-Brain:
  • Test Set Sample Size: 75 unique head-neck-brain cancer patients.
  • Data Provenance: MR (T1-Dixon) and CT images acquired in the treatment position at four different hospitals.
  • Geographic Distribution: 55% of patient images from the US, 45% from outside the US.
  • Demographics: 39% female, 64% male, age range 41-85 years.
  • MRI Scanner Data: Acquired at six different MRI scanner models from two different vendors (1.5T and 3T field strengths).

For Auto Contouring Bench Test

• Test Set Sample Size: 51 unique male prostate cancer patients.
• Data Provenance: MR (T2w) images acquired in the treatment position at five different hospitals.
• Geographic Distribution: 39% of patient images from the US, 61% from outside the US.
• Demographics: Male, age range 51-88 years.

3. Number of Experts and Qualifications for Ground Truth (Auto Contouring)

• Number of Experts: Two expert truthers.
• Qualifications: "Expert truthers" involved in the product development and training dataset generation. No specific professional qualifications (e.g., "Radiologist with X years of experience") are provided, but their involvement in product development and training data generation suggests specialized knowledge.
• Employment: Both truthers were employed by the manufacturer (Spectronic Medical AB) at the time of performing the manual delineations for the bench test.

4. Adjudication Method for Ground Truth (Auto Contouring)

• Adjudication Method: Consensus approach ("using the consensus approach"). This implies the two experts worked together to agree on the final ground truth delineations.
• Guidelines: The consensus was based on US clinical guidelines.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• The provided text does not mention a multi-reader multi-case (MRMC) comparative effectiveness study to evaluate human readers' improvement with or without AI assistance. The studies performed are standalone performance evaluations against ground truth (for auto-contouring) and conventional CT (for synthetic CT dose accuracy).

6. Standalone Algorithm Performance

• Yes, standalone performance was done for both components.
  • Synthetic CT Generation: The dosimetric bench tests directly evaluate the performance of the MRI Planner's generated synthetic CT images against conventional CT (which serves as a form of ground truth for dose calculation), without human intervention in the generation process.
  • Auto Contouring: The auto-contouring bench tests evaluate the automatically generated delineations against expert manual delineations (ground truth), representing the algorithm's performance without human-in-the-loop during contour generation.

7. Type of Ground Truth Used

For Dose Accuracy Bench Test (Synthetic CT)

• Type of Ground Truth: Dosimetric agreement was evaluated by comparing the dose calculated from synthetic CT (sCT) with dose calculated from conventional CT (CT). In this context, the conventional CT images serve as the ground truth for accurate dose calculation and attenuation estimation.

For Auto Contouring Bench Test

• Type of Ground Truth: Expert consensus manual delineations. These refer to the manually generated contours of bladder, colon, and femoral heads by two expert truthers based on US clinical guidelines.

8. Sample Size for the Training Set

For Synthetic CT Generation

• Training Set Sample Size: 244 patients. This dataset comprised MR and CT images.

For Auto Contouring

• Training Set Sample Size: 175 patients. This dataset comprised MR images.

9. How Ground Truth for the Training Set was Established

For Synthetic CT Generation

• Ground Truth Establishment: The training data for synthetic CT generation comprised paired MR and CT images. The CT images inherently serve as the ground truth for tissue properties and attenuation values. These images were acquired in the treatment position at four hospitals.

For Auto Contouring

• Ground Truth Establishment: The training data for auto-contouring included MR images together with "in-house generated expert manual contours." This indicates that expert(s) within Spectronic Medical manually delineated the structures (bladder, colon, femoral heads) on the MR images to create the ground truth for training the model. The text also states that the truthers involved in the bench test were involved in the development of the product and the generation of the training dataset. It is specifically mentioned that these expert manual delineations for the training set were generated at a different time than those used for the bench test.

Ask a specific question about this device