Search Results

icobrain aria is a computer-assisted detection (CADe) and diagnosis (CADx) software device to be used as a concurrent reading aid to help trained radiologists in the detection, assessment and characterization of Amyloid Related Imaging Abnormalities (ARIA) from a set of brain MR images. The software provides information about the presence, location, size, severity and changes of ARIA-E (brain edema or sulcal effusions) and ARIA-H (hemosiderin deposition, including microhemorrhage and superficial siderosis). Patient management decisions should not be made solely on the basis of analysis by icobrain aria.

icobrain aria is a software-only device for assisting radiologists with the detection of amyloid-related imaging abnormalities (ARIA) on brain MRI scans of Alzheimer's disease patients under an amyloid beta-directed antibody therapy. The device utilizes 2D fluid-attenuated inversion recovery (FLAR) for the detection of ARIA-E (edema/sulcal effusion) and 2D T2* gradient echo (T2*-GRE) for the detection of ARIA-H (hemosiderin deposition).

icobrain aria automatically processes input brain MRI scans in DICOM format from two time points and generates annotated DICOM images and an electronic report.

Here's a summary of the acceptance criteria and study that proves the device meets them, based on the provided text:

icobrain aria: Acceptance Criteria and Performance Study Summary

1. Table of Acceptance Criteria and Reported Device Performance

The acceptance criteria are not explicitly listed in a single, dedicated table with pass/fail thresholds. Instead, they are implicitly defined by the statistically significant improvements demonstrated in the clinical (MRMC) study, and the "in line with human experts" conclusion from standalone performance. The document focuses on showing the effect size of the improvement rather than pre-defined absolute thresholds for sensitivity, specificity, or AUC for human-AI combined performance. For standalone metrics, it reports specific values and concludes they are "in line with the performance of human experts," suggesting the internal acceptance criteria were met.

Therefore, the table below will summarize the reported performance results from the clinical study, which implicitly met the acceptance criteria by demonstrating significant improvement over unassisted reading.

2. Sample Size Used for the Test Set and Data Provenance

• Test Set Sample Size: 199 cases.
• Data Provenance: MRI datasets from subjects diagnosed with Alzheimer's disease. To guarantee independence, test data subjects were not included in the training set.
  • Country of Origin: More than 100 sites in 20 countries. Approximately half the data originated from the US and the other half from outside the US.
  • Retrospective/Prospective: The study used retrospective data from clinical trials (aducanumab clinical trials PRIME (NCT02677572), EMERGE (NCT02484547), and ENGAGE (NCT02477800)). This data provenance applies to both training and testing datasets.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

• Number of Experts: A consensus of 3 experts was used for the clinical (MRMC) study ground truth. For standalone testing, the ground truth was established by unspecified "expert neuroradiologists."
• Qualifications of Experts:
  • Clinical Study (MRMC): Experts who performed "safety ARIA reading in clinical trials for Aβ-directed antibody therapies in AD."
  • Standalone Testing: "expert neuroradiologists (with experience performing safety ARIA reading in clinical trials for Aβ-directed antibody therapies in AD) manually segmented both ARIA-H findings." This indicates they had prior, relevant experience.

4. Adjudication Method for the Test Set

• Adjudication Method: "A consensus of 3 experts" was used to establish the ground truth for the clinical (MRMC) study. The specific consensus method (e.g., majority vote, discussion to agreement) is not detailed, but the term "consensus" implies a collective agreement process.

5. If a Multi Reader Multi Case (MRMC) Comparative Effectiveness Study was Done, and Effect Size of Improvement

• MRMC Study Done: Yes, a fully-crossed MRMC retrospective reader study was conducted.
• Effect Size (AUC difference, Assisted vs. Unassisted):
  • ARIA-E Detection: +0.051 AUC (95% CI [0.020, 0.083]), p=0.001
  • Pooled ARIA-H Detection: +0.044 AUC (95% CI [0.017, 0.070]), p=0.001
  • ARIA-H Microhemorrhages: +0.029 AUC (95% CI [0.002, 0.055]), p=0.032
  • ARIA-H Superficial Siderosis: +0.063 AUC (95% CI [0.023, 0.102]), p=0.003

Readers also showed significant increases in sensitivity, significant decreases in inter-reader variability, and were on average faster when assisted.

6. If a Standalone (i.e. Algorithm only without human-in-the-loop performance) was Done

• Standalone Study Done: Yes, "icometrix conducted standalone performance assessments."
  • Standalone Performance Highlights (Main Test Set on 199 cases):
    • ARIA-E Diagnosis: Sensitivity 0.94, Specificity 0.67, AUC 0.84
    • ARIA-H Diagnosis: Sensitivity 0.87, Specificity 0.66, AUC 0.81
    • ARIA-E Finding-level: True Positive Rate 69.1%, False Positive findings per case 0.7
    • ARIA-H New Microhemorrhages Finding-level: True Positive Rate 66.1%, False Positive findings per case 0.9
    • ARIA-H New Superficial Siderosis Finding-level: True Positive Rate 62.5%, False Positive findings per case 0.1
  • The document concludes that standalone performance was "in line with the performance of human experts."

7. The Type of Ground Truth Used

• Ground Truth Type: Expert consensus for the clinical study (MRMC) and expert manual annotations for the standalone testing.
  • Details: For standalone testing, "expert neuroradiologists ... manually segmented both ARIA-E and ARIA-H findings. Ground truth ARIA measurements were derived from the expert manual annotated masks." For the MRMC study, ground truth was obtained via "a consensus of 3 experts."

8. The Sample Size for the Training Set

• Training Set Sample Size:
  • FLAIR images (for ARIA-E): 475 image pairs from 172 subjects.
  • T2-GRE images (for ARIA-H):* 326 image pairs from 177 subjects.

9. How the Ground Truth for the Training Set Was Established

• Ground Truth Establishment for Training Set: The data used for developing the algorithms "have been manually annotated by expert neuroradiologists with prior experience of reading ARIA in clinical trials of amyloid beta-directed antibody drugs." This implies manual annotation by experts served as the ground truth for training.

Ask a specific question about this device

icobrain ctp is an image processing software package to be used by trained professionals, including but not limited to physicians and medical technicians. The software runs on a standard "off-the-shelf" computer or a virtual platform, such as VM ware, and can be used to perform image processing, and communication of computed tomography (CT) perfusion scans of the brain. Data and images are acquired through DICOM-compliant imaging devices.

icobrain ctp provides both analysis and communication capabilities for dynamic imaging datasets that are acquired with CT Perfusion imaging protocols. Analysis includes calculation of parameters related to tissue flow (perfusion) and tissue blood volume. Results of image processing which include CT perfusion parameter maps generated from a raw CTP scan are exported in the standard DICOM format and may be viewed on existing radiological imaging viewers.

The input images are CT perfusion images. During the pre-processing, each scan is loaded from the DICOM format: the image data and relevant dicom tags are extracted. The image processing block calculates the perfusion parameters and the volumes of the Tmax abnormality (defined as tissue with delayed arrival) and the CBF abnormality (defined as tissue with delayed arrival and critically decreased cerebral blood flow). Finally, the computed measurements are summarized into an electronic report. Optionally if requested, Tmax and CBF abnormalities segmentations are overlaid on the input images and image volumes of perfusion parameters maps are sent.

Here's an analysis of the acceptance criteria and study details for the icobrain-ctp device, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The document describes several performance tests. The specific numerical acceptance criteria are not always explicitly stated in the same detail as the results, but the type of metric used and the general outcome can be inferred.

2. Sample Sizes Used for the Test Set and Data Provenance

• Sample Size for Clinical Test Set: Not explicitly stated, but mentioned as "a dataset of clinical CTP scans."
• Sample Size for Digital Phantom Test Set: Not explicitly stated, but described as "a wide range of clinically relevant values of perfusion parameters."
• Data Provenance (Clinical Test Set): "The subjects upon whom the software was tested include stroke patients." No specific country of origin is mentioned. It is described as a "clinical dataset," which typically implies retrospective use of existing patient data, but it is not definitively stated as prospective or retrospective.
• Data Provenance (Digital Phantom Test Set): Generated by simulating tracer kinetic theory.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications

The document mentions "manually delineated DWI images" and "manually annotated ROI" for establishing ground truth in the clinical accuracy experiments. However:

• Number of Experts: Not specified.
• Qualifications of Experts: Not specified (e.g., specific medical specialty, years of experience).

4. Adjudication Method for the Test Set

The adjudication method is not explicitly stated. The text refers to "manually delineated DWI images" and "manually annotated ROI," which suggests expert involvement, but whether multiple experts were involved and an adjudication process (like 2+1 or 3+1) was used is not detailed.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done

No, an MRMC comparative effectiveness study involving human readers with and without AI assistance was not described in the provided text. The performance evaluations focus on the algorithm's standalone performance against established ground truth (either from reference devices, manual expert delineation, or digital phantoms).

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done

Yes, the studies described are standalone (algorithm-only) performance evaluations. The device's output (e.g., Tmax and CBF abnormality volumes, perfusion parameter maps) is compared directly to reference data or ground truth without human interaction as part of the primary outcome assessment.

7. The Type of Ground Truth Used

• Clinical Dataset:
  • Comparison to a "reference device" for CBF and Tmax abnormality volumes.
  • "Manually delineated DWI images" for unbiased CBF abnormality volume.
  • "Manually annotated ROI" for ROI volume.
• Digital Phantom Dataset: Ground truth generated by "simulating tracer kinetic theory" for perfusion parameters (CBV, CBF, MTT).

8. The Sample Size for the Training Set

The document does not provide any information about the training set size or methodology. The performance testing section focuses solely on validation data.

9. How the Ground Truth for the Training Set Was Established

As no information is provided about the training set, there is also no information on how its ground truth was established.

Ask a specific question about this device

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR or NCCT images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR or NCCT images. icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

icobrain cross is intended to provide volumes from MR or NCCT images acquired at a single time point. icobrain long is intended to provide changes in volumes between two MR images that were acquired on the same scanner, with the same image acquisition protocol and with same contrast at two different timepoints. The results of icobrain cross cannot be compared with the results of icobrain long.

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR or NCCT images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR or NCCT images.

icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

• icobrain cross is intended to provide volumes from MR or NCCT images acquired at a single time point.
• icobrain long is intended to provide changes in volumes between two MR images that were acquired on the same scanner, with the same image acquisition protocol and with same contrast at two different timepoints.

The results of icobrain cross cannot be compared with the results of icobrain long.

The input images can be MR images (current icobrain software - KI6I 148 and KI80326) or CT images (current icobrain software - K181939). During the pre-processing, each scan is converted from DICOM format to NIFTI format. The image processing then performs the actual segmentation and calculates the measurements of the brain structures and abnormalities. Finally, the computed measurements are summarized into an electronic report and (some) segmentations are overlaid on the input images.

Here's a breakdown of the acceptance criteria and study details for the icobrain device, based on the provided text:

1. Acceptance Criteria and Reported Device Performance

The document states that a literature review was performed to set relevant acceptance criteria for each type of experiment. The acceptance criteria were based on the 90th percentile of the absolute differences in comparison to the validation threshold, and all experiments passed the acceptance criteria.

While specific numerical acceptance thresholds for individual metrics (e.g., specific volume differences) are not explicitly stated in the provided text, the overall performance is summarized by correlation coefficients:

Table: Reported Device Performance

Note: The text explicitly states that all experiments passed the acceptance criteria, implying that the device's performance metrics were within the predefined thresholds.

2. Sample Sizes and Data Provenance

• Sample Size for Test Set:
  • 463 MR subject datasets
  • 618 CT subject datasets
• Data Provenance: The document does not explicitly state the country of origin for the data. It mentions subjects included:
  • Healthy subjects
  • Alzheimer's disease patients
  • Multiple sclerosis patients
  • Traumatic brain injury patients
  • Depression patients
• Retrospective or Prospective: The document does not explicitly state whether the data was retrospective or prospective.

3. Number of Experts and Qualifications for Ground Truth (Test Set)

The document states that in accuracy experiments, the volumes/volume changes were compared to manually labeled ground truth volumes/volume changes. However, it does not specify the number of experts used or their qualifications (e.g., radiologist with X years of experience) for establishing this manual ground truth.

4. Adjudication Method (Test Set)

The document does not specify an adjudication method (e.g., 2+1, 3+1, none) for the test set ground truth. It only mentions "manually labeled ground truth."

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

The provided text does not mention a multi-reader multi-case (MRMC) comparative effectiveness study. Therefore, no effect size of human readers improving with AI vs. without AI assistance is reported. The study focuses on comparing the algorithm's performance against ground truth and test-retest reproducibility.

6. Standalone Performance Study

Yes, a standalone performance study was done. The accuracy experiments described compare the device's measured volumes and volume changes to simulated and/or manually labeled ground truth. The reproducibility experiments compare the device's output on test-retest imaging data sets. This directly assesses the algorithm's performance without human-in-the-loop.

7. Type of Ground Truth Used

The ground truth used for the accuracy experiments was:

• Simulated ground truth
• Manually labeled ground truth

For reproducibility experiments, test-retest consistency was used as a measure, rather than an independent ground truth.

8. Sample Size for the Training Set

The document does not provide the sample size used for the training set. It only describes the test set.

9. How Ground Truth for the Training Set Was Established

The document does not describe how the ground truth for the training set was established. The information provided focuses solely on the performance testing.

Ask a specific question about this device

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR or NCCT images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR or NCCT images.

icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

icobrain cross is intended to provide volumes from MR or NCCT images acquired at a single time point. icobrain long is intended to provide changes in volumes between two MR images that were acquired on the same scanner, with the same image acquisition protocol and with same contrast at two different timepoints. The results of icobrain cross cannot be compared with the results of icobrain long.

The input images can be MR images (current icobrain software - K161148 and K180326) or CT images. During the pre-processing, the modality and/or sequence of each scan is detected and each scan is converted from DICOM format to NIFTI format. The image processing then performs the actual segmentation and calculates the measurements of the brain structures and abnormalities. Finally, the computed measurements are summarized into an electronic report and (some) segmentations are overlaid on the input images, generating output images in DICOM format.

Since the processing of MR images remains unchanged compared to the currently approved icobrain software (see KI 6 I 148 and K180326), the remainder of this file will focus on the design of the software that processes CT images. We refer to the overall architecture focused on (pre)processing CT images as the CT pipeline.

Here's a summary of the acceptance criteria and the study that proves the device meets them, based on the provided text:

Acceptance Criteria and Device Performance

Study Details

1. Sample sizes used for the test set and data provenance:

   • Test Set Sample Size: 544 subject datasets.
   • Data Provenance: The subjects included TBI patients and potential dementia patients. The specific country of origin is not explicitly stated, but the submission is from a Belgian company (icometrix NV). The study appears to be retrospective, using existing subject datasets.

2. Number of experts used to establish the ground truth for the test set and their qualifications:

   • The document states that some ground truth was "manually segmented." However, it does not specify the number of experts, their qualifications (e.g., radiologist with X years of experience), or the specific process for this manual segmentation.
   • For lateral ventricles and whole brain volumes, the ground truth was "MR images segmented by the cleared icobrain 3.0 software." This implies a form of software-generated ground truth, rather than human expert ground truth for these specific metrics.

3. Adjudication method for the test set:

   • The document does not explicitly state an adjudication method (e.g., 2+1, 3+1). It only mentions that lesions, basal cisterns, lateral ventricles, and midline shift were compared to "manually segmented ground truth." Without further detail, it's impossible to determine the adjudication method for the manual segmentation.

4. If a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was done:

   • No, a multi-reader multi-case (MRMC) comparative effectiveness study was not explicitly mentioned or described. The study focuses on the standalone performance of the device against established ground truth and reproducibility, not on comparing human readers with and without AI assistance.

5. If a standalone performance (algorithm only without human-in-the-loop performance) was done:

   • Yes, a standalone performance study was done. The performance testing "demonstrate[s] the performance of the CT pipeline of icobrain 4.0" by validating its accuracy and reproducibility against ground truth. This is a direct assessment of the algorithm's performance without a human in the loop during the measurement generation.

6. The type of ground truth used:

   • Expert Consensus/Manual Segmentation: For lesions, basal cisterns, lateral ventricles, and midline shift, the ground truth was established by "manually segmented ground truth." This implies human expert input, but the specifics of consensus are not detailed.
   • Software-generated Ground Truth (Predicate Device): For lateral ventricles and whole brain volumes, the ground truth was "MR images segmented by the cleared icobrain 3.0 software."

7. The sample size for the training set:

   • The document does not provide the sample size for the training set. It only mentions the test set size of 544 subject datasets.

8. How the ground truth for the training set was established:

   • The document does not specify how the ground truth for the training set was established, as the size and details of the training set are not provided. The technical characteristics state "segmentation by classical machine learning and deep learning (in our case supervised voxel classification with Convolutional Neural Networks)," which implies that a training set with established ground truth would have been necessary for supervised learning.

Ask a specific question about this device

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR images. icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

icobrain cross is intended to provide volumes from images acquired at a single timepoint icobrain long is intended to provide changes in volumes between two images that were acquired on the samer, with the same image acquisition protocol and with same contrast at two different timepoints The results of icobrain cross cannot be compared with the results of icobrain long.

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR images.

icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

• icobrain cross is intended to provide volumes from images acquired at a single timepoint
• · icobrain long is intended to provide changes in volumes between two images that were acquired on the same scanner, with the same image acquisition protocol and with same contrast at two different timepoints

The results of icobrain cross cannot be compared with the results of icobrain long.

As input, icobrain uses TI-weighted and a fluid-attenuated inversion recovery (FLAIR) DICOM MR images from a single or from multiple time points. In case of multiple time points, i.e. multiple MRI scans from the same subject, for each time point one FLAIR and one TI image are used as input. During the pre-processing, the scan type (TI, FLAIR) is detected for every input image before it is converted from DICOM format to NIFTI format. The image processing then performs the actual segmentation and calculates the volumes of the brain structures. In case MRI scans from the same subject on multiple time points are available, the changes in volume of the brain structures are calculated as well. Finally, the computed volumes and volume changes (in case of multiple time points) are summarized into an electronic report and (some) segmentations are overlaid on the input images.

The provided text describes the performance testing of icobrain, focusing on its accuracy and reproducibility for volumetric quantification of brain structures from MR images.

Here's a breakdown of the requested information based on the provided text:

1. A table of acceptance criteria and the reported device performance

The document mentions that a literature review was performed to set relevant acceptance criteria for each type of experiment. However, the specific numerical acceptance criteria (e.g., minimum Pearson correlation or ICC values) are not explicitly stated in the provided text.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

• Sample Size (Test Set): "The experiments encompassed 463 subject datasets in total."
• Data Provenance: The document states, "The subjects upon whom the device was tested include healthy subjects, Alzheimer's disease patients, traumatic brain injury patients, depression patients." The country of origin and whether the data was retrospective or prospective are not specified.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

• The document states, "In the accuracy experiments, the volumes / volume changes are compared to simulated and/or manually labeled ground truth volumes / volume changes".
• The number of experts and their qualifications for establishing the ground truth are not specified.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

• The document mentions "manually labeled ground truth" but does not specify any adjudication method used for cases where multiple experts might have been involved in labeling.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

• The document does not mention a multi-reader multi-case (MRMC) comparative effectiveness study or any assessment of human reader improvement with AI assistance. The study focuses on the standalone performance of the device against a ground truth.

6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done

• Yes, a standalone performance study was done. The description indicates the device's measured volumes/volume changes were compared to ground truth, which implies an algorithm-only evaluation. The statement "This software is intended to automate the current manual process" further supports this.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc.)

• The ground truth used was "simulated and/or manually labeled ground truth volumes / volume changes." This implies expert manual segmentation and/or simulated data.

8. The sample size for the training set

• The document does not specify the sample size used for the training set. It only discusses the "463 subject datasets" used for testing.

9. How the ground truth for the training set was established

• The document does not specify how the ground truth for the training set was established, as the details focus on the "experiments" (performance testing).

Ask a specific question about this device

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR images.

icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

Icobrain cross is intended to provide volumes from images acquired at a single timepoint

icobrain long is intended to provide changes in volumes between two images that were acquired on the same scanner, with the same image acquisition protocol and with same contrast at two different timepoints

The results of icobrain cross cannot be compared with the results of icobrain long.

icobrain is intended for automatic labeling, visualization and volumetric quantification of segmentable brain structures from a set of MR images. This software is intended to automate the current manual process of identifying, labeling and quantifying the volume of segmentable brain structures identified on MR images.

icobrain consists of two distinct image processing pipelines: icobrain cross and icobrain long.

• icobrain cross is intended to provide volumes from images acquired at a single timepoint
• icobrain long is intended to provide changes in volumes between two images that were acquired on the same scanner, with the same image acquisition protocol and with same contrast at two different timepoints

The results of icobrain cross cannot be compared with the results of icobrain long.

As input, icobrain uses TI-weighted and fluid-attenuated inversion recovery (FLAIR) DICOM MR images from a single or from multiple time points. In case of multiple time points, i.e. multiple MRI scans from the same subject, for each time point one FLAIR and one TI image are used as input. During the pre-processing, the scan type (TI, FLAIR) is detected for every input image before it is converted from DICOM format to NIFTI format. The image processing then performs the actual segmentation and calculates the volumes of the brain structures. In case MRI scans subject on multiple time points are available, the changes in volume of the brain structures are calculated as well. Finally, the computed volumes and volume changes (in case of multiple time points) are summarized into an electronic report and (some) segmentations are overlaid on the input images.

The software displays the following volumetric measures:

• normalized volume and volume changes of the whole brain (sum of white and grey matter),
• normalized volume and volume changes of grey matter,
• unnormalized volume and volume changes of FLAIR white matter hyperintensities.

Normalized whole brain and grey matter volumes are corrected for head size and are compared to a healthy population using a statistical model. The reported FLAIR white matter hyperintensities volumes are not normalized since they are not comparable to a reference population.

Here's a breakdown of the acceptance criteria and study information for the icobrain device, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance:

The document broadly mentions acceptance criteria but does not explicitly list them with numerical targets. Instead, it reports aggregated performance metrics.

2. Sample Size Used for the Test Set and Data Provenance:

• Sample Size: 349 subject datasets in total (encompassing all experiments: accuracy and reproducibility).
• Data Provenance: The document states the subjects include "healthy subjects, Alzheimer's disease patients, multiple sclerosis patients, traumatic brain injury patients, depression patients." It does not specify the country of origin, nor whether the data was retrospective or prospective.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts:

The document mentions "manually labeled ground truth volume changes" for accuracy experiments but does not specify the number of experts, their qualifications, or how the manual labeling was performed.

4. Adjudication Method for the Test Set:

Not specified in the provided text.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

• Was one done? Not explicitly mentioned. The study focuses on comparing the device's measurements to ground truth or test-retest data, not on human readers' performance with and without AI assistance.
• Effect size of human readers improvement: Not applicable, as an MRMC comparative effectiveness study involving human readers' improvement with AI assistance is not described.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study:

Yes, the study appears to be a standalone performance evaluation of the icobrain algorithm. It validates the "measured volume changes of the segmentable brain structures for accuracy and reproducibility" by comparing them to "simulated and/or manually labeled ground truth volume changes" and "test-retest image data sets." There is no mention of human-in-the-loop performance in the performance testing section.

7. Type of Ground Truth Used:

• Accuracy Experiments: "Simulated and/or manually labeled ground truth volume changes."
• Reproducibility Experiments: Test-retest image data sets.

8. Sample Size for the Training Set:

Not specified in the provided text. The document focuses on the performance testing dataset (349 subject datasets).

9. How the Ground Truth for the Training Set Was Established:

Not specified in the provided text.

Ask a specific question about this device