(142 days)
AI-Rad Companion Brain MR is a post-processing image analysis software that assists clinicians in viewing, analyzing, and evaluating MR brain images.
Al-Rad Companion Brain MR provides the following functionalities:
- Automated segmentation and quantitative analysis of includual brain structures and white matter hyperintensities
- Quantitative comparison of brain structure with normative data from a healthy population
- Presentation of results of reporting that includes all numerical values as well as visualization of these results
AI-Rad Companion Brain MR VA40 is an enhancement to the predicate. AI-Rad Companion Brain MR VA20 (K193290). Just as in the predicate, AI-Rad Companion Brain MR addresses the automatic quantification and visual assessment of the volumetric properties of various brain structures based on T1 MPRAGE datasets. In AI-Rad Companion Brain MR VA40, the quantification and visual assessment extends to white matter hyperintensities on the basis of T1 MPRAGE and T2 weighted FLAIR datasets. These datasets are acquired as part of a typical head MR acquisition. The results are directly archived in PACS as this is the standard location for reading by radiologist. From a predefined list of 30 structures (e.g. Hippocampus, Left Frontal Grey Matter, etc.), volumetric properties are calculated as absolute and normalized volumes with respect to the total intercranial volume. The normalized values for a given patient are compared against age-matched mean and standard deviations obtained from a population of healthy reference subjects.
The white matter hyperintensities can be visualized as a 3D overlay map and the quantification in count and volume as per 4 brain regions in the report.
As an update to the previously cleared device, the following modifications have been made:
-
- Modified Intended Use Statement
-
- Addition of white matter hyperintensities overlay map, count and volume as per 4 brain regions
-
- Enhanced DICOM Structured Report (DICOM SR)
-
- Updated deployment structure
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
1. A table of acceptance criteria and the reported device performance:
| Validation Type | Acceptance Criteria | Reported Device Performance AVG | Reported Device Performance 95% CI |
|---|---|---|---|
| Volumetric Segmentation Accuracy (PCC) | PCC 95% Confidence Interval includes 0.91 | 0.98 | [0.97, 0.99] |
| Volumetric Segmentation Accuracy (ICC) | ICC 95% Confidence Interval includes 0.95 | 0.97 | [0.96, 0.98] |
| Voxel-wise Segmentation Accuracy | Mean Dice score >= 0.58 | 0.60 | [0.53, 0.63] |
| WMH Lesion-wise Segmentation Accuracy | Mean F1-score >= 0.57 | 0.60 | [0.57, 0.64] |
| Reproducibility | Lower Bound of the 95% Bootstrap CI Dice >= 0.63 | 0.79 | [0.77, 0.81] |
All reported device performance metrics meet or exceed the specified acceptance criteria, as their 95% Confidence Intervals either include the criterion or are entirely above it (for metrics requiring a minimum value).
Study Details
2. Sample size used for the test set and the data provenance:
- Test Set Sample Size: 64 subjects for the main testing cohort, and 25 subjects for the reproducibility cohort.
- Total Subjects: 89 subjects (64 + 25)
- Total Studies: 164 studies (64 for testing cohort, and 100 for reproducibility cohort)
- Data Provenance (Country of Origin): United States (Cleveland, Baltimore, New York, ADNI), Switzerland (Lausanne, CLEMENS), France (Montpellier).
- Retrospective or Prospective: The text does not explicitly state whether the data was retrospective or prospective, but the description of "test data" and "training data" suggests retrospective data collection from existing datasets.
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- Number of Experts: Three distinct groups involved in establishing ground truth for each dataset: an annotator, a reviewer, and a clinical expert. The text implies a total of three individuals per case, forming a disjoint group (i.e., three different people for annotation, review, and expert correction).
- Qualifications of Experts: The text refers to them as "in-house annotators," "in-house reviewer," and "referred clinical expert." Specific qualifications (e.g., years of experience, board certification) are not explicitly detailed beyond the "clinical expert" designation.
4. Adjudication method for the test set:
- Adjudication Method: A multi-step process: "For each dataset, three sets of white matter hyperintensity ground truth are annotated manually. Each set is annotated by a disjoint group of annotator, reviewer, and clinical expert with the expert randomly assigned per case to minimize annotation bias. For each test dataset, the three initial annotations are annotated by three different in-house annotators. Then, each initial annotation is reviewed by the in-house reviewer. Afterwards, each initial annotation is reviewed by the referred clinical expert. The clinical expert reviews and corrects the initial annotation of the WMH according to the annotation protocol."
- This is a form of cascading/sequential review and consensus, rather than a direct voting or "X+Y" adjudication, with the clinical expert performing the final review and correction.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:
- No, an MRMC comparative effectiveness study was not done. The study focuses on the standalone performance of the AI algorithm against expert-established ground truth, not on how human readers' performance improves with or without AI assistance.
6. If a standalone (i.e., algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance study was done. The testing validated the "AI-Rad Companion Brain MR WMH" algorithm's performance by comparing its outputs directly to manually annotated ground truth. The results table explicitly presents the "Volumetric Segmentation Accuracy," "Voxel-wise Segmentation Accuracy," "WMH Lesion-wise Segmentation Accuracy," and "Reproducibility" of the device.
7. The type of ground truth used:
- Expert Consensus / Expert-Annotated Ground Truth. The ground truth was established through a multi-step manual annotation, review, and correction process by "in-house annotators," "in-house reviewer," and a "clinical expert."
8. The sample size for the training set:
- The sample size for the training set is not explicitly stated. The text only mentions: "The training data used for the training of the White matter hyperintensity algorithm is independent of the data used to test the white matter hyperintensity algorithm."
9. How the ground truth for the training set was established:
- The text does not provide details on how the ground truth for the training set was established. It only ensures that the training data and testing data are independent.
{0}------------------------------------------------
April 15, 2022
Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: a symbol on the left and the FDA acronym with the agency's name on the right. The symbol on the left is the Department of Health & Human Services logo. The right side of the logo has the FDA acronym in a blue square, followed by the words "U.S. FOOD & DRUG ADMINISTRATION" in blue text.
Siemens Healthcare GmBh % Kira Kuzmenchuk Regulatory Affairs Specialist Siemens Medical Solutions USA, Inc. 40 Liberty Blvd., Mail Code 65-1 MALVERN PA 19355
Re: K213706
Trade/Device Name: AI-Rad Companion Brain MR Regulation Number: 21 CFR 892.2050 Regulation Name: Medical image management and processing system Regulatory Class: Class II Product Code: QIH Dated: March 11, 2022 Received: March 15, 2022
Dear Kira Kuzmenchuk:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for
{1}------------------------------------------------
devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely.
For
Thalia T. Mills, Ph.D. Director Division of Radiological Health OHT7: Office of In Vitro Diagnostics and Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
{2}------------------------------------------------
Indications for Use
510(k) Number (if known)
Device Name
AI-RAD Companion Brain MR
Indications for Use (Describe)
AI-Rad Companion Brain MR is a post-processing image analysis software that assists clinicians in viewing, analyzing, and evaluating MR brain images.
Al-Rad Companion Brain MR provides the following functionalities:
- Automated segmentation and quantitative analysis of includual brain structures and white matter hyperintensities
- Quantitative comparison of brain structure with normative data from a healthy population
- Presentation of results of reporting that includes all numerical values as well as visualization of these results
Type of Use (Select one or both, as applicable)
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
{3}------------------------------------------------
Image /page/3/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots.
510(k) SUMMARY FOR AI-Rad Companion Brain MR
Submitted by: Siemens Medical Solutions USA, Inc. 40 Liberty Boulevard Malvern, PA 19355 Date Prepared: November 22, 2021
This summary of 510(k) safety and effectiveness information is being submitted in accordance with the requirements of Safe Medical Devices Act of 1990 and 21 CFR §807.92.
1. Submitter
| Importer/Distributor | Siemens Medical Solutions USA, Inc.40 Liberty BoulevardMalvern, PA 19355Mail Code: 65-1ARegistration Number: 2240869 |
|---|---|
| Manufacturing Site | Siemens Healthcare GmbHHenkestrasse 127Erlangen, Germany 91052Registration Number: 3002808157 |
2. Contact Person
Kira Kuzmenchuk Regulatory Affairs Specialist Siemens Medical Solutions USA, Inc. 40 Liberty Boulevard Mail Code: 65-1A Malvern, PA 19335 Phone: +1 (484) 901 - 9471 Email: kira.kuzmenchuk@siemens-healthineers.com
3. Device Name and Classification
| Product Name: | AI-Rad Companion Brain MR |
|---|---|
| Trade Name: | AI-Rad Companion Brain MR |
| Classification Name: | Medical Image Management and Processing System |
{4}------------------------------------------------
Image /page/4/Picture/0 description: The image shows the Siemens Healthineers logo. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange. To the right of the word "Healthineers" is a graphic of orange dots arranged in a circular pattern.
| Classification Panel: | |
|---|---|
| CFR Section: | |
| Device Class: | |
| Product Code: |
Radiology 21 CFR §892.2050 Class II OIH
4. Predicate Device
| Product Name: | AI-Rad Companion Brain MR |
|---|---|
| Propriety Trade Name: | AI-Rad Companion Brain MR |
| 510(k) Number: | K193290 |
| Clearance Date: | June 17, 2020 |
| Classification Name: | Picture Archiving and Communication System |
| Classification Panel: | Radiology |
| CFR Section: | 21 CFR §892.2050 |
| Secondary CFR Section: | 21 CFR §892.1000 |
| Device Class: | Class II |
| Primary Product Code: | LLZ |
| Secondary Product Code: | LNH |
| Recall Information: | N/A |
5. Indications for Use
AI-Rad Companion Brain MR is a post-processing image analysis software that assists clinicians in viewing, analyzing and evaluating MR brain images.
AI-Rad Companion Brain MR provides the following functionalities:
- Automatic segmentation and quantitative analysis of individual brain structures and white . matter hyperintensities
- Quantitative comparison of each brain structure with normative data from a healthy population
- Presentation of results for reporting that includes all numerical values as well as visualization of these results.
6. Device Description
AI-Rad Companion Brain MR VA40 is an enhancement to the predicate. AI-Rad Companion Brain MR VA20 (K193290). Just as in the predicate, AI-Rad Companion Brain MR addresses the automatic quantification and visual assessment of the volumetric properties of various brain structures based on T1 MPRAGE datasets. In AI-Rad Companion Brain MR VA40, the quantification and visual assessment extends to white matter hyperintensities on the basis of T1 MPRAGE and T2 weighted FLAIR datasets. These datasets are acquired as part of a typical head MR acquisition. The results are directly archived in PACS as this is the standard location for reading by radiologist. From a predefined list of 30 structures (e.g. Hippocampus, Left Frontal
{5}------------------------------------------------
Healthineer
Grey Matter, etc.), volumetric properties are calculated as absolute and normalized volumes with respect to the total intercranial volume. The normalized values for a given patient are compared against age-matched mean and standard deviations obtained from a population of healthy reference subjects.
The white matter hyperintensities can be visualized as a 3D overlay map and the quantification in count and volume as per 4 brain regions in the report.
As an update to the previously cleared device, the following modifications have been made:
-
- Modified Intended Use Statement
-
- Addition of white matter hyperintensities overlay map, count and volume as per 4 brain regions
-
- Enhanced DICOM Structured Report (DICOM SR)
-
- Updated deployment structure
7. Substantially Equivalent (SE) and Technological Characteristics
The intended use of the predicate device and the subject device are equivalent. The main difference is that AI-Rad Companion Brain MR VA40 adds the additional analysis of white matter hyperintensities compared to the predicate, AI-Rad Companion Brain MR VA20.
The subject device, AI-Rad Companion Brain MR VA40 is substantially equivalent with regard to the intended use and technical characteristics compared to the predicate device, AI-Rad Companion Brain MR VA20 (K193290), with respect to the software features, functionalities, and core algorithms. The additional features, enhancements and improvements provided in AI-Rad Companion Brain MR VA40 increase the usability and reduce the complexity of the imaging workflow for the clinical user. The white matter hyperintensity algorithm within AI-Rad Companion Brain MR VA40 is equivalent to the algorithm in icobrain (K192130). Icobrain serves as a reference device within this submission and a dedicated comparison of technological characteristics is provided.
| Subject Device:AI-Rad CompanionBrain MR VA40 | Predicate Device:AI-Rad CompanionBrain MR VA20(K193290) | Reference Device:icobrain (K192130) | |
|---|---|---|---|
| Indications forUse | AI-Rad CompanionBrain MR is a post-processing imageanalysis software thatassists clinicians inviewing, analyzing, andevaluating MR brainimages. | AI-Rad CompanionBrain MR is a post-processing imageanalysis software thatassists clinicians inviewing, analyzing, andevaluating MR brainimages. | icobrain is intendedfor automaticlabeling,visualization andvolumetricquantification ofsegmentable brainstructures |
The risk analysis and non-clinical data support that both devices perform equivalently and do not raise different questions of the safety and effectiveness.
{6}------------------------------------------------
Image /page/6/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
{7}------------------------------------------------
Image /page/7/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
| and volumetry ofMPRAGE data. | and volumetry ofMPRAGE data. | volumetry ofMPRAGE data. | |
|---|---|---|---|
| BrainMorphometryQuantification | Calculation of labelmaps (display of brainsegmentation) andpartially combined labelmaps (fused with theprocessed MPRAGEdata). | Calculation of labelmaps (display of brainsegmentation) andpartially combinedlabel maps (fused withthe processedMPRAGE data). | Normalized andunnormalizedvolume and volumechanges of differentbrain structures. |
| BrainMorphometry:Deviation Map | Calculation of deviationmap (representation ofbrain status in relation toreference data) andpartially combineddeviation maps (fusedwith the processedMPRAGE data) Usercustomizable color labelsfor the overlay map. | Calculation ofdeviation map(representation of brainstatus in relation toreference data) andpartially combineddeviation maps (fusedwith the processedMPRAGE data) Usercustomizable colorlabels for the overlaymap. | Not available |
| Brain WhiteMatterHyperintensitiesSegmentation | Pre-processingfunctionality forautomatic segmentationand volumetry ofMPRAGE and FLAIRdata. | Not available | Image processing forautomaticsegmentation andvolumetry of FLAIRdata. |
| Brain WhiteMatterHyperintensitiesQuantification | Calculation of whitematter hyperintensitiescount and volume as per4 brain regions. | Not available | Unnormalizedvolume and volumechanges of FLAIRwhite matterhyperintensities asper 4 brain regions |
| Brain WhiteMatterHyperintensitiesMap | Calculation of whitematter hyperintensitiesmap fused with theprocessed FLAIR dataUser customizable colorlabels for the overlaymap. | Not available | Calculation of whitematterhyperintensities mapoverlaid with theFLAIR data |
| Distribution &Archiving | Creation of an imageseries for a morphometry | Creation of an imageseries for a | Automatic transferof generated image |
{8}------------------------------------------------
Image /page/8/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots arranged in a pattern.
| report. Automatictransfer of generatedmaps and morphometryreport to a PACS system. | morphometry report.Automatic transfer ofgenerated maps andmorphometry report toa PACS system. | series and report to aPACS system. | |
|---|---|---|---|
| User InterfaceConfirmation | Confirmation UI withbasic visualizationfunctionality | Confirmation UI withbasic visualizationfunctionality | Not available. |
| User InterfaceConfiguration | Configuration UI | Configuration UI | Not available |
| Architecture | Cloud solution and Edgecomponents deployed oncustomer premise. | Cloud only solutionwith no componentsdeployed on customerpremise. | Cloud only solutionwith no componentsdeployed oncustomer premise. |
| DICOM SR | DICOM structuredreport representation of anatural language report | Basic morphometryreport | DICOM structuredreport |
Table 1: Comparison table for Al-Rad Companion Brain MR VA40, predicate device Al-Rad Companion Brain MR VA20 (K193290) and reference device icobrain (K192130)
Siemens Healthineers has determined that icobrain (K192130) has similar technological and performance characteristics with respect to the segmentation and quantification of white matter hyperintense lesions. Icobrain produces reports that identify unnormalized volumes and volume changes of FLAIR white matter hyperintensities of four different regions (juxtacortical, periventricular, infratentorial, deep white matter) using equivalent methodology used in the subject device, AI-Rad Companion Brain MR VA40.
The conclusions from all verification and validation data suggest that these enhancements are equivalent with respect to safety and effectiveness of the predicate device. These modifications do not change the intended use of the product. Siemens is of the opinion that AI-Rad Companion Brain MR VA40 is substantially equivalent to the currently marketed device, AI-Rad Companion Brain MR VA20 (K193290).
8. Nonclinical Tests
Non-clinical tests were conducted to test the functionality of AI-Rad Companion Brain MR. Software validation and bench testing have been conducted to assess the performance claims as well as the claim of substantial equivalence to the predicate device.
AI-Rad Companion has been tested to meet the requirements of conformity to multiple industry standards. Non-clinical performance testing demonstrates that AI-Rad Companion Brain MR complies with the FDA guidance document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" (May 11, 2005) as well as with the following voluntary FDA recognized Consensus Standards listed in Section 9.
{9}------------------------------------------------
Image /page/9/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the word "Healthineers" is a graphic of orange dots.
Verification and Validation
Software documentation for a Moderate Level of Concern software, per FDA's Guidance Document "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" issued on May 11, 2005, is also included as part of this submission. The performance data demonstrates continued conformance with special controls for medical devices containing software. Non-clinical tests were conducted on the subject device during product development.
Software "bench" testing in the form of Unit, System and Integration tests were performed to evaluate the performance and functionality of the new features and software updates. All testable requirements in the Requirement Specifications and the Risk Analysis have been successfully verified and traced in accordance with the Siemens Healthineers DH product development (lifecycle) process. Human factor usability validation is addressed in system testing and usability validation test records. Software verification and regression testing have been performed successfully to meet their previously determined acceptance criteria as stated in the test plans.
Siemens Healthineers adheres to the cybersecurity requirements as defined the FDA Guidance "Content of Premarket Submissions for Management for Cybersecurity in Medical Devices," issued October 2, 2014 by implementing a process of preventing unauthorized access, modifications, misuse or denial of use, or the unauthorized use of information that is stored, accessed, or transferred from a medical device to an external recipient.
9. Performance Software Validation
To validate the AI-Rad Companion Brain MR software from clinical perspective, the white matter hyperintensities segmentation and analysis algorithm underwent a scientific evaluation. The results of clinical data-based software validation for the subject device AI-Rad Companion Brain demonstrated equivalent performance in comparison to the reference device. A complete scientific evaluation report is provided in support of the device modifications. The brain morphometry algorithm, unchanged from the predicate, did not undergo a new scientific evaluation.
Performance testing for AI-Rad Companion Brain MR WMH was performed on Siemens Healthineers test data from 89 subjects, which included Multiple Sclerosis patients (MS), Alzheimer's patients (AD), cognitive impaired (CI) and healthy controls (HC). Testing data has balanced distribution with respect to gender and age of the patient according to target patient population and field strength of the MR scanner used. Accuracy was validated by comparing the results of the subject device to manual annotated ground truth from three radiologists. Three sets of white matter hyper-intensity ground truth were annotated manually by a disjoint group of annotator, reviewer, and clinical expert, with each expert randomly assigned per case to minimize annotation bias. Reproducibility studies were conducted to demonstrate the robustness of the WMH segmented by our device with respect to instrumental and patient noise.
Acceptance Criteria:
{10}------------------------------------------------
Image /page/10/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
| Validation Type | Acceptance Criteria |
|---|---|
| Volumetric Segmentation Accuracy | PCC 95% Confidence Interval includes 0.91ICC 95% Confidence Interval includes 0.95 |
| Voxel-wise Segmentation Accuracy | Mean Dice score >= 0.58 |
| WMH Lesion-wise Segmentation Accuracy | Mean F1-score >= 0.57 |
| Reproducibility | Lower Bound of the 95% Bootstrap CI Dice >=0.63 |
Summary Performance data, Standard Deviations & CIs:
| Volumetric Segmentation | Voxel-wiseSegmentation | WMH Lesion-wiseSegmentation | Reproducibility | ||
|---|---|---|---|---|---|
| PCC | ICC | Dice | F1-score | Dice | |
| AVG | 0.98 | 0.97 | 0.60 | 0.60 | 0.79 |
| STD | n.a. | n.a. | 0.18 | 0.14 | 0.11 |
| 95% CI | [0.97,0.99] | [0.96,0.98] | [0.53,0.63] | [0.57,0.64] | [0.77,0.81] |
Testing Data Information:
| Reproducibility Cohort | Testing Cohort | |
|---|---|---|
| # Subjects | 25 | 64 |
| # Studies | 100 | 64 |
| # of Females | 12 | 35 |
| # of Males | 13 | 29 |
| Age Range | 23-55 | 19-83 |
{11}------------------------------------------------
Image /page/11/Picture/0 description: The image shows the Siemens Healthineers logo. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots arranged in a circular pattern.
| Medical Indication | MS – all | MS – 36Cognitive Impairment - 17Other neurological disease - 5Cognitive Normal – 5Unknown – 1 |
|---|---|---|
| Scan Protocol | 3D T1w MPRAGE3D T2w FLAIR | T1w MPRAGET2w FLAIR |
| Field Strength | 3T | 1.5T: 303.0T: 34 |
| Manufacturer | Siemens | Siemens |
| Data Origin | Cleveland (US): 16Baltimore (US): 9 | New York (US): 25ADNI (US): 12Lausanne (CH): 8CLEMENS (CH): 10Montpellier (FR): 9 |
Standard Annotation Process:
For each dataset, three sets of white matter hyperintensity ground truth are annotated manually. Each set is annotated by a disjoin group of annotator, reviewer, and clinical expert with the expert randomly assigned per case to minimize annotation bias. For each test dataset, the three initial annotations are annotated by three different in-house annotators. Then, each initial annotation is reviewed by the in-house reviewer. Afterwards, each initial annotation is reviewed by the referred clinical expert. The clinical expert reviews and corrects the initial annotation of the WMH according to the annotation protocol.
Testing & Training Data Independence:
The training data used for the training of the White matter hyperintensity algorithm is independent of the data used to test the white matter hyperintensity algorithm.
10. Clinical Tests
No clinical tests were conducted to test the performance and functionality of the modifications introduced within AI-Rad Companion Brain MR. Verification and validation of the enhancements and improvements have been performed and these modifications have been validated for their intended use. The data from these activities were used to support the subject device and the substantial equivalence argument. No animal testing has been performed on the subject device.
{12}------------------------------------------------
Image /page/12/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots arranged in a circular pattern.
11. Safety and Effectiveness
The device labeling contains instructions for use and any necessary cautions and warnings to ensure safe and effective use of the device.
Risk management is ensured via ISO 14971:2019 compliance to identify and provide mitigation of potential hazards in a risk analysis early in the design phase and continuously throughout the development of the product. These risks are controlled via measures realized during software development, testing and product labeling.
Furthermore, the device is intended for healthcare professionals familiar with the post processing of magnetic resonance images.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).