(142 days)
Yes
The device name "AI-Rad Companion Brain MR" explicitly includes "AI", and the description mentions "training data used for the training of the White matter hyperintensity algorithm" and "Performance testing for AI-Rad Companion Brain MR WMH was performed on Siemens Healthineers test data". This strongly indicates the use of AI/ML algorithms for image analysis and segmentation.
No.
This device is a post-processing image analysis software and its intended use is to assist clinicians in viewing, analyzing, and evaluating MR brain images by providing automated segmentation and quantitative analysis of brain structures. It does not provide treatment or direct therapy.
Yes
The device assists clinicians in viewing, analyzing, and evaluating MR brain images by providing automated segmentation, quantitative analysis, and comparison with normative data, which are all integral parts of the diagnostic process.
Yes
The device is described as "post-processing image analysis software" and its functionalities are purely software-based (segmentation, analysis, comparison, reporting). The description focuses on the software's capabilities and modifications, with no mention of accompanying hardware components required for its function beyond the input MR images.
Based on the provided information, this device is not an IVD (In Vitro Diagnostic).
Here's why:
- Intended Use: The intended use clearly states that the device is "post-processing image analysis software that assists clinicians in viewing, analyzing, and evaluating MR brain images." This describes a tool for analyzing medical images, not for performing tests on biological samples (like blood, urine, or tissue) outside of the body.
- Device Description: The description focuses on the analysis of MR brain images (T1 MPRAGE and T2 weighted FLAIR datasets) to quantify and visualize brain structures and white matter hyperintensities. This is image processing and analysis, not in vitro testing.
- Input: The input is MR brain images, which are generated by a medical imaging device, not biological samples.
- Functionality: The functionalities listed (segmentation, quantitative analysis, comparison with normative data, reporting) are all related to the analysis and interpretation of medical images.
IVD devices are specifically designed to perform tests on specimens derived from the human body to provide information for the diagnosis, treatment, or prevention of disease. This device operates on medical images, which are a representation of the body's internal structures, not a biological specimen itself.
No
The letter does not state that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device.
Intended Use / Indications for Use
AI-Rad Companion Brain MR is a post-processing image analysis software that assists clinicians in viewing, analyzing, and evaluating MR brain images.
Al-Rad Companion Brain MR provides the following functionalities:
- Automated segmentation and quantitative analysis of includual brain structures and white matter hyperintensities
- Quantitative comparison of brain structure with normative data from a healthy population
- Presentation of results of reporting that includes all numerical values as well as visualization of these results
Product codes
QIH
Device Description
AI-Rad Companion Brain MR VA40 is an enhancement to the predicate, AI-Rad Companion Brain MR VA20 (K193290). Just as in the predicate, AI-Rad Companion Brain MR addresses the automatic quantification and visual assessment of the volumetric properties of various brain structures based on T1 MPRAGE datasets. In AI-Rad Companion Brain MR VA40, the quantification and visual assessment extends to white matter hyperintensities on the basis of T1 MPRAGE and T2 weighted FLAIR datasets. These datasets are acquired as part of a typical head MR acquisition. The results are directly archived in PACS as this is the standard location for reading by radiologist. From a predefined list of 30 structures (e.g. Hippocampus, Left Frontal Grey Matter, etc.), volumetric properties are calculated as absolute and normalized volumes with respect to the total intercranial volume. The normalized values for a given patient are compared against age-matched mean and standard deviations obtained from a population of healthy reference subjects.
The white matter hyperintensities can be visualized as a 3D overlay map and the quantification in count and volume as per 4 brain regions in the report.
As an update to the previously cleared device, the following modifications have been made:
-
- Modified Intended Use Statement
-
- Addition of white matter hyperintensities overlay map, count and volume as per 4 brain regions
-
- Enhanced DICOM Structured Report (DICOM SR)
-
- Updated deployment structure
Mentions image processing
Yes
Mentions AI, DNN, or ML
Yes
Input Imaging Modality
MR brain images, T1 MPRAGE datasets, T2 weighted FLAIR datasets
Anatomical Site
Brain
Indicated Patient Age Range
Not Found
Intended User / Care Setting
Clinicians, healthcare professionals familiar with the post processing of magnetic resonance images.
Description of the training set, sample size, data source, and annotation protocol
The training data used for the training of the White matter hyperintensity algorithm is independent of the data used to test the white matter hyperintensity algorithm. Specifics of the training set (sample size, data source, and annotation protocol) are not provided.
Description of the test set, sample size, data source, and annotation protocol
Performance testing for AI-Rad Companion Brain MR WMH was performed on Siemens Healthineers test data from 89 subjects, which included Multiple Sclerosis patients (MS), Alzheimer's patients (AD), cognitive impaired (CI) and healthy controls (HC). Testing data has balanced distribution with respect to gender and age of the patient according to target patient population and field strength of the MR scanner used.
Testing Cohort:
Subjects: 64
Studies: 64
of Females: 35
of Males: 29
Age Range: 19-83
Medical Indication: MS – 36, Cognitive Impairment - 17, Other neurological disease - 5, Cognitive Normal – 5, Unknown – 1
Scan Protocol: T1w MPRAGE, T2w FLAIR
Field Strength: 1.5T: 30, 3.0T: 34
Manufacturer: Siemens
Data Origin: New York (US): 25, ADNI (US): 12, Lausanne (CH): 8, CLEMENS (CH): 10, Montpellier (FR): 9
Reproducibility Cohort:
Subjects: 25
Studies: 100
of Females: 12
of Males: 13
Age Range: 23-55
Medical Indication: MS – all
Scan Protocol: 3D T1w MPRAGE, 3D T2w FLAIR
Field Strength: 3T
Manufacturer: Siemens
Data Origin: Cleveland (US): 16, Baltimore (US): 9
Annotation Protocol:
Accuracy was validated by comparing the results of the subject device to manual annotated ground truth from three radiologists. Three sets of white matter hyper-intensity ground truth were annotated manually by a disjoint group of annotator, reviewer, and clinical expert, with each expert randomly assigned per case to minimize annotation bias. For each dataset, three sets of white matter hyperintensity ground truth are annotated manually. Each set is annotated by a disjoin group of annotator, reviewer, and clinical expert with the expert randomly assigned per case to minimize annotation bias. For each test dataset, the three initial annotations are annotated by three different in-house annotators. Then, each initial annotation is reviewed by the in-house reviewer. Afterwards, each initial annotation is reviewed by the referred clinical expert. The clinical expert reviews and corrects the initial annotation of the WMH according to the annotation protocol.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Study Type: Software validation.
Sample Size: 89 subjects (64 in testing cohort, 25 in reproducibility cohort).
Standalone Performance: Accuracy was validated by comparing the results of the subject device to manual annotated ground truth from three radiologists. Reproducibility studies were conducted to demonstrate the robustness of the WMH segmented by our device with respect to instrumental and patient noise.
Key Results:
Volumetric Segmentation Accuracy: PCC 95% Confidence Interval includes 0.91, ICC 95% Confidence Interval includes 0.95
Voxel-wise Segmentation Accuracy: Mean Dice score >= 0.58
WMH Lesion-wise Segmentation Accuracy: Mean F1-score >= 0.57
Reproducibility: Lower Bound of the 95% Bootstrap CI Dice >= 0.63
Summary Performance data, Standard Deviations & CIs:
Volumetric Segmentation:
PCC: AVG 0.98, 95% CI [0.97,0.99]
ICC: AVG 0.97, 95% CI [0.96,0.98]
Voxel-wise Segmentation Dice: AVG 0.60, STD 0.18, 95% CI [0.53,0.63]
WMH Lesion-wise Segmentation F1-score: AVG 0.60, STD 0.14, 95% CI [0.57,0.64]
Reproducibility Dice: AVG 0.79, STD 0.11, 95% CI [0.77,0.81]
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
PCC (Pearson Correlation Coefficient), ICC (Intraclass Correlation Coefficient), Dice score, F1-score.
Predicate Device(s)
Reference Device(s)
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).
0
April 15, 2022
Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: a symbol on the left and the FDA acronym with the agency's name on the right. The symbol on the left is the Department of Health & Human Services logo. The right side of the logo has the FDA acronym in a blue square, followed by the words "U.S. FOOD & DRUG ADMINISTRATION" in blue text.
Siemens Healthcare GmBh % Kira Kuzmenchuk Regulatory Affairs Specialist Siemens Medical Solutions USA, Inc. 40 Liberty Blvd., Mail Code 65-1 MALVERN PA 19355
Re: K213706
Trade/Device Name: AI-Rad Companion Brain MR Regulation Number: 21 CFR 892.2050 Regulation Name: Medical image management and processing system Regulatory Class: Class II Product Code: QIH Dated: March 11, 2022 Received: March 15, 2022
Dear Kira Kuzmenchuk:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for
1
devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely.
For
Thalia T. Mills, Ph.D. Director Division of Radiological Health OHT7: Office of In Vitro Diagnostics and Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known)
Device Name
AI-RAD Companion Brain MR
Indications for Use (Describe)
AI-Rad Companion Brain MR is a post-processing image analysis software that assists clinicians in viewing, analyzing, and evaluating MR brain images.
Al-Rad Companion Brain MR provides the following functionalities:
- Automated segmentation and quantitative analysis of includual brain structures and white matter hyperintensities
- Quantitative comparison of brain structure with normative data from a healthy population
- Presentation of results of reporting that includes all numerical values as well as visualization of these results
Type of Use (Select one or both, as applicable)
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
Image /page/3/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots.
510(k) SUMMARY FOR AI-Rad Companion Brain MR
Submitted by: Siemens Medical Solutions USA, Inc. 40 Liberty Boulevard Malvern, PA 19355 Date Prepared: November 22, 2021
This summary of 510(k) safety and effectiveness information is being submitted in accordance with the requirements of Safe Medical Devices Act of 1990 and 21 CFR §807.92.
1. Submitter
| Importer/Distributor | Siemens Medical Solutions USA, Inc.
40 Liberty Boulevard
Malvern, PA 19355
Mail Code: 65-1A
Registration Number: 2240869 |
|----------------------|--------------------------------------------------------------------------------------------------------------------------------------|
| Manufacturing Site | Siemens Healthcare GmbH
Henkestrasse 127
Erlangen, Germany 91052
Registration Number: 3002808157 |
2. Contact Person
Kira Kuzmenchuk Regulatory Affairs Specialist Siemens Medical Solutions USA, Inc. 40 Liberty Boulevard Mail Code: 65-1A Malvern, PA 19335 Phone: +1 (484) 901 - 9471 Email: kira.kuzmenchuk@siemens-healthineers.com
3. Device Name and Classification
| Product Name: | AI-Rad Companion Brain MR |
|---|---|
| Trade Name: | AI-Rad Companion Brain MR |
| Classification Name: | Medical Image Management and Processing System |
4
Image /page/4/Picture/0 description: The image shows the Siemens Healthineers logo. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange. To the right of the word "Healthineers" is a graphic of orange dots arranged in a circular pattern.
| Classification Panel: | |
|---|---|
| CFR Section: | |
| Device Class: | |
| Product Code: |
Radiology 21 CFR §892.2050 Class II OIH
4. Predicate Device
| Product Name: | AI-Rad Companion Brain MR |
|---|---|
| Propriety Trade Name: | AI-Rad Companion Brain MR |
| 510(k) Number: | K193290 |
| Clearance Date: | June 17, 2020 |
| Classification Name: | Picture Archiving and Communication System |
| Classification Panel: | Radiology |
| CFR Section: | 21 CFR §892.2050 |
| Secondary CFR Section: | 21 CFR §892.1000 |
| Device Class: | Class II |
| Primary Product Code: | LLZ |
| Secondary Product Code: | LNH |
| Recall Information: | N/A |
5. Indications for Use
AI-Rad Companion Brain MR is a post-processing image analysis software that assists clinicians in viewing, analyzing and evaluating MR brain images.
AI-Rad Companion Brain MR provides the following functionalities:
- Automatic segmentation and quantitative analysis of individual brain structures and white . matter hyperintensities
- Quantitative comparison of each brain structure with normative data from a healthy population
- Presentation of results for reporting that includes all numerical values as well as visualization of these results.
6. Device Description
AI-Rad Companion Brain MR VA40 is an enhancement to the predicate. AI-Rad Companion Brain MR VA20 (K193290). Just as in the predicate, AI-Rad Companion Brain MR addresses the automatic quantification and visual assessment of the volumetric properties of various brain structures based on T1 MPRAGE datasets. In AI-Rad Companion Brain MR VA40, the quantification and visual assessment extends to white matter hyperintensities on the basis of T1 MPRAGE and T2 weighted FLAIR datasets. These datasets are acquired as part of a typical head MR acquisition. The results are directly archived in PACS as this is the standard location for reading by radiologist. From a predefined list of 30 structures (e.g. Hippocampus, Left Frontal
5
Healthineer
Grey Matter, etc.), volumetric properties are calculated as absolute and normalized volumes with respect to the total intercranial volume. The normalized values for a given patient are compared against age-matched mean and standard deviations obtained from a population of healthy reference subjects.
The white matter hyperintensities can be visualized as a 3D overlay map and the quantification in count and volume as per 4 brain regions in the report.
As an update to the previously cleared device, the following modifications have been made:
-
- Modified Intended Use Statement
-
- Addition of white matter hyperintensities overlay map, count and volume as per 4 brain regions
-
- Enhanced DICOM Structured Report (DICOM SR)
-
- Updated deployment structure
7. Substantially Equivalent (SE) and Technological Characteristics
The intended use of the predicate device and the subject device are equivalent. The main difference is that AI-Rad Companion Brain MR VA40 adds the additional analysis of white matter hyperintensities compared to the predicate, AI-Rad Companion Brain MR VA20.
The subject device, AI-Rad Companion Brain MR VA40 is substantially equivalent with regard to the intended use and technical characteristics compared to the predicate device, AI-Rad Companion Brain MR VA20 (K193290), with respect to the software features, functionalities, and core algorithms. The additional features, enhancements and improvements provided in AI-Rad Companion Brain MR VA40 increase the usability and reduce the complexity of the imaging workflow for the clinical user. The white matter hyperintensity algorithm within AI-Rad Companion Brain MR VA40 is equivalent to the algorithm in icobrain (K192130). Icobrain serves as a reference device within this submission and a dedicated comparison of technological characteristics is provided.
| | Subject Device:
AI-Rad Companion
Brain MR VA40 | Predicate Device:
AI-Rad Companion
Brain MR VA20
(K193290) | Reference Device:
icobrain (K192130) |
|------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
| Indications for
Use | AI-Rad Companion
Brain MR is a post-
processing image
analysis software that
assists clinicians in
viewing, analyzing, and
evaluating MR brain
images. | AI-Rad Companion
Brain MR is a post-
processing image
analysis software that
assists clinicians in
viewing, analyzing, and
evaluating MR brain
images. | icobrain is intended
for automatic
labeling,
visualization and
volumetric
quantification of
segmentable brain
structures |
The risk analysis and non-clinical data support that both devices perform equivalently and do not raise different questions of the safety and effectiveness.
6
Image /page/6/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
7
Image /page/7/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
| | and volumetry of
MPRAGE data. | and volumetry of
MPRAGE data. | volumetry of
MPRAGE data. |
|-------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
| Brain
Morphometry
Quantification | Calculation of label
maps (display of brain
segmentation) and
partially combined label
maps (fused with the
processed MPRAGE
data). | Calculation of label
maps (display of brain
segmentation) and
partially combined
label maps (fused with
the processed
MPRAGE data). | Normalized and
unnormalized
volume and volume
changes of different
brain structures. |
| Brain
Morphometry:
Deviation Map | Calculation of deviation
map (representation of
brain status in relation to
reference data) and
partially combined
deviation maps (fused
with the processed
MPRAGE data) User
customizable color labels
for the overlay map. | Calculation of
deviation map
(representation of brain
status in relation to
reference data) and
partially combined
deviation maps (fused
with the processed
MPRAGE data) User
customizable color
labels for the overlay
map. | Not available |
| Brain White
Matter
Hyperintensities
Segmentation | Pre-processing
functionality for
automatic segmentation
and volumetry of
MPRAGE and FLAIR
data. | Not available | Image processing for
automatic
segmentation and
volumetry of FLAIR
data. |
| Brain White
Matter
Hyperintensities
Quantification | Calculation of white
matter hyperintensities
count and volume as per
4 brain regions. | Not available | Unnormalized
volume and volume
changes of FLAIR
white matter
hyperintensities as
per 4 brain regions |
| Brain White
Matter
Hyperintensities
Map | Calculation of white
matter hyperintensities
map fused with the
processed FLAIR data
User customizable color
labels for the overlay
map. | Not available | Calculation of white
matter
hyperintensities map
overlaid with the
FLAIR data |
| Distribution &
Archiving | Creation of an image
series for a morphometry | Creation of an image
series for a | Automatic transfer
of generated image |
8
Image /page/8/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots arranged in a pattern.
| | report. Automatic
transfer of generated
maps and morphometry
report to a PACS system. | morphometry report.
Automatic transfer of
generated maps and
morphometry report to
a PACS system. | series and report to a
PACS system. |
|---------------------------------|------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------|
| User Interface
Confirmation | Confirmation UI with
basic visualization
functionality | Confirmation UI with
basic visualization
functionality | Not available. |
| User Interface
Configuration | Configuration UI | Configuration UI | Not available |
| Architecture | Cloud solution and Edge
components deployed on
customer premise. | Cloud only solution
with no components
deployed on customer
premise. | Cloud only solution
with no components
deployed on
customer premise. |
| DICOM SR | DICOM structured
report representation of a
natural language report | Basic morphometry
report | DICOM structured
report |
Table 1: Comparison table for Al-Rad Companion Brain MR VA40, predicate device Al-Rad Companion Brain MR VA20 (K193290) and reference device icobrain (K192130)
Siemens Healthineers has determined that icobrain (K192130) has similar technological and performance characteristics with respect to the segmentation and quantification of white matter hyperintense lesions. Icobrain produces reports that identify unnormalized volumes and volume changes of FLAIR white matter hyperintensities of four different regions (juxtacortical, periventricular, infratentorial, deep white matter) using equivalent methodology used in the subject device, AI-Rad Companion Brain MR VA40.
The conclusions from all verification and validation data suggest that these enhancements are equivalent with respect to safety and effectiveness of the predicate device. These modifications do not change the intended use of the product. Siemens is of the opinion that AI-Rad Companion Brain MR VA40 is substantially equivalent to the currently marketed device, AI-Rad Companion Brain MR VA20 (K193290).
8. Nonclinical Tests
Non-clinical tests were conducted to test the functionality of AI-Rad Companion Brain MR. Software validation and bench testing have been conducted to assess the performance claims as well as the claim of substantial equivalence to the predicate device.
AI-Rad Companion has been tested to meet the requirements of conformity to multiple industry standards. Non-clinical performance testing demonstrates that AI-Rad Companion Brain MR complies with the FDA guidance document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" (May 11, 2005) as well as with the following voluntary FDA recognized Consensus Standards listed in Section 9.
9
Image /page/9/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the word "Healthineers" is a graphic of orange dots.
Verification and Validation
Software documentation for a Moderate Level of Concern software, per FDA's Guidance Document "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" issued on May 11, 2005, is also included as part of this submission. The performance data demonstrates continued conformance with special controls for medical devices containing software. Non-clinical tests were conducted on the subject device during product development.
Software "bench" testing in the form of Unit, System and Integration tests were performed to evaluate the performance and functionality of the new features and software updates. All testable requirements in the Requirement Specifications and the Risk Analysis have been successfully verified and traced in accordance with the Siemens Healthineers DH product development (lifecycle) process. Human factor usability validation is addressed in system testing and usability validation test records. Software verification and regression testing have been performed successfully to meet their previously determined acceptance criteria as stated in the test plans.
Siemens Healthineers adheres to the cybersecurity requirements as defined the FDA Guidance "Content of Premarket Submissions for Management for Cybersecurity in Medical Devices," issued October 2, 2014 by implementing a process of preventing unauthorized access, modifications, misuse or denial of use, or the unauthorized use of information that is stored, accessed, or transferred from a medical device to an external recipient.
9. Performance Software Validation
To validate the AI-Rad Companion Brain MR software from clinical perspective, the white matter hyperintensities segmentation and analysis algorithm underwent a scientific evaluation. The results of clinical data-based software validation for the subject device AI-Rad Companion Brain demonstrated equivalent performance in comparison to the reference device. A complete scientific evaluation report is provided in support of the device modifications. The brain morphometry algorithm, unchanged from the predicate, did not undergo a new scientific evaluation.
Performance testing for AI-Rad Companion Brain MR WMH was performed on Siemens Healthineers test data from 89 subjects, which included Multiple Sclerosis patients (MS), Alzheimer's patients (AD), cognitive impaired (CI) and healthy controls (HC). Testing data has balanced distribution with respect to gender and age of the patient according to target patient population and field strength of the MR scanner used. Accuracy was validated by comparing the results of the subject device to manual annotated ground truth from three radiologists. Three sets of white matter hyper-intensity ground truth were annotated manually by a disjoint group of annotator, reviewer, and clinical expert, with each expert randomly assigned per case to minimize annotation bias. Reproducibility studies were conducted to demonstrate the robustness of the WMH segmented by our device with respect to instrumental and patient noise.
Acceptance Criteria:
10
Image /page/10/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
| Validation Type | Acceptance Criteria |
|---|---|
| Volumetric Segmentation Accuracy | PCC 95% Confidence Interval includes 0.91 |
| ICC 95% Confidence Interval includes 0.95 | |
| Voxel-wise Segmentation Accuracy | Mean Dice score >= 0.58 |
| WMH Lesion-wise Segmentation Accuracy | Mean F1-score >= 0.57 |
| Reproducibility | Lower Bound of the 95% Bootstrap CI Dice >= |
| 0.63 |
Summary Performance data, Standard Deviations & CIs:
| | Volumetric Segmentation | | Voxel-wise
Segmentation | WMH Lesion-
wise
Segmentation | Reproducibility |
|--------|-------------------------|-------------|----------------------------|-------------------------------------|-----------------|
| | PCC | ICC | Dice | F1-score | Dice |
| AVG | 0.98 | 0.97 | 0.60 | 0.60 | 0.79 |
| STD | n.a. | n.a. | 0.18 | 0.14 | 0.11 |
| 95% CI | [0.97,0.99] | [0.96,0.98] | [0.53,0.63] | [0.57,0.64] | [0.77,0.81] |
Testing Data Information:
| Reproducibility Cohort | Testing Cohort | |
|---|---|---|
| # Subjects | 25 | 64 |
| # Studies | 100 | 64 |
| # of Females | 12 | 35 |
| # of Males | 13 | 29 |
| Age Range | 23-55 | 19-83 |
11
Image /page/11/Picture/0 description: The image shows the Siemens Healthineers logo. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots arranged in a circular pattern.
| Medical Indication | MS – all | MS – 36
Cognitive Impairment - 17
Other neurological disease - 5
Cognitive Normal – 5
Unknown – 1 |
|--------------------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------|
| Scan Protocol | 3D T1w MPRAGE
3D T2w FLAIR | T1w MPRAGE
T2w FLAIR |
| Field Strength | 3T | 1.5T: 30
3.0T: 34 |
| Manufacturer | Siemens | Siemens |
| Data Origin | Cleveland (US): 16
Baltimore (US): 9 | New York (US): 25
ADNI (US): 12
Lausanne (CH): 8
CLEMENS (CH): 10
Montpellier (FR): 9 |
Standard Annotation Process:
For each dataset, three sets of white matter hyperintensity ground truth are annotated manually. Each set is annotated by a disjoin group of annotator, reviewer, and clinical expert with the expert randomly assigned per case to minimize annotation bias. For each test dataset, the three initial annotations are annotated by three different in-house annotators. Then, each initial annotation is reviewed by the in-house reviewer. Afterwards, each initial annotation is reviewed by the referred clinical expert. The clinical expert reviews and corrects the initial annotation of the WMH according to the annotation protocol.
Testing & Training Data Independence:
The training data used for the training of the White matter hyperintensity algorithm is independent of the data used to test the white matter hyperintensity algorithm.
10. Clinical Tests
No clinical tests were conducted to test the performance and functionality of the modifications introduced within AI-Rad Companion Brain MR. Verification and validation of the enhancements and improvements have been performed and these modifications have been validated for their intended use. The data from these activities were used to support the subject device and the substantial equivalence argument. No animal testing has been performed on the subject device.
12
Image /page/12/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots arranged in a circular pattern.
11. Safety and Effectiveness
The device labeling contains instructions for use and any necessary cautions and warnings to ensure safe and effective use of the device.
Risk management is ensured via ISO 14971:2019 compliance to identify and provide mitigation of potential hazards in a risk analysis early in the design phase and continuously throughout the development of the product. These risks are controlled via measures realized during software development, testing and product labeling.
Furthermore, the device is intended for healthcare professionals familiar with the post processing of magnetic resonance images.