(77 days)
Yes
The device description explicitly states that it "utilizes deep learning algorithms" and mentions an "Enhanced AI Algorithm" that was improved through "retraining the algorithm." The "Mentions AI, DNN, or ML" section further confirms the use of "deep learning based segmentation" and "Deep learning image to image network."
No.
Explanation: The device is an image processing software that provides quantitative analysis from previously acquired Computed Tomography DICOM images to support radiologists and physicians in the evaluation and assessment of musculoskeletal disease. It aids in diagnosis and assessment, but does not directly provide therapy.
Yes
The device aids in the evaluation and assessment of musculoskeletal disease by segmenting and labeling vertebrae, measuring heights, and providing Hounsfield values, all of which are used to support radiologists and physicians in diagnosis.
Yes
The device is explicitly described as "image processing software" and its functionality is solely based on analyzing previously acquired DICOM images. While it has deployment options (cloud and on-premise "edge"), these refer to the software's execution environment, not the inclusion of physical medical device hardware.
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs analyze biological specimens: In Vitro Diagnostics are designed to examine samples taken from the human body, such as blood, urine, tissue, etc., to provide information about a person's health.
- This device analyzes medical images: The AI-Rad Companion (Musculoskeletal) processes previously acquired Computed Tomography (CT) DICOM images. It does not interact with or analyze biological specimens.
- Its intended use is image processing and analysis: The device's purpose is to provide quantitative analysis from medical images to support clinicians in evaluating musculoskeletal disease.
Therefore, while it is a medical device that aids in diagnosis, it falls under the category of medical image processing software, not an In Vitro Diagnostic.
No
The input letter does not contain any explicit statement that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device.
Intended Use / Indications for Use
AI-Rad Companion (Musculoskeletal) is an image processing software that provides quantitative andysis from previously acquired Computed Tomography DICOM images to support radiologists and physicians from emergency medicine, specialty care, urgent care, and general practice in the evaluation and assessment of musculoskeletal disease.
It provides the following functionality:
- Segmentation of vertebras
- Labelling of vertebras
- Measurements of heights in each vertebra and indication if they are critically different
- Measurement of mean Hounsfield value in volume of interest within vertebra.
Only DICOM images of adult patients are considered to be valid input.
Product codes
JAK
Device Description
AI-Rad Companion (Musculoskeletal) SW version VA20 is an enhancement to the previously cleared device AI-Rad Companion (Musculoskeletal) K193267 that utilizes deep learning algorithms to provide quantitative and qualitative analysis to computed tomography DICOM images to support qualified clinicians in the evaluation and assessment of the spine.
As an update to the previously cleared device, the following modifications have been made:
- Enhanced AI Algorithm The vertebrae segmentation accuracy has been improved through retraining the algorithm.
- DICOM Reports
The reports generated out of the system have been enhanced to support both human and machine-readable formats. Additionally, an update version of the system changed the DICOM structured report format to TID 1500 for applicable content.
- Architecture Enhancement for on premise Edge deployment The system supports the existing cloud deployment as well as an on premise "edge" deployment. The system remains hosted in the teamplay digital health platform and remains driven by the AI-Rad Companion Engine. Now the edge deployment implies that the processing of clinical data and the generation of results can be performed onpremises within the customer network. The edge system is fully connected to the cloud for monitoring and maintenance of the system from remote.
Mentions image processing
Yes
Mentions AI, DNN, or ML
- Enhanced AI Algorithm The vertebrae segmentation accuracy has been improved through retraining the algorithm.
- AI-Rad Companion (Musculoskeletal) SW version VA20 is an enhancement to the previously cleared device AI-Rad Companion (Musculoskeletal) K193267 that utilizes deep learning algorithms to provide quantitative and qualitative analysis to computed tomography DICOM images to support qualified clinicians in the evaluation and assessment of the spine.
- The system remains hosted in the teamplay digital health platform and remains driven by the AI-Rad Companion Engine.
- Deep learning based segmentation of vertebras
- Deep learning image to image network for 3D segmentation
Input Imaging Modality
Computed Tomography DICOM images
Anatomical Site
Musculoskeletal, Spine, Vertebras
Indicated Patient Age Range
adult patients
Intended User / Care Setting
radiologists and physicians from emergency medicine, specialty care, urgent care, and general practice.
Description of the training set, sample size, data source, and annotation protocol
The training data used for the training of the post-processing algorithm is independent of the data used to test the algorithm.
Description of the test set, sample size, data source, and annotation protocol
- Sample Size: 140 Chest CTs (1,553 thoracic vertebrae)
- Data Source: KUM (N=80), NLST (N=60)
- KUM Pathologies / Patient info: Primary indications: Lung/airways 10; infect focus 10; malignancy 22, follow-up 10; (cardio-)vascular 14; ischemia 2; bleeding 4; trauma 1; lymph nodes 3; inflammation 1; unknown 3.
- NLST Pathologies / Patient info: Comorbidities: diabetes 7; heart disease 10; hypertension 24; cancer 9; emphysema/COPD 14; asthma 2; pneumonia 13; chron. bronchitis 1. Smoking history (pack years): median: 47, IQR: [38, 70]
- Sex: male: 47, female: 93
- Age [yrs]: ≤55: 16, (55, 65]: 59, (65, 75]: 38, >75: 25 median: 65, IQR: [60, 72]
- Manufacturer: GE: 32, Philips: 20, Siemens: 68, Toshiba: 20
- Slice Thickness [mm]: ≤1.0: 60, (1.0, 1.5]: 24, (1.5, 2.0]: 56
- Dose: KUM: CTDIVol [mGy]: median 4.8, IQR: [1.3, 10.7] NLST: low dose (screening)
- Reconstruction method: Filtered backprojection: 107 Iterative reconstruction: 33
- Reconstruction kernel: soft: 25, medium: 68, hard: 47
- Contrast Enhancement: enhanced: 59, native: 81
- Annotation protocol: For ground truth annotations, four board-certified radiologists were selected. Vertebra heights and average density (HU) values were measured manually and loaded into the application and automatic detection and labelling was performed. The 140 cases were randomly distributed across the four readers such that each case was read independently by two radiologists in randomized order. For outliers, a third annotation was blindly provided by one of the radiologist who had not annotated before. The ground truth was generated by the average of the two most concordant measurements. For all other cases, the two annotations were used as ground truth.
Summary of Performance Studies
- Study Type: Performance Software Validation, Nonclinical tests
- Sample Size: 140 subjects
- Key Results:
- Mislabeling of Vertebrae or absence of height measurement: Failure Rate of 8.6% (Acceptance Criteria: Ratio of cases that are mislabeled or missing measurements shall be 1.0mm, the difference was 92.6% (Acceptance Criteria: For cases with slice thickness >1.0mm, the difference should be within the LoA for ≥ 85% of cases)
- Consistency of height and density measurement across critical sub-groups: Overall failure rate of the subject device was consistent with the predicate as well as having the results of all sub-group analysis rated equal or superior to the predicate (Acceptance Criteria: For each sub-group, the ratio of measurements within the corresponding LoA should not drop by more than 5% compared to the ratio for all data sets)
Key Metrics
- Failure Rate: 8.6% for mislabeling or missing measurements.
- For cases with slice thickness ≤1.0 mm, 95.5% of differences were within the LoA.
- For cases with slice thickness >1.0mm, 92.6% of differences were within the LoA.
Predicate Device(s)
Reference Device(s)
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 892.1750 Computed tomography x-ray system.
(a)
Identification. A computed tomography x-ray system is a diagnostic x-ray system intended to produce cross-sectional images of the body by computer reconstruction of x-ray transmission data from the same axial plane taken at different angles. This generic type of device may include signal analysis and display equipment, patient and equipment supports, component parts, and accessories.(b)
Classification. Class II.
0
Image /page/0/Picture/0 description: The image contains the logos of the Department of Health & Human Services and the Food and Drug Administration (FDA). The Department of Health & Human Services logo is on the left, and the FDA logo is on the right. The FDA logo is a blue square with the letters "FDA" in white, followed by the words "U.S. FOOD & DRUG ADMINISTRATION" in blue.
October 20, 2022
Siemens Medical Solutions USA, Inc. % Kira Kuzmenchuk Regulatory Affairs Manager 40 Liberty Blvd. MALVERN PA 19355
Re: K222361
Trade/Device Name: AI-Rad Companion (Musculoskeletal) Regulation Number: 21 CFR 892.1750 Regulation Name: Computed Tomography X-Ray System Regulatory Class: Class II Product Code: JAK Dated: October 4, 2022 Received: October 12, 2022
Dear Kira Kuzmenchuk:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal
1
statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Laurel Burk, Ph.D. Assistant Director Diagnostic X-Ray Systems Team DHT8B: Division of Imaging Devices and Electronic Products OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known)
K222361
Device Name AI-Rad Companion (Musculoskeletal)
Indications for Use (Describe)
AI-Rad Companion (Musculoskeletal) is an image processing software that provides quantitative andysis from previously acquired Computed Tomography DICOM images to support radiologists and physicians from emergency medicine, specialty care, urgent care, and general practice in the evaluation and assessment of musculoskeletal disease.
It provides the following functionality:
- · Segmentation of vertebras
- · Labelling of vertebras
- · Measurements of heights in each vertebra and indication if they are critically different
- · Measurement of mean Hounsfield value in volume of interest within vertebra.
Only DICOM images of adult patients are considered to be valid input.
Type of Use (Select one or both, as applicable)
X Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW!
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
3
SIEMENS Healthineers
510(k) SUMMARY FOR AI-RAD COMPANION (Musculoskeletal) SW version VA20
Submitted by: Siemens Medical Solutions USA, Inc. 40 Liberty Boulevard Malvern, PA 19355 Date Prepared: October 20, 2022
This summary of 510(k) safety and effectiveness information is being submitted in accordance with the requirements of Safe Medical Devices Act of 1990 and 21 CFR §807.92.
1. Submitter
| Importer/Distributor | Siemens Medical Solutions USA, Inc.
40 Liberty Boulevard
Malvern, PA 19355
Mail Code: 65-1A
Registration Number: 2240869 |
|----------------------|--------------------------------------------------------------------------------------------------------------------------------------|
| Manufacturing Site | Siemens Healthcare GmbH
Henkestrasse 127
Erlangen, Germany 91052
Registration Number: 3002808157 |
2. Contact Person
Kira Kuzmenchuk Regulatory Affairs Specialist Siemens Medical Solutions USA, Inc. 40 Liberty Boulevard Malvern, PA 19335 Email: Kira.Kuzmenchuk@siemens-healthineers.com Phone: (484) 901-9471
3. Device Name and Classification
| Product Name: | AI-Rad Companion (Musculoskeletal) |
|---|---|
| Trade Name: | AI-Rad Companion (Musculoskeletal) |
4
Image /page/4/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots arranged in a circular pattern.
Classification Name: Classification Panel: CFR Section: Device Class: Product Code:
Computed Tomography X-Ray System Radiology 21 CFR §892.1750 Class II JAK
4. Predicate Device
| Product Name: | AI-Rad Companion (Musculoskeletal) |
|---|---|
| Propriety Trade Name: | AI-Rad Companion (Musculoskeletal) |
| 510(k) Number: | K193267 |
| Clearance Date: | March 16, 2020 |
| Classification Name: | Computed Tomography X-Ray System |
| Classification Panel: | Radiology |
| CFR Section: | 21 CFR §892.1750 |
| Device Class: | Class II |
| Primary Product Code: | JAK |
| Recall Information: | N/A |
5. Indications for Use
AI-Rad Companion (Musculoskeletal) is an image processing software that provides quantitative and qualitative analysis from previously acquired Computed Tomography DICOM images to support radiologists and physicians from emergency medicine, specialty care, urgent care, and general practice in the evaluation and assessment of musculoskeletal disease.
It provides the following functionality:
- Segmentation of vertebras
- Labelling of vertebras ●
- Measurements of heights in each vertebra and indication if they are critically different
- Measurement of mean Hounsfield value in volume of interest within vertebra. ●
Only DICOM images of adult patients are considered to be valid input.
6. Device Description
AI-Rad Companion (Musculoskeletal) SW version VA20 is an enhancement to the previously cleared device AI-Rad Companion (Musculoskeletal) K193267 that utilizes deep learning algorithms to provide quantitative and qualitative analysis to computed tomography DICOM images to support qualified clinicians in the evaluation and assessment of the spine.
5
Image /page/5/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. To the right of the words is a graphic of orange dots.
As an update to the previously cleared device, the following modifications have been made:
- Enhanced AI Algorithm The vertebrae segmentation accuracy has been improved through retraining the algorithm.
- DICOM Reports
The reports generated out of the system have been enhanced to support both human and machine-readable formats. Additionally, an update version of the system changed the DICOM structured report format to TID 1500 for applicable content.
- Architecture Enhancement for on premise Edge deployment The system supports the existing cloud deployment as well as an on premise "edge" deployment. The system remains hosted in the teamplay digital health platform and remains driven by the AI-Rad Companion Engine. Now the edge deployment implies that the processing of clinical data and the generation of results can be performed onpremises within the customer network. The edge system is fully connected to the cloud for monitoring and maintenance of the system from remote.
7. Substantially Equivalent (SE) And Technological Characteristics
The intended use of the predicate device and the subject device are equivalent. The subject device, AI-Rad Companion (Musculoskeletal) VA20 is substantially equivalent with regard to the intended use and technical characteristics compared to the predicate device, AI-Rad Companion (Musculoskeletal) (K193267), with respect to the software features, functionalities, and core algorithms. The additional features, enhancements and improvements provided in AI-Rad Companion (Musculoskeletal) VA20 increase the usability and reduce the complexity of the imaging workflow for the clinical user.
The risk analysis and non-clinical data support that both devices perform equivalently and do not raise different questions of the safety and effectiveness.
| Feature | Subject Device
AI-Rad Companion
(Musculoskeletal)
VA20 | Predicate Device
AI-Rad Companion
(Musculoskeletal)
(K193267) |
|------------------------------|-----------------------------------------------------------------|------------------------------------------------------------------------|
| Modality | CT | CT |
| Detection of
Vertebrae | Detection of Vertebras | Detection of Vertebras |
| Labeling of
Vertebrae | Labeling of Vertebras | Labeling of Vertebras |
| Segmentation
of Vertebrae | Deep learning based segmentation of
vertebras | Deep learning based segmentation of
vertebras |
The comparison between the above referenced predicate device are listed at a high-level in the following table.
6
Image /page/6/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots arranged in a circular pattern.
| Measurement
of Heights | Distance measurements based on
segmentation results and comparison
with neighboring measurements | Distance measurements based on
segmentation results and comparison
with neighboring measurements |
|------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------|
| Measurement
of
Hounsfield
(HU) Values | HU measurements based on
segmentation results | HU measurements based on
segmentation results |
| Algorithm | Deep learning image to image
network for 3D segmentation | Deep learning image to image
network for 3D segmentation |
| Deployment | Cloud and on-premise deployment | Cloud deployment |
| Reports | Quantitative, Structured and Text
reports with DICOM secondary
capture & TID 1500 in both human
and machine readable formats. | Quantitative, Structured and Text
reports with DICOM secondary
capture images |
Table 1: Technological Comparisons
The conclusions from all verification and validation data suggest that these enhancements are equivalent with respect to safety and effectiveness of the predicate device. These modifications do not change the intended use of the product. Siemens is of the opinion that AI-Rad Companion (Musculoskeletal) VA20 is substantially equivalent to the currently marketed device, AI-Rad Companion (Musculoskeletal) (K193267).
8. Nonclinical Tests
Non-clinical tests were conducted to test the functionality of AI-Rad Companion (Musculoskeletal). Software validation and bench testing have been conducted to assess the performance claims as well as the claim of substantial equivalence to the predicate device.
AI-Rad Companion has been tested to meet the requirements of conformity to multiple industry standards. Non-clinical performance testing demonstrates that AI-Rad Companion (Musculoskeletal) complies with the FDA guidance document, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" (May 11, 2005) as well as with the following voluntary FDA recognized Consensus Standards listed in Table 2.
| Recognition
Number | Product
Area | Title of Standard | Reference
Number and
Date | Standards
Development
Organization |
|-----------------------|-----------------|---------------------------------------------------------------------------------------------------------------------|---------------------------------|------------------------------------------|
| 5-114 | General | Medical Devices – Application
of usability engineering to
medical devices [including
Corrigendum 1 (2016)] | 62366-1: 2015-
02 | IEC |
| 5-125 | General | Medical Devices – application
of risk management to
medical devices | 14971:2007 | ISO |
7
Image /page/7/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the text is a graphic of several orange dots arranged in a circular pattern.
| 13-79 | Software/
Informatics | Medical device software –
software life cycle processes
[Including Amendment 1
(2016)] | 62304:
2006/A1:2016 | AAMI
ANSI
IEC |
|--------|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------|---------------------|
| 12-300 | Radiology | Digital Imaging and
Communications in Medicine
(DICOM) Set | PS 3.1 – 3.20
(2016) | NEMA |
| 12-261 | Radiology | Information Technology –
Digital Compression and
coding of continuous -tone
still images: Requirements
and Guidelines [including:
Technical Corrigendum
1(2005)] | 10918-1 1994-
02-15 | ISO
IEC |
| 5-134 | General | Medical devices – symbols to
be used with information to
be supplied by the
manufacturer – Part 1:
General Requirements | 15223-1
Fourth edition
2021-07 | ISO
IEC |
| 13-97 | Software/
Informatics | Health software – Part 1:
General requirements for
product safety | 82304-1
Edition 1.0
2016-10 | IEC |
Table 2: List of recognized standards
Verification and Validation
Software documentation for a Moderate Level of Concern software, per FDA's Guidance Document "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices" issued on May 11, 2005, is also included as part of this submission. The performance data demonstrates continued conformance with special controls for medical devices containing software. Non-clinical tests were conducted on the subject device during product development.
Software "bench" testing in the form of Unit, System and Integration tests were performed to evaluate the performance and functionality of the new features and software updates. All testable requirements in the Requirement Specifications and the Risk Analysis have been successfully verified and traced in accordance with the Siemens Healthineers DH product development (lifecycle) process. Human factor usability validation is addressed in system testing and usability validation test records. Software verification and regression testing have been performed successfully to meet their previously determined acceptance criteria as stated in the test plans.
Siemens Healthineers adheres to the cybersecurity requirements as defined the FDA Guidance "Content of Premarket Submission for Management of Cybersecurity in Medical Devices: Guidance for Industry and Food and Drug Administration Staff" (October 18, 2018) by implementing a process of preventing unauthorized access, modifications, misuse or denial of use, or the unauthorized use of information that is stored, accessed, or transferred from a medical device to an external recipient.
8
Image /page/8/Picture/0 description: The image contains the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots.
9. Performance Software Validation
To validate the AI-Rad Companion (Musculoskeletal) VA40 software from a clinical perspective, the algorithm contained in the product underwent a scientific evaluation. The results of clinical data-based software validation for the subject device AI-Rad Companion (Musculoskeletal) demonstrated equivalent performance in comparison to the reference device. A complete scientific evaluation report is provided in support of the device modifications.
Performance testing for AI-Rad Companion (Musculoskeletal) was performed on 140 subjects (clinically relevant patient data shown in Table 2) during product development. Additionally, the segmentation and height measurements of the thoracic vertebral bodies were reassessed for accuracy.
| Validation Type | Acceptance Criteria |
|---|---|
| Mislabeling of Vertebrae or absence of height | |
| measurement | Ratio of cases that are mislabeled or missing |
| measurements shall be 1.0mm, the | |
| difference should be within the LoA for $\ge$ 85% | |
| of cases | |
| Consistency of height and density | |
| measurement across critical sub-groups | For each sub-group, the ratio of |
| measurements within the corresponding LoA | |
| should not drop by more than 5% compared | |
| to the ratio for all data sets |
Acceptance Criteria
Table 2: Acceptance Criteria Information
9
Image /page/9/Picture/0 description: The image shows the Siemens Healthineers logo. The word "SIEMENS" is in teal, and the word "Healthineers" is in orange. There are also some orange dots to the right of the words.
Summary Performance Data
| Validation Type | Results |
|---|---|
| Mislabeling of Vertebrae or absence of height | |
| measurement | Failure Rate of 8.6% |
| Inter-reader variability: heights calculated by | |
| AIRC & the ground truth should be within the | |
| LoA reported | For cases with slice thickness ≤1.0 mm, the |
| difference was 95.5% | |
| For cases with slice thickness >1.0mm, the | |
| difference was 92.6% | |
| Consistency of height and density | |
| measurement across critical sub-groups | Overall failure rate of the subject device was |
| consistent with the predicate as well as having | |
| the results of all sub-group analysis rated | |
| equal or superior to the predicate |
Table 3: Summary Performance Data
Testing Data Information
| # data sets | 140 Chest CTs (1,553 thoracic vertebrae) |
|---|---|
| Pathologies / | |
| Patient info | KUM (N=80): |
| Primary indications: Lung/airways 10; infect focus 10; malignancy 22, | |
| follow-up 10; (cardio-)vascular 14; ischemia 2; bleeding 4; trauma 1; | |
| lymph nodes 3; inflammation 1; unknown 3. | |
| NLST (N=60): | |
| Comorbidities: diabetes 7; heart disease 10; hypertension 24; cancer 9; | |
| emphysema/COPD 14; asthma 2; pneumonia 13; chron. bronchitis 1. | |
| Smoking history (pack years): median: 47, IQR: [38, 70] | |
| Sex | male: 47, female: 93 |
| Age [yrs] | ≤55: 16, (55, 65]: 59, (65, 75]: 38, >75: 25 |
| median: 65, IQR: [60, 72] | |
| Manufacturer | GE: 32, Philips: 20, Siemens: 68, Toshiba: 20 |
| Slice Thickness | |
| [mm] | ≤1.0: 60, (1.0, 1.5]: 24, (1.5, 2.0]: 56 |
| Dose | KUM: CTDIVol [mGy]: median 4.8, IQR: [1.3, 10.7] |
| NLST: low dose (screening) | |
| Reconstruction | |
| method | Filtered backprojection: 107 |
| Iterative reconstruction: 33 | |
| Reconstruction | |
| kernel | soft: 25, medium: 68, hard: 47 |
| Contrast | |
| Enhancement | enhanced: 59, native: 81 |
Table 4: Testing Data Information
10
Image /page/10/Picture/0 description: The image shows the logo for Siemens Healthineers. The word "SIEMENS" is written in teal, and the word "Healthineers" is written in orange below it. To the right of the words is a graphic of orange dots arranged in a circular pattern.
Standard Annotation Process
For ground truth annotations, four board-certified radiologists were selected. Vertebra heights and average density (HU) values were measured manually and loaded into the application and automatic detection and labelling was performed. The 140 cases were randomly distributed across the four readers such that each case was read independently by two radiologists in randomized order. For outliers, a third annotation was blindly provided by one of the radiologist who had not annotated before. The ground truth was generated by the average of the two most concordant measurements. For all other cases, the two annotations were used as ground truth.
Testing & Training Data Independence
The training data used for the training of the post-processing algorithm is independent of the data used to test the algorithm.
10. Clinical Tests
No clinical tests were conducted to test the performance and functionality of the modifications introduced within AI-Rad Companion (Musculoskeletal). Verification and validation of the enhancements and improvements have been performed and these modifications have been validated for their intended use. The data from these activities were used to support the subject device and the substantial equivalence argument.
No animal testing has been performed on the subject device.
11. Safety and Effectiveness
The device labeling contains instructions for use and any necessary cautions and warnings to ensure safe and effective use of the device.
Risk management is ensured via ISO 14971:2019 compliance to identify and provide mitigation of potential hazards in a risk analysis early in the design phase and continuously throughout the development of the product. These risks are controlled via measures realized during software development, testing and product labeling.
Furthermore, the device is intended for healthcare professionals familiar with the post processing of CT images.