(167 days)
Clarius AI is intended to semi-automatically place calipers for non-invasive measurements of musculoskeletal structures (e.g., Achilles' tendon, plantar fascia, patellar tendon) on ultrasound data acquired by the Clarius Ultrasound Scanner (i.e., L 7 and L15). The user shall be a healthcare professional trained and qualified in MSK (musculoskeletal) ultrasound. The user shall retain the ultimate responsibility of ascertaining the measurements based on standard practices and clinical iudgment.
Clarius AI is a radiological (ultrasound) image processing software application which implements artificial intelligence (Al), including non-adaptive machine learning algorithms, and is incorporated into the Clarius App software for use as part of the complete Clarius Ultrasound Scanner system product offering in musculoskeletal (MSK) ultrasound imaging applications. Clarius Al (MSK model) is intended for use by trained healthcare practitioners for non-invasive measurements of ultrasound data from musculoskeletal (MSK) ultrasound imaging acquired by the Clarius Ultrasound Scanner system using an artificial intelligence (AI) image segmentation algorithm. Clarius AI (MSK model) is intended to semi-automatically place adjustable calipers and provide supplementary information to the user regarding tendon thickness measurements (i.e., foot/plantar fascia, ankle/Achilles' tendon, knee/patellar tendon). Clarius Al is intended to inform clinical management and is not intended to replace clinical decision-making. The clinician retains the ultimate responsibility of ascertaining the measurements based on standard practices and clinical judgment. Clarius Al is indicated for use in adult patients only.
During the ultrasound imaging procedure, the anatomical site is selected through a preset software selection (e.g., foot, ankle, knee), in which the Clarius Al will engage to segment the correlating tendon. Clarius Al analyzes ultrasound images in real-time and outputs probabilities for each pixel within the image for determination of the particular tendon thickness.
The combination of all the pixels meeting a programmed threshold will render an overlay being displayed on top of the ultrasound image with a pre-programmed transparency so that the ultrasound greyscale is still visible. Once the user has obtained the best view, imaging can be manually paused, in which the Clarius Al will further analyze the tendon segmentation to determine the greatest thickness, in number of pixels, and subsequently place two measurement calipers that correspond to the top and bottom of the tendon at its thickest region, outputting a value in millimeters. The user can then manually alter the measurement calipers to make any necessary adjustments if desired. Clarius Al does not perform any functions that could not be accomplished manually by a trained and qualified user.
Clarius AI (MSK model) is incorporated into the Clarius App software and is intended for use with the following Clarius Ultrasound Scanner system transducers (previously 510(k)-cleared in K180799, K192107, and K213436):
Clarius Ultrasound Transducers: L7 and L15
Clarius App Software: Clarius Ultrasound App (Clarius App) for iOS; Clarius Ultrasound App (Clarius App) for Android
The Clarius AI device is intended for semi-automatic placement of calipers for non-invasive measurements of musculoskeletal structures (e.g., Achilles' tendon, plantar fascia, patellar tendon) on ultrasound data. The device's performance was evaluated through non-clinical and clinical testing to demonstrate its safety and effectiveness.
Here’s a breakdown of the acceptance criteria and the study that proves the device meets them:
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the Clarius AI device, particularly regarding measurement accuracy, are based on achieving non-inferiority to manual measurements made by expert clinicians. The specific criterion for clinical significance was defined as a difference of greater than 20% from normal thickness measurements.
| Acceptance Criteria (Measurement Accuracy) | Reported Device Performance |
|---|---|
| Automatic thickness measurement to be non-inferior to mean manual measurements. | Non-inferior (p-value of 9.0 x 10^-5). The mean difference between automated and manual measurements was 0.03% (95% CI: -0.05% to -0.01%). |
| Clinically significant difference defined as >20% absolute percent difference between automatic measurement and mean reviewer measurement, corresponding to: |
- 0.6 mm for plantar fascia
- 1 mm for patellar tendon
- 1.2 mm for Achilles' tendon. | The automatic measurement was found to be non-inferior to manual measurements within this clinically significant margin. |
| Automatic segmentation to have a high degree of overlap with ground truth. | Average Dice score of 96% and mean IoU of 94% for tendon segmentation. |
2. Sample Sizes and Data Provenance
-
Test Set (Validation Phase):
- Sample Size: 73 subjects, resulting in a total of 2,503 ultrasound images/frames.
- Data Provenance: Retrospective analysis of anonymized ultrasound images. Images were captured in-house from volunteer subjects and by clinical partners, mainly located in the USA. The data was anonymized and queried from Clarius Cloud storage.
-
Training Set:
- Sample Size: A total of 20,287 images.
- Data Provenance: Images were captured in-house from volunteer subjects and by clinical partners, mainly located in the USA. The data was anonymized and queried from Clarius Cloud storage.
3. Number of Experts and Qualifications for Ground Truth - Test Set
The document mentions "expert clinicians" and "licensed clinicians with relevant (i.e., musculoskeletal) ultrasound experience" were involved in establishing the ground truth for the test set (manual measurements). However, the exact number of experts and their specific qualifications (e.g., years of experience, board certification) are not explicitly stated in the provided text. The study states "reviewer pairs" for calculating manual measurement differences, implying at least two reviewers per case for comparison.
4. Adjudication Method for the Test Set
The provided text does not explicitly detail an adjudication method (e.g., 2+1, 3+1). It states that the "difference between auto-measurements and mean manual measurements" was compared to the "mean difference between manual measurements within the clinically significant margin." This suggests that multiple manual measurements were obtained, and their mean was used for comparison, rather than a specific consensus or adjudication process described.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
The provided text describes a verification test where "Clarius AI auto-measurements are non-inferior to manual measurements performed by licensed clinicians." While this compares AI performance against human performance, it doesn't describe a typical MRMC study designed to assess how human readers improve with AI assistance versus without AI assistance (i.e., an effect size of AI assistance on human readers). The focus was on the AI's standalone accuracy relative to human measurements.
6. Standalone Performance (Algorithm Only without Human-in-the-Loop)
Yes, a standalone performance evaluation was largely conducted for the AI algorithm. The core of the "Measurement Accuracy" testing compared the automatic thickness measurement directly against the mean manual measurements, indicating the algorithm's performance without specific human-in-the-loop assistance influencing the AI's initial output for that specific evaluation. The segmentation accuracy (Dice score and IoU) also represents standalone algorithm performance. While the AI semi-automatically places calipers and allows for manual adjustment, the non-inferiority study specifically evaluated the AI's auto-measurement against expert manual measurements.
7. Type of Ground Truth Used
- Expert Consensus / Manual Measurements: For the measurement accuracy study, the ground truth was established by "mean manual measurements performed by licensed clinicians with relevant (i.e., musculoskeletal) ultrasound experience."
- Expert Annotation (Segmentation): For the segmentation evaluation, the ground truth was described as "segmentation ground truth" where "the tendon regions in the images were annotated by a clinical scientist as the ground truth."
8. Sample Size for the Training Set
A total of 20,287 images were used for the training phase.
9. How the Ground Truth for the Training Set was Established
For the training phase, "the tendon regions in the images were annotated by a clinical scientist as the ground truth." This indicates that human experts (clinical scientists) manually outlined or labeled the tendon regions in the ultrasound images to create the reference data used to train the machine learning model.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).