(144 days)
MEDO-Thyroid is designed to view and quantify ultrasound thyroid image data using machine learning techniques to aid in analysis of thyroid lobes and identify thyroid nodules, including evaluation, quantification and documentation of any such nodule. The device is intended to be used on adult patient images of 18 years or older.
MEDO-Thyroid is a cloud-based standalone software as a medical device (SaMD) that helps qualified users with image-based assessment of thyroid ultrasound images in adult patients of 18 years and older. It is designed to support the workflow by helping the radiologist to evaluate, quantify, and generate reports for thyroid ultrasound images.
MEDO-Thyroid Software takes as an input imported Digital Imaging and Communications in Medicine (DICOM) images from ultrasound scanners and allows users to upload, browse, and view images, measure thyroid lobes and thyroid nodule volumes of single frame and multi-frame ultrasound images, as well as create and finalize examination reports. It provides users with a specific toolset for viewing ultrasound Thyroid images, placing landmarks, and creating reports.
Key features of the software are:
- Single and multi-frame visualization .
- Cross Referencing .
- . Manual and semi-automatic landmark placements
- Thyroid Lobes (left and right) and thyroid nodule volume measurements ●
- TI-RADS Score and Classification (based on user manual input) .
- Report generation ●
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Device: MEDO-Thyroid (Automated Radiological Image Processing Software)
Intended Use: To view and quantify ultrasound thyroid image data using machine learning techniques to aid in analysis of thyroid lobes and identify thyroid nodules, including evaluation, quantification and documentation of any such nodule. Intended for adult patients 18 years or older.
1. Table of Acceptance Criteria and Reported Device Performance
The document doesn't explicitly state "acceptance criteria" in a separate section. However, the "Performance Data" section implies that the device's ability to accurately measure thyroid lobe and nodule volume, demonstrated by high Intraclass Correlation Coefficient (ICC) and acceptable maximum percentage volume error compared to reference data, are the key performance metrics. This implies the acceptance criteria were defined by thresholds for these metrics.
| Acceptance Criterion (Implied) | Performance Metric | Reported Device Performance (Overall) |
|---|---|---|
| Thyroid Lobe Volume Measurement Accuracy | Intraclass Correlation Coefficient (ICC) between AI and Reference Data | 0.972 (95% CI 0.969-0.975) |
| Maximum % Volume Error between AI and Reference Data | 21.7% (95% CI 19.0-24.8) | |
| Thyroid Nodule Volume Measurement Accuracy | Intraclass Correlation Coefficient (ICC) between AI and Reference Data | 0.973 (95% CI 0.971-0.975) |
| Maximum % Volume Error between AI and Reference Data | 22.9% (95% CI 20.0-26.0) |
The document also notes the device performed successfully on nodules ranging from 0.13 cc to 36.5 cc, implying the performance is independent of nodule size within this range.
2. Sample Size Used for the Test Set and Data Provenance
The document does not explicitly state the exact sample size for the test set. However, it indicates that the device was trained and tested on images from Philips, GE, and Siemens ultrasound devices. The performance tables (5.7.2 and 5.7.3) show aggregated data for "All" manufacturers, implying the test set includes images from these vendors.
Data Provenance:
- Country of Origin: Not explicitly stated, but the company is based in Singapore with an address in Canada provided in the FDA letter. The ultrasound machines listed (Philips, GE, Siemens) are globally used.
- Retrospective or Prospective: Not explicitly stated. Given that the data is being used for performance analysis and summarized, it is likely retrospective data collected from existing image archives.
3. Number of Experts Used to Establish Ground Truth for the Test Set and Qualifications
The document does not explicitly state the number of experts or their specific qualifications (e.g., "Radiologist with 10 years of experience") for establishing the ground truth. It refers to "Ref. data" for volume measurements, indicating that ground truth was established by a reference method, presumably by experts.
4. Adjudication Method for the Test Set
The document does not explicitly describe an adjudication method (e.g., 2+1, 3+1). The "Ref. data" suggests a single established ground truth measurement for comparison.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No. The document presents a standalone (algorithm only) performance evaluation against "Ref. data." There is no mention of a human-in-the-loop study or human reader improvement with AI assistance.
6. If a Standalone (i.e., Algorithm Only Without Human-in-the-Loop Performance) Was Done
Yes. The tables 5.7.2 and 5.7.3 directly compare the AI's measurements ("AI") against reference data ("Ref. data") for thyroid lobe and nodule volumes. This represents a standalone performance evaluation of the algorithm.
7. The Type of Ground Truth Used
The type of ground truth used is reference data for volume measurements. This implies that precise volume measurements were obtained by an established, reliable method, likely manual measurements performed by expert clinicians or a highly accurate segmentation method, to serve as the benchmark for the AI.
8. The Sample Size for the Training Set
The document does not specify the exact sample size used for the training set. It states: "MEDO Thyroid-AI has been primarily trained and tested on the Philips, GE and Siemens ultrasound devices."
9. How the Ground Truth for the Training Set Was Established
The document does not explicitly state how the ground truth for the training set was established. It only refers to "Ref. data" for the test set performance evaluation. However, it is a standard practice in machine learning for medical imaging that similar reference data establishment methods (e.g., expert annotations, manual measurements) would be used for training data as well.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).