(179 days)
Yes
The device description explicitly states that the software "uses machine learning algorithms to extract specific information."
No
Explanation: This device is a diagnostic software that assists medical professionals in analyzing thyroid ultrasound images and generating reports; it does not directly treat or alleviate a disease or condition.
Yes
The device assists in analyzing thyroid ultrasound images, localizing nodules, and outputting lexicon-based descriptors and TI-RADS levels, all of which are steps in the diagnostic process. While patient management decisions should not be made solely based on its output, it directly contributes to the diagnostic evaluation of thyroid conditions.
Yes
The device is described as "stand-alone, web-based image processing and reporting software" that runs on a standard "off-the-shelf" computer and is accessed via a web browser. It processes images acquired from DICOM-compliant ultrasound devices but does not include any hardware components itself.
Based on the provided information, the See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is not an In Vitro Diagnostic (IVD) device.
Here's why:
- IVD Definition: In Vitro Diagnostic devices are used to examine specimens taken from the human body, such as blood, urine, or tissue, to provide information about a person's health.
- SMART-T's Function: SMART-T analyzes medical images (ultrasound images) of the thyroid. It does not analyze biological specimens taken from the patient.
- Intended Use: The intended use is to assist medical professionals in analyzing thyroid ultrasound images and generating reports based on that image analysis. This is a function related to medical imaging interpretation and reporting, not the analysis of in vitro specimens.
Therefore, while SMART-T is a medical device that uses image processing and AI to aid in diagnosis, it falls under the category of medical imaging software or a clinical decision support tool based on imaging, rather than an In Vitro Diagnostic device.
No
The provided text does not contain any explicit statement that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device.
Intended Use / Indications for Use
See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is a stand-alone reporting software to assist trained medical professionals in analyzing thyroid ultrasound images of adult (>=22 years old) patients who have been referred for an ultrasound examination.
Output of the device includes regions of interest (ROIs) placed on the thyroid ultrasound images assisting healthcare professionals to localize nodules in thyroid studies. The device also outputs ultrasonographic lexicon-based descriptors based on ACR TI-RADS. The software generates a report based on the image analysis results to be reviewed and approved by a qualified clinician after performing quality control.
SMART-T may also be used as a structured reporting software for further ultrasound studies. The software includes tools for reading measurements and annotations from the images that can be used for generating a structured report.
Patient management decisions should not be made solely on the basis of analysis by See-Mode Augmented Reporting Tool, Thyroid.
Product codes
QDQ, QIH
Device Description
See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is a stand-alone, web-based image processing and reporting software for localization, characterization and reporting of thyroid ultrasound images.
The software analyzes thyroid ultrasound images and uses machine learning algorithms to extract specific information. The algorithms can identify and localize suspicious soft tissue nodules and also generate lexicon-based descriptors, which are classified according to ACR TI-RADS (composition, echogenicity, shape, margin, and echogenic foci) with a calculated TI-RADS level according to the ACR TI-RADS chart.
SMART-T may also be used as a structured reporting software for further ultrasound studies. The software includes tools for reading measurements and annotations from the images that can be used for generating a structured report.
The software then generates a report based on the image analysis results to be reviewed and approved by a qualified clinician after performing quality control. Any information within this report can be changed and modified by the clinician if needed during quality control and before finalizing the report.
The software runs on a standard "off-the-shelf" computer and can be accessed within the client web browser to perform the reporting of ultrasound images. Input data and images for the software are acquired through DICOM-compliant ultrasound imaging devices.
Mentions image processing
Yes
Mentions AI, DNN, or ML
Yes
Input Imaging Modality
Ultrasound images
Anatomical Site
Thyroid
Indicated Patient Age Range
Adult (>=22 years old)
Intended User / Care Setting
Trained medical professionals (physicians, medical technicians) analyzing thyroid ultrasound images.
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
Test Set (MRMC Study):
- Sample Size: 600 cases from 600 unique patients.
- Data Source: Cases were sourced from institutions or sources not part of the model training or development datasets, with 74% of the data acquired from the US. Data was curated from images sourced from accepted ultrasound systems, such as GE Healthcare, Philips Medical Systems and Siemens Medical Systems.
- Annotation Protocol:
- Ground truth labels of benign or malignant status were assigned to the nodule for each case, sourced from the reference standard of FNA or 2-year follow-up for benign status.
- Ground truth labels for localization, ACR TI-RADS lexicon descriptors, and TI-RADS level agreement were based on the labels of two expert US-board certified radiologists and an adjudicator (also US-board certified radiologist with the most years of experience).
Data Selection Criteria:
- The dataset contained both negative and positive biopsy results with all TI-RADS levels represented.
- The data is sampled such that the distribution of the data accounts for patient sex, age, nodule size, malignancy, a priori clinical ACR TI-RADS level and a minimum sample is present for each subgroup.
- Patient distribution:
- Female: 489 cases
- Male: 111 cases
- 0.5):** 0.703 (0.642, 0.762)
- Localisation Accuracy (IOU > 0.5): 95.1%
- TI-RADS Descriptor Accuracy:
- Composition: 86.7%
- Echogenicity: 68.2%
- Shape: 93.4%
- Margin: 58.4%
- Echogenic Foci: 70.3%
- TI-RADS Level Agreement:
- Overall: 63.8 (60.0, 67.7)
- TR-1: 61.9 (40.0, 82.6)
- TR-2: 41.1 (31.7, 50.4)
- TR-3: 71.7 (64.9, 78.3)
- TR-4: 65.5 (59.1, 71.6)
- TR-5: 77.0 (66.1, 87.3)
- The standalone performance of the device is on-par with the aided use of the device, indicating the validity of the algorithms.
2. Multi-reader Multi-Case (MRMC) Study:
- Study Type: Compares the performance of expert readers (radiologists) with and without the aid of the device. The study evaluated the readers' localization and characterization of thyroid nodules in scenarios both unaided and aided.
- Sample Size: 18 radiologists read 600 cases from 600 patients twice (once aided, once unaided), with a one-month washout period.
- Key Results:
- AULROC (IOU > 0.5):
- Average Aided AUC: 0.758 (0.711, 0.803)
- Average Unaided AUC: 0.736 (0.693, 0.780)
- Difference (Aided - Unaided): 0.022 (-0.012, 0.056)
- AULROC (various IOU thresholds):
- IOU > 0.6: Aided 0.734 (0.682, 0.781), Unaided 0.682 (0.632, 0.730), Difference 0.052 (0.008, 0.093)
- IOU > 0.7: Aided 0.686 (0.629, 0.740), Unaided 0.548 (0.490, 0.610), Difference 0.138 (0.082, 0.195)
- IOU > 0.8: Aided 0.593 (0.529, 0.658), Unaided 0.356 (0.293, 0.423), Difference 0.237 (0.168, 0.307)
- Localisation Accuracy (IOU > 0.5):
- Average Aided: 95.6% (94.1, 97.0)
- Average Unaided: 93.6% (92.1, 95.0)
- TI-RADS Descriptor Accuracy:
- Composition: Aided 84.9% (82.2, 87.5), Unaided 80.4% (77.3, 83.4)
- Echogenicity: Aided 77.4% (74.4, 80.3), Unaided 70.0% (67.0, 72.8)
- Shape: Aided 90.8% (88.2, 93.1), Unaided 86.4% (83.7, 88.8)
- Margin: Aided 73.5% (70.2, 76.7), Unaided 57.3% (53.3, 61.2)
- Echogenic Foci: Aided 75.2% (71.9, 78.5), Unaided 71.1% (67.1, 74.9)
- TI-RADS Level Agreement:
- Overall: Aided 60.0% (56.8, 63.3), Unaided 51.1% (47.8, 54.5)
- TR-1: Aided 59.0% (42.3, 74.9), Unaided 52.9% (37.3, 68.3)
- TR-2: Aided 38.1% (31.1, 45.6), Unaided 31.2% (24.6, 38.1)
- TR-3: Aided 68.9% (62.6, 74.9), Unaided 58.8% (52.2, 65.4)
- TR-4: Aided 61.4% (56.5, 66.3), Unaided 52.1% (47.2, 57.0)
- TR-5: Aided 71.3% (61.8, 80.5), Unaided 62.0% (52.2, 71.5)
- The use of the device results in superior performance of the readers on nodule localization, TI-RADS lexicon characterization, and TR level agreement.
- Improved AULROC performance across different IOU thresholds compared to unaided readers.
- Significant improvement and superiority in characterizing all lexicon descriptors according to ACR TI-RADS.
- Significant improvement in TR level agreement across all TR levels, especially higher TR levels (TR-3, TR-4, TR-5).
- Consistent performance across different subgroups of patient sex, patient age, nodule size, ultrasound machine, reader category, reader experience, and source of data.
- AULROC (IOU > 0.5):
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
- AUC / AULROC
- Localisation Accuracy
- TI-RADS Descriptor Accuracy (Composition, Echogenicity, Shape, Margin, Echogenic Foci)
- TI-RADS Level Agreement (Overall, TR-1 to TR-5)
Predicate Device(s)
Reference Device(s)
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 892.2090 Radiological computer-assisted detection and diagnosis software.
(a)
Identification. A radiological computer-assisted detection and diagnostic software is an image processing device intended to aid in the detection, localization, and characterization of fracture, lesions, or other disease-specific findings on acquired medical images (e.g., radiography, magnetic resonance, computed tomography). The device detects, identifies, and characterizes findings based on features or information extracted from images, and provides information about the presence, location, and characteristics of the findings to the user. The analysis is intended to inform the primary diagnostic and patient management decisions that are made by the clinical user. The device is not intended as a replacement for a complete clinician's review or their clinical judgment that takes into account other relevant information from the image or patient history.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithm, including a description of the algorithm inputs and outputs, each major component or block, how the algorithm and output affects or relates to clinical practice or patient care, and any algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will provide improved assisted-read detection and diagnostic performance as intended in the indicated user population(s), and to characterize the standalone device performance for labeling. Performance testing includes standalone test(s), side-by-side comparison(s), and/or a reader study, as applicable.
(iii) Results from standalone performance testing used to characterize the independent performance of the device separate from aided user performance. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, positive and negative predictive values, and diagnostic likelihood ratio). Devices with localization output must include localization accuracy testing as a component of standalone testing. The test dataset must be representative of the typical patient population with enrichment made only to ensure that the test dataset contains a sufficient number of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant disease, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Results from performance testing that demonstrate that the device provides improved assisted-read detection and/or diagnostic performance as intended in the indicated user population(s) when used in accordance with the instructions for use. The reader population must be comprised of the intended user population in terms of clinical training, certification, and years of experience. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, positive and negative predictive values, and diagnostic likelihood ratio). Test datasets must meet the requirements described in paragraph (b)(1)(iii) of this section.(v) Appropriate software documentation, including device hazard analysis, software requirements specification document, software design specification document, traceability analysis, system level test protocol, pass/fail criteria, testing results, and cybersecurity measures.
(2) Labeling must include the following:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the device instructions for use, including the intended reading protocol and how the user should interpret the device output.
(iii) A detailed description of the intended user, and any user training materials or programs that address appropriate reading protocols for the device, to ensure that the end user is fully aware of how to interpret and apply the device output.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations must include situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) A detailed summary of the performance testing, including test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders, such as anatomical characteristics, patient demographics and medical history, user experience, and imaging equipment.
0
Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: the Department of Health & Human Services logo on the left and the FDA logo on the right. The FDA logo is a blue square with the letters "FDA" in white, followed by the words "U.S. Food & Drug Administration" in blue.
See-Mode Technologies Pte. Ltd. % Sadaf Monajemi Co-founder and Director 32 Carpenter Street. #03-01 Singapore, 059911 Singapore
September 9, 2024
Re: K240697
Trade/Device Name: See-Mode Augmented Reporting Tool, Thyroid (SMART-T) Regulation Number: 21 CFR 892.2090 Regulation Name: Radiological Computer Assisted Detection And Diagnosis Software Regulatory Class: Class II Product Code: QDQ, QIH Dated: August 2, 2024 Received: August 2, 2024
Dear Sadaf Monajemi:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
1
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30. Design controls; 21 CFR 820.90. Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatory
2
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Jessica Lamb
Jessica Lamb, Ph.D. Assistant Director Imaging Software Team DHT 8B: Division of Radiological Imaging Devices and Electronic Products OHT 8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
3
Indications for Use
Submission Number (if known)
Device Name
See-Mode Augmented Reporting Tool, Thyroid (SMART-T)
Indications for Use (Describe)
See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is a stand-alone reporting software to assist trained medical professionals in analyzing thyroid ultrasound images of adult (>=22 years old) patients who have been referred for an ultrasound examination.
Output of the device includes regions of interest (ROIs) placed on the thyroid ultrasound images assisting healthcare professionals to localize nodules in thyroid studies. The device also outputs ultrasonographic lexicon-based descriptors based on ACR TI-RADS. The software generates a report based on the image analysis results to be reviewed and approved by a qualified clinician after performing quality control.
SMART-T may also be used as a structured reporting software for further ultrasound studies. The software includes tools for reading measurements and annotations from the images that can be used for generating a structured report.
Patient management decisions should not be made solely on the basis of analysis by See-Mode Augmented Reporting Tool, Thyroid.
Type of Use (Select one or both, as applicable)
Prescription Use (Part 21 CFR 801 Subpart D)
Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
4
Image /page/4/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, teal-colored graphic above the text "See-Mode". The graphic appears to be two overlapping wave-like shapes, possibly representing a signal or data visualization. The text "See-Mode" is in a simple, sans-serif font.
This "510(k) Summary" was prepared per section 807.92(c).
ADMINISTRATIVE INFORMATION 1.
| Date of Preparation: | September 5, 2024 |
|---|---|
| Prepared by: | Sadaf Monajemi, PhD. Co-founder and Director |
| Manufacturer: | See-Mode Technologies Pte. Ltd. |
| 32 Carpenter Street #03-01 | |
| Singapore 059911 | |
| SINGAPORE | |
| Email: sadaf@see-mode.com | |
| Tel: +61 415 952 782 | |
| www.see-mode.com | |
| Official Contact: | Dr. Sadaf Monajemi, PhD, Co-founder and Director |
| See-Mode Technologies | |
| 32 Carpenter Street #03-01 | |
| Singapore 059911 | |
| SINGAPORE | |
| Email: sadaf@see-mode.com | |
| www.see-mode.com |
2. DEVICE NAME AND CLASSIFICATION
Trade/Proprietary Name: See-Mode Augmented Reporting Tool, Thyroid (SMART-T) Regulation Number: 21 CFR 892.2090 Regulation Name: Radiological computer-assisted detection and diagnosis software Classification Name: System, Image Processing, Radiological Review Panel: Radiology Regulatory Class: Class II Product Code: QDQ/QIH
3. INTENDED USE
Localization and characterization of thyroid ultrasound images.
5
Image /page/5/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized wave-like graphic above the text "See-Mode." Both the graphic and the text are in a teal color. The wave graphic appears to be two overlapping sine waves.
4. INDICATIONS FOR USE
See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is a stand-alone reporting software to assist trained medical professionals in analyzing thyroid ultrasound images of adult (>=22 years old) patients who have been referred for an ultrasound examination.
Output of the device includes regions of interest (ROIs) placed on the thyroid ultrasound images assisting healthcare professionals to localize nodules in thyroid studies. The device also outputs ultrasonographic lexicon-based descriptors based on ACR TI-RADS. The software generates a report based on the image analysis results to be reviewed and approved by a qualified clinician after performing quality control.
SMART-T may also be used as a structured reporting software for further ultrasound studies. The software includes tools for reading measurements and annotations from the images that can be used for generating a structured report.
Patient management decisions should not be made solely on the basis of analysis by See-Mode Augmented Reporting Tool, Thyroid.
DEVICE DESCRIPTION 5.
See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is a stand-alone, web-based image processing and reporting software for localization, characterization and reporting of thyroid ultrasound images.
The software analyzes thyroid ultrasound images and uses machine learning algorithms to extract specific information. The algorithms can identify and localize suspicious soft tissue nodules and also generate lexicon-based descriptors, which are classified according to ACR TI-RADS (composition, echogenicity, shape, margin, and echogenic foci) with a calculated TI-RADS level according to the ACR TI-RADS chart.
SMART-T may also be used as a structured reporting software for further ultrasound studies. The software includes tools for reading measurements and annotations from the images that can be used for generating a structured report.
The software then generates a report based on the image analysis results to be reviewed and approved by a qualified clinician after performing quality control. Any information within this report can be changed and modified by the clinician if needed during quality control and before finalizing the report.
The software runs on a standard "off-the-shelf" computer and can be accessed within the client web browser to perform the reporting of ultrasound images. Input data and images for the software are acquired through DICOM-compliant ultrasound imaging devices.
6
Image /page/6/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized wave-like graphic above the text "See-Mode". The graphic is a teal color and appears to be two overlapping sine waves. The text "See-Mode" is also teal and in a sans-serif font.
The data produced by the software is intended to be used by trained medical professionals, including but not limited to physicians and medical technicians. The software is not intended to be used as an independent source of medical advice or to determine or recommend a course of action or treatment for patients.
6. SUBSTANTIAL EQUIVALENCE
6.1. Predicate Device
Manufacturer: TaiHao Medical Inc. Trade Name: BU-CAD 510(k) Identifier: K210670 Regulation Number: 21 CFR 892.2090 Regulation Name: Radiological Computer Assisted Detection/Diagnosis Software For Lesions Suspicious For Cancer Classification Name: Radiological Computer Assisted Detection/Diagnosis Software For Lesions Suspicious For Cancer Classification Panel: Radiology Regulatory Class: Class II Product Code: QDQ, LLZ Date Cleared: December 21, 2021
6.2. Tabular Comparison of Features and Specifications of the Subject Device, Predicate Device, and Reference Device
| | Subject Device
See-Mode Augmented
Reporting Tool, Thyroid
(SMART-T)
(K240697) | Predicate Device
BU-CAD (K210670) | Reference Device
Koios DS (K212616) |
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| | Administrative information | | |
| Regulation | 21 CFR 892.2090 | 21 CFR 892.2090 | 21 CFR 892.2060 |
| | Radiological
computer-assisted detection
and diagnosis software for
lesions suspicious for cancer | Radiological
computer-assisted detection
and diagnosis software for
lesions suspicious for cancer | Radiological
computer-assisted diagnostic
software for lesions
suspicious of cancer |
| Regulatory
Class | Class II | Class II | Class II |
| Subject Device | Predicate Device | Reference Device | |
| See-Mode Augmented
Reporting Tool, Thyroid
(SMART-T)
(K240697) | BU-CAD (K210670) | Koios DS (K212616) | |
| Product
Code | QDQ, QIH | QDQ, LLZ | POK, QIH |
| 510(k)
Number | K240697 | K210670 | K212616 |
| Intended Use | | | |
| Indications
for Use | See-Mode Augmented
Reporting Tool, Thyroid
(SMART-T) is a stand-alone
reporting software to assist
trained medical
professionals in analyzing
thyroid ultrasound images
of adult (>=22 years old)
patients who have been
referred for an ultrasound
examination.
Output of the device
includes regions of interest
(ROIs) placed on the thyroid
ultrasound images assisting
healthcare professionals to
localize nodules in thyroid
studies. The device also
outputs ultrasonographic
lexicon-based descriptors
based on ACR TI-RADS. The
software generates a report
based on the image analysis
results to be reviewed and
approved by a qualified
clinician after performing
quality control.
SMART-T may also be used
as a structured reporting
software for further
ultrasound studies. The | BU-CAD is a software
application indicated to assist
trained interpreting
physicians in analyzing the
breast ultrasound images of
patients with soft tissue
breast lesions suspicious for
breast cancer who are being
referred for further diagnostic
ultrasound examination.
Output of the device includes
regions of interest (ROIs) and
lesion contours placed on
breast ultrasound images
assisting physicians to identify
suspicious soft tissue lesions
from up to two orthogonal
views of a single lesion, and
region-based analysis of
lesion malignancy upon the
physician's query. The
region-based analysis
indicates the score of lesion
characteristics (SLC), and
corresponding BI-RADS
categories in user-selected
ROIs or ROIs automatically
identified by the software. In
addition, BU-CAD also
automatically classifies lesion
shape, orientation, margin,
echo pattern, and posterior | Koios DS is an artificial
intelligence (AI)/machine
learning (ML)-based
computer-aided diagnosis
(CADx) software device
intended for use as an
adjunct to diagnostic
ultrasound examinations of
lesions or nodules
suspicious for breast or
thyroid cancer.
Koios DS allows the user to
select or confirm regions of
interest (ROIs) within an
image representing a single
lesion or nodule to be
analyzed. The software then
automatically characterizes
the selected image data to
generate an AI/ML-derived
cancer risk assessment and
selects applicable
lexicon-based descriptors
designed to improve overall
diagnostic accuracy as well
as reduce interpreting
physician variability.
Koios DS may also be used as
an image viewer of
multi-modality digital
images, including ultrasound |
| Subject Device
See-Mode Augmented
Reporting Tool, Thyroid
(SMART-T)
(K240697) | Predicate Device
BU-CAD (K210670) | Reference Device
Koios DS (K212616) | |
| annotations from the images
that can be used for
generating a structured
report.
Patient management
decisions should not be
made solely on the basis of
analysis by See-Mode
Augmented Reporting Tool,
Thyroid. | BU-CAD may also be used as
an image viewer of
multi-modality digital images,
including ultrasound and
mammography. The software
includes tools that allow users
to adjust, measure and
document images, and output
into a structured report (SR).
Patient management decisions
should not be made solely on
the basis of analysis by
BU-CAD. | allow users to adjust,
measure and document
images, and output into a
structured report.
Koios DS software is
designed to assist trained
interpreting physicians in
analyzing the breast
ultrasound images of adult
(>= 22 years) female
patients with soft tissue
breast lesions and/or
thyroid ultrasounds of all
adult (>= 22 years) patients
with thyroid nodules
suspicious for cancer. When
utilized by an interpreting
physician who has
completed the prescribed
training, this device provides
information that may be
useful in recommending
appropriate clinical
management. | |
| Intended
Population | Patients with thyroid
nodules
who are being referred
for ultrasound scan
(Prescription only) | Patients with soft
tissue breast lesions
who are being referred
for ultrasound
interpreting
(Prescription only) | Patients with thyroid nodules
suspicious for cancer
(Prescription only) |
| Image
Source | Ultrasound images | Ultrasound images | Ultrasound images |
| Rx only? | Yes | Yes | Yes |
| | Subject Device | Predicate Device | Reference Device |
| | See-Mode Augmented
Reporting Tool, Thyroid
(SMART-T)
(K240697) | BU-CAD (K210670) | Koios DS (K212616) |
| Application
Description | The subject device is a
stand-alone, web-based
image processing and
reporting software for
localization, characterization
and reporting of thyroid
ultrasound images.
The software analyzes
thyroid ultrasound images
and uses machine learning
algorithms to extract specific
information. The algorithms
can identify and localize
suspicious soft tissue
nodules and also generate
lexicon-based descriptors,
which are classified
according to ACR TI-RADS
(composition, echogenicity,
shape, margin, and echogenic
foci) with a calculated
TI-RADS category according
to the ACR TI-RADS chart.
The software then generates
a report based on the image
analysis results to be
reviewed and approved by a
qualified clinician after
performing quality control.
Any information within this
report can be changed and
modified by the clinician if
needed during quality
control and before finalizing
the report. | BU-CAD is a software system
designed to assist users in
analyzing breast ultrasound
images including identification
of regions suspicious for breast
cancer and assessment of their
malignancy. BU-CAD consists
of a viewer, a lesion
identification module, and a
lesion analysis module.
The lesion identification
module identifies regions of
interest (automated ROIs) of a
single suspicious soft tissue
lesion in up to two orthogonal
views of breast ultrasound
images for assisting users in
detecting soft tissue lesions.
Additionally, the lesion
identification module
generates an ROI and a lesion
contour on each breast
ultrasound image.
The lesion analysis module
analyzes given ROIs of a breast
lesion on ultrasound images,
and generates a score of lesion
characteristics (SLC) in terms
of malignancy or benignity of a
lesion, BI-RADS category, and
BI-RADS descriptors. | Koios DS is a
computer-aided diagnosis
(CADx) software device
intended for use as an
adjunct to diagnostic
ultrasound examinations of
lesions or nodules
suspicious for breast or
thyroid cancer.
Koios DS allows the user to
select or confirm regions of
interest (ROIs) within an
image representing a single
lesion or nodule to be
analyzed. Koios DS software
contains functionality for
automatically classifying
thyroid nodules suspicious
for cancer.
The system generates an
output aligned to either the
TI-RADS or ATA classification
guidelines. The system
automatically generates
user-modifiable thyroid
nodule descriptors
(Composition, Echogenicity,
Shape, Margin, Echogenic
Foci) and a direct,
image-derived cancer risk
assessment that is translated
into an optional
lexicon-specific (TI-RADS or
ATA) modifier. |
| Anatomical
Location | Thyroid | Breast | Thyroid and Breast |
| Input | Medical images
provided in a DICOM
format | Medical images
provided in a DICOM
format | Medical images
provided in a DICOM
format |
| | Subject Device
See-Mode Augmented
Reporting Tool, Thyroid
(SMART-T)
(K240697) | Predicate Device
BU-CAD (K210670) | Reference Device
Koios DS (K212616) |
| Output | ROIs placed on thyroid
nodules
TI-RADS lexicon descriptors
TI-RADS category according
to the ACR TI-RADS chart | ROIs and lesion
contours placed on
suspicious soft
tissue lesion
BI-RADS lexicon descriptors
A region-based score
of lesion malignancy
BI-RADS category | Thyroid
TI-RADS lexicon descriptors
based on user-selected ROIs
TI-RADS category according
to the ACR TI-RADS chart
A direct, deep-learning
derived cancer risk
assessment that is translated
into an optional
lexicon-specific modifier.
The software's direct,
non-descriptor-based cancer
risk assessment is presented
as the Koios "AI Adapter" that
can be used in conjunction
with the ACR TI-RADS or ATA
guidelines for nodule risk
stratification
Breast
Categorical and continuous
outputs (confidence level
indicator) that align to
BI-RADScategories
Auto classification of
BI-RADS lexicon descriptors
(shape and orientation) |
| Operating
Platform | Client-server technology. | Client-server technology | Client-server technology |
| Image
Format | DICOM | DICOM | DICOM |
| 2D viewing
capabilities | Yes | Yes | Yes |
| Image
storage and
report
generation | Yes | Yes | Yes |
7
Image /page/7/Picture/1 description: The image contains the logo for See-Mode. The logo consists of a stylized, abstract symbol above the text "See-Mode". The symbol is composed of two overlapping, curved lines that resemble waves or a stylized letter 'M'. The color of both the symbol and the text is a light teal or green.
8
Image /page/8/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, green, wave-like symbol above the text "See-Mode", which is also in green. The wave symbol appears to be two overlapping curves, creating a visual representation of movement or flow.
9
Image /page/9/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, connected, wavy line in a light green color, resembling a sine wave or a stylized letter 'M'. Below the graphic is the text "See-Mode" in a sans-serif font, also in the same light green color.
10
Image /page/10/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized wave-like graphic above the text "See-Mode". The graphic is a connected line that forms two peaks and valleys, resembling a simplified waveform. The text "See-Mode" is in a sans-serif font and is positioned directly below the graphic.
11
Image /page/11/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized wave-like graphic above the text "See-Mode". The graphic is a teal color and appears to be two overlapping sine waves. The text "See-Mode" is also teal and in a sans-serif font.
6.3. Comparison with Predicate and Reference Devices
Similarities
- · Intended Use: The intended use of SMART-T is the same as that of the legally marketed predicate device, BU-CAD. Both are intended to be used by clinicians interpreting radiological images to help them localize and characterize soft tissue lesions. SMART-T and the predicate device are both intended to be used concurrently with the reading of images and are not intended as a replacement for the review of a clinician or their clinical judgment.
- Target Population: SMART-T, BU-CAD, and Koios DS share the same intended ● population. All devices are intended to be used for assisting trained interpreting clinicians in analyzing patients with soft tissue lesions or suspicious nodules that are being referred for diagnostic ultrasound examination.
- Localization and Characterization: See-Mode Augmented Reporting Tool, Thyroid . (SMART-T) is similar to BU-CAD, the legally marketed predicate device, in its intent to localize soft tissue lesions in ultrasound images. While the reference Koios DS device analyzes thyroid lesions, it relies on users to manually identify the nodules, whereas the subject device automatically localizes the nodules.
Additionally, similar to the predicate and reference devices, the subject device characterizes nodules based on lexicon descriptors. The subject device (SMART-T), predicate device (BU-CAD), and reference device (Koios DS) all use established classification systems for their ultrasonographic lexicon descriptors. They all use the American College of Radiology Systems for describing soft tissue lesions or nodules. The ACR atlases provide standardized imaging terminology, report organization, assessment structure, and a classification system, which enables radiologists to communicate results clearly and consistently. The predicate BU-CAD device uses ACR BI-RADS for breast ultrasound images. Both SMART-T and the reference Koios DS device use ACR TI-RADS for analyzing and reporting thyroid ultrasound images.
Both BU-CAD and SMART-T automatically localize soft tissue findings, with the findings classified in line with lexicon descriptors. Similar to BU-CAD, our subject device provides automatic regions of interest (ROIs), rather than manually selected, to localize thyroid nodules. The ROI highlights the bounding box around the detected nodules for the user to easily review and make their own assessment.
- Performance Testing: When comparing clinical validation between SMART-T, ● BU-CAD and Koios DS, the devices were evaluated using similar endpoints in their clinical studies. The Area Under the Curve (AUC) shift was used when comparing the performance of users with and without the aid of the device.
12
Image /page/12/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, wavy line in a teal color, resembling two interconnected 'M' shapes, positioned above the text "See-Mode". The text is also in the same teal color as the wavy line above it. The logo has a clean and modern design.
The number of cases evaluated in the MRMC reader study for BU-CAD was 628, the number of cases for Koios DS was 650 and the reader study for SMART-T included 600 cases. The number of readers included in the BU-CAD reader study was 16 (14 radiologists and 2 breast surgeons), the number of readers for Koios DS was 15 (14 radiologists and 1 endocrinologist) and the number of readers for SMART-T was 18 (all radiologists).
Following the primary endpoint, the AUC shift between predicate and subject device was similar. Similarly, the MRMC for BU-CAD showed an improvement of readers' determination of BI-RADS descriptors (Shape, Orientation, Margin, Echo Pattern, and Posterior Features) for at least one or more subcategories for each descriptor. The MRMC study for SMART-T, however, showed reader improvement in all TI-RADS descriptors (Composition, Echogenicity, Shape, Margin and Echogenic Foci).
The SMART-T MRMC reader study demonstrated substantially equivalent performance to BU-CAD and Koios DS by showing similar study design, success criteria and performance.
Differences
- Anatomical Regions: BU-CAD is specifically focused on breast ultrasound images, whereas See-Mode Augmented Reporting Tool, Thyroid (SMART-T) analyzes thyroid ultrasound images. This is aligned with our reference device, Koios DS, which assesses and characterizes both breast lesions and thyroid nodules using ultrasound image data. SMART-T localizes thyroid nodules and characterizes the nodule with their lexicon descriptors (composition, echogenicity, shape, margin, echogenic foci) in line with ACR TI-RADS.
- Modality: Contrary to BU-CAD and Koios DS, SMART-T only interprets ultrasound . images. SMART does not analyze mammography images or other multi-modality digital images.
The technological characteristics of the subject device and its predicate and reference device have been evaluated to determine equivalence. Upon reviewing and comparing intended use, design, materials, principle of operation, and overall technological characteristics, the subject device is determined to be substantially equivalent to predicate and reference devices.
Both devices (subject device and predicate) have the same intended use and are indicated for the same use. The subject device uses standard principles of operation, methods, and algorithms for processing, measurement, and quantification of the images and is intended to be used by trained professionals, which is similar to the predicate and reference devices.
13
Image /page/13/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, teal-colored symbol resembling two interconnected sine waves or mountain peaks. Below the symbol is the company name, "See-Mode," also in teal, with a hyphen connecting the two words.
The comparison of technological characteristics, non-clinical performance data, clinical data, and software validation data demonstrate that See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is as safe and effective when compared to the predicate and reference devices that are currently marketed for similar intended use and indications for use. The table above provides a comparison between the proposed subject device, the predicate device, and the reference device.
7. PERFORMANCE DATA - CLINICAL AND BENCH TESTING
The performance of our device has been validated both in an MRMC reader study and standalone, as described below:
- Standalone Study: To evaluate the standalone performance of our device, where the . output of the models are directly compared against ground truth labels.
- Multi-reader Multi-Case (MRMC) Study: To compare the performance of expert readers (radiologists) with and without the aid of our device. The study evaluated the readers' localization and characterisation of thyroid nodules in scenarios both unaided and aided. In the MRMC study, we had 18 radiologists read 600 cases from 600 patients twice, once with the aid of the device and once without. There was a one-month washout period in between the two reads.
To ensure the generalizability and satisfactory performance of our models on new, unseen data, all cases in our MRMC study were sourced from institutions or sources not part of the model training or development datasets. The preceding results of this study, demonstrating the models' success with data from novel institutions, have been presented.
The performance of the device was evaluated across different sub-groups of patient sex, patient age, nodule size, ultrasound machine, data sources (US vs. Non-US), reader category (US vs. Non-US), and reader experience.
Data Selection
The dataset has been collected retrospectively from thyroid ultrasound images of patients who have been referred for an ultrasound examination. The study consisted of 600 cases from unique patients with 74% of the data acquired from the US. The age range is respective of the target population with an 81.5% female cohort. The ethnicity distribution in our dataset is representative of the broader US population.
14
Image /page/14/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized wave-like graphic above the text "See-Mode". The graphic is a teal color and appears to be two sine waves connected together. The text "See-Mode" is also teal and is in a sans-serif font.
The cases in the dataset contained both negative and positive biopsy results with all TI-RADS levels represented. The data is sampled such that the distribution of the data accounts for patient sex, age, nodule size, malignancy, a priori clinical ACR TI-RADS level and a minimum sample is present for each subgroup. The data was curated from images sourced from accepted ultrasound systems, such as GE Healthcare, Philips Medical Systems and Siemens Medical Systems.
The patient distribution of the dataset:
- Patient sex
- o Female: 489 cases
- o Male: 111 cases
- Patient age
- o 0.5 as the criterion for successful location detection in the LROC analysis. In this analysis, the malignant cases where the detection IOU is below or equal to 0.5 are penalised as false negative.
| IOU Criteria | Average Aided AUC
(95% CI) | Average Unaided AUC
(95% CI) | Standalone
(95% CI) |
|--------------|-------------------------------|---------------------------------|------------------------|
| IOU > 0.5 | 0.758 (0.711, 0.803) | 0.736 (0.693, 0.780) | 0.703 (0.642, 0.762) |
To provide a comprehensive analysis and evaluate the performance of our device with various IOU thresholds, we have obtained AULROCs for different IOU thresholds ranging from 0.5 to 0.8 with 0.1 increments as shown in the table below. It was observed that the use of the device results in an improved AULROC performance for the readers across different IOU thresholds. Specifically, AULROC is improved significantly when applying more rigorous IOU criteria (>0.6, >0.7, and >0.8), showing the promise of the device to improve readers' performance.
| Average Reader AUC (95% CI) | |||
|---|---|---|---|
| Analysis | Aided | Unaided | Difference (Aided - Unaided) |
| AURLOC IOU > 0.5 | 0.758 (0.711, 0.803) | 0.736 (0.693, 0.780) | 0.022 (-0.012, 0.056) |
| IOU > 0.6 | 0.734 (0.682, 0.781) | 0.682 (0.632, 0.730) | 0.052 (0.008, 0.093) |
| IOU > 0.7 | 0.686 (0.629, 0.740) | 0.548 (0.490, 0.610) | 0.138 (0.082, 0.195) |
| IOU > 0.8 | 0.593 (0.529, 0.658) | 0.356 (0.293, 0.423) | 0.237 (0.168, 0.307) |
Identification and localisation of nodules
To evaluate the localisation performance of the device, we calculated the average accuracy of the readers on localisation in aided and unaided scenarios, as well as the standalone performance. Localisation accuracy for each reader is calculated as the number of cases
16
Image /page/16/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, teal-colored graphic above the text "See-Mode". The graphic appears to be two connected sine waves, forming a shape reminiscent of the letter 'M' or a mountain range. The text "See-Mode" is in a bold, sans-serif font, also in teal, and is positioned directly below the graphic.
where the reader's bounding box has an overlap greater than 0.5 with the ground truth, divided by the total number of cases.
| | Average Aided
(95% CI) | Average Unaided
(95% CI) | Standalone (95% CI) |
|--------------------------|---------------------------|-----------------------------|---------------------|
| Localisation
Accuracy | 95.6% (94.1, 97.0) | 93.6% (92.1, 95.0) | 95.1% |
As demonstrated in the table above, the aid of our device results in superior performance of the readers for localising thyroid nodules. It was also observed that the results of the algorithm are consistent across different sub-groups.
Thyroid lexicon descriptors
Aside from localisation, the other output of our device is characterisation of thyroid nodules according to ACR TI-RADS lexicon descriptors. In the table below, we have calculated the accuracy of the readers in determining each of the ACR TI-RADS descriptors (composition, echogenicity, shape, margin, and echogenic foci). We have also evaluated the standalone performance of the device on TI-RADS lexicon descriptors. As described above, the ground truth for the TI-RADS descriptors is the consensus labels of two expert US-board certified radiologists and an adjudicator (also US-board certified radiologist with the most years of experience).
| TI-RAD Descriptor | Average Aided
(95% CI) | Average Unaided
(95% CI) | Standalone (95%
CI) |
|-------------------|---------------------------|-----------------------------|------------------------|
| Composition | 84.9% (82.2, 87.5) | 80.4% (77.3, 83.4) | 86.7% |
| Echogenicity | 77.4% (74.4, 80.3) | 70.0% (67.0, 72.8) | 68.2% |
| Shape | 90.8% (88.2, 93.1) | 86.4% (83.7, 88.8) | 93.4% |
| Margin | 73.5% (70.2, 76.7) | 57.3% (53.3, 61.2) | 58.4% |
| Echogenic Foci | 75.2% (71.9, 78.5) | 71.1% (67.1, 74.9) | 70.3% |
As can be seen from the table above, the device provides significant improvement and achieves superiority for characterizing all lexicon descriptors according to ACR TI-RADS. It
17
Image /page/17/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, abstract symbol above the text "See-Mode". The symbol is composed of two wavy lines that intersect in the middle, forming a shape reminiscent of mountains or waves. The color of both the symbol and the text is a muted green.
was also observed that the results of the algorithm are consistent across different sub-groups.
Attaining superiority across all TI-RADS descriptors is a strong proof point for the performance of our device, as it aligns with its primary function and intended use according to the ACR TI-RADS guideline.
TI-RADS level agreement
The accuracy of TI-RADS (TR) levels is critical as it directly informs clinical decisions regarding patient management according to the ACR TI-RADS guideline. To evaluate the impact of our device on TR level agreement, we have compared the overall TR level agreement percentage of each of the readers against the ground truth in both aided and unaided scenarios. As described above, the ground truth for the TI-RADS descriptors and, therefore the TR level, is based on the consensus labels of two expert US-board certified radiologists and an adjudicator (also US-board certified radiologist with the most years of experience).
To further evaluate the level of agreement, we also calculated the agreement for each TR level (TR-1 to TR-5). The agreement percentage for each TR level for each reader is calculated by dividing the number of cases where a reader's TR level matches the ground truth TR level by the total number of cases with that specific TR level. The average TR level agreement is then calculated as the average over all the readers.
| TI-RADS | Average Aided
(95% CI) | Average Unaided
(95% CI) | Standalone (95%
CI) |
|---------|---------------------------|-----------------------------|------------------------|
| Overall | 60.0% (56.8, 63.3) | 51.1% (47.8, 54.5) | 63.8 (60.0, 67.7) |
| TR-1 | 59.0% (42.3, 74.9) | 52.9% (37.3, 68.3) | 61.9 (40.0, 82.6) |
| TR-2 | 38.1% (31.1, 45.6) | 31.2% (24.6, 38.1) | 41.1 (31.7, 50.4) |
| TR-3 | 68.9% (62.6, 74.9) | 58.8% (52.2, 65.4) | 71.7 (64.9, 78.3) |
| TR-4 | 61.4% (56.5, 66.3) | 52.1% (47.2, 57.0) | 65.5 (59.1, 71.6) |
| TR-5 | 71.3% (61.8, 80.5) | 62.0% (52.2, 71.5) | 77.0 (66.1, 87.3) |
18
Image /page/18/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized wave-like symbol above the text "See-Mode". The wave symbol is made up of two connected curves, resembling an "M". The color of the logo is a light green.
The table above highlights a significant improvement in TR level agreement among readers with the use of the device. It is important to note that TR level agreement has been improved across all TR levels, from TR-1 to TR-5, with particularly significant gains at higher TR levels (TR-3, TR-4, and TR-5). Notably, these higher TR levels (TR-3) are the levels that are of high clinical importance with follow-up or FNA considerations. It was also observed that the results of the algorithm are consistent across different sub-groups.
Subgroup analysis
Subgroup analysis of patient sex (female, male), patient age (=25mm), ultrasound machine (GE, Philips, Samsung, Siemens, Canon/Toshiba), reader category (US/Canada, Non-US/Canada), reader experience (0-3 years, 4-7 years, 8-10 years, >=11 years), and source of data (US, non-US) from the MRMC reader study were performed. The readers aided by SMART-T achieved consistent performance across all of the subgroups.
Conclusion
The results of our MRMC reader study shows improvement in reader performance with the aid of our device. Based on these results it was observed that the use of the device results in superior performance of the readers on nodule localisation, TI-RADS lexicon characterisation, and TR level agreement. The use of the device can also improve the performance of the readers with regards to malignancy and benignity status.
The standalone performance of the device is also on-par with the aided use of our device, indicating the validity of our algorithms. The outputs of our study are in line with the intended use of See-Mode Augmented Reporting Tool, Thyroid.
NON-CLINICAL DATA 8.
The design and development of SMART-T has been implemented according to recognised standards. See-Mode Technologies has performed software verification and validation testing for the subject device according to the FDA's guidance document "Content of Premarket Submissions for Device Software Functions," as well as "IEC 62304:2006/AC: 2015 - Medical Device Software - Software Lifecycle Processes". Special Controls were added according to 21 CFR 892.2090, "DEN180005 Evaluation of automatic class III designation for OsteoDetect – Decision summary with special controls".
19
Image /page/19/Picture/1 description: The image shows the logo for See-Mode. The logo consists of a stylized, teal-colored graphic resembling two overlapping sine waves above the text "See-Mode" in a bold, sans-serif font, also in teal. The sine wave graphic is smooth and continuous, creating a visual representation of waves or signals.
The risk analysis was completed and the hazard risk analysis has been submitted as part of this application. It has been observed that all the risks identified with the subject device are acceptable and have been reduced as far as possible in accordance with ISO 14971:2019 Medical devices - Application of risk management to medical devices.
9. CONCLUSIONS
A detailed analysis has been conducted between See-Mode Augmented Reporting Tool, Thyroid (SMART-T) and its predicate and reference devices. It is noted that no new additional questions of safety and effectiveness are raised by these technologies. Through reviewing the similarities and differences of the intended use, technological characteristics and principles of operation, as well as assessing the clinical and non-clinical performance data, See-Mode Augmented Reporting Tool, Thyroid (SMART-T) is as safe, as effective and performs as well as the predicate and reference devices.