Koios Decision Support (DS) is an artificial intelligence (AI)/machine learning (ML)-based computer-aided diagnosis (CADx) software device intended for use as an adjunct to diagnostic ultrasound examinations of lesions or nodules suspicious for breast or thyroid cancer.

Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an AI/ML-derived cancer risk assessment and selects applicable lexicon-based descriptors designed to improve overall diagnostic accuracy as well as reduce interpreting physician variability.

Koios DS software may also be used as an image viewer of multi-modality digital images, including ultrasound and mammography. The software includes tools that allow users to adjust, measure and document images, and output into a structured report.

Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. When utilized by an interpreting physician who has completed the prescribed training, this device provides information that may be useful in recommending appropriate clinical management.

Limitations:
· Patient management decisions should not be made solely on the results of the Koios DS analysis.
· Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them.
· Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images.
· The software does not predict the presence of the thyroid nodule margin descriptor, extra-thyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list.

Device Description

Koios Decision Support (DS) is a software application designed to assist trained interpreting physicians in analyzing breast and thyroid ultrasound images. The software device is a web application that is deployed to a Microsoft IIS web server and accessed by a user through a compatible client. Once logged in and granted access to the Koios DS application, the user examines selected breast or thyroid ultrasound DICOM images. The user selects Regions of Interest (ROIs) of orthogonal views of a breast lesion or thyroid nodule for processing by Koios DS. The ROI(s) are transmitted electronically to the Koios DS server for image processing and the results are returned to the user for review.

AI/ML Overview

Here's a summary of the acceptance criteria and the study proving the device meets them, based on the provided text:

Device Name: Koios DS Version 3.6

1. Table of Acceptance Criteria and Reported Device Performance (Combining Breast and Thyroid where applicable):

Acceptance Criteria Category	Specific Metric (Breast Engine)	Reported Device Performance (Breast Engine)	Specific Metric (Thyroid Engine)	Reported Device Performance (Thyroid Engine)	Acceptance Criteria (Smart Click)	Reported Device Performance (Smart Click)	Acceptance Criteria (Image Registration & Matching)	Reported Device Performance (Image Registration & Matching)	Acceptance Criteria (OCR)	Reported Device Performance (OCR)
Standalone Performance (AI Engine)	Malignancy Risk Classifier AUC	0.945 [0.932, 0.959] (increased from 0.929)	AUC (ACR TI-RADS, with AI Adapter)	79.8% (significant increase over average physician AUC)	Non-inferiority Test - Sensitivity / Specificity	Sensitivity: Difference = -0.009 [-0.036, 0.018] (Non-inferior) Specificity: Difference = -0.018 [-0.041, 0.005] (Non-inferior)	No Match Rate	0.32%	Breast Freetext Identification (by field)	Breast Side: 0.983 Location Type: 0.948 Clock Hour: 0.926 Clock Minute: 0.934 CMFN: 0.944 Plane: 0.976
	Categorical Output Sensitivity	0.976 [0.960, 0.992] (increased from 0.97)	Sensitivity (ACR TI-RADS, biopsy rec., with AI Adapter)	0.644 [0.545, 0.744] (non-significant improvement over avg physician)	Non-inferiority Test - AUC	Difference = -0.012 [-0.029, 0.006] (Non-inferior)	Average Time for Study Preprocessing	2.39 +/- 0.48 seconds	Thyroid Freetext Identification (by field)	Thyroid Side: 0.965 Pole: 0.976 Region: 0.998 Plane: 0.970
	Categorical Output Specificity	0.632 [0.588, 0.676] (increased from 0.61)	Specificity (ACR TI-RADS, biopsy rec., with AI Adapter)	0.612 [0.566, 0.658] (significant improvement over avg physician)	Sub-optimal ROI Test	Difference = 0.026 [-0.009, 0.062] (Non-inferior)	Average Time for Image Matching	0.22 +/- 0.12 seconds	Measurement Text Identification (by field)	Measurement Description: 0.943 Measurement Value: 0.948 Unit of Measurement: 0.967
	Sensitivity to Region of Interest	0.012 (decreased from 0.019)	Sensitivity (ACR TI-RADS, follow-up rec., with AI Adapter)	0.879 [0.812, 0.946] (non-significant improvement)	Detection DICE Coefficient	DICE = 0.913 +/- 0.075 (demonstrating precise approximation to physician ROIs)	End-to-End Breast Engine Performance	AUC = 0.946 Sensitivity = 0.975 Specificity = 0.637
	Sensitivity to Transducer Frequency (High freq, >=15MHz)	AUC = 0.948 [0.917, 0.978] (increased from 0.940)	Specificity (ACR TI-RADS, follow-up rec., with AI Adapter)	0.495 [0.446, 0.544] (significant improvement)	Non-inferiority Test - Descriptor Agreement (per descriptor, e.g., Composition)	Demonstrated non-inferiority for all listed descriptors (Composition, Echogenicity, Shape, Margin, Echogenic Foci subcategories). Examples: Composition: 0.018 [0.001, 0.035]; Echogenicity: -0.005 [-0.022, 0.011]	End-to-End Thyroid Engine Performance	AUC = 0.801 Sensitivity = 0.670 Specificity = 0.603
	Sensitivity to Transducer Frequency (Low freq, <15MHz)	AUC = 0.940 [0.925, 0.956] (increased from 0.924)	AUC (ATA, with AI Adapter)	Significant increase of 9.135% [5.975, 12.294] over physician AUC	Overall Thyroid Engine Met or Exceeded Performance Requirements in all tests.		Breast Image Matching Outcomes	Successful Match: 99.5% (2018/2028 ROIs) No Match: 0.5% (10/2028 ROIs) Incorrect Match: 0.0% Incorrect Image: 0.0%
	Single Image vs Orthogonal Image Pair	Single Image: 0.932 [+/- 0.003] (not directly comparable, but improved standalone AUC overall)	Sensitivity (ATA, with AI Adapter)	Non-significant increase of 0.511% [-5.182, 6.204]			Breast Image Matching DICE Coefficient	0.995 +/- 0.005
	Operating Point (PLR, NLR, PPV, NPV)	Improved from predicate: PLR = 2.661 [2.338, 2.984]; NLR = 0.039 [0.013, 0.064]; PPV = 0.708 [0.672, 0.743]; NPV = 0.966 [0.944, 0.988]	Specificity (ATA, with AI Adapter)	Significant increase of 18.741% [9.885, 27.596]			Thyroid Image Matching Outcomes	Successful Match: 100% (1288/1288 ROIs) No Match: 0.0% Incorrect Match: 0.0% Incorrect Image: 0.0%
	Overall Breast Engine Met or Exceeded Performance Requirements in all tests.		Overall Thyroid Engine Met or Exceeded Performance Requirements in all tests.				Thyroid Image Matching DICE Coefficient	0.996 +/- 0.004

2. Sample Sizes and Data Provenance for Test Sets:

Breast Engine Standalone/Clinical Test Set:
- Sample Size: 900 lesions from 900 different patients. An expanded validation set of 1014 cases (900 + 114 additional) was used for dataset drift.
- Data Provenance: Retrospective. Images sourced from a wide variety of ultrasound hardware. Patient demographic distribution was based upon data from the Breast Cancer Surveillance Consortium (2006-2009) to ensure representativeness of diverse populations.
Thyroid Engine Standalone/Clinical Test Set:
- Sample Size: 650 retrospectively collected cases (nodules) from 650 different patients. Each lesion represented by two orthogonal images, totaling 1000 images for standalone testing.
- Data Provenance: Retrospective. Data analysis cases involved images from both the US (500 cases, 77%) and Europe (150 cases). Tested on images from a wide variety of ultrasound hardware.
Thyroid Smart Click Test Set:
- Sample Size: 650 nodules.
- Data Provenance: Not explicitly stated, but likely the same validation dataset as the Thyroid Engine, derived retrospectively from US and European locations.
Image Registration and Matching Test Set:
- Sample Size: 1,600 ultrasound studies (950 breast, 650 thyroid), involving 2028 breast ROIs and 1288 thyroid ROIs.
- Data Provenance: Not explicitly stated, but likely drawn from the same validation datasets for breast and thyroid as mentioned above.
OCR Test Set:
- Sample Size: 1910 ultrasound B-Scans (mix of thyroid and breast images). A subset of 1226 images from supported machines was used for evaluation.
- Data Provenance: Not explicitly stated, but derived from a variety of machines.

3. Number of Experts and Qualifications for Ground Truth - Test Set:

Breast Engine Standalone: Not applicable for malignancy risk classification ground truth, which was pathology or 1-year follow-up. For categorical agreement metrics (Shape, Orientation), it mentions "agreement with the subjective categorizations assigned by physicians," implying experts, but the number and specific qualifications are not detailed beyond "trained interpreting physicians."
Thyroid Engine Standalone: Not applicable for malignancy risk classification ground truth, which was "pathology results only." For descriptor predictions, it states they were tested "objectively – against ground truth pathology" and "subjectively and met the requirements for agreement with readers' descriptor categorizations," implying experts, but number and specific qualifications are not detailed.
Breast Clinical Study: 15 readers. Qualifications varied (Diagnostic Radiology, Breast Surgeon, OB/GYN). Years of experience ranged from 0 to 30 years. Some were Breast Fellowship Trained and/or Dedicated Breast Imagers, and some were MQSA Qualified Interpreting Physicians.
Thyroid Clinical Study: 15 readers (11 US-based, 4 European-based). Qualifications included Endocrinologists (End) and Radiologists (Rad). Experience ranged from < 10 years to ≥ 20 years (post-residency).

4. Adjudication Method for Test Set:

Breast Engine Standalone/Clinical:
- Malignancy Ground Truth: Determined by pathology or 1-year follow-up. No explicit adjudication method amongst multiple experts for this final ground truth is mentioned, implying a single, definitive source.
- Reader Study: Not explicitly stated for establishing a ground truth for individual cases based on reader input. The study collected reader interpretations and compared them to the established ground truth (pathology/follow-up).
Thyroid Engine Standalone/Clinical:
- Malignancy Ground Truth: Determined by "pathology results only." No explicit adjudication method amongst multiple experts is mentioned.
- Reader Study: Not explicitly stated for establishing a ground truth for individual cases based on reader input. The study collected reader interpretations and compared them to pathological ground truth.
Other Standalone Tests (Smart Click, Image Registration, OCR): Ground truth was based on defined metrics (e.g., DICE coefficient for ROI matching, manual annotation for OCR, physician-drawn ROIs, pathology for descriptor agreement). No multi-expert adjudication mentioned.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study:

Yes, for both Breast and Thyroid.
Effect Size of Human Readers Improve with AI vs without AI Assistance:
- Breast (from K190442, still applicable to K242130 as performance is superior):
  - Change in average AUC (USE + DS vs. USE Alone): 0.0370 [0.030, 0.044] at α = .05 (a significant increase).
  - Average Kendall Tau-B (measure of inter-operator variability):
    - USE Alone: 0.5404 (.5301, .5507)
    - USE + DS: 0.6797 (.6653, .6941) => Significant increase in agreement.
  - Intra-operator variability (class switching rate):
    - USE Alone: 13.6%
    - USE + DS: 10.8% (p = 0.042) => Statistically significant reduction.
- Thyroid (CRRS-3 Study):
  - Change in average AUC (USE + DS vs. USE Alone, all readers, all data): +0.083 [0.066, 0.099] (parametric) / +0.079 [0.062, 0.096] (non-parametric)
  - Specifically for US readers, US data: +0.074 [0.051, 0.098] (parametric) / +0.073 [0.049, 0.096] (non-parametric). This demonstrates a statistically significant improvement in overall reader performance.
  - Change in average Sensitivity/Specificity of FNA (with AI Adapter + size criteria):
    - All readers, all data: +0.084 (sensitivity), +0.140 (specificity)
    - US readers, US data: +0.058 (sensitivity), +0.130 (specificity)
  - Change in average Sensitivity/Specificity of Follow-up (with AI Adapter + size criteria):
    - All readers, all data: +0.060 (sensitivity), +0.206 (specificity)
    - US readers, US data: +0.053 (sensitivity), +0.180 (specificity)
  - Inter-Reader Variability (relative change in TI-RADS points association): 40.7% (all readers, all data), 37.4% (US readers, US data), 49.7% (EU Readers, EU Data)
  - Impact on Interpretation Time: -23.6% (all readers, all data), -22.7% (US readers, US data), -32.4% (EU Readers, EU Data).

6. Standalone (Algorithm Only without Human-in-the-loop) Performance:

Yes, for both Breast and Thyroid AI Engines, Smart Click, Image Registration and Matching, and OCR.
- Breast Engine: AUC = 0.945; Sensitivity = 0.976; Specificity = 0.632.
- Thyroid Engine (ACR TI-RADS, biopsy recommendation): Sensitivity = 0.644; Specificity = 0.612.
- Thyroid Smart Click: Demonstrated non-inferiority for Sensitivity, Specificity, AUC, and descriptor agreement compared to physician-selected calipers. Detection DICE = 0.913.
- Image Registration and Matching: Very high DICE coefficients (Breast 0.995, Thyroid 0.996) and successful match rates (>99.5%).
- OCR Engine: High accuracy rates for identification of various freetext and measurement fields (e.g., Breast Side 0.983, Measurement Value 0.948).

7. Type of Ground Truth Used:

Malignancy Risk Classification (Breast & Thyroid AI Engines):
- Breast: Pathology or 1-year follow-up.
- Thyroid: Pathology results only (for standalone). Clinical study also used cyto-/histological or excisional pathology.
Descriptor Predictions (Thyroid Standalone): Tested objectively against ground truth pathology and subjectively for agreement with readers' descriptor categorizations.
Smart Click, Image Registration, OCR: Ground truth was established by manual annotations, physician-drawn ROIs, or defined objective metrics (like DICE coefficient against a reference ROI).

8. Sample Size for Training Set:

Not explicitly stated for either Breast or Thyroid engines. The text mentions drawing upon a "large database of known cases" for the underlying engines and that the test sets were "set aside from the system's training data." However, the exact number of cases/images in the training set is not provided.

9. How Ground Truth for Training Set was Established:

Not explicitly detailed for either Breast or Thyroid engines. The text states the engines "draw upon knowledge learned from a large database of known cases, tying image features to their eventual diagnosis, to form a predictive model." This implies that the training data had associated definitive diagnoses (e.g., from pathology or follow-up), but the process of establishing this ground truth (e.g., expert review, adjudication) for the training data is not described.

Summary

{0}------------------------------------------------

November 15, 2024

Image /page/0/Picture/1 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: the Department of Health & Human Services logo on the left and the FDA logo on the right. The FDA logo features the letters 'FDA' in a blue square, followed by the words 'U.S. FOOD & DRUG ADMINISTRATION' in blue text.

Koios Medical, Inc. % Michael Bocchinfuso Director of Regulatory Compliance and Quality 242 West 38th Street 14th Floor New York, NY 10018

Re: K242130

Trade/Device Name: Koios DS Regulation Number: 21 CFR 892.2060 Regulation Name: Radiological computer-assisted diagnostic software for lesions suspicious of cancer Regulatory Class: Class II Product Code: POK, QIH Dated: August 13, 2024 Received: August 19, 2024

Dear Michael Bocchinfuso:

We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

{1}------------------------------------------------

Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).

Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review. the OS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.

All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rue"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-device-advicecomprehensive-regulatory-assistance/unique-device-identification-system-udi-system.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

{2}------------------------------------------------

For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely.

YANNA S. KANG -S

Yanna Kang, Ph.D. Assistant Director Mammography and Ultrasound Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

{3}------------------------------------------------

Indications for Use

510(k) Number (if known) K242130

Device Name Koios DS

Indications for Use (Describe)

Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an AI/MLderived cancer risk assessment and selects applicable lexicon-based to improve overall diagnostic accuracy as well as reduce interpreting physician variability.

Limitations:

· Patient management decisions should not be made solely on the results of the Koios DS analysis.

· Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them.

· Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images.

· The software does not predict the presence of the thyroid nodule margin descriptor, extra-thyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list.

Type of Use (Select one or both, as applicable)
-------------------------------------------------	--

X Prescription Use (Part 21 CFR 801 Subpart D)

| | Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{4}------------------------------------------------

Image /page/4/Picture/1 description: The image shows the logo for Koios. The logo consists of a stylized owl head on the left and the word "koios" in lowercase letters on the right. The owl head is dark gray with light blue eyes, and the word "koios" is light blue.

510(k) Summary of Safety and Effectiveness

This 510(k) summary of safety and effectiveness information is submitted as part of the Premarket Notification in accordance with the requirements of 21 CFR Part 807, Subpart E and Section 807.92.

1. Identification of Submitter:

Submitter:	Koios Medical Inc.
Address:	242 West 38th Street, 14th FloorNew York, NY 10018
Phone:	732-529-5755
Fax:	732-529-5757
Contact:	Michael Bocchinfuso
Title:	Director of Regulatory Compliance and Quality
Phone:	732-529-5755
Fax:	732-529-5757
Summary Date:	October 18, 2024

2. Identification of Product:

Device Name:	Koios DSVersion 3.6
Device Common Name:Device Classification:	Radiological Computer-Assisted Diagnostic Software21 CFR 892.2060, Class II, POK (primary)21 CFR 892.2050, Class II, QIH (secondary)
Classification Name:	Radiological Computer-Assisted Diagnostic Software (CADx) forLesions Suspicious for Cancer
Manufacturer:	Koios Medical, Inc.

3. Marketed Devices

In terms of safety and performance, this software medical device is substantially equivalent to the devices listed below:

Predicate device:	Koios DS
Manufacturer:	Koios Medical, Inc.

510(k) Summary

{5}------------------------------------------------

510(k) Number: K212616

4. Device Description

Breast Functionality:

Koios DS software automatically classifies breast lesions suspicious for cancer based on image data into one of four ACR BI-RADS® Atlas2 or European U1-U5 Classification System-aligned categories (Benign, Probably Benign, Suspicious or Indeterminate, or Probably Malignant) and also displays a continuous graphical Confidence Level Indicator depicting where the lesion falls within its respective category and its relation to neighboring categories. The software automatically classifies the shape (Round, Oval, Irregular) and orientation (Parallel) of the selected lesion.

Thyroid Functionality:

Koios DS is a software medical device used to analyze ultrasound data to classify user-selected regions containing thyroid nodules suspicious for cancer. The software generates a set of usereditable sonographic nodule descriptor recommendations (Composition, Echogenicity, Shape, Margin, Echogenic Foci) along with an optional, deep-learning derived cancer risk assessment of the suspected nodule from two orthogonal views. Nodule descriptor recommendations are subsequently mapped to a categorical assessment and risk level rating via the ACR TI-RADS™ ATLAS or American Thyroid Association (ATA) risk stratification systems (RSSs) based on user preference. The software's direct, non-descriptor-based cancer risk assessment is presented as the Koios "AI Adapter" that, when used in conjunction with the ACR TI-RADS or ATA guidelines for nodule risk stratification, is shown to improve overall diagnostic performance of both systems. The Al Adapter operates as an optional lexicon-specific input used to modify the final categorization in the ACR TI-RADS and ATA RSSs. The AI adapter positively impacts performance through either a point-based modification (either positive or negative) or a risk-shift modification (either positive or negative) for ACR TI-RADS and the ATA systems, respectively.

{6}------------------------------------------------

This process creates an Al-augmented categorization that is meant to be used with no other modifications to the decision-making pathway of either RSS. A trained interpreting physician may choose to incorporate or exclude the Koios AI Adapter from the overall assessment when finalizing their diagnostic interpretation.

Koios DS enables the following functionality:

Breast and Thyroid Diagnostic Core AI Engines enabled by state-of-the-art computer . vision and machine learning techniques capable of reading, interpreting, analyzing, classifying and generating findings from ultrasound image data resulting in an automated risk assessment for breast lesions and thyroid nodules suspicious for cancer.
. Automatic classification of thyroid nodule TI-RADS and ATA Descriptors of: Composition, Echogenicity, Shape, Margin, and Echogenic Foci based on user-selected regions of interest (ROIs).
. Automatic classification of breast lesion BI-RADS and U1-U5 Descriptors Shape and Orientation based on user-selected or confirmed regions of interest (ROIs).
. Annotation and description of ultrasound images based on ACR BI-RADS Breast Imaging Atlas and U1-U5 for Koios DS Breast and ACR TI-RADS Atlas for thyroid lexicon classification forms and ATA classification guidelines for Koios DS Thyroid.
. Reporting forms for breast lesion or thyroid nodule identification and tracking in the Electronic Health Record.
. Smart Calipers - extraction of user-supplied ROI data (alternately referred to as Calipers) embedded in DICOM SR files from the ultrasound modality.
. Smart Click - for streamlining the manual ROI selection process. The Smart Click functionality enables the user to click on the center of a lesion in order to activate a system-generated region of interest surrounding the selected lesion for the user.
. Image Registration and Matching - allows users to select images and regions of interest through their own image viewers when interacting with Koios DS Breast and Koios DS Thyroid, and facilitates a flexible viewer agnostic workflow. When the Image Matching Engine is given a screenshot of a medical image with coordinates for a region of interest, it identifies the original full quality image and translates the coordinates to its frame of reference.
. Automatic Size and Position population using Optical Character Recognition (OCR) - the Koios DS Optical Character Recognition engine uses machine learning and rulebased methods to create a system which is capable of retrieving fast, accurate transcriptions of the text overlaid on ultrasound images. Given an ultrasound image that has been annotated by a radiologist or technician, the OCR function identifies all text in the image and extracts relevant information to the documentation of lesions or nodules.

{7}------------------------------------------------

This allows users to quickly interpret and transcribe the locations and measurements of ultrasound findings.

. Remote analysis interface to generate and view results within compatible software (e.g. ultrasound equipment or PACS workstation software).
Installer and Configuration Wizard.
Single Sign-on (SSO) Windows and LDAP Authentication. ●
Operating system and platform-agnostic usage.
. Zero-footprint web-based HTML5 DICOM image viewer with image manipulation and annotation tools.
Ability to save findings to PACS.
Ability to export findings to reporting software. ●

User Profile:

Koios DS is for use by trained professionals only. Koios DS is not for use by patients. Users must have appropriate medical professional competence, such as trained sonographers and interpreting physicians.

Use Environment:

Koios DS is a software application for use within the healthcare setting (in a clinic or hospital) for the examination and assessment of breast lesions or thyroid nodules using ultrasound. It is a platform-agnostic web application that queries and accepts DICOM compliant digital medical files from any compliant device subject to the specified DICOM Conformance Statement for Koios DS. Processing of the image(s) occurs in conjunction with a trained interpreting physician's typical diagnostic case read. The output of the system is a digital display to be used as a concurrent read and report input that may be added as an addendum to the DICOM series selected for processing or exported directly into a patient's draft report.

Operating Principle:

Koios DS is an ASP.NET web application deployed to a web server inside a Windows operating system environment accessed by a user through a compatible client. The application provides image-derived data via web triggering and remote analysis.

Once logged in and granted access to the Koios DS application, the user examines selected breast and thyroid ultrasound DICOM images. For breast functionality, the user selects or confirms up to two ROIs, from up to two orthogonal views that represent a single breast lesion for processing by the system. For thyroid functionality, two ROIs are required for processing by the system. The first ROI must be drawn on the transverse view, with the second on the

{8}------------------------------------------------

longitudinal view of the nodule. For breast functionality, bench testing has verified a single ROI does not significantly decrease system AUC performance. The ROI(s) are transmitted electronically to the Koios DS server by the Koios DS Breast or Thyroid software for image processing and the results are returned to the user for review in the respective interface. Images and data can be stored, communicated, processed, and displayed within the system and/or across computer networks at distributed locations.

The Koios DS Client is an optional workflow enhancement tool installed as a desktop application on the user workstation that enables a user to draw ROIs natively within their image viewing software. The Koios DS Client captures a screenshot of the ROI selected by the user instead of being directly drawn on and captured with DICOM data. The ROI screenshot is transmitted electronically to the Image Matching Engine within the Koios DS Server. The Image Matching Engine processes the ROI screenshot and data, identifying and matching the correct DICOM image, and overlaying the ROI on that image. Once matched, the ROIs are returned to the user for review in the Koios DS Breast or Thyroid interface.

The software does not require any specialized hardware to return a diagnostic output, but the time to process ROIs will vary depending on the hardware specifications.

Koios DS contains two distinct Al/ML engines to characterize breast lesions and thyroid nodules. Based on the structured data that exists within the DICOM header for a patient study, the Koios DS system calls the corresponding engine for analysis of the identified lesion or nodule. Each system uses computer vision and machine learning techniques embedded within an engine capable of reading, interpreting, and generating findings from ultrasound data. The underlying Breast and Thyroid engines draw upon knowledge learned from a large database of known cases, tying image features to their eventual diagnosis, to form a predictive model.

Koios DS results can be saved or transferred in three separate ways: in-transmission, saving to Picture Archiving and Communication System (PACS), and exporting results to thirdparty reporting software. In-transit transmission may be utilized when users wish to share analyses across viewing workstations. Results can be stored in in-transit memory for a preset period of time defined by a system administrator. After that preset period of time, all results are wiped from the local memory. Another method of saving is storing a report in the patient study on the PACS. After single or multiple lesion or nodule analyses have been performed and ultimately accepted by a trained interpreting physician, Koios DS can export a summary report to PACS as an addendum to the DICOM study that was selected for processing. This report serves as future reference and aid in the comparison of cases requiring follow up. This functionality is strictly reserved for approved users and must be configured by a site administrator.

{9}------------------------------------------------

Koios DS also supports exporting results to third-party reporting software to facilitate the reporting process. Saving or exporting preferences can be configured by the system administrator and user.

5. Indications for Use

Koios DS allows the user to select or confirm regions of interest (ROIs) within an image representing a single lesion or nodule to be analyzed. The software then automatically characterizes the selected image data to generate an Al/ML-derived cancer risk assessment and selects applicable lexicon-based descriptors designed to improve overall diagnostic accuracy as well as reduce interpreting physician variability.

Limitations:

• Patient management decisions should not be made solely on the results of the Koios DS analysis.

Koios DS software is not to be used for the evaluation of normal tissue, on sites of postsurgical excision, or images with doppler, elastography, or other overlays present in them.
Koios DS software is not intended for use on portable handheld devices (e.g. smartphones or tablets) or as a primary diagnostic viewer of mammography images.

{10}------------------------------------------------

• The software does not predict the presence of the thyroid nodule margin descriptor, extrathyroidal extension. In the event that this condition is present, the user may select this category manually from the margin descriptor list.

{11}------------------------------------------------

6. Substantial Equivalence Chart

Product	Koios DS 3.0(K212616)	Koios DS 3.6(subject device)
PhysicalCharacteristics	Software PackageOperates on off-the-shelf hardware	Software PackageOperates on off-the-shelf hardware
Storage	Storage not supported	Storage not supported
Image Input	DICOM	DICOM
Characteristics	Decision support device used to assistin the assessment andcharacterization of breast lesions andthyroid nodules using US image data.	Decision support device used to assist inthe assessment and characterization ofbreast lesions and thyroid nodules using USimage data.
IntendedUse/Indicationsfor Use	Koios Decision Support (DS) is anartificial intelligence (AI)/machinelearning (ML)-based computer-aideddiagnosis (CADx) software deviceintended for use as an adjunct todiagnostic ultrasound examinations oflesions or nodules suspicious forbreast or thyroid cancer.	Koios Decision Support (DS) is an artificialintelligence (AI)/machine learning (ML)- based computer-aided diagnosis (CADx)software device intended for use as anadjunct to diagnostic ultrasoundexaminations of lesions or nodulessuspicious for breast or thyroid cancer.Koios DS allows the user to select orconfirm regions of interest (ROIs) within animage representing a single lesion ornodule to be analyzed. The software thenautomatically characterizes the selectedimage data to generate an AI/ML-derivedcancer risk assessment and selectsapplicable lexicon-based descriptorsdesigned to improve overall diagnosticaccuracy as well as reduce interpretingphysician variability.Koios DS software may also be used as animage viewer of multi-modality digitalimages, including ultrasound andmammography. The software includestools that allow users to adjust, measure

	includes tools that allow users to adjust, measure and document images, and output into a structured report.	structured report.Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer. When utilized by an interpreting physician who has completed the prescribed training, this device provides information that may be useful in recommending appropriate clinical management.
Target Population(subset of abovefor comparisonpurposes)	Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer.	Koios DS software is designed to assist trained interpreting physicians in analyzing the breast ultrasound images of adult (>= 22 years) female patients with soft tissue breast lesions and/or thyroid ultrasounds of all adult (>= 22 years) patients with thyroid nodules suspicious for cancer.
Limitations for Use(subset of abovefor comparisonpurposes)	Limitations:• Patient management decisions should not be made solely on the results of the Koios DS analysis.• Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them.	Limitations:• Patient management decisions should not be made solely on the results of the Koios DS analysis.• Koios DS software is not to be used for the evaluation of normal tissue, on sites of post-surgical excision, or images with doppler, elastography, or other overlays present in them.

	for use on portable handheld devices	on portable handheld devices (e.g.
	(e.g. smartphones or tablets) or as a	smartphones or tablets) or as a primary
	primary diagnostic viewer of	diagnostic viewer of mammography
	mammography images.	images.
	• The software does not predict the	• The software does not predict the
	presence of the thyroid nodule	presence of the thyroid nodule margin
	margin descriptor, extra-thyroidal	descriptor, extra-thyroidal extension. In
	extension. In the event that this	the event that this condition is present, the
	condition is present, the user may	user may select this category manually
	select this category manually from the	from the margin descriptor list.
	margin descriptor list.
Modality Used for	Breast Ultrasound Data	Breast Ultrasound Data
Analysis	Thyroid Ultrasound Data	Thyroid Ultrasound Data
Input	Medical images provided in a DICOM	Medical images provided in a DICOM
	format	format
ROI	Breast	Breast
Requirements	The software requires a user to select	The software requires a user to select up to
	up to two ROIs, from up to two	two ROIs, from up to two orthogonal
	orthogonal views, that represent a	views, that represent a single lesion to be
	single lesion to be selected and	selected and processed.
	processed.
		Thyroid
	Thyroid	Two ROIs that represent a single lesion to
	Two ROIs that represent a single	be selected and processed are required for
	lesion to be selected and processed	analysis.
	are required for analysis.
		The first ROI is drawn on the transverse
	The first ROI is drawn on the	view of the nodule. The second is drawn on
	transverse view of the nodule. The	the longitudinal view.
	second is drawn on the longitudinal
	view.
Output (Breast)	Koios defined categorical and	Koios defined categorical and continuous
	continuous outputs (confidence level	outputs (confidence level indicator) that
	indicator) that align to BI-RADS, U1-	align to BI-RADS, U1-U5, and auto-
	U5, and auto-classified shape and	classified shape and orientation.
	orientation.
Output (Thyroid)	Koios DS software automatically	Koios DS software automatically classifies
	classifies thyroid nodules suspicious	thyroid nodules suspicious for cancer

	for cancer based on image datagenerating an output aligned to eitherthe TI-RADS or ATA classificationguidelines. The system automaticallygenerates user-modifiable noduledescriptors (Composition,Echogenicity, Shape, Margin,Echogenic Foci) and a direct, image-derived cancer risk assessment that istranslated into an optional lexicon-specific modifier.	based on image data generating an outputaligned to either the TI-RADS or ATAclassification guidelines. The systemautomatically generates user-modifiablenodule descriptors (Composition,Echogenicity, Shape, Margin, EchogenicFoci) and a direct, image-derived cancerrisk assessment that is translated into anoptional lexicon-specific modifier.
ComparativeClinicalPerformanceTesting (Breast)	Metric: AUCCases: 900Readers: 15	Metric: AUCCases: 900Readers: 15
ComparativeClinicalPerformanceTesting (Thyroid)	Metric: AUCCases: 650Readers: 15	Metric: AUCCases: 650Readers: 15

{12}------------------------------------------------

{13}------------------------------------------------

{14}------------------------------------------------

{15}------------------------------------------------

7. Description of Similarities and/or Differences

Intended Use/Indications for Use (IFU)

The IFU of the subject and predicate devices are the same.

Target Patient Population

Both the Koios DS predicate and the subject device are software applications designed to assist trained interpreting physicians in analyzing the ultrasound images of patients with soft tissue lesions who are being referred for further diagnostic ultrasound examination.

Technological Characteristics

Modality

Koios DS shares the ultrasound modality requirements of Koios DS (K212616)

Technological Characteristics Comparison Table

	Koios DS 3.0 K212616	Koios DS 3.6 K242130
Diagnostic Engine	Breast (v 1.1.0)Thyroid (v2.2.0)	Breast (v. 3.0.0)Thyroid (v. 2.2.0)
Workflow Enhancements	Breast Smart Click	Breast Smart Click
	Breast Smart Calipers	Breast Smart Calipers
	Thyroid Smart Calipers	Thyroid Smart Click
		Thyroid Smart Calipers
		Image Registration andMatching

{16}------------------------------------------------

		OCR Automatic Size andPosition Population

Input

Per the respective device descriptions of Koios DS 3.6 and Koios DS (K212616), the following technical characteristics are included in the Koios DS 3.6 (K242130) version:

Thyroid Smart Click

If using this feature, the user can specify a Region of Interest with a single click near the center of the nodule. The Thyroid Smart Click Engine, based on this click, estimates the coordinates of the intended ROI in order to present these to the user. The Thyroid Smart Click engine produces an ROI which best matches the user's Smart Click selection. The resulting ROI contains coordinates for the region. The user can adjust or overrule the resulting ROI if they deem it necessary, and the ROI can then be used as an input to the thyroid diagnostic engine.

Image Registration and Matching

If using this optional feature, the user draws ROIs on images within their image viewing software, which generates a screenshot. The Koios DS Client transmits the ROI screenshot to the Koios DS Server Image Matching Engine to generate the ROI on the underlying DICOM image and displays it to the user for review. The ROI and associated DICOM data are sent to the Koios DS Server for processing by the appropriate Diagnostic Engine.

The screenshots (or image region replicas) utilized during an image matching query are not saved. Image region replicas are captured by the software and stored as uncompressed bitmap data in memory and processed by the Image Matching engine as such. This process is agnostic to the underlying body part and functions generically for any breast and/or thyroid image selection.

The image matching engine performs its search independently for each query region based purely and solely on image data, without external information pertaining to modality or other orthogonal views. The search candidates are limited in scope to database images from within the same study as the search query. As a result, if a search query is created from a thyroid image

{17}------------------------------------------------

from a specified thyroid study, only that specific thyroid study is searched for potential matching image regions. The same search query process is utilized for breast images; only the specific breast study is searched for matching image regions.

Auto-populate Size and Position using Optical Character Recognition (OCR)

If using this feature, the user provides an ultrasound image which has previously been annotated by a radiologist or technician. The OCR engine then identifies all the image and parses out text which is relevant to the study of lesions or nodules. This functionality allows the user to quickly interpret and transcribe the locations and measurements of ultrasound findings.

Output

When comparing breast functionality, the subject Koios DS Breast Engine demonstrates a significant categorical output performance increase in AUC (1.6%), a significant increase in sensitivity (0.6%), and a significant increase (2.2%) in specificity. Koios DS retains the identical descriptor outputs and performance for the assessment of shape and orientation.

The Koios DS thyroid performance is unchanged from the predicate device including when accounting for new technological characteristics. Performance testing outlined in the next section demonstrates equivalence and non-inferiority for each new technical characteristic.

Auto-populate Size and Position using Optical Character Recognition (OCR)

The output of the OCR Engine consists of two categories - measurements and text. The measurements output consists of three elements: description (e.g. the length, height, or width of a nodule), value (a field representing the numerical value of measurement), and unit (the modifier to the value specifying a unit of measurement, e.g. cm or mm). The text output consists of two elements: a dictionary (with fields corresponding to each possible category of information present in the findings text), and values (corresponding to the predicted output for each field).

8. Performance Testing - Standalone Testing

Breast Engine

To compare the performance of the subject device to the predicate device (K212616, product codes POK, QIH), the subject device's updated breast classification engine was compared to the

{18}------------------------------------------------

predicate device on the same 900 case validation set. Bench testing was performed on the Koios DS 3.6 breast engine to ascertain the degree of concordance with trained interpreting physicians. Ground truth for malignancy risk classification was determined by pathology or 1year follow-up for cases that were not biopsied. The system was analyzed on 900 lesions from 900 different patients set aside from the system's training data for the purpose of validating performance. Each lesion was represented by two orthogonal images (e.g. radial and antiradial), providing a total of 1800 images. An expanded validation set of 1014, including these 900 and an additional 114 cases is used to test for dataset drift. System performance on the 900 cases reported an AUC of 94.5%, with a Sensitivity of 0.976 [0.960, 0.992] and a Specificity of 0.632 [0.588, 0.676].

The table below provides a detailed evaluation of the breast diagnostic engine across key performance metrics. Direct comparison with the Koios DS v3.0 breast engine's performance determined there is a significant increase in AUC (1.6%), a significant increase in sensitivity (0.6%), and a significant increase (2.2%) in specificity.

{19}------------------------------------------------

Koios DS Engine (Breast) Test	Engine Version 1.1.0 (Previous)	Engine Version 3.0.0(Current)
1: Malignancy Risk Classifier AUC	0.929 [0.913, 0.945]	0.945 [0.932, 0.959]
2: Categorical Output
Sensitivity	0.97 [0.96, 0.99]	0.976 [0.960, 0.992]
Specificity	0.61 [0.57, 0.66]	0.632 [0.588, 0.676]
3: Sensitivity to Region of Interest	0.019	0.012
4. Sensitivity to TransducerFrequency	High frequency (>=15MHz),AUC = 0.940 [0.907, 0.974]Low frequency (<15MHz),AUC = 0.924 [0.904, 0.944]	High frequency (>=15MHz),AUC = 0.948 [0.917, 0.978]Low frequency (<15MHz),AUC = 0.940 [0.925, 0.956]
5. Single Image vs OrthogonalImage Pair	Single Image: 0.914 [0.910 -0.918]Orthogonal Pair: 0.929 [0.913,0.945]	Single Image: 0.932 [+/- 0.003]
6. Assessment of CategoricalAgreement - Shape(prior results continue to apply)	0.738 [0.679, 0.797]
7. Assessment of CategoricalAgreement - Orientation(prior results continue to apply)	0.744 [0.675, 0.813]
8. Operating Point
	PLR:System= 2.52 [2.26, 2.79]NLR:System=0.04 [0.02, 0.07]PPV:System= 0.70 [0.67, 0.73]NPV:System= 0.96 [0.94, 0.98]
	PLR:System= 2.661 [2.338, 2.984]NLR:System= 0.039 [0.013, 0.064]PPV:System= 0.708 [0.672, 0.743]NPV:System= 0.966 [0.944, 0.988]
9. Data Set Drift Analysis -Malignancy Risk Classifier AUC
	ROCAUC = 0.930 [0.914, 0.946]
	ROCAUC = 0.949 [0.936, 0.962]
10. Data Set Drift Analysis -Categorical Output	Sensitivity = 0.977 [0.967, 0.987]	Sensitivity = 0.973 [0.958, 0.989]
	Specificity = 0.615 [0.578, 0.651]	Specificity = 0.659 [0.619, 0.700]

510(k) Summary

{20}------------------------------------------------

{21}------------------------------------------------

{22}------------------------------------------------

{23}------------------------------------------------

Bench testing demonstrates that the system exceeds physician performance measured by AUC, sensitivity, and specificity. The engine's shape and orientation predictions have not been modified from the previously cleared device (which demonstrated the required level of agreement with the subjective categorizations assigned by physicians). Testing characterizes the system's sensitivity to shifts in the selected region of interests (ROI) and transducer frequency. Testing characterizes the system's Positive Value (PPV), Negative Predictive Value (NPV), Positive Likelihood Ratio (PLR) and Negative Likelihood Ratio (NLR) in comparison with physicians. Testing demonstrates that the performance of the engine does not demonstrate degradation when regions of interest are provided by the Smart Caliper system, as compared to manually drawn regions of interest. Therefore, the diagnosis engine is agnostic to the source of input ROI (Smart Click, Smart Calipers, physician drawn) and robust to shifts in ROI. In all tests, the Breast engine met or exceeded performance requirements.

Thyroid Engine

Bench testing was performed on the thyroid engine to ascertain the degree of concordance with trained interpreting physicians utilizing both the ACR TI-RADS and ATA classification systems. Ground truth for malignancy risk classification was determined by pathology results only. The system was analyzed on 500 lesions from 500 different patients set aside from the system's training data for the purpose of validating performance. Each lesion was represented by two orthogonal images (e.g. radial and anti-radial), providing a total of 1000 images.

When applied to diagnoses made using ACR TI-RADS guidelines, the Al Adapter and descriptor predictors achieved an AUC of 79.8%, demonstrating a significant increase over the average physician AUC. When recommending biopsy, the system's sensitivity is 0.644 [0.545, 0.744] and specificity is 0.612 [0.566, 0.658]. When recommending follow-up, the system's sensitivity and specificity are 0.879 [0.812, 0.946] and 0.495 [0.446, 0.544], respectively. In both scenarios, bench testing of the system demonstrates a non-significant improvement in sensitivity and a significant improvement in specificity over the physician average.

Tests demonstrating Al Adapter impact on ATA classifications yielded similarly improved performance. With application of the Al Adapter, physician AUC demonstrates a significant increase of 9.135% [5.975, 12.294]. Sensitivity shows a non-significant increase of 0.511% [-5.182, 6.204], while specificity shows a significant increase of 18.741% [9.885, 27.596].

{24}------------------------------------------------

Bench testing included verification of standalone performance with TI-RADS and ATA outputs, as well as performance when compared to a separate data set including data from independent sites (separate and apart from the sites/data used to train and tune the algorithm).

Testing demonstrates that application of the Koios DS Al Adapter exceeds physician performance as measured by AUC, sensitivity, and specificity. Descriptor predictions were tested objectively – against ground truth pathology. Testing demonstrated that performance requirements were met under ACR TI-RADS and ATA reporting systems as well as when compared against independent site data. Outputs were additionally tested subjectively and met the requirements for agreement with readers' descriptor categorizations. Testing characterized the sensitivity of the system with respect to shifts in the region of interest and variation in performance between high and low transducer frequencies. System performance on data acquired from independent sites meets performance requirements. In all tests, the Thyroid engine met or exceeded performance requirements.

Thyroid Smart Click

A dataset of 650 nodules with corresponding physician-drawn ROI's were used for testing the Koios DS Thyroid Smart Click Engine. These ROI's are the reference ROI's used to validate the Koios DS Thyroid Engine. A user "click" will be simulated for testing the performance of the Smart Click engine by calculating the center of each nodule.

Non-inferiority testing is used to demonstrate that the use of the Smart Click engine does not degrade diagnostic performance when compared to physician-selected calipers. Each test will demonstrate that the lower 95% confidence bound of the difference in performance falls above a designated equivalence margin, delta (δ). In each case, delta is defined using the measured uncertainty in our performance metric and the variability observed on physicians' assessment of the data. Measurement uncertainty is computed via bootstrapping.

Testing further provides quantitative metrics which demonstrate how closely the automated Smart Click ROIs match the manual physician-drawn ROI's using standard segmentation metrics. In this case, the Dice Similarity Coefficient will be used. Specifically, the test will measure and report the average DICE score between the Smart Click and physician ROY's across all of the images present in the validation set. The objective of this test is to demonstrate the similarity between Smart Click ROI's and manually drawn ROI's.

Finally, testing demonstrates concretely that descriptors generated from Smart Click ROI's are not impacted by differences between them and manually drawn ROI's. Non-inferiority testing was used on a per-descriptor basis. Specifically, testing will show that the rate of agreement between the system's and the physicians' descriptors when utilizing smart click is non-inferior to the system's descriptors when using the manual ROI's. The Cohen's Kappa metric will be used to characterize agreement between system and reader in either case.

Superiority is evaluated using similar methodology. In this case, the lower bound of the difference in performance must fall above zero.

{25}------------------------------------------------

Koios DS Smart Click Engine Test	Smart Click Engine Version (TOR.2.0.X)
1: Non-inferiority Test - Sensitivity / Specificity	Sensitivity:Difference = -0.009 [-0.036, 0.018]Result: Non-inferior Specificity:Difference = -0.018 [-0.041, 0.005]Result: Non-inferior
2: Non-inferiority Test - AUC	Difference = -0.012 [-0.029, 0.006]Result: Non-inferior
3: Sub-optimal ROI Test	Difference = 0.026 [-0.009, 0.062]Result: Non-inferior
4: Detection DICE Coefficient	DICE= 0.913 +/- 0.075
5: Non-inferiority Test - Descriptor Agreement	Composition:Difference = 0.018 [0.001, 0.035]Result: Non-inferiorEchogenicity:Difference = -0.005 [-0.022, 0.011]Result: Non-inferior
	Shape:
	Difference = -0.033 [-0.093, 0.027]
	Result: Non-inferior
	Margin:
	Difference = -0.013 [-0.032, 0.007]
	Result: Non-inferior
	Echogenic Foci:
	- Large Comet-Tail Artifacts
	Difference = -0.019 [-0.048, 0.010]
	Result: Non-inferior
	- Macrocalcifications
	Difference = 0.017 [-0.035, 0.068]
	Result: Non-inferior
	- Peripheral (Rim) Calcifications
	Difference = 0.024 [-0.072, 0.120]
	Result: Non-inferior
	- Punctate Echogenic Foci
	Difference = 0.007 [-0.038, 0.053]
	Result: Non-inferior

{26}------------------------------------------------

Testing demonstrates that system performance does not fall below the computed value for the equivalence margin, delta. The confidence interval of the difference falls within the expected bounds.

The high value (DICE = 0.913 +/- 0.075) of the DICE coefficient demonstrates that Smart Click ROls are, on average, a precise approximation to the ROls that a physician would select. Test 5 contains a quantitative perdescriptor comparison of the predictions generated when using Smart Click ROIs to those generated using physician ROIs, via Non-Inferiority testing. Noninferiority was demonstrated for each descriptor using Smart

{27}------------------------------------------------

Click ROls. Together, these results clearly demonstrate descriptor performance is not negatively impacted by the use of the Smart Click engine.

Image Registration and Matching

A dataset consisting of 1,600 ultrasound studies of lesions in both breast (950 cases) and thyroid (650 cases) was used to evaluate the performance of the Koios DS Client ROI Match Engine in Tests 1-4 outlined below. Each study consists of one or more ultrasound images, wherein one or more contain a region of interest (RO)) that signifies the location of a lesion within the image.

Tests 5 & 6 utilized the breast validation dataset of 1014 cases. Tests 7 & 8 utilized the thyroid validation dataset of 650 cases.

Tests 5 & 7 measure the rate of incidence of each of the possible outcomes of the matching process.

Successful Match: The system correctly identifies the image region corresponding to the query screenshot.
. No Match: The system is unable to identify a matching region of interest and returns no match for the query screenshot. Note: This is considered a positive outcome. If the system identifies that it cannot match a query screenshot, it should return that status rather than an incorrect result.
Incorrect Match: The system identifies the correct image but returns an incorrect region within that image. Incorrect is defined as under 0.5 intersect-over-union with respect to the correct region.
Incorrect Image: The system selects the wrong image as a match for the query screenshot.

These tests were run on each of the ROI's contained in the test dataset, which corresponds to 2028 breast ROI's and 1288 thyroid ROI's.

These tests are to tabulate how often the image matching process results in each of the four possible outcomes. This is reported both by total count and percent incidence.

Tests 6 & 8 investigate the quality of the resulting registration. This is measured by computing the average DICE coefficient for the set of matches which are in the successful match category.

A summary of performance statistics is outlined below.

Koios DS Image Registration and MatchingEngine Test	Image Registration and Matching Engine Version(LAM.1.X.X)
1: No Match Rate	No Match Rate = 0.32%
2: Match Time	Average Time for Study Preprocessing: 2.39 +/- 0.48secondsAverage Time for Image Matching: 0.22 +/- 0.12secondsNote: The average study size which was processed inthis experiment was 53 +/- 11 images. The measureddurations scale roughly linearly with study size.
3: End-to-End Breast Engine Performance	AUC = 0.946Sensitivity = 0.975Specificity = 0.637
4: End-to-End Thyroid Engine Performance	AUC = 0.801Sensitivity = 0.670Specificity = 0.603
5: Breast Image Matching Outcomes	Successful Match:Count: 2018Fraction: 0.995No Match:Count: 10Fraction: 0.005Incorrect Match:
	Count: 0Fraction: 0.000Incorrect Image:Count: 0Fraction: 0.000
6: Breast Image Matching DICE Coefficient	DICE = 0.995 =/- 0.005
7: Thyroid Image Matching Outcomes	Successful Match:Count: 1288Fraction: 1.000No Match:Count: 0Fraction: 0.000Incorrect Match:Count: 0Fraction: 0.000Incorrect Image:Count: 0Fraction: 0.000
8: Thyroid Image Matching DICE Coefficient	DICE = 0.996 =/- 0.004

{28}------------------------------------------------

{29}------------------------------------------------

{30}------------------------------------------------

Auto-populate Size and Position using Optical Character Recognition (OCR)

A dataset of 1910 ultrasound B-Scans was manually annotated to test the OCR engine. These are a mix of thyroid and breast images that come from a variety of machines. Of these, a subset of 1226 images that come from the supported list of machines is used to carry out this test.

Tests will measure accuracy (percent correct) for each of the structured fields predicted by the engine. False positive, false negative, and misread text fields will count against the accuracy measurement of the engine.

Koios DS OCR Engine Test	Optical Character Recognition (OCR) Engine Version(GNO.1.1.X)
1: Breast Freetext Identification	Breast Side: 0.983Location Type: 0.948Clock Hour: 0.926Clock Minute: 0.934CMFN: 0.944Plane: 0.976
2: Thyroid Freetext Identification	Thyroid Side: 0.965Pole: 0.976Region: 0.998Plane: 0.970
3: Measurement Text Identification	Measurement Description: 0.943Measurement Value: 0.948

{31}------------------------------------------------

Unit of Measurement:	0.967
----------------------	-------

In conclusion, the subject device has demonstrated substantially equivalent performance to the predicate by showing statistically significant results against similar success criteria in bench testing comparisons.

9. Performance Testing - Clinical

Breast

A clinical study was previously executed to determine the effect of Koios DS Breast (K190442) on reader performance. There were no new clinical studies performed for the design and development of Koios DS 3.0 (K212616), which used Koios DS Breast (K190442) as a predicate device during its regulatory submission. As discussed in the prior section, the performance of Koios DS 3.0 has been met or significantly improved across all measured metrics by Koios DS 3.6. This data continues to apply to the breast functionality within the subject device, with the understanding that its performance is superior, and it would therefore provide an equivalent or greater benefit. The below summary of the clinical study data has been included for ease of reference.

The study objective was to determine the impact on Interpreting Physician (Reader) performance as defined by the area under the Receiver Operating Characteristic (ROC) Curve (AUC) when Koios DS Breast and an ultrasound examination are combined (USE + DS), compared to USE Alone in patients that present with a soft tissue breast lesion through any form of imaging or physical examination and are referred for diagnostic ultrasound.

The study consisted of 15 readers with varying levels of training and experience providing analysis on a randomized set of 900 patient cases presented with USE + DS and USE Alone in two reading periods separated by a 1-month wash-out, totaling 1800 cases analyzed per reader. The reader set and dataset were distributed in accordance with FDA guidance and are explained in detail below:

{32}------------------------------------------------

Reader Background

ReaderID	BoardCertification/Specialty	BreastFellowshipTrainedand/orDedicatedBreast Imager	Years ofExperience -Mammographyand/or BreastUltrasound	AcademicInstitutionAffiliation(Yes/No)	MQSAQualifiedInterpretingPhysician
1	DiagnosticRadiology	No	13 years	No	Yes
2	DiagnosticRadiology	No	4 years	No	No
3	DiagnosticRadiology	Yes	7 years	Yes	Yes
4	BreastSurgeon	No	0 years	No	No
5	OB/GYN	No	20 years	No	No
6	DiagnosticRadiology	No	13 years	Yes	No
7	DiagnosticRadiology	No	3 years	Yes	No
8	OB/GYN	No	0 years	No	No
9	DiagnosticRadiology	Yes	15 years	No	Yes
10	DiagnosticRadiology	No	13 years	No	No
11	DiagnosticRadiology	Yes	30 years	No	Yes
12	DiagnosticRadiology	Yes	10 years	Yes	Yes
13	DiagnosticRadiology	No	0 years	No	No
14	InterventionalRadiology	No	4 years	No	No
15	BreastSurgeon	No	25 years	Yes	No

{33}------------------------------------------------

Dataset Demographic Information

The Koios DS Breast engine was tested on images sourced from a wide variety of ultrasound hardware and data with the following patient demographics to ensure the system performance is generalizable to and representative of diverse populations. Patient demographic distribution was based upon data from the Breast Cancer Surveillance Consortium (2006-2009)2.

Image /page/33/Figure/2 description: This bar chart shows the count of benign and cancer diagnoses. The x-axis represents the diagnosis type, with 'benign' and 'cancer' as the categories. The y-axis represents the count, ranging from 0 to 500. The 'benign' category has a count of 470 (52.2%), while the 'cancer' category has a count of 430 (47.8%).

The following figures represent the final validation dataset (900 cases):

Image /page/33/Figure/4 description: The image is a bar chart showing the distribution of BI-RADS categories. The x-axis represents the BI-RADS category, and the y-axis represents the count. The categories are 2, 3, 4A, 4B, 4C, and 5, with corresponding counts of 68 (7.6%), 173 (19.2%), 222 (24.7%), 153 (17.0%), 187 (20.8%), and 97 (10.8%) respectively. The highest count is in category 4A.

Distribution of Malignancy in Final Validation Set

Image /page/33/Figure/6 description: The image is a bar chart that shows the distribution of ethnicity. The x-axis shows the different ethnicities, including white, black, hispanic, asian, and other. The y-axis shows the count of each ethnicity. The bar chart shows that the majority of the population is white, with 592 people (65.8%), followed by asian with 133 people (14.8%).

Distribution of BI-RADS Category in Final Validation Set

Image /page/33/Figure/8 description: The image is a bar chart showing the distribution of ages. The x-axis represents age groups, including 'n/a', '<40', '40-49', '50-59', '60-74', '75-84', and '85+'. The y-axis represents the count. The bar chart indicates that the age group '60-74' has the highest count of 235 (26.1%), while the age group '85+' has the lowest count of 9 (1.0%).

Distribution of Ethnicity in Final Validation Set

Distribution of Age in Final Validation Set

4 Data were obtained from the Breast Cancer Surveillance Consortium, funded by the National Cancer Institute (HHSN261201100031C). From the Breast Cancer Surveillance Consortium website, http://www.bcsc-research.org/

{34}------------------------------------------------

Image /page/34/Figure/0 description: The image contains two bar charts. The first bar chart shows the distribution of tumor sizes, with the x-axis representing size in millimeters and the y-axis representing the count. The tumor sizes are grouped into bins: n/a (7, 0.8%), <10 (332, 36.9%), 10-14 (229, 25.4%), 15-19 (132, 14.7%), and 20+ (200, 22.2%). The second bar chart shows the distribution of invasive cancer types, with the x-axis representing invasive cancer and the y-axis representing the count: invasive (369, 85.8%) and dcis/nos (61, 14.2%).

Distribution of Lesion Size in Final Validation Set

Distribution of Invasive Cancer in Final Validation Set

Image /page/34/Figure/3 description: This bar graph shows the BI-RADS density. The x-axis shows the density, and the y-axis shows the count. The bar graph shows that 343 (38.1%) are scattered fibroglandular density, 299 (33.2%) are heterogeneously dense, 143 (15.9%) are extremely dense, and 115 (12.8%) are fatty.

Distribution of BI-RADS Density in Final Validation Set

{35}------------------------------------------------

Image /page/35/Figure/0 description: The image is a bar chart titled "Distribution of US Machines - Validation Set". The chart shows the count of different types of machines on the y-axis and the machine types on the x-axis. The most common machine type is "ge logiq e" with a count of 1334, followed by "philips hdi 5000" with a count of 282.

Distribution of US Machines in Final Validation Set

Per the primary endpoint of the study, ROC curves were generated and analyzed. All AUCs were computed via the trapezoidal approximation. Based on the standard error measurements, the error can be propagated to estimate the mean performance interface and 95% confidence interval. This was found to be 0.0370 (0.030, 0.044) at α = .05, satisfying the success criteria for the primary endpoint.

To characterize the effect of Koios DS (USE + DS) system on inter-operator variability, the Kendall Tau-B correlation coefficient was computed in a pairwise manner for all readers. The metric is > 0 for all reader pairs. The standard error for USE + DS and USE Alone was computed to assess if the shifts in the metric were significant. The average Kendall Tau-B of USE Alone was .5404 (.5301, .5507) and the average Kendall Tau-B of USE + DS was .6797 (.6653, .6941) with 95% Cl demonstrating a significant increase in the metric (α = .05).

Also assessed was the effect of Koios DS on intra-operator variability leveraging 150 reads that did not switch from USE Alone to USE + DS across the washout session in the reader study (75 each). USE Alone class switching rate was 13.6% and the USE + DS class switching rate was 10.8% (p = 0.042), demonstrating a statistically significant reduction in intra-reader variability when using USE + DS.

{36}------------------------------------------------

Thyroid

An observational case-controlled, Multi-Reader, Multi-Case (MRMC) retrospective clinical trial (CRRS-3) was executed to determine the effect of Koios DS Thyroid on reader performance.

Effect on performance was defined by measuring the area under the Receiver Operating Characteristic (ROC) Curve (AUC) when Koios DS and an ultrasound examination were combined (USE + DS), compared to unassisted TI-RADS based Reader performance (USE Alone). All data analysis cases consisted of USE Alone and USE + DS image readings in patients that presented with a thyroid abnormality through any form of imaging or physical examination and were referred for diagnostic ultrasound where a nodule was subsequently discovered.

Data analysis in the CRRS-3 study was based on 650 retrospectively collected cases that were assigned a TI-RADS Assessment Category 1 through 5 at the of initial review at study entry based upon the interpreting physician of the ultrasound evaluation. The study consisted of 15 readers reviewing and interpreting 650 cases twice (1300 total cases per reader). All data analysis was based on two randomized evaluations of each case with and without the assistance of Koios DS software with a 1-month washout period between corresponding presentations of the case and interpretations by physicians.

The study design called for a mixed population of physician readers (11/15 or 73% US based) and cases (500 or 77% US based) coming from both the US and Europe. Readers with a current medical license who met inclusion criteria and completed the study training protocol were considered trained interpreting physicians for study purposes. Readers possessed varying levels of training and experience, as detailed below:

Reader ID	Reader Category	Experience (post-residency)
R1	Domestic Endocrinologist (End)	< 10 years
R2	Domestic Radiologist (Rad)	≥ 20 years
R3	Domestic Rad	≥ 20 years
R4	Domestic Rad	≥ 10 and < 20 years
R5	Domestic Rad	≥ 10 and < 20 years
R6	Domestic Rad	≥ 10 and < 20 years
R7	Domestic Rad	≥ 20 years
R8	Domestic Rad	< 10 years
R9	Domestic Rad	≥ 20 years
R10	Domestic Rad	≥ 20 years
R11	Domestic End	< 10 years
R12	European Rad	≥ 20 years
R13	European Rad	≥ 20 years
R14	European End	≥ 20 years
R15	European End	≥ 20 years

Reader Experience

{37}------------------------------------------------

Dataset Demographic Information

The Koios DS thyroid engine was tested on images sourced from a wide variety of ultrasound hardware and data with the following patient demographics to ensure the system performance is generalizable to and representative of diverse populations.

The following ultrasound hardware represents the final validation dataset (650 cases).

Image /page/37/Figure/3 description: The image is a bar chart titled "Distribution of US Machines - Final Validation Set". The x-axis represents different machine types, and the y-axis represents the count. The bar chart shows the distribution of different machine types, with "siemens iu22" having the highest count of 256, followed by "siemens acuson" with 246 and "toshiba aplio mx" with 238.

Distribution of ultrasound machine models in the final validation set, by image

{38}------------------------------------------------

The final validation set data is divided into 2 subsets; 500 cases from United States locations and 150 cases from European locations. The following figures represent the United States patient demographics:

Image /page/38/Figure/1 description: The image is a bar chart titled "Diagnosis - Final Validation Set (US)". The x-axis is labeled "Diagnosis" and has two categories: "Benign" and "Malignant". The y-axis is labeled "Count" and ranges from 0 to 400. The bar chart shows that there are 400 (80.0%) Benign diagnoses and 100 (20.0%) Malignant diagnoses.

Image /page/38/Figure/2 description: The image shows the title of a document or presentation. The title is "Distribution of Malignancy in the Final Validation Set". The subtitle is "(United States)". The text is centered on the image.

Image /page/38/Figure/3 description: The image is a bar graph titled "Ethnicity - Final Validation Set (US)". The x-axis represents ethnicity, and the y-axis represents count. The bar graph shows the distribution of ethnicities, with 326 (65.2%) White, 62 (12.4%) Black or African American, and 61 (12.2%) Other.

Distribution of Patient Ethnicity in the Final Validation Set (United States)

Image /page/38/Figure/5 description: The image is a bar chart titled "Diagnostic Assessments - Final Validation Set (US)". The x-axis is labeled "TI-RADS" and has the categories TR1, TR2, TR3, TR4, and TR5. The y-axis is labeled "Count" and ranges from 0 to 175. The bar chart shows the counts for each category: TR1 has a count of 23 (4.6%), TR2 has a count of 48 (9.6%), TR3 has a count of 103 (20.6%), TR4 has a count of 186 (37.2%), and TR5 has a count of 140 (28.0%).

Distribution of TI-RADS Assessment in the Final Validation Set (United States)

Image /page/38/Figure/7 description: This bar chart titled "Sex - Final Validation Set (US)" shows the distribution of sex in the final validation set. The x-axis shows the sex, with two categories: Male and Female. The y-axis shows the count, with the number of males being 103 (20.6%) and the number of females being 394 (78.8%).

Distribution of Patient Sex in the Final Validation Set (United States)

{39}------------------------------------------------

Image /page/39/Figure/0 description: The image is a bar chart titled "Age - Final Validation Set (US)". The x-axis represents age groups, and the y-axis represents the count. The age groups are: <25, 25-29, 30-34, 35-39, 40-44, 45-49, 50-54, 55-59, 60-64, 65-69, 70-74, and 75+. The counts for each age group are: 16 (3.2%), 28 (5.6%), 24 (4.8%), 32 (6.4%), 44 (8.8%), 51 (10.2%), 79 (15.8%), 61 (12.2%), 63 (12.6%), 50 (10.0%), 22 (4.4%), and 27 (5.4%).

Distribution of Patient Age in the Final Validation Set (United States)

Image /page/39/Figure/2 description: The image is a bar chart titled "Age - Final Validation Set (EU)". The x-axis represents age groups, and the y-axis represents the count. The age groups are: <25, 25-29, 30-34, 35-39, 40-44, 45-49, 50-54, 55-59, 60-64, 65-69, 70-74, and 75+. The counts for each age group are: 5, 1, 7, 7, 11, 12, 18, 18, 18, 19, 17, and 17.

The following figures represent the European patient demographics:

Image /page/39/Figure/4 description: The image shows the title of a figure. The title is "Distribution of Patient Age in the Final Validation Set (European)". This title indicates that the figure will likely show the distribution of patient ages within a specific dataset used for validation, focusing on a European population.

Image /page/39/Figure/5 description: This bar chart titled "Sex - Final Validation Set (EU)" shows the distribution of sex in the final validation set. The x-axis shows the sex, and the y-axis shows the count. The bar chart shows that there are 29 males (19.3%) and 121 females (80.7%).

Distribution of Patient Sex in the Final Validation Set (European)

Koios Medical, Inc.

{40}------------------------------------------------

The primary CRRS-3 analysis was performed on the Readers' TI-RADS point total gradings from their review of the USE Alone and their review of the USE + DS for the Non-Cancer Case Set and Cancer Case Set. For each Reader, two ROC curves (Sensitivity vs. 1 – Specificity) were plotted using the USE Alone and the USE + DS primary analysis cases. Reader-specific AUC values for the primary analysis were derived from the trapezoidal approximation, whereas the mean AUC values and associated standard errors within- and between-modality across all Readers were derived from the DBM (Dorfman-Berbaum-Metz ANOVA after jackknife) method. This approach captures both reader variability and case variability and is the standard methodology for comparing AUCs in MRMC studies. All ROC curve analysis was done with respect to cyto-/histological or excisional pathology.

Analysis	Overview	Result
Primary Endpoint 1	Change in average AUC with Koios DS(all readers, all data)	+0.083 [0.066, 0.099] (parametric)+0.079 [0.062, 0.096] (non-parametric)
Primary Endpoint 2	Change in average AUC with Koios DS(US readers, US data)	+0.074 [0.051, 0.098] (parametric)+0.073 [0.049, 0.096] (non-parametric)
Secondary Analysis 1	Change in average Sensitivity andSpecificity of FNA with Koios DS(all readers, all data)	+ 0.084 [0.054, 0.113] (sensitivity)+ 0.140 [0.125, 0.155] (specificity)
	Change in average Sensitivity andSpecificity of FNA with Koios DS(US readers, US data)	+ 0.058 [0.017, 0.098] (sensitivity)+ 0.130 [0.110, 0.151] (specificity)
	Change in average Sensitivity andSpecificity of FNA with Koios DS(EU readers, EU data)	+0.125 [0.014, 0.237] (sensitivity)+0.171 [0.109, 0.233] (specificity)
Secondary Analysis 2 -- excluding casesrecommended forFNA	Change in average Sensitivity andSpecificity of Follow-up with Koios DS(all readers, all data)	+ 0.092 [0.043, 0.141] (sensitivity)+ 0.242 [0.220, 0.264] (specificity)
	Change in average Sensitivity andSpecificity of Follow-up with Koios DS(US readers, US data)	+ 0.087 [0.023, 0.151] (sensitivity)+ 0.206 [0.176, 0.235] (specificity)
	Change in average Sensitivity andSpecificity of Follow-up with Koios DS(EU readers, EU data)	+0.084 [-0.133, 0.300] (sensitivity)+0.350 [0.267, 0.434] (specificity)
Secondary Analysis 2a– including casesrecommended for	Change in average Sensitivity andSpecificity of Follow-up with Koios DS(all readers, all data)	+0.060 [0.040, 0.080] (sensitivity)+0.206 [0.192, 0.219] (specificity)
FNA	Change in average Sensitivity andSpecificity of Follow-up with Koios DS(US readers, US data)	+0.053 [0.026, 0.080] (sensitivity)+0.180 [0.161, 0.198] (specificity)
	Change in average Sensitivity andSpecificity of Follow-up with Koios DS(EU readers, EU data)	+0.060 [-0.009, 0.129] (sensitivity)+0.296 [0.238, 0.354] (specificity)
Secondary Analysis 3	Change in average AUC with Koios DS(EU Readers, EU Data)	+ 0.079 [0.024, 0.134] (parametric)+ 0.066 [0.014, 0.118] (non-parametric)
Secondary Analysis 4	Inter-Reader Variability measuring theassociation of TI-RADS points assignedwith and without decision supportDifference (Relative Change %)	40.7% (all readers, all data)37.4% (US readers, US data)49.7% (EU Readers, EU Data)
Secondary Analysis 5	Impact on Interpretation Time	-23.6% (all readers, all data)-22.7% (US readers, US data)-32.4% (EU Readers, EU Data)
Secondary Analysis 6	Change in average AUC with Koios DSdescriptor classifiers only (without AlAdapter) (parametric)	+0.022 [0.005, 0.039](all readers, all data)
		+0.017 [-0.007, 0.041](US readers, US data)
		+0.010 [-0.051, 0.071](EU Readers, EU Data)
		+0.019 [0.001, 0.037](all readers, all data)
	Change in average AUC with Koios DSdescriptor classifiers only (without AlAdapter) (non-parametric)	+0.015 [-0.010, 0.039](US readers, US data)
		+0.004 [-0.054, 0.062](EU Readers, EU Data)
	Change in average sensitivity andspecificity of FNA with Koios DSdescriptor classifiers only (without AlAdapter)	Sensitivity:+0.052 [0.022, 0.081](all readers, all data)
		+0.026 [-0.014, 0.066](US readers, US data)
		+0.109 [-0.004, 0.221](EU Readers, EU Data)
		Specificity-0.009 [-0.024, 0.006](all readers, all data)-0.001 [-0.022, 0.019](US readers, US data)-0.032 [-0.095, 0.031](EU Readers, EU Data)
	Change in average sensitivity andspecificity of Follow-up with Koios DSdescriptor classifiers only (without AlAdapter) — excluding casesrecommended for FNA	Sensitivity0.079 [0.031, 0.128](all readers, all data)0.072 [0.008, 0.135](US readers, US data)0.133 [-0.068, 0.334](EU Readers, EU Data)Specificity0.015 [-0.010, 0.040](all readers, all data)0.012 [-0.021, 0.045](US readers, US data)0.010 [-0.093, 0.113](EU Readers, EU Data)
	Change in average sensitivity andspecificity of Follow-up with Koios DSdescriptor classifiers only (without AlAdapter) - including casesrecommended for FNA	Sensitivity+0.047 [0.026, 0.067](all readers, all data)+0.037 [0.009, 0.065](US readers, US data)+0.067 [0.000, 0.134](EU Readers, EU Data)Specificity+0.000 [-0.013, 0.012](all readers, all data)+0.003 [-0.014, 0.019]
		(US readers, US data)-0.012 [-0.065, 0.041](EU Readers, EU Data)

Summary of All Primary Study Endpoints and Secondary Analyses (US data in bold)

{41}------------------------------------------------

{42}------------------------------------------------

{43}------------------------------------------------

Summary of System Clinical Performance Using TI-RADS RSS

	All Readers, All Data	US Readers, US Data	EU Readers, EU Data
Change in average Sensitivity/Specificity of FNA
TI-RADS categorizationw/Al Adapter + sizecriteria	+0.084 [0.054, 0.113](sensitivity)+0.140 [0.125, 0.155](specificity)	+0.058 [0.017, 0.098](sensitivity)+0.130 [0.110, 0.151](specificity)	+0.125 [0.014, 0.237](sensitivity)+0.171 [0.109, 0.233](specificity)
TI-RADS categorization +size criteria	+0.052 [0.022, 0.081](sensitivity)-0.009 [-0.024, 0.006](specificity)	+0.026 [-0.014, 0.066](sensitivity)-0.001 [-0.022, 0.019](specificity)	+0.109 [-0.004, 0.221](sensitivity)-0.032 [-0.095, 0.031](specificity)
Change in average Sensitivity/Specificity of Follow-up
TI-RADS categorizationw/Al Adapter + sizecriteria	+0.060 [0.040, 0.080](sensitivity)+0.206 [0.192, 0.219](specificity)	+0.053 [0.026, 0.080](sensitivity)+0.180 [0.161, 0.198](specificity)	+0.060 [-0.009, 0.129](sensitivity)+0.296 [0.238, 0.354](specificity)
TI-RADS categorization +size criteria	+0.047 [0.026, 0.067](sensitivity)+0.000 [-0.013, 0.012](specificity)	+0.037 [0.009, 0.065](sensitivity)+0.003 [-0.014, 0.019](specificity)	+0.067 [0.000, 0.134](sensitivity)-0.012 [-0.065, 0.041](specificity)

Page 40

{44}------------------------------------------------

Image /page/44/Figure/0 description: The image is a scatter plot titled "Change in ROCAUC". The x-axis is labeled "AUCus" and the y-axis is labeled "AUCus + Ds". There are 11 data series plotted on the graph, labeled R1 through R11. The data points are clustered between x values of 0.65 and 0.8 and y values of 0.75 and 0.82.

Reader US (TI-RADS categorization) vs. US+DS (TI-RADS categorization w/AI Adapter)

Per reader non-parametric AUC comparing US to US+DS. The dashed line represents equivocal results with all points above this line demonstrating an improvement for the US+DS reading condition.

Image /page/44/Figure/3 description: The image is a scatter plot titled "Change in Operating Point". The x-axis is labeled "Specificity" and ranges from 0.0 to 1.0, while the y-axis is labeled "Sensitivity" and also ranges from 0.0 to 1.0. There are 15 different data series plotted on the graph, labeled R1 through R15, each represented by a different color and an arrow indicating the change in operating point.

Reader US (TI-RADS categorization + size criteria) vs. US+DS (TI-RADS categorization w/Al Adapter + size criteria) Change in Operating Point (FNA)

Change in Sensitivity and Specificity of FNA Recommendations for all readers. The base of the arrow represents the initial operating point, while the arrowhead represents the sensitivity and specificity of US+DS

Koios Medical, Inc.

{45}------------------------------------------------

Image /page/45/Figure/0 description: The image shows a plot titled "Change in Operating Point". The plot shows the sensitivity on the y-axis and specificity on the x-axis, both ranging from 0 to 1. There are 15 different lines plotted on the graph, labeled R1 through R15, each represented by a different color. Each line has an arrow indicating the direction of change.

Reader US (TI-RADS categorization + size criteria) vs. US+DS (TI-RADS categorization w/Al Adapter + size criteria) Change in Operating Point (Follow-up)

Change in Sensitivity and Specificity of Follow-Up Recommendations for all readers. The base of the arrow represents the initial operating point, while the arrowhead represents the sensitivity and specificity of US+DS

Primary endpoints were successfully met, demonstrating a statistically significant improvement of 0.074 [0.051, 0.098] (95% confidence interval) in overall reader performance of US-based readers when utilizing Koios DS for the interpretation of US-based thyroid ultrasound studies.

10. Special Controls

Design verification and validation and product labelling include all requirements proscribed in the 21 CFR 892.2060 Special Controls.

11. Conclusion

Nonclinical performance tests demonstrate that the Koios DS software device is as safe, as effective, and performs as well as or better than the legally marketed predicate Koios DS software. It has similar intended use, indications for use, technological characteristics, and principles of operation as its predicate device. The Koios DS product is substantially equivalent to K212616.

Regulation Number and Section

§ 892.2060 Radiological computer-assisted diagnostic software for lesions suspicious of cancer.

(a)
Identification. A radiological computer-assisted diagnostic software for lesions suspicious of cancer is an image processing prescription device intended to aid in the characterization of lesions as suspicious for cancer identified on acquired medical images such as magnetic resonance, mammography, radiography, or computed tomography. The device characterizes lesions based on features or information extracted from the images and provides information about the lesion(s) to the user. Diagnostic and patient management decisions are made by the clinical user.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Design verification and validation must include:
(i) A detailed description of the image analysis algorithms including, but not limited to, a detailed description of the algorithm inputs and outputs, each major component or block, and algorithm limitations.
(ii) A detailed description of pre-specified performance testing protocols and dataset(s) used to assess whether the device will improve reader performance as intended.
(iii) Results from performance testing protocols that demonstrate that the device improves reader performance in the intended use population when used in accordance with the instructions for use. The performance assessment must be based on appropriate diagnostic accuracy measures (
e.g., receiver operator characteristic plot, sensitivity, specificity, predictive value, and diagnostic likelihood ratio). The test dataset must contain sufficient numbers of cases from important cohorts (e.g., subsets defined by clinically relevant confounders, effect modifiers, concomitant diseases, and subsets defined by image acquisition characteristics) such that the performance estimates and confidence intervals of the device for these individual subsets can be characterized for the intended use population and imaging equipment.(iv) Standalone performance testing protocols and results of the device.
(v) Appropriate software documentation (
e.g., device hazard analysis; software requirements specification document; software design specification document; traceability analysis; and description of verification and validation activities including system level test protocol, pass/fail criteria, results, and cybersecurity).(2) Labeling must include:
(i) A detailed description of the patient population for which the device is indicated for use.
(ii) A detailed description of the intended reading protocol.
(iii) A detailed description of the intended user and recommended user training.
(iv) A detailed description of the device inputs and outputs.
(v) A detailed description of compatible imaging hardware and imaging protocols.
(vi) Warnings, precautions, and limitations, including situations in which the device may fail or may not operate at its expected performance level (
e.g., poor image quality or for certain subpopulations), as applicable.(vii) Detailed instructions for use.
(viii) A detailed summary of the performance testing, including: Test methods, dataset characteristics, results, and a summary of sub-analyses on case distributions stratified by relevant confounders (
e.g., lesion and organ characteristics, disease stages, and imaging equipment).