DeepContour is a deep learning based medical imaging software that allows trained healthcare professionals to use DeepContour as a tool to automatically process CT images. In addition, DeepCoutour is suitable for the following conditions:

Creation of contours using deep-learning algorithms , support quantitative analysis, organ HU distribution statistics, transfer contour files to TPS, and create management archives for patients.
Analvze the anatomical structure at different anatomical positions.
Rigid and elastic registration based on CT.
3D reconstruction, editing and other visual tools based on organ contours

Device Description

DeepContour is a deep learning based medical imaging software that allows trained healthcare professionals to use DeepContour as a tool to automatically process CT images. DeepContour contouring workflow supports CT input data and produces RTSTRUCT outputs. The organ segmentation can also be combined into templates, which can be customized by different hospitals according to their needs. DeepContour provides an interactive contouring application to edit and review the contours automatically generated by DeepContour.

AI/ML Overview

Here's a breakdown of the acceptance criteria and study details for the DeepContour (V1.0) device, based on the provided FDA 510(k) Summary:

Acceptance Criteria and Reported Device Performance

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state "acceptance criteria" as a set of predefined quantitative thresholds the device must meet. Instead, the study's aim is to demonstrate that DeepContour's performance is equivalent to or better than the predicate devices. The primary metric used for this comparison is the Dice coefficient, and the implicit acceptance criterion is that DeepContour's performance is not significantly worse than the predicates.

The equivalence definition is stated as: "the lower bound of 95th percentile confidence interval of the subject device segmentation is greater than 0.1 Dice lower than the mean of predicate device segmentation."

Below is a table summarizing the reported Dice coefficients for DeepContour and the predicate devices for a selection of structures. It also includes the summary Average Symmetric Surface Distance (ASSD) comparison.

Table 1: Acceptance Criteria (Implicit) and Reported Device Performance

Metric	Implicit Acceptance Criteria	DeepContour Reported Performance (Mean ± Std (95% CI Lower Bound))	Predicate (AI-Rad CAI-Rad Companion Organs RT) Reported Performance (Mean ± Std)	Predicate (Contour ProtégéAI) Reported Performance (Mean ± Std)
Dice Coefficient	Lower 95th percentile CI of DeepContour segmentation > (Mean of Predicate Segmentation - 0.1 Dice)	See "Clinical performance comparison" tables below for specific structures.	See "Clinical performance comparison" tables below for specific structures.	See "Clinical performance comparison" tables below for specific structures.
ASSD (median)	Median ASSD comparable to predicate devices.	0.95 (95% CI: [0.85, 1.13])	0.96 (95% CI: [0.84, 1.15])	0.95 (95% CI: [0.86, 1.17])

Table 5: Clinical performance comparison (Peking Union Medical College Hospital) - Selected Structures

Structure:	DeepContour	AI-Rad CAI-RadCompanion Organs RT(K221305)	Contour ProtégéAI(K223774)
Brain	0.98±0.01(0.97)	0.93±0.11	0.98 ± 0.01
BrainStem	0.91±0.03(0.89)	0.90±0.02	0.82 ± 0.09
Eye_L	0.89±0.02(0.88)	0.81±0.06	0.87 ± 0.06
Lung_L	0.98±0.05(0.96)	0.92±0.16	0.96 ± 0.02
Heart	0.93±0.16(0.90)	0.91±0.06	0.90 ± 0.07
Liver	0.96±0.07(0.95)	0.86±0.17	0.93 ± 0.07
Kidney_L	0.92±0.03(0.91)	0.82±0.13	0.92 ± 0.05
Pancreas	0.86±0.01(0.86)	0.87±0.03	0.45 ± 0.22
Bladder	0.95±0.15(0.93)	0.87±0.15	0.52 ± 0.19
Prostate	0.87±0.02(0.85)	0.74 ± 0.12	0.85 ± 0.06
SpinalCord	0.93±0.01(0.92)	0.66 ± 0.14	0.63±0.16

Table 6: Clinical performance comparison (LCTSC American public datasets) - Selected Structures

Structure:	DeepContour	AI-Rad CAI-RadCompanion Organs RT(K221305)	ContourProtégéAI(K223774)
SpinalCord	0.92±0.02(0.91)	0.64±0.13	0.62 ± 0.21
Lung L	0.97±0.15(0.96)	0.90±0.13	0.95 ± 0.05
Heart	0.92±0.11(0.90)	0.91±0.04	0.90 ± 0.04
Esophagus	0.89±0.13(0.86)	0.75±0.13	0.68 ± 0.19

Table 7: Clinical performance comparison (Pancreas-CT American public datasets) - Selected Structures

Structure:	DeepContour	AI-Rad CAI-RadCompanion Organs RT(K221305)	ContourProtégéAI(K223774)
Spleen	0.90±0.05(0.88)	0.91±0.12	0.89 ± 0.08
Pancreas	0.85±0.03(0.83)	0.84±0.02	0.43 ± 0.25
Kidney_L	0.93±0.02(0.91)	0.84±0.03	0.92 ± 0.17
Liver	0.97±0.03(0.97)	0.85±0.13	0.92 ± 0.06
Stomach	0.85±0.02(0.84)	0.80±0.05	0.81 ± 0.17

2. Sample sized used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

Sample Size: 203 CT images.
- 100 clinical datasets
- 103 American public datasets (60 from LCTSC, 43 from Pancreas-CT)
Data Provenance:
- 100 clinical datasets: Retrospectively collected from Peking Union Medical College Hospital (China).
- 103 American public datasets: Publicly available datasets originally from American sources.
  - 2017 Lung CT Segmentation Challenge (LCTSC): 60 thoracic CT scan patients.
  - Pancreas-CT (PCT): 43 abdominal contrast-enhanced CT scan patients.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

For the 100 clinical datasets (China): Two radiation oncologists with more than 10 years of clinical practice established the ground truth annotations. Their detailed CVs are in Appendix 2 (not provided in the input, but referenced).
For the 103 American public datasets: Annotated by American doctors. (Specific qualifications not detailed in the provided text).

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

For the 100 clinical datasets (China): The ground truth was established by two radiation oncologists. A third qualified internal staff member was available to adjudicate if needed. This implies a 2+1 adjudication method if there was disagreement.
For the 103 American public datasets: No explicit adjudication method is mentioned, only that they were "annotated by American doctors."

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

The provided text does not describe a Multi-Reader Multi-Case (MRMC) comparative effectiveness study involving human readers with and without AI assistance to measure improvement in human performance. The study focuses on the standalone performance of the AI algorithm (DeepContour) compared to predicate devices.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

Yes, a standalone performance study was done. The entire "Performance comparison" section (Tables 5, 6, 7, and 8) details the Dice coefficients and ASSD values for the DeepContour algorithm, directly comparing its segmentation performance against the ground truth and the predicate devices. There is no human reader involved in generating the DeepContour results reported in these tables.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

For the 100 clinical datasets (China): Expert consensus (two radiation oncologists applying RTOG and clinical guidelines using manual annotation, with a third available for adjudication).
For the 103 American public datasets: Expert annotation by American doctors. (Implied expert consensus or single expert annotation from the original dataset creation process, as described by the original publications).

8. The sample size for the training set

# of Datasets: 800 CT images.
- 200 for head and neck region
- 200 for chest region
- 200 for abdomen region
- 200 for pelvic region
- (Out of these, 160 cases per region were used for training, and 40 cases per region for validation.)

9. How the ground truth for the training set was established

The initial segmentations were reviewed and corrected by two radiation oncologists for model training, with a third qualified internal staff member available to adjudicate if needed. This indicates an expert review and correction process, likely similar to the 2+1 adjudication method used for the test set ground truth.

Summary

{0}------------------------------------------------

May 7, 2024

Image /page/0/Picture/1 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left, there is a seal with an abstract design and the text "DEPARTMENT OF HEALTH & HUMAN SERVICES-USA" arranged around it. To the right, there is a blue square with the letters "FDA" in white, followed by the words "U.S. FOOD & DRUG" in blue, and "ADMINISTRATION" in a smaller font size below it.

Wisdom Technologies., Inc. % Wei Wang Regulatory Consultant 11 Longstreet IRVINE, CA 92620

Re: K232928

Trade/Device Name: DeepContour (V1.0) Regulation Number: 21 CFR 892.2050 Regulation Name: Medical Image Management And Processing System Regulatory Class: Class II Product Code: QKB Dated: April 5, 2024 Received: April 5, 2024

Dear Wei Wang:

We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).

Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming

{1}------------------------------------------------

product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely,

Loran Werchner

Lora D. Weidner, Ph.D. Assistant Director Radiation Therapy Team DHT8C: Division of Radiological Imaging and Radiation Therapy Devices OHT8: Office of Radiological Health Office of Product Evaluation and Quality Center for Devices and Radiological Health

{2}------------------------------------------------

Indications for Use

Submission Number (if known)

K232928

Device Name

DeepContour (V1.0)

Indications for Use (Describe)

Creation of contours using deep-learning algorithms , support quantitative analysis, organ HU distribution statistics, transfer contour files to TPS, and create management archives for patients.
Analvze the anatomical structure at different anatomical positions.
Rigid and elastic registration based on CT.
3D reconstruction, editing and other visual tools based on organ contours

Type of Use (Select one or both, as applicable)

Prescription Use (Part 21 CFR 801 Subpart D)

he-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{3}------------------------------------------------

K232928

Image /page/3/Picture/1 description: The image shows a logo with a stylized human figure in the center, surrounded by radiating lines. The figure is placed within a blue, semi-circular shape. To the right of the figure, there are Chinese characters followed by the words "WISDOM TECH" in blue font. The logo appears to represent a technology company, possibly focused on human-centered solutions.

510(k) Summary

The following information is provided as required by 21 CFR 807.92

1. SUBMITTER

Name: Wisdom Technologies., Inc.

Address: 4th Floor, Building F2, Phase II, Innovation Industrial Park, Hefei, Anhui, China 230088

Phone: +86-0551-65116387

Email: registration(@wisdom-tech.online

Contact Person: Wei Wang, Consultant, Regulatory Affairs Phone: 949-7849283

Date Prepared: August 24, 2023

2. DEVICE

Subject Device Name: DeepContour v1.0 Common/Trade Name: DeepContour Product Code and Classification: Medical Image Management And Processing System 21 CFR 892.2050 | QKB | Class II

3. PREDICATE DEVICE

Primary: AI-Rad CAI-Rad Companion Organs RT (K221305) Siemens Reference: Contour ProtégéAI (K223774) MIM Software

4. DEVICE DESCRIPTION

5. INDICATIONS FOR USE

{4}------------------------------------------------

1). Creation of contours using deep-learning algorithms , support quantitative analysis, organ HU distribution statistics, transfer contour files to TPS, and create management archives for patients.

2). Analyze the anatomical structure at different anatomical positions.

3). Rigid and elastic registration based on CT.
4). 3D reconstruction, editing and other visual tools based on organ contours

6. COMPARISON OF TECHNOLOGICAL CHARACTERISTICS WITH PREDICATE DEVICE

The primary technological components of DeepContour and its predicate device are to achieve the deep learning based medical imaging software functions that allows trained healthcare professionals to automatically process CT images. Both are software devices that receive inputs related to radiological images; Both generate contours as output that may be used as input for radiation Treatment Planning Systems and interactive contouring applications to review and edit; Both are software devices for prescription use in a professional environment with no patient contact.

There are no known differences in technological characteristics between the subject device and the predicate device that raise any questions of safety or effectiveness. The technological characteristics of the subject device are believed to be substantially equivalent to the predicate device.

Area ofComparison	Subject Device-DeepContour	Primary-AI-Rad CAI-Rad Companion OrgansRT (K221305) Siemens	Reference-ContourProtégéAI(K223774)MIM Software
RegulationNumber/code	21 CFR 892.2050 QKB	22 CFR 892.2050 QKB	21 CFR 892.2050 QKB
Regulation Name	Medical Image ManagementAnd Processing System	Medical Image ManagementAnd Processing System	Medical ImageManagement AndProcessing System
Indications forUse	DeepContour is a deeplearning based medicalimaging software that allowstrained healthcareprofessionals to useDeepContour as a tool toautomatically process CTimages. In addition,DeepCoutour is suitable forthe following conditions:1. Creation of contours usingdeep-learning algorithms ,support quantitative analysis,organ HU distributionstatistics, transfer contour filesto TPS, and createmanagement archives forpatients.2. Analyze the anatomicalstructure at differentanatomical positions.3. Rigid and elastic registrationbased on CT.4. 3D reconstruction, editingand other visual tools based onorgan contours	AI-Rad Companion OrgansRT is a post-processingsoftware intended toautomatically contourDICOM CT imaging datausing deep-learning-basedalgorithms.Contours that are generatedby AI-Rad CompanionOrgans RT may be used asinput for clinical workflowsincluding external beamradiation therapy treatmentplanning. AI-RadCompanion Organs RT mustbe used in conjunction withappropriate software such asTreatment Planning Systemsand Interactive Contouringapplications, to review, edit,and accept contoursgenerated by AI-RadCompanion Organs RT.The output of AI-RadCompanion Organs RT inthe format of RTSTRUCTobjects are intended to beused by trained medicalprofessionals.The software is not intendedto automatically detect orcontour lesions. OnlyDICOM images of adultpatients are considered to bevalid input.	Trained medicalprofessionals use ContourProtégéAI as a tool toassist in the automatedprocessing of digitalmedical images ofmodalities CT and MR, assupported by ACR/NEMADICOM 3.0. In addition,Contour ProtégéAIsupports the followingindications:• Creation of contoursusing machine-learningalgorithms for applicationsincluding, but not limitedto, quantitativeanalysis, aiding adaptivetherapy, transferringcontours to radiationtherapy treatment planningsystems, and archivingcontours for patientfollow-up andmanagement.• Segmenting anatomicalstructures across a varietyof CT anatomic locations.• And segmenting theprostate, the seminalvesicles, and the urethrawithin T2-weighted MRimages.Appropriate imagevisualization softwaremust be used to reviewand, if necessary, edit
			results automaticallygenerated by ContourProtégéAI.
Devicedescription	DeepContour is a deeplearning based medicalimaging software that allowstrained healthcareprofessionals to useDeepContour as a tool toautomatically process CTimages. DeepContourcontouring workflow supportsCT input data and producesRTSTRUCT outputs. Theorgan segmentation can alsobe combined into templates,which can be customized bydifferent hospitals according totheir needs.DeepContour provides aninteractive contouringapplication to edit and reviewthe contours automaticallygenerated by DeepContour.	AI-Rad Companion OrgansRT is a post-processingsoftware used toautomatically contourDICOM CT imaging datausing deep-learning-basedalgorithms. AI-RadCompanion Organs RTcontouring workflowsupports CT input data andproduces RTSTRUCToutputs. Theconfiguration of the organdatabase and organtemplates defining theorgans and structures to becontoured based on the inputDICOM data is managed viaa configuration interface.Contours that are generatedby AI-Rad CompanionOrgans RT may be used asinput for clinical workflowsincluding external beamradiation therapy treatmentplanning.The output of AI-RadCompanion Organs RT, inthe form of RTSTRUCTobjects, are intended to beused by trained medicalprofessionals. The output ofAI-Rad Companion OrgansRT must be used inconjunction with appropriatesoftware such as TreatmentPlanning Systems andInteractive Contouringapplications, to review, edit,and accept contoursgenerated by AI-RadCompanion Organs RTapplication.At a high-level, AI-RadCompanion Organs RTincludes the followingfunctionality: 1. Automatedcontouring of Organs at Risk(OAR) workflowa. Input -DICOM CTb. Output - DICOM	Contour ProtégéAI is anaccessory to MIMsoftware that automaticallycreates contours onmedical images throughthe use of machine-learning algorithms. It isdesigned for use in theprocessing of medicalimages and operates onWindows, Mac, and Linuxcomputer systems.Contour ProtégéAI isdeployed on a remoteserver using the MIMcloudservice for datamanagement and transfer;or locally on theworkstation or serverrunning MIM software.
		RTSTRUCT2. Organ Templatesconfiguration (incl. OrganDatabase)3. Web-based preview ofcontouring results to acceptor reject the generatedcontours.
Algorithm	Deep Learning	Deep Learning	Machine-learning
Segmentation ofOrgan at Risk inthe AnatomicRegions	Head & Neck, Thorax,Abdomen & Pelvis(82 OARs)	Head & Neck, Thorax,Abdomen & PelvisHead & Neck lymphnodes(108 OARs)	Head & Neck, Prostate,Thorax, Abdomen, Lungs& Liver, MRT structures(spleen, pelvic lymphnodes, descendingaorta, bone)
CompatibleModality	CT Images	CT Images	CT & MR
CompatibleScanner Models	No Limitation on scannermodel,DICOM compliance required.	No Limitation on scannermodel,DICOM compliancerequired.	No Limitation on scannermodel,DICOM compliancerequired.
CompatibleTreatmentPlanning System	No Limitation on TPS model,DICOMcompliance required.	No Limitation on TPSmodel, DICOMcompliance required.	No Limitation on TPSmodel, DICOMcompliance required.
Contraindications	Adult use only	Adult use only	Adult use only
TargetPopulation	DeepContour is designed foruse only in adult populationsfor whom relevant modalityscans , including head andneck, thorax, abdomen, andpelvis, are available .	AI-Rad Companion OrgansRT is designed for use onlyin adult populations. AI-RadCompanion Organs RT isdesigned for any patient forwhom relevant modalityscans are available. Morespecifically, the software isvalidated on previouslyacquired CT DICOMvolumes for radiationtherapy treatment planning,including, head and neck,	No public record found
Clinical conditionthe device isintended todiagnose, treat ormanage	Limited to patients previouslyselected for RadiationTherapy.	Limited to patientspreviously selected forRadiation Therapy.	Limited to patientspreviously selected forRadiation Therapy.
SoftwareArchitecture	Server-based applicationsupportingWindows and Localdeployment on Windows.	AI-Rad Companion(Engine) architectureenabling the deployment ofAI Rad Companion OrgansRT using Edge and in theCloud. The UI is providedusing a webbased interface.	Server-based applicationsupportingLinux-based OS and Localdeployment on Windowsor Mac
DeploymentFeature	locally deployed or Cloud-based	Edge & Cloud Deployment	Cloud-based or locallydeployed
Organ Templates	Creating, editing and deletionof organ templates. Customizepredefined structure databasewith mapping to internationalnomenclature schemes.	Creating, editing anddeletion of organ templates.Customize predefinedstructure database withmapping to internationalnomenclature schemes.	No public record found
Automatedworkflow	DeepContour automaticallyprocesses input image data andsends the results as DICOM-RT Structure Sets to a user-configurable target node.	AI-Rad Companion OrgansRT automatically processesinput image data and sendsthe results as DICOM-RTStructure Sets to a user-configurable target node.	Automatic contouringworking using machine-learning
Contourvisualization andediting feature	DeepContour provides basicresult preview of automaticsegmentation results, andediting feature of theautomatic segmented contour.	AI-Rad Companion OrgansRT provides basic resultpreview of automaticsegmentation results, and noediting feature of theautomatic segmentedcontour.	No public record found

Table 1. Substantial Equivalence Comparison

{5}------------------------------------------------

{6}------------------------------------------------

{7}------------------------------------------------

{8}------------------------------------------------

{9}------------------------------------------------

SegmentationPerformance	The target performance wasvalidated using 100 cases. Themean and standard deviationDice coefficients, along withthe lower 95th percentileconfidence bound werecalculated.	The target performance wasvalidated using 113 casesdistributed to two cohorts.Cohort A is clinical routinetreatment planning CT and itis split into two sub-cohortand Cohort B is PET-CTdata. To objectively evaluatethe target performance, theDICE coefficient, theabsolute symmetric surfacedistance (ASSD) and the failrate was evaluated. Thesegmentation performanceof the subject and referencedevice were equivalent aswell as the overallperformance compared tothe predicate device.	739 CT Images from 12clinical sites were used fortesting. The mean andstandard deviation Dicecoefficients, along with thelower 95th percentileconfidence bound werecalculated.
User Interface -Results Preview(Confirmation)	Basic visualizationfunctionality of original dataand generated contours	Basic visualizationfunctionality of original dataand generated contours	Basic visualizationfunctionality of originaldata and generatedcontours
User InterfaceConfiguration	Configuration UI	Configuration UI	Configuration UI
AutomatedWorkflow to TPS	Results send to ConfirmationUI & Optional bypassing ofConfirmation UI to TPS	Results send toConfirmation UI & Optionalbypassing of ConfirmationUI to TPS	Results send toConfirmation UI &Optional bypassing ofConfirmation UI to TPS
Human Factors	Design to be used bytrained clinicians.	Design to be used bytrained clinicians.	Design to be used bytrained clinicians.
Patient Contact	None	None	None

7. PERFORMANCE AND NONCLINICAL TESTS

Software verification and validation were conducted, and the process was documented per FDA's Guidance for Industry and FDA Staff, "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices."

Verification test results demonstrate conformance to applicable requirements and specifications. Testing against the predicate device demonstrates good agreement, proving that tested device can be used as equivalent contouring software for clinical purposes. No animal studies or clinical tests were required for validation of the software.

{10}------------------------------------------------

7.1 Training Datasets

For DeepContour, the deep learning models were trained on a pool of training data that did not include any patients from the same institution as the test subjects. The data was collected from 35 hospitals across China over a period of about 3 years. The data collection time is random, including patients with different ages, races, size, et al. Data collected in a hospital tends to be a collection of consecutive cases. Due to the different time periods applicable to different hospitals, the time for collecting data varies among different hospitals. The specific patient subgroups were not the special consideration when collecting the patients' data.

The training data included 800 CT images (200 for each region of head and neck region, chest region, abdomen region and pelvic region) at various ages, no ethnicities or genders were excluded from training. Within 200 cases collected in each region, 160 cases were used for training and 40 cases were used for validation to establish the auto-segmentation models. The auto-segmentation process includes two steps: first step to distinguish the four different regions (head and neck region, chest region, abdomen region and pelvic region), second step to call the model of this region to realize auto-segmentation. The initial segmentations were then reviewed and corrected by two radiation oncologists for model training, with a third qualified internal staff member available to adjudicate if needed.

# of Datasets	800
Data Origin	35 hospitals across China
Sex	Male:372
	Female:428
Age	<50: 289
	50-70: 415
	>70:70
	Unknown:26 (unknown due to data minimization on customer site)
Body Region	Head and neck region: 200
	chest region: 200
	abdomen region: 200
	pelvic region: 200
CT Scanner	Philips: 301
	GE: 226
	Simens: 128
	Unknown:145 (unknown due to data minimization on customer site)

|--|

{11}------------------------------------------------

7.2 Verification datasets

The performance of the DeepContour were verified by a total of 203 CT images with 2 datasets:

(1) clinical datasets retrospectively collected from multiple clinical regions across the Peking Union Medical College Hospital (N=100), consisting of four selected parts based on their location, 25 cases for head and neck, 25 cases for chest, 25 cases for abdomen and 25 cases for pelvic; These clinical datasets are not used in the training datasets at all. (2) American public datasets (N=103);

(a)The 2017 lung CT segmentation challenge (LCTSC), which contains 60 thoracic CT scan patients with five segmented organs (left lung, right lung, heart, spinal cord, and esophagus), (b) Pancreas-CT (PCT), which contains 43 abdominal contrast enhanced CT scan patients with seven segmented organs (the spleen, left kidney, esophagus, liver, stomach, pancreas, and duodenum). The American public datasets were annotated by the American doctors. The article was published in the journal (Medical Physics), doi: 10.1002/mp.14131. (Please refer to more information about the two publicly available datasets: https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=24284539, https://wiki.cancerimagingarchive.net/display/Public/Pancreas-CT)

The auto-segmentation process is the same with what was applied to the training dataset with two steps: first step to distinguish the four different regions (head and neck region, chest region, abdomen region and pelvic region), second step to call the model of this region to realize auto-segmentation. The ground truth annotations for the 100 cases in China were established by two different radiation oncologists with more than 10 years of clinical practice (See Appendix 2 for their detailed CVs) following RTOG and clinical guidelines using manual annotation. A third qualified internal staff member is also available to adjudicate if needed. American public datasets (N=103) were annotated by the American doctors.

The CT images in the 100 test cases were curated for suitability and to avoid bias through rich pre-processing and post-processing methods. Due to the differences between patients and the differences in CT machine image data, the data augmentation was used during training to simulate possible data situations in clinical practice, thereby increasing the model's generalization ability. The strategy of first localization and then segmentation is also used to improve the accuracy of the model.

# of Datasets	100	103
Data Origin	Peking Union MedicalCollege Hospital	The 2017 lung CT segmentationchallenge (LCTSC): 60Pancreas-CT (PCT): 43
Sex	Male: 43Female: 57	Unknow:103 (unknown due to dataminimization on customer site)

Table 3: Verification Data Information

{12}------------------------------------------------

Age	<50: 31	Unknown:103 (unknown due to dataminimization on customer site)
	50-70: 46
	>70: 23
Body Region	Head and neck region: 25	60 thoracic CT scan: 5 segmentedorgans (left lung, right lung, heart, spinalcord, and esophagus)
	chest region: 25
	abdomen region: 25	43 abdominal CT scan: 7 segmentedorgans (the spleen, left kidney, esophagus,liver, stomach, pancreas, and duodenum)
	pelvic region: 25
CT Scanner	Philips: 34	Unknown 103 (unknown due to dataminimization on customer site)
	GE: 36
	Simens: 30

7.3 Rigid and deformable registration

The rigid and deformable registration methods are mainly used for contour mapping in the delineation of target areas and organs at risk. The following Table showed the results between planning CT images (moving image) and CBCT images (Fixed images), rpCT represents rigid planning CT, dpCT1 represents deformed planning CT with our algorithms, and dpCT2 represents deformed planning CT with predicate device. The metrics include the mean absolute error (MAE), root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM).

	rpCT-CBCT	dpCT1- CBCT	dpCT2- CBCT
MAE(HU)	51.23±13.67	43.98±10.74	46.71±12.71
RMSE	121.09±30.23	117.58±28.22	127.96±30.76
PSNR	20.01±2.74	22.23±2.61	20.00±3.77
SSIM	0.623±0.084	0.680±0.050	0.685±0.055

Table 4: Rigid and deformable registration

7.4 Performance comparison

The mean and standard deviation of Dice coefficients was calculated for each organ in the subject device compared with the predicate device. The results of the subject device demonstrate equivalent or better performance compared to the predicate device when aggregate performance over all organs is considered with known limitations described in the labeling. Here equivalence is defined such that the lower bound of 95th percentile confidence interval of the subject device segmentation is greater than 0.1 Dice lower than the mean of predicate device segmentation.

{13}------------------------------------------------

Dice coefficients of 100 clinical datasets collected from multiple clinical regions across the Peking Union Medical College Hospital compared to the predicate device are presented here:

Structure:	DeepContour	AI-Rad CAI-RadCompanion Organs RT(K221305)	Contour ProtégéAI(K223774)
Brain	0.98±0.01(0.97)	0.93±0.11	0.98 ± 0.01
BrainStem	0.91±0.03(0.89)	0.90±0.02	0.82 ± 0.09
Cochlea_L	0.86±0.03(0.84)	0.84±0.03	0.27 ± 0.17
Cochlea_R	0.85±0.01(0.84)	0.86±0.07	0.29 ± 0.18
Eye_L	0.89±0.02(0.88)	0.81±0.06	0.87 ± 0.06
Eye_R	0.88±0.03(0.86)	0.89±0.13	0.87 ± 0.06
Lens_L	0.89±0.02(0.88)	0.85±0.05	0.61±0.17
Lens_R	0.88±0.02(0.85)	0.81±0.12	0.63 ± 0.15
Larynx	0.88±0.03(0.83)	0.84±0.08	0.50 ± 0.16
Larynx_extend	0.92±0.02(0.91)	0.90±0.11	None
Mandible	0.95±0.02(0.91)	0.91±0.07	0.85 ± 0.07
Optic_Chiasm	0.88±0.03(0.82)	0.63±0.11	0.12±0.11
OpticalNerve_L	0.86±0.04(0.79)	0.66±0.06	0.53 ± 0.13
OpticalNerve_R	0.89±0.02(0.81)	0.59±0.10	0.52 ± 0.12
OralCavity	0.92±0.03(0.89)	0.82±0.09	0.77 ± 0.12
OralCavity_WithGum	0.91±0.06(0.89)	0.71±0.06	None
Parotid_L	0.88±0.05(0.82)	0.80±0.15	0.80 ± 0.10
Parotid_R	0.86±0.03(0.81)	0.81±0.04	0.80 ± 0.06
Pituitary	0.78±0.04(0.69)	0.68±0.14	0.49±0.15
Temporal_Lobe_L	0.92±0.02(0.90)	0.82±0.09	0.68 ± 0.17
Temporal_Lobe_R	0.91±0.11(0.86)	0.81±0.07	0.79 ± 0.18
TMJ_L	0.85±0.02(0.83)	0.84±0.02	0.84± 0.06
TMJ_R
	0.86±0.15(0.81)	0.85±0.15	0.83 ± 0.06
InternalAcousticCanal_L	0.76±0.02(0.71)	0.73±0.12	0.71 ± 0.17
InternalAcousticCanal_R	0.79±0.02(0.75)	0.77±0.05	0.73±0.15
MiddleEar_L	0.82±0.12(0.78)	0.72±0.02	0.70±0.16
MiddleEar_R	0.85±0.03(0.81)	0.80±0.13	0.77 ± 0.17
TemporalLobe_withHippo_L	0.89±0.08(0.85)	0.87±0.14	0.85 ± 0.18
TemporalLobe_withHippo_R	0.90±0.05(0.86)	0.88±0.09	0.87 ± 0.06
Submandibular_L	0.91±0.06(0.88)	0.81±0.11	0.75 ± 0.10
Submandibular_R	0.92±0.03(0.89)	0.72±0.16	0.74±0.09
PharyngealConstrictors_U	0.82±0.12(0.76)	0.77±0.13	None
PharyngealConstrictors_M	0.86±0.14(0.83)	0.76±0.11	None
PharyngealConstrictors_L	0.84±0.17(0.80)	0.74±0.07	None
BrachialPlexus_L	0.81±0.13(0.79)	0.71±0.08	0.37±0.13
BrachialPlexus_R	0.82±0.11(0.69)	0.52±0.06	0.36±0.16
Hippocampus_L	0.76±0.03(0.73)	0.75 ± 0.12	0.75 ± 0.02
Hippocampus_R	0.79±0.02(0.76)	0.73 ± 0.09	0.76 ± 0.02
EustachianTubeBone_L	0.85±0.05(0.84)	0.81±0.05	0.88 ± 0.07
EustachianTubeBone_R	0.87±0.07(0.80)	0.77±0.09	0.72 ± 0.17
TympanicCavity_L	0.84±0.03(0.83)	0.80±0.13	0.75 ± 0.15
TympanicCavity_R	0.86±0.09(0.81)	0.76±0.05	0.74 ± 0.17
Vestibule_L	0.85±0.06(0.82)	0.80±0.16	0.79 ± 0.10
Vestibule_R	0.87±0.02(0.86)	0.77±0.10	0.74±0.10
InnerEar_L	0.87±0.10(0.83)	0.83±0.11	0.82 ± 0.07
InnerEar_L	0.86±0.13(0.83)	0.87±0.03	0.91 ± 0.07
Lung_L	0.98±0.05(0.96)	0.92±0.16	0.96 ± 0.02
Lung_R	0.99±0.03(0.98)	0.95±0.08	0.96 ± 0.02
Lung_All	0.98±0.14(0.97)	0.91±0.04	None
Heart	0.93±0.16(0.90)	0.91±0.06	0.90 ± 0.07
Trachea	0.89±0.03(0.88)	0.89±0.03	0.73 ± 0.17
Esophagus	0.88±0.11(0.85)	0.78±0.07	0.70 ± 0.15
Breast_L	0.92±0.08(0.86)	0.82±0.05	0.74 ± 0.17
Breast_R	0.93±0.01(0.92)	0.83±0.04	0.77 ± 0.10
Aorta	0.89±0.09(0.87)	0.70 ± 0.08	0.74 ± 0.10
Liver	0.96±0.07(0.95)	0.86±0.17	0.93 ± 0.07
Kidney_L	0.92±0.03(0.91)	0.82±0.13	0.92 ± 0.05
Kidney_R	0.93±0.04(0.91)	0.88±0.07	0.91 ± 0.06
Duodenum	0.88±0.16(0.83)	0.81±0.012	None
Pancreas	0.86±0.01(0.86)	0.87±0.03	0.45 ± 0.22
Smallintestine	0.89±0.12(0.85)	0.88±0.08	None
Bowelbag	0.93±0.16(0.88)	0.83 ± 0.08	0.68 ± 0.08
Bladder	0.95±0.15(0.93)	0.87±0.15	0.52 ± 0.19
Stomach	0.86±0.01(0.86)	0.79 ± 0.21	0.81 ± 0.11
Femur_Head_L	0.92±0.12(0.89)	0.90±0.16	0.93 ± 0.05
Femur_Head_R	0.91±0.14(0.86)	0.90±0.09	0.93 ± 0.04
Pelvis	0.87±0.01(0.87)	0.88±0.08	0.93 ± 0.11
Marrow	0.85±0.13(0.84)	0.81±0.03	None
Sigmoid	0.82±0.02(0.81)	0.70 ± 0.17	0.60 ± 0.26
Rectum	0.87±0.15(0.83)	0.73 ± 0.18	0.83 ± 0.11
Spleen	0.91±0.01(0.90)	0.92±0.07	0.95 ± 0.03
SeminalVesicle	0.86±0.02(0.85)	0.78 ± 0.27	0.68 ± 0.15
Testis	0.87±0.03(0.84)	0.79 ± 0.16	0.63 ± 0.16
Prostate	0.87±0.02(0.85)	0.74 ± 0.12	0.85 ± 0.06
Ovid_L	$0.85\pm0.03(0.82)$	$0.65\pm0.03$	$0.39 \pm 0.17$
Ovid_R	$0.86\pm0.01(0.85)$	$0.66\pm0.01$	$0.43 \pm 0.15$
Bladder-Brt	$0.86\pm0.02(0.84)$	$0.82 \pm 0.23$	$0.91 \pm 0.12$
SmallIntestine-Brt	$0.87\pm0.04(0.85)$	$0.76 \pm 0.14$	$0.73 \pm 0.11$
Rectum-Brt	$0.86\pm0.03(0.84)$	$0.85 \pm 0.18$	$0.83 \pm 0.11$
Sigmoid-Brt	$0.79\pm0.02(0.78)$	$0.70 \pm 0.17$	$0.60 \pm 0.26$
SpinalCord	$0.93\pm0.01(0.92)$	$0.66 \pm 0.14$	$0.63\pm0.16$
Body	$0.98\pm0.05(0.96)$	$0.97\pm0.03$	None

Table 5: Clinical performance comparison (Peking Union Medical College Hospital)

{14}------------------------------------------------

{15}------------------------------------------------

{16}------------------------------------------------

Data format: Mean ± Std Dice coefficient (lower 95th percentile confidence bound based on normal distribution in parentheses)

Dice coefficients of LCTSC American public datasets compared to the predicate device are presented here:

Structure:	DeepContour	AI-Rad CAI-RadCompanion Organs RT(K221305)	ContourProtégéAI(K223774)
SpinalCord	0.92±0.02(0.91)	0.64±0.13	0.62 ± 0.21
Lung L	0.97±0.15(0.96)	0.90±0.13	0.95 ± 0.05
Lung R	0.98±0.06(0.98)	0.93±0.11	0.94 ± 0.08
Heart	0.92±0.11(0.90)	0.91±0.04	0.90 ± 0.04
Esophagus	0.89±0.13(0.86)	0.75±0.13	0.68 ± 0.19

Table 6: Clinical performance comparison (LCTSC American public datasets)

Data format: Mean ± Std Dice coefficient (lower 95th percentile confidence bound based on normal distribution in parentheses)

Dice coefficients of Pancreas-CT American public datasets compared to the predicate device are presented here:

Table 7: Clinical performance comparison (Pancreas-CT American public datasets)

{17}------------------------------------------------

Structure:	DeepContour	AI-Rad CAI-RadCompanion Organs RT(K221305)	ContourProtégéAI(K223774)
Spleen	0.90±0.05(0.88)	0.91±0.12	0.89 ± 0.08
Pancreas	0.85±0.03(0.83)	0.84±0.02	0.43 ± 0.25
Kidney_L	0.93±0.02(0.91)	0.84±0.03	0.92 ± 0.17
Esophagus	0.88±0.02(0.87)	0.80±0.06	0.70 ± 0.06
Liver	0.97±0.03(0.97)	0.85±0.13	0.92 ± 0.06
Stomach	0.85±0.02(0.84)	0.80±0.05	0.81 ± 0.17
Duodenum	0.86±0.02(0.85)	0.82±0.12	None

Data format: Mean ± Std Dice coefficient (lower 95th percentile confidence bound based on normal distribution in parentheses)

A comparison of the median Average Symmetric Surface Distance (ASSD) between the subject device and the predicate device is also performed and the results are comparable as shown in the following table. The subject device achieved a median ASSD of 0.95 in comparison to the predicate device achieving a median ASSD of 0.96 for existing organs.

Table 8: Rigid and deformable registration

	DeepContour		AI-Rad CAI-RadCompanion Organs RT(K221305)		Contour ProtégéAI(K223774)
	Median	95% CI(Bootstrap)	Median	95% CI(Bootstrap)	Median	95% CI(Bootstrap)
ASSD	0.95	[0.85,1.13]	0.96	[0.84,1.15]	0.95	[0.86,1.17]

8 CONCLUSION

DeepContour is believed to be substantially equivalent to predicate device in terms of its indications for use, technical characteristics, and overall performance. The information provided in this submission indicates the subject device is as safe, is as effective, and performs as well as predicate device. Therefore, it is in the opinion of Wisdom Technologies, Inc. that the medical device, DeepContour, is substantially equivalent to predicate device.

Regulation Number and Section

§ 892.2050 Medical image management and processing system.

(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).