The OrthoVis Web Portal is intended for use as a software interface and for the transfer of imaging information from a medical scanner such as a CT scanner. It is also intended as pre-operative software for simulating implant placement and surgical treatment options. The OrthoVis Web Portal is intended for use with the Glenoid Intelligent Reusable Instrument system and with the OrthoVis Preoperative Plan. It is also intended for use with the Arthrex and DePuy shoulder replacement implant systems listed below.

· Arthrex Univers Apex
· Arthrex Univers II
· Arthrex Univers Revers
· DePuy Global AP
DePuy Global StepTech
· DePuy Delta Xtend Reverse

The Glenoid Intelligent Reusable Instrument System ("Glenoid IRIS") is a patient specific manual instrument system intended for use to facilitate preoperative planning and intraoperative placement of the central glenoid guide pin used in the preparation of the glenoid in total shoulder systems that utilize a central guide pin for preparing the glenoid to receive the glenoid implant.

Device Description

The OrthoVis Web Portal is composed of software intended for use to facilitate upload of medical images and preoperative planning and plan approval of placement and orientation of Arthrex and DePuy total shoulder replacement systems. Each surgeon user's uploaded images are grouped into cases and associated with that user's profile. Uploaded images can be downloaded from the portal by COS technicians and used to create preoperative plans (see 510(k) K13367 and K151568). The user is then able to login to the OrthoVis Web Portal to review the preoperative plan and either approve or modify the location and/or orientation of the shoulder replacement component and then approved plan is then available to download by COS technicians for further preoperative planning production (see 510(k) K123122 and K142072) and for download by the user/surgeon.

AI/ML Overview

The provided document does not contain explicit acceptance criteria or a study that proves the device meets specific performance criteria. The document is a 510(k) premarket notification letter from the FDA to Custom Orthopaedics Solutions, Incorporated for their OrthoVis Web Portal.

The letter focuses on the FDA's determination of substantial equivalence to legally marketed predicate devices, rather than a clinical performance study with predefined acceptance criteria for the device itself. The device, OrthoVis Web Portal, is described as a software interface for transferring imaging information and pre-operative simulation, essentially a communication tool between surgeons and technicians.

However, the "NON-CLINICAL TESTING" section (page 6) describes the testing performed to demonstrate substantial equivalence, which can be interpreted as fulfilling the requirements for market clearance for this type of device.

Here's a breakdown based on the information provided, recognizing that it's not a typical clinical performance study:

1. A table of acceptance criteria and the reported device performance

The document does not present a table of acceptance criteria with corresponding device performance for a typical clinical study (e.g., sensitivity, specificity, accuracy). Instead, the "performance" demonstrated relates to software functionality and safety in comparison to existing methods.

Acceptance Criteria (Inferred from non-clinical testing)	Reported Device Performance (Summary of testing results)
Software Verification and Validation: Software functions correctly, securely, and reliably as intended. Ensures data integrity and proper operation.	Comprehensive coverage of application requirements demonstrated through automated unit tests, automatic integration testing, and manual testing. Constraints and protections verified for correct results and error handling.
Dimensional Validation: Accuracy of reading and display of CT scans and derived measurements is maintained.	The same code responsible for reading and display of CT scans is used as in already cleared OrthoVis Shoulder software (K123122, K133367), implying maintained accuracy.
Simulated Use Comparison: The OrthoVis Web Portal process is comparable or superior to existing manual processes (CD mailer/Go-To-Meeting/Email) for surgeon interaction, planning, and approval.	Surgeons with experience in the old process were surveyed to compare their experience with the Web Portal process. While specific results are not detailed, the implication is that the comparison was favorable for substantial equivalence.
Security and Data Handling: Secure authentication, confidential communication, encrypted storage, and appropriate notifications.	Secure portal login with hashed password, HTTPS/SSH network communication, SFTP for local office uploads/downloads, encrypted database on server, email for status change (no PHI).

2. Sample size used for the test set and the data provenance

Software Verification and Validation:
- Test Set Sample Size: Not explicitly stated as a number of "cases" or "patients." This testing involved numerous automated unit tests and integration tests, as well as manual tests covering various software functionalities and scenarios.
- Data Provenance: Not applicable in the traditional sense of patient data. The "data" would be test cases and simulated inputs designed to exercise the software's capabilities and constraints.
Dimensional Validation:
- Test Set Sample Size: Not specified. This validation likely involved testing the software's rendering and measurement capabilities rather than a patient dataset.
- Data Provenance: Not applicable.
Simulated Use Comparison:
- Test Set Sample Size: "Surgeons with experience" were asked to participate in a survey. The exact number of surgeons is not specified.
- Data Provenance: Prospective, as surgeons were asked to compare their experiences with the old and new processes. The origin of the data is implicit as part of the manufacturer's testing in the US.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts

For the software verification and dimensional validation, "ground truth" would be the expected correct software behavior and measurement accuracy, established by software engineers and potentially medical imaging experts involved in the development and previous clearances of OrthoVis. Specific numbers and qualifications are not provided.
For the simulated use comparison, the "experts" were the participating surgeons. Their qualifications are stated as "Surgeons with experience in using the OrthoVis Preoperative Plan via the CD Mailer/Go-To-Meeting/Email process." No specific number is given.

4. Adjudication method for the test set

Not applicable in the context of a clinical performance study with human reviewers.
Software testing would involve test engineers and potentially QA personnel verifying test outcomes against expected results.
For the surgeon survey, presumably, the survey responses were collected and analyzed directly, not adjudicated among experts to establish a "ground truth."

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

No, an MRMC comparative effectiveness study was not done.
The OrthoVis Web Portal is described as a software interface for communication and pre-operative simulation, not an AI-assisted diagnostic or interpretative tool. Therefore, the concept of improving human readers with AI assistance does not apply directly to this device's intended use according to the provided text. The "Simulated Use Comparison" was about process efficiency and user experience, not diagnostic accuracy.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

The device is a human-in-the-loop system by design, facilitating communication and planning between surgeons and technicians. The software itself is designed to display, transfer, and allow modification of plans by human users.
The "standalone" performance would be related to its software functionality (e.g., ability to upload, store, display data correctly), which was addressed by the software verification and validation. However, this is not a clinical "standalone" performance in the sense of making a medical decision without human input.

7. The type of ground truth used

Software Verification and Validation: Expected software behavior, functional requirements, and logical outputs.
Dimensional Validation: Known measurement standards and calculations, validated against established (previously cleared) software code.
Simulated Use Comparison: User experience and feedback from surgeons, evaluated against the existing manual processes.

8. The sample size for the training set

Not applicable. The OrthoVis Web Portal is a communication and simulation tool, not a machine learning or AI algorithm that requires a training set of data.

9. How the ground truth for the training set was established

Not applicable, as there is no training set for this device.

Summary

{0}------------------------------------------------

Image /page/0/Picture/1 description: The image shows the logo for the U.S. Department of Health & Human Services. The logo consists of a circular seal with the text "DEPARTMENT OF HEALTH & HUMAN SERVICES - USA" around the perimeter. Inside the circle is a stylized image of three faces in profile, stacked on top of each other, representing the department's focus on people and health.

Food and Drug Administration 10903 New Hampshire Avenue Document Control Center - WO66-G609 Silver Spring, MD 20993-0002

Custom Orthopaedics Solutions, Incorporated Mr. Keith Grafmeyer Product Engineer 10000 Cedar Avenue Cleveland, Ohio 44106

October 2, 2015

Re: K151501

Trade/Device Name: OrthoVis Web Portal Regulation Number: 21 CFR 892.2050 Regulation Name: Picture archiving and communications system Regulatory Class: Class II Product Code: LLZ, KWS, PHX Dated: May 26, 2015 Received: June 4, 2015

Dear Mr. Grafmeyer:

We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food. Drug. and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device-related adverse events) (21 CFR 803); good manufacturing practice requirements as set

{1}------------------------------------------------

forth in the quality systems (QS) regulation (21 CFR Part 820); and if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.

If you desire specific advice for your device on our labeling regulation (21 CFR Part 801), please contact the Division of Industry and Consumer Education at its toll-free number (800) 638-2041 or (301) 796-7100 or at its Internet address

http://www.fda.gov/MedicalDevices/Resourcesfor You/Industry/default.htm. Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to

http://www.fda.gov/MedicalDevices/Safety/ReportaProblem/default.htm for the CDRH's Office of Surveillance and Biometrics/Division of Postmarket Surveillance.

You may obtain other general information on your responsibilities under the Act from the Division of Industry and Consumer Education at its toll-free number (800) 638-2041 or (301) 796-7100 or at its Internet address

http://www.fda.gov/MedicalDevices/ResourcesforYou/Industry/default.htm.

Sincerely yours,

Mark N. Melkerson -S

Mark N. Melkerson Director Division of Orthopedic Devices Office of Device Evaluation Center for Devices and Radiological Health

Enclosure

{2}------------------------------------------------

Indications for Use

510(k) Number (if known) K151501

Device Name OrthoVis Web Portal

Indications for Use (Describe)

· Arthrex Univers Apex
· Arthrex Univers II
· Arthrex Univers Revers
· DePuy Global AP
DePuy Global StepTech
· DePuy Delta Xtend Reverse

Type of Use (Select one or both, as applicable)
X Prescription Use (Part 21 CFR 801 Subpart D)	Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

{3}------------------------------------------------

510(k)

Summary

510(k) SPONSOR / MANUFACTURER:	Custom Orthopaedic Solutions, Inc.10000 Cedar AvenueCleveland, Ohio44106
CONTACT PERSON:	Keith GrafmeyerProduct EngineerCustom Orthopaedic Solutions10000 Cedar Ave.Cleveland, OH 44106Tele: (216) 445 -3403Email: kgrafmeyer@customorthopaedics.com
TRADE NAME:	OrthoVis Web Portal
DATE PREPARED:	26-May -2015
DATE REVISED:	08-Aug-2015
COMMON NAMES:	-Image processing system and preoperative software forsimulating / evaluating implant placement and surgicaltreatment options-Prosthesis, Shoulder, Semi-Constrained, Metal/PolymerCemented-Shoulder Prosthesis, Reverse Configuration

Product	Product Code	Regulation and Classification Name	Device Class
OrthoVis Web Portal	LLZ	21 CFR 892.2050Picture Archiving andCommunications System	II
Secondary Product Names and Codes
OrthoVis Web Portal -Accesory to Glenoid IRIS	KWS	21 CFR 888.3660	II
OrthoVis Web Portal -Accesory to Glenoid IRIS	PHX	21 CFR 888.3660	II

{4}------------------------------------------------

Predicate Devices:

K133367: OrthoVis Preoperative Plan

K142072: Glenoid Intelligent Reusable Instrument System (Glenoid IRIS)

DEVICE DESCRIPTION

INTENDED USE AND INDICATIONS FOR USE

The OrthoVis Web Portal is intended for use as a software interface and for the transfer of imaging information from a medical scanner such as a CT scanner. It is also intended as pre-operative software for simulating / evaluating implant placement and surgical treatment options. The OrthoVis Web Portal is intended for use with the Glenoid Intelligent Reusable Instrument system and with the OrthoVis Preoperative Plan. It is also intended for use with the Arthrex and DePuy shoulder replacement implant systems listed below.

Arthrex Univers Apex
· Arthrex Univers II
Arthrex Univers Revers
DePuy Global AP
DePuy Global StepTech
DePuy Delta Xtend Reverse

BASIS OF SUBSTANTIAL EQUIVALENCE

The OrthoVis Web Portal has nearly the same intended use and indications for use as the OrthoVis Preoperative Plan (K133367) and the Glenoid Intelligent Reusable Instrument System (Glenoid IRIS) (K142072). The main difference between the original OrthoVis software and the OrthoVis Web Portal is that the Web Portal facilitates the communication between the COS planning technician and surgeon.

Technological Comparison to Predicate

The device described here is intended to collect images and preferences from a surgeon relative to a case, move those images to an in-house workstation for processing, and to allow those plans to be displayed for a surgeon to either request adjustments to the plan or approve the plan, followed by distribution of the final plan materials.

These functions were previously done by a largely manual process involving secure email and physical mail. This table compares the new technology to the old process, describing the use of the web portal, SFTP transfer, and email to do these functions.

{5}------------------------------------------------

(Note that email is only used for indication of status change, and does not ever contain PHI.)

Function	Predicates	Subject
Surgeon authentication	Phone conversation.	Secure portal login with hashedpassword.
Confidential communication	Postal mail, phone call, or secure email.	1. HTTPS/SSH networkcommunication for all web portal andsurgeon upload/downloads2. STFP for local officeuploads/downloads.3. Email for status change notification(e.g. plan ready, etc.)
Create case	Phone conversation, file on localcomputer.	Web portal case creation page.
Store case	Local files	Encrypted database on server
Submit images	CD/DVD in postal mail, files put onlocal server	1. Files uploaded via HTTP web portalupload page to protected directory.2. Local office is notified that files areuploaded via case status change.3. Files are then downloaded via SFTPsecure protocol from network serverto local server.
Create surgical plan	Planning software on localworkstation.	Planning software on localworkstation.
Preview surgical plan with surgeon	Surgeon uses commercial screensharing software (i.e. PC Anywhere) toview local workstation screen duringphone conversation.	1. Files uploaded by office to webportal protected directory via SFTPsecure protocol.2. Web portal software imports planinformation into web database, andimports images into web image store.3. Web portal notifies surgeon of planavailability via case status changeemail.4. Surgeon logs into portal.5. Surgeon views plan interactively in3D on plan view page.
Surgeon requests adjustments to plan	During phone conversation, surgeonverbally requests adjustments and seeschanges on shared screen.	1. Surgeon uses web controls to change3D image on plan view page.2. Surgeon indicates on plan view pagevia web controls that changes arerequested.3. Local office is notified via email ofstatus change.
Surgeon approves plan	During phone conversation or secureemail, surgeon verbally approves plan.	1. Surgeon views plan on plan viewpage.2. Surgeon indicates on plan via webcontrols that the plan is approved.3. Local office is notified via email ofstatus change.
Function	Predicates	Subject
Surgeon gets final copy of approvedplan	The Approved Preoperative Plan iscreated in OrthoVis and sent in a PDFfile via secure email.	1. Local office prepares PDF file onOrthoVis and uploads via STFP secureprotocol.2. Surgeon is notified via email ofstatus change.3. Surgeon logs into web portal.4. Surgeon downloads PDF containingfinal approved plan.
Surgeon is reminded of plan	Reminder phone call.	Periodic generation of reminder emailsabout status change until downloadoccurs.

{6}------------------------------------------------

NON-CLINICAL TESTING

The following testing was performed to demonstrate substantial equivalency of the OrthoVis Web Portal to the predicate device

Software verification and validation
Dimensional validation
Simulated Use Comparison between OrthoVis Web Portal Process and CD Mailer/Go-To-Meeting/Email Process

Software Verification and Validation

Software Verification and Validation was performed in order to establish substantial equivalence to the predicate device.

Dimensional Validation

Dimensional validation of the OrthoVis system was performed (see Exhibit A). The same code responsible for reading and display of CT scans is used in the cleared OrthoVis Shoulder software (K123122, and K133367) as is used in the OrthoVis Web Portal software.

Simulated Use Comparison between OrthoVis Web Portal Process and CD Mailer/Go-To-Meeting/Email Process

Surgeons with experience in using the OrthoVis Preoperative Plan via the CD Mailer/Go-To-Meeting/Email process were asked to compare that experience with the OrthoVis Web Portal Process and fill out a survey based on the two experiences.

CLINICAL TESTING

Clinical testing was not necessary to determine substantial equivalence between the OrthoVis Web Portal and the predicate device.

NON-CLINICAL TESTING SUMMARY AND CONCLUSION

The testing done for the web portal consisted of three parts:

The first part consisted of automated unit tests. -
-The second part used an automatic integration-testing tool to test the Web Portal user interface.
-The third part manually tested remaining errors.

The sum of our automated and manual tests described below provide comprehensive coverage of the application requirements, and proved that our device is as safe and effective as the predicate device.

We tested each constraint and protection using an automated testing framework that created test cases that followed the constraint rules to verify correct results, and by creating test cases that violate these

{7}------------------------------------------------

rules to verify that the constraints and error messages were working correctly.

Operational behavior of the user interface (those part of the requirements that could be observed by a user of the application) was tested using the WebDriver testing tool (which is now a W3C standard) to drive the user interface in a large number of scenarios. By "robotically" operating the user interface, the application was tested exactly as if a user (using Chrome or Firefox) was executing a series of tests. Automated tests were made from prepared data to test each requirement. By using rigorous automation, tests can be repeated many times when moving to each specific environment, to test servers or to run regression tests when features are added.

Since some errors are difficult to mitigate with automated testing, we verified, by manual code inspection, that error handlers are in place and operational. We completed manual testing that was executed as usual during each testing phase.

Regulation Number and Section

§ 892.2050 Medical image management and processing system.

(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).