(85 days)
Ortho-CMS is an orthopaedic analysis software tool. It has been developed to optimize preoperative planning through digital prosthesis templating and to enable preoperative and postoperative measurements in digital or digitized X-Ray images. Ortho-CMS software is meant solely for use by trained medical personnel.
The intended purposes of Ortho-CMS are:
- Displaying of X-Ray images
- Supporting planning of joint replacement operations
- Supporting clinical diagnoses on implant loosening
- Enabling preoperative and postoperative measurements for clinical and research purposes
In orthopaedics, radiographs are used to diagnose and analyse various kinds of orthopaedic disorders, generative joint conditions, and bone fractures, and to evaluate orthopaedic treatments such as osteotomies and total joint arthroplasty. These arthroplasties need to be evaluated in order to assess the quality of the procedure. Measurements on radiographs of endoprostheses include the assessment of radiolucent lines around the prosthesis, bone growth or bone resorption, position of the prosthesis, motion of the prosthesis relative to the surrounding bone, and the determination of wear of the polyethylene components.
Since the measurements on radiographs are commonly performed manually, considerable intra-observer and inter-observer variation exists. Automation of the measurements might increase the objectivity and speed of the analysis, and decrease the variation of the results. In radiology, digital roentgen imaging techniques are increasingly being used over plain film radiographs. The digital roentgen images (DICOM CR or DX) are easily accessible from a medical picture archive (PACS) through a network connection.
Ortho-CMS supports the radiologist by facilitating the diagnosis of orthopaedic digital images and allows the orthopaedic specialist to perform a pre-surgical planning and a post-surgical evaluation on these images. Further, Ortho-CMS can be deployed as a measurement tool for core-labs that focus on quality assessment of orthopaedic implants or it can be used by bone centres that need to make measurements in donor bone images for joints replacement purposes.
Here's a breakdown of the acceptance criteria and study information for the Ortho-CMS device based on the provided text:
It's important to note that the provided text is a 510(k) summary for a medical device submitted in 2004. As such, it focuses on demonstrating substantial equivalence to a predicate device rather than presenting a detailed, statistically robust study design with explicit acceptance criteria and detailed performance metrics as might be seen for novel technologies today. The document heavily emphasizes the safety and effectiveness through risk management, verification, and validation, rather than a specific comparative effectiveness study with human readers or standalone AI performance.
1. Table of Acceptance Criteria and Reported Device Performance
The provided 510(k) summary does not explicitly state specific quantitative acceptance criteria or detailed device performance metrics in the way one might expect for a modern AI-driven device. Instead, the "acceptance criteria" are implicitly tied to demonstrating substantial equivalence and ensuring safety and effectiveness through general software development lifecycle processes.
The document discusses the limitations of manual measurements (intra-observer and inter-observer variation) and suggests that automation might increase objectivity, speed, and decrease variation. However, it doesn't provide threshold values for these improvements.
| Acceptance Criteria (Implicit from Document) | Reported Device Performance |
|---|---|
| Safety and Effectiveness: Demonstrate that Ortho-CMS is safe and effective for its intended use and does not introduce new hazards compared to predicate devices. | "It is the opinion of Medis medical imaging systems bv that Ortho-CMS is safe and potential hazards are controlled by a risk management plan for the software development process... The software package Ortho-CMS itself will not have any adverse effects on health." "The use of Ortho-CMS software does not change the intended use of X-ray equipment in practice, nor does the use of software result in any new potential hazards." |
| Substantial Equivalence: Demonstrate substantial equivalence to predicate devices (Meridian Technique Ltd. 6132401 "Orthoview™" and eFilm Medical Inc., K020995 "eFilm" Workstation™) using the same technological technique for the same intended use. | "Ortho-CMS is substantially equivalent to the Predicate Devices... using the same technological technique for the same intended use." This is the core claim of the 510(k) submission and is affirmed by the FDA's decision to grant 510(k) clearance. No specific metrics from a comparative study are detailed, but rather the general capabilities (displaying, planning, supporting diagnosis, enabling measurements) are considered similar. |
| Accuracy (Implicit): Enable accurate preoperative and postoperative measurements, implying the results are reliable enough for clinical and research purposes. | The document states it allows specialists "to perform a pre-surgical planning and a post-surgical evaluation." It also states "The analyses results will be interpreted by the operator, who can choose to accept or reject the tools results." This implies the device provides information that is verifiable and useful, but no quantitative accuracy metrics (e.g., error margins, inter/intra-observer variability reduction) are provided in this summary. The mention of "Evaluations by hospitals and literature (See Appendix F) support this statement" suggests that external validation was part of the submission, but the details are not in this summary. |
| Functionality: Perform stated functions: Displaying X-Ray images, supporting planning of joint replacement operations, supporting clinical diagnoses on implant loosening, and enabling preoperative/postoperative measurements. | The device description outlines these functions, and its clearance implies these functions were validated as working. |
2. Sample Size Used for the Test Set and Data Provenance
The provided 510(k) summary does not specify a numerical sample size for a test set. It mentions "verification and validation tests (See Appendix E)" and "Evaluations by hospitals and literature (See Appendix F)." This suggests that testing was conducted, but the details of the dataset used (number of cases, data provenance, etc.) are not included in this public summary. Given the year (2004) and the nature of medical device submissions focusing on substantial equivalence for software tools, a large, independent test set with explicit details might not have been a mandatory requirement to the same extent as for a novel diagnostic algorithm today.
3. Number of Experts Used to Establish Ground Truth and Qualifications
The document does not specify the number or qualifications of experts used to establish ground truth for any test set. It states the software "supports the radiologist" and "allows the orthopaedic specialist" to perform tasks, and that "The analyses results will be interpreted by the operator, who can choose to accept or reject the tools results." This implies that medical professionals would be involved in evaluating the output, but details on how ground truth was established for a formal validation study are absent from this summary.
4. Adjudication Method for the Test Set
The provided text does not describe any specific adjudication method (e.g., 2+1, 3+1, none) for a test set.
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
The 510(k) summary does not mention a Multi-Reader Multi-Case (MRMC) comparative effectiveness study demonstrating how much human readers improve with AI vs. without AI assistance. The device is presented as a tool to facilitate measurements and planning, rather than a diagnostic AI that provides independent interpretations or directly augments human reader performance in a controlled study comparing assisted vs. unassisted reading. The focus is on increasing objectivity and speed, and decreasing variation, but not a quantified improvement in diagnostic accuracy via an MRMC study.
6. Standalone (Algorithm Only) Performance Study
The document describes Ortho-CMS as an "orthopaedic analysis software tool" that "supports the radiologist" and "allows the orthopaedic specialist" to perform measurements and evaluations. It explicitly states, "The analyses results will be interpreted by the operator, who can choose to accept or reject the tools results." This clearly indicates that the device is not intended to operate in a standalone (algorithm only) manner for making clinical decisions without human oversight. Its role is as a measurement and planning aid for trained medical personnel. Therefore, a standalone performance study as understood for fully automated diagnostic AI would not be applicable or expected for this device.
7. Type of Ground Truth Used
The document does not explicitly state the type of ground truth used for any validation studies. Given the device's function involves making measurements and supporting clinical decisions on images, it's plausible that ground truth would involve:
- Expert Consensus/Manual Measurements: Manual measurements performed by multiple experienced orthopaedic specialists or radiologists, potentially with reconciliation, could have served as a reference standard.
- Pathology/Outcomes Data: For supporting diagnoses on implant loosening, potentially clinical outcomes or follow-up pathology could have been considered, but this is speculative given the limited information.
The mention of "Evaluations by hospitals and literature (See Appendix F)" suggests that real-world clinical data and established medical knowledge were used for validation, but the specifics are not detailed.
8. Sample Size for the Training Set
The provided text does not specify a sample size for the training set. The Ortho-CMS, described in 2004, is more likely a rule-based software tool or one that employs traditional image processing algorithms rather than a machine learning or AI model in the modern sense that requires a "training set" for model parameters. If any "training" occurred, it would likely refer to internal development and calibration using a limited set of representative cases, rather than a large, diverse dataset for supervised learning.
9. How the Ground Truth for the Training Set Was Established
As the document does not specify a training set or explicitly describe a machine learning model, it also does not detail how ground truth for such a set was established. If traditional image processing techniques were "trained" or calibrated, it would have involved internal testing and parameter tuning, likely based on expert-annotated or known-measurement cases developed by the manufacturer.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).