Search Results

The OrthoNext™ Platform System is indicated for assisting healthcare professionals in preoperative planning of orthopedic surgery and post-operative planning of orthopedic treatment. The device allows for overlaying of Orthofix Product templates on radiological images, and includes tools for performing measurements on the image and for positioning the template. Clinical judgments and experience are required to properly use the software.

The subject OrthoNext™ Platform System is a web-based modular software system, indicated for assisting healthcare professionals in planning of orthopedic surgery and treatment both preoperatively and postoperatively, including deformity analysis and correction with several Orthofix products.
The subject software system is intended for use by Healthcare Professionals (HCP), with full awareness of the appropriate orthopedic procedures, in the operating theatre only.
The subject software functions are intended to inform the HCP on orthopedic procedure treatment planning when the Orthofix external or internal fixation systems are used. These functions are evidence-based tools that support HCP when considering treatment digital planning options for a patient. The software functions do not treat a patient or determine a patient's treatment.
The software enables the HCP to import radiological images, display 2D views (frontal and lateral) of the radiological images, overlay the positioning of the template and simulate the treatment plan option, and to generate parameters and/or measurements to be verified or adjusted by the HCP based on their clinical judgment.

Here's a breakdown of the acceptance criteria and the study details for the OrthoNext Platform System, based on the provided document:

Acceptance Criteria and Device Performance

The OrthoNext Platform System was evaluated for measurement accuracy and an AI/ML algorithm's performance for automatic marker detection.

Table of Acceptance Criteria and Reported Device Performance

• Mean error: 0.1 mm for linear measurements, 0.05 degrees for angular measurements.
• Mean percentage error: 0.24% (threshold 0.27% within 95th percentile) for linear measurements, 0.08% (threshold 1% within 95th percentile) for angular measurements.

For measurements made using mechanical axes:

• Mean error: 0.4 mm for linear measurements, 0.06 degrees for angular measurements.
• Mean percentage error: 1.73% (threshold 16.67% within 95th percentile) for linear measurements, 0.28% (threshold 1.27% within 95th percentile) for angular measurements. |
  | Sensitivity of Measurements | Not explicitly stated as a separate acceptance criterion, but device performance reported. | 1 mm for linear measurements, 1 degree for angular measurements. |
  | Specificity of Measurements | Not explicitly applicable as a direct acceptance criterion due to manual nature. | The device requires active user involvement for each measurement, relying on user expertise. |
  | AI/ML Algorithm Accuracy | Not explicitly stated as an isolated acceptance criterion, but reported. | 0.8 |
  | AI/ML Algorithm Specificity | Goal: zero false positives in the test set, resulting in a precision of 1. | 1 (Precision) |
  | AI/ML Algorithm Sensitivity | Not explicitly stated as an isolated acceptance criterion, but reported. | 0.75 (TPR/Recall) |
  | AI/ML Algorithm FPR | Not explicitly stated as an isolated acceptance criterion, but reported. | 0 |
  | AI/ML Algorithm F1 Score | Not explicitly stated as an isolated acceptance criterion, but reported. | 0.86 |
  | AI/ML Algorithm Intersection over Union (IoU) | Not explicitly stated as a direct acceptance criterion, but reported. | 79% |
  | AI/ML Algorithm Center MAE | Not explicitly stated as a direct acceptance criterion, but reported. | 4.83 px |
  | AI/ML Algorithm Center MAPE | Not explicitly stated as a direct acceptance criterion, but reported. | 0.38% |
  | AI/ML Algorithm Radius MAE | Not explicitly stated as a direct acceptance criterion, but reported. | 1.29 px |
  | AI/ML Algorithm Radius MAPE | Satisfying 3% Radius MAPE. | 3% Radius MAPE, contributing to an error of less than 1 mm. |

Study Details: Magnification Marker Detection Algorithm

The document focuses on the performance testing for the magnification marker detection algorithm, which is an AI/ML component of the OrthoNext Platform System.

1. Sample size used for the test set and the data provenance:

   • Sample Size: 1000 X-ray images. Of these, 800 images depicted a magnification marker, and 200 images did not.
   • Data Provenance: The test set consisted of real X-ray images. The document does not specify the country of origin of the data or whether it was retrospective or prospective. It only states that these images were "not used during training, ensuring independence."

2. Number of experts used to establish the ground truth for the test set and the qualifications of those experts:

   • Number of Experts: Not explicitly stated. The document mentions "qualified personnel" were used for the truthing process.
   • Qualifications of Experts: "Qualified personnel" are mentioned, but specific qualifications (e.g., radiologist with X years of experience) are not provided.

3. Adjudication method for the test set:

   • The document describes a "review and discard process" implemented to ensure the quality of the annotations, but it does not specify an adjudication method like "2+1" or "3+1". This suggests a quality control step for annotations rather than a formal consensus process among multiple readers for ground truth establishment.

4. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:

   • No MRMC comparative effectiveness study was done or reported in this document. The study evaluates the standalone performance of the AI/ML algorithm for magnification marker detection, not its impact on human reader performance.

5. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:

   • Yes, a standalone performance evaluation of the AI/ML algorithm for magnification marker detection was explicitly done and reported. The performance metrics (Precision, Accuracy, TPR/Recall, FPR, F1 Score, IoU, MAE, MAPE) are all indicative of standalone algorithm performance.

6. The type of ground truth used:

   • Expert Consensus (Annotation): The ground truth was established through a "truthing process... conducted by qualified personnel, who carefully overlaid a circular shape on each magnification marker using annotation software."

7. The sample size for the training set:

   • Training Set Composition: 1500 X-ray images with random areas depicting magnification markers.
   • These 1500 magnification markers were randomly overlaid on top of 4000 X-ray images.
   • Image augmentation techniques (random rotations, brightness adjustments) generated 24,000 unique X-ray images for training.

8. How the ground truth for the training set was established:

   • The training set involved synthetic images. Specifically, "1500 X-ray images with random areas depicting magnification markers" were created, and these markers were "randomly overlaid on top of 4000 X-ray images." Image augmentation was then applied. This suggests that the ground truth for the training set markers was generated as part of the synthetic image creation process (i.e., the location and characteristics of the overlaid markers were known by design). A "hash check" was used to ensure the uniqueness of these synthetic images.

Ask a specific question about this device

The OrthoNext ™ Platform system is indicated for assisting healthcare professionals in preoperative planning of orthopedic surgery. The device allows for overlaying of Orthofix Product templates on radiological images, and includes tools for performing measurements on the image and for positioning the template. Clinical judgments and experience are required to properly use the software.
The OrthoNext ™ Platform system is not to be used for mammography.

The OrthoNextTM Platform is a web-based platform module system, to allow surgeons to evaluate digital images while performing various pre-operative treatment planning, evaluation of images and post-operative treatment planning. This software application enables surgeons to import radiological images, display various 2D views of the images, overlays the positioning of the Orthofix devices template and simulate the treatment options, generate parameters and/or measurements to be verified or adjusted by the surgeons based on their clinical judgment.

The provided document is a 510(k) summary for the OrthoNext™ Platform System. This type of submission generally focuses on demonstrating substantial equivalence to a predicate device rather than presenting a full clinical study with specific acceptance criteria and detailed performance metrics as would be found in a PMA or de novo submission.

Based on the document, here's what can be extracted and what is not explicitly provided:

1. A table of acceptance criteria and the reported device performance

The document does not explicitly state quantitative acceptance criteria or detailed reported device performance in a table format. Instead, it relies on demonstrating equivalence to the predicate device and successful non-clinical testing. The "Performance Analysis" section states: "Subject device has similar configuration, and operating principle as the predicate device. Non-clinical software testing on operative treatment planning of orthopedic surgery using OrthoNext ™ Platform system produces results comparable to planning using acetate overlays but with the additional advantages of digital planning and simulations including ease of use, library, case documentation, access to a wider arrange of tools, and secure accessibility."

The "Conclusion" section indirectly describes the "performance" by stating that "The successful non-clinical testing demonstrates the safety and effectiveness of the OrthoNext ™ Platform system when used for the defined indications for use and demonstrates that the subject device, for which this Traditional 510(k) is submitted, performs as well as or better than the legally marketed predicate devices."

The types of testing performed are listed: "Unit, System/Integration and Acceptance test levels. Testing included also security, negative testing, error message handling, stress testing, platform testing, workflow testing, functional testing, multi-user/external access testing, data integrity testing, compatibility testing, load testing, regression testing, and hazard mitigation testing." However, specific acceptance criteria for each of these tests are not provided.

2. Sample size used for the test set and the data provenance (e.g. country of origin of the data, retrospective or prospective)

This information is not provided in the document. The filing focuses on non-clinical software testing: "Non-clinical software testing on operative treatment planning of orthopedic surgery using OrthoNext ™ Platform system produces results comparable to planning using acetate overlays..." The nature of this "testing" doesn't seem to involve a "test set" of patient data in the typical sense of a clinical trial.

3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts (e.g. radiologist with 10 years of experience)

This information is not provided. The document mentions that the device is "indicated for assisting healthcare professionals in preoperative planning of orthopedic surgery" and that "Clinical judgments and experience are required to properly use the software." However, it does not detail any expert review process for a test set or ground truth establishment.

4. Adjudication method (e.g. 2+1, 3+1, none) for the test set

This information is not provided.

5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance

An MRMC comparative effectiveness study was not performed or at least not reported in this 510(k) summary. The document states: "The review of clinical literatures on similar devices support the clinical performance of the Subject device with no additional clinical data." This indicates that no new clinical study (like an MRMC) was conducted for this submission.

6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done

A standalone performance study of the algorithm alone (without human-in-the-loop) was not explicitly described with specific performance metrics. The nature of the device, which "assists healthcare professionals" and requires "clinical judgments and experience," implies a human-in-the-loop interaction as its primary mode of use. The software testing mentioned is "non-clinical software testing," which would assess the software's functionality and accuracy in performing its intended tasks (e.g., measurements, template overlay) rather than a diagnostic standalone performance.

7. The type of ground truth used (expert consensus, pathology, outcomes data, etc)

For the non-clinical software testing, the document suggests the "ground truth" or reference for comparison was "planning using acetate overlays." This implies that the accuracy of the software's measurements and template positioning was compared against established practices using physical overlays. No mention of expert consensus, pathology, or outcomes data for establishing ground truth is made for the device's performance evaluation in this document.

8. The sample size for the training set

This information is not provided. The document describes "non-clinical software testing," and given the nature of the device (a planning and measurement tool, not an AI for diagnosis), a "training set" in the context of machine learning model development is most likely not applicable or not disclosed. The device performs functions like overlaying templates and performing measurements, which are rule-based software operations rather than typically requiring a "training set" in the AI sense.

9. How the ground truth for the training set was established

As a training set is likely not applicable or not disclosed, the method for establishing its ground truth is also not provided.

Ask a specific question about this device

Ask a specific question about this device