(106 days)
SMART Bun-Yo-Matic CT software is to be used by orthopaedic healthcare professionals for diagnosis and surgical planning in a hospital or clinic environment. The medical imaging type intended to be used as the input of the software is Computed Tomography (CT).
SMART Bun-Yo-Matic CT software provides:
· Visualization report of the three-dimensional mathematical models of the anatomical structures of foot and ankle and three-dimensional models of orthopaedic fixation devices,
· Measurement templates containing radiographic measures of foot and ankle,
· Surgical planning application for visualization of foot and ankle anatomical three-dimensional structures, radiographic measures, and surgical instrument parameters.
The visualization report containing the measurements can be used for the diagnosis of orthopaedic healthcare conditions. The surgical planning application containing the visualizations of the threedimensional structural models, orthopaedic fixation device models and surgical instrument parameters combined with the measurements can be used for the planning of treatments and operations to correct orthopaedic healthcare conditions of foot and ankle.
The SMART Bun-Yo-Matic CT device is an automatic software tool that segments foot and ankle bones from computed tomography (CT) images and provides a case report showing images of a 3D model of the segmented structures with pre-operative and post-correction measurements. The correction is for hallux valgus through a Lapidus Arthrodesis procedure. The case report also provides parameters of an orthopedic surgical instrument and an example of an implant construct for the procedure.
The device includes machine learning derived outputs. Details on the validation are summarized below. The testing for 82 CT image series presented 100% correctly identified bones of foot and ankle. The existence of metal was identified correctly for 98.8% of the images (specificity 98%, sensitivity 100%).
Here's an analysis of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| Bone Identification | 100% correctly identified bones of foot and ankle. |
| Metal Identification (Specificity) | 98% (accuracy 98.8%) |
| Metal Identification (Sensitivity) | 100% (accuracy 98.8%) |
| Model Conformance (3D models) | 95% within 1.0mm distance to reference model. |
| Angular Measurements (for surgical planning) | 2.0 degrees standard deviation. |
| Angular Measurements (estimated correction) | ±1 degree. |
| Distance Measurements (estimated correction) | ±1.0 mm. |
Study Details
2. Sample size used for the test set and data provenance:
- Test Set Sample Size: 82 CT image studies.
- Data Provenance: The CT image series were collected from various sites across the USA and Europe, with a minimum of 50% of the images originating from the USA.
- Patient Demographics: Patients of different ages and racial groups, with a minimum of 35% male/female within each dataset. Mean age approximately 47 years (SD 15 years). Representatives from White, (Non-)Hispanic, African American, and Native racial groups.
- Clinical Conditions: Balanced in terms of subjects with different foot alignment, demographics, imaging devices, and with subjects from clinical subgroups ranging from control/normal feet (44% of test data) to pre-/post-operative clinical conditions such as Hallux Valgus, Progressive Collapsing Foot Deformity, fractures, or with metal implants (40% of test data).
3. Number of experts used to establish the ground truth for the test set and their qualifications:
- Number of Experts: Three (3).
- Qualifications: U.S. Orthopedic surgeons.
4. Adjudication method for the test set:
- Adjudication Method: Majority vote. Two same responses were required from the three experts to establish a ground truth for the presence of a bone and metal in each DICOM series.
5. If a multi-reader multi-case (MRMC) comparative effectiveness study was done:
- No MRMC comparative effectiveness study was explicitly mentioned or detailed in the provided text. The study described focuses on standalone algorithm performance against expert-established ground truth.
6. If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance assessment was conducted for the SMART Bun-Yo-Matic CT software. The reported performance metrics (100% bone identification, 98.8% metal identification, model conformance, and measurement accuracy) refer to the algorithm's performance without human intervention in the interpretation phase.
7. The type of ground truth used:
- Expert consensus (majority vote of three U.S. Orthopedic surgeons).
8. The sample size for the training set:
- Bone identification algorithm: 145 CT image studies.
- Metal identification algorithm: 130 CT image studies.
9. How the ground truth for the training set was established:
- The document states that "The AI algorithm for bone identification was developed using 145 CT image studies and metal identification was developed using 130 CT image studies." It does not explicitly detail the method for establishing ground truth for the training data, beyond implying it was part of the algorithm development process. However, given the ground truth methodology for the test set, it is highly probable that a similar expert review or gold standard was used for training data labeling.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).