(106 days)
SMART Bun-Yo-Matic X-Ray software is to be used by orthopaedic healthcare professionals for diagnosis and surgical planning in a hospital or clinic environment. The medical imaging type intended to be used as the input of the software is X-ray.
The SMART Bun-Yo-Matic X-Ray software provides:
· Visualization report of the three-dimensional mathematical models of the anatomical structures of the foot and ankle and three-dimensional models of orthopaedic fixation devices.
· Measurement templates containing radiographic measures of foot and ankle,
· Surgical planning application for visualization of foot and ankle anatomical three-dimensional structures, radiographic measures, and surgical instrument parameters.
The visualization report containing the measurements can be used for the diagnosis of orthopaedic healthcare conditions. The surgical planning application containing the visualizations of the measurements in the context of three-dimensional models, orthopaedic fixation device models and surgical instrument parameters can be used for the planning of treatments and operations to correct orthopaedic healthcare conditions of foot and ankle.
The SMART Bun-Yo-Matic X-Ray device is a software tool that takes x-rays of the foot and produces 3D axes on contextual bone models to help a user plan for hallux valgus correction. The final output of the device is a case report that provides images of the patient's axes, as well as measurements prior to correction and following a surgical correction selected by the user.
Device Acceptance Criteria and Performance Study: SMART Bun-Yo-Matic X-Ray
This response details the acceptance criteria and the study that proves the SMART Bun-Yo-Matic X-Ray device meets these criteria, based on the provided FDA 510(k) summary.
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria | Reported Device Performance |
|---|---|
| 95% model conformance within 1.0mm distance to reference model (for image analytics) | The subject device meets the predicate's established acceptance criteria. Specific percentage met for this device is not explicitly stated, but "Results showed the subject device performed as intended." |
| 2.0 degrees standard deviation for angular measurements (for image analytics) | The subject device meets the predicate's established acceptance criteria. Specific performance is not explicitly stated, but "Results showed the subject device performed as intended." |
| Surgical planning executes mathematical operations for estimated correction ± 1 degree for angular measurements | "Surgery planning executes mathematical operations for estimated correction ± 1 degree for angular measurements". The results indicated the device performed as intended. |
| Surgical planning executes mathematical operations for estimated correction ± 1.0 mm for distance measurements | "Surgery planning executes mathematical operations for estimated correction ± 1.0 mm for distance measurements". The results indicated the device performed as intended. |
2. Sample Size Used for the Test Set and Data Provenance
- Test Set Sample Size: 97 x-ray and DRR (Digitally Reconstructed Radiographs) images.
- Data Provenance: The x-ray and CBCT DRR were collected from various sites across USA, Germany, UK, Finland, and Korea. The data was collected from patients with different ages and racial groups, with a minimum of 5% male/female within each dataset, mean age approximately 35 years, and representatives from White (Non-)Hispanic, Hispanic, and Native American racial groups. Each dataset was balanced in terms of subjects with different foot alignment, demographics, imaging devices, and subjects from clinical subgroups ranging from control/normal feet to pre-/post-operative clinical conditions such as Hallux Valgus, and undefined indications. This implicitly suggests a retrospective collection for the purpose of algorithm development and testing.
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
- Number of Experts: 2 clinicians.
- Qualifications: "Over five (5) years of experience practicing medicine."
4. Adjudication Method for the Test Set
The adjudication method for establishing ground truth on the test set is not explicitly detailed beyond "Each clinician was given the same image data to review dorsoplantar and lateral x-ray images. Each clinician then marks on a spreadsheet the presence of the bone in the image." This suggests either independent marking or a simple consensus approach, but no specific adjudication rule (e.g., 2+1, 3+1) is mentioned.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done
No, an MRMC comparative effectiveness study involving human readers assisting with or without AI and their improvement was not reported in this summary. The performance testing focused on the AI system's ability to meet preset technical/measurement accuracy criteria and its comparison to ground truth and manual measurements.
6. If a Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study Was Done
Yes, a standalone performance assessment study was done. The document states: "Performance testing was conducted. Testing included the following: AI/ML Testing. Comparison of the 2D-3D construction to manual measurements as well as ground truth. Comparison of the clinical acceptability of axes placement. Comparison of the planned surgical correction to the actual surgical correction." This indicates the algorithm's performance was evaluated against ground truth and manual measurements without direct human-in-the-loop interaction for the specific performance metrics. The training, tuning, and validation data were independent for this standalone assessment.
7. The Type of Ground Truth Used
The ground truth for the testing data was established by expert consensus (implied by 2 clinicians marking the presence of bone) and also involved manual measurements for comparison with the 2D-3D construction and the actual surgical correction for comparison with planned surgical correction.
8. The Sample Size for the Training Set
- AI algorithm for bone identification: 1,5776 (likely a typo, assumed to be 1,576 or 15,776) x-ray and CBCT DRR images.
- Metal identification: 15 x-ray and CBCT DRR images.
9. How the Ground Truth for the Training Set Was Established
The document states that the "AI algorithm for bone identification was developed using 1,5776 x-ray and CBCT DRR and metal identification was developed using 15 x-ray and CBCT DRR." While it mentions the training and tuning data were independent, it does not explicitly describe how the ground truth for the training set was established. It can be inferred that a similar expert labeling process was likely used, but the details are not provided in this summary.
§ 892.2050 Medical image management and processing system.
(a)
Identification. A medical image management and processing system is a device that provides one or more capabilities relating to the review and digital processing of medical images for the purposes of interpretation by a trained practitioner of disease detection, diagnosis, or patient management. The software components may provide advanced or complex image processing functions for image manipulation, enhancement, or quantification that are intended for use in the interpretation and analysis of medical images. Advanced image manipulation functions may include image segmentation, multimodality image registration, or 3D visualization. Complex quantitative functions may include semi-automated measurements or time-series measurements.(b)
Classification. Class II (special controls; voluntary standards—Digital Imaging and Communications in Medicine (DICOM) Std., Joint Photographic Experts Group (JPEG) Std., Society of Motion Picture and Television Engineers (SMPTE) Test Pattern).