(372 days)
Rho is a software application intended for use opportunistically with standard frontal radiographs of the lumbar spine, thoracic spine, chest, pelvis, knee, or hand/wrist performed in patients aged 50 years and older. Rho provides a notification in the form of a report to aid radiologists and/or physician interpreters in identifying patients with possible low bone mineral density (BMD) at L1-L4 or the femoral neck to prompt a clinical assessment of bone health. Rho should not be used to rule out low BMD. Radiologists and referring clinicians should follow recommended practices for screening and assessment, regardless of the absence of Rho report.
Rho is a machine learning-based software-as-a-medical device that interfaces with institutional Picture Archiving and Communications Systems (PACS) to identify patients 50 years and older undergoing x-ray with possible low bone mineral density (BMD). Eligible x-rays are frontal projections of the lumbar spine, thoracic spine, chest, pelvis, knee or hand/wrist. Rho uses the xray DICOM and DICOM tags of age and sex as inputs into a locked machine learning algorithm. The locked machine learning algorithm is trained on a patient-based dataset (True North Imaging. TNI13). The algorithm presents a binary output to indicate whether or not the patient likely has low BMD at either the femoral neck or L1-L4. A Rho Report is generated for positive cases that can be sent back to the PACS for physician interpretation or viewed through a browser-based interface.
Radiologists can choose to include this finding in their report to the referring physician. Inclusion may trigger a referring physician to conduct a clinical fracture risk assessment related to bone health.
For cases where Rho algorithm outputs a negative result, no Rho Report will be sent, and neither the radiologist nor the referring physician will receive any device output.
Here's a breakdown of the acceptance criteria and the study proving the device meets them, based on the provided text:
Acceptance Criteria and Reported Device Performance
| Acceptance Criteria (Performance Goals Proposed) | Reported Device Performance (Specificity) | Reported Device Performance (AUC) | Reported Device Performance (Sensitivity) |
|---|---|---|---|
| Lower end of the two-sided 95% confidence interval of specificity >0.775 (for TN16 and OMN cohorts) | Met for all datasets and most subgroups. |
-
Overall:
- TNI: 0.90 (0.88-0.92)
- OMN: 0.93 (0.90-0.96)
- OAI: 0.94 (0.92-0.96)
-
Most subgroups (specificity generally above 0.85).
-
Lowest reported specificity:
- Hand/wrist (OMN): 0.81 (0.67-0.95)
- 80+ (TNI): 0.71 (0.62-0.80) | Met for all datasets and all subgroups, as per statement.
-
Overall:
- TNI: 0.88 (0.88-0.90)
- OMN: 0.85 (0.82-0.88)
- OAI: 0.82 (0.79-0.85)
-
Most subgroups (AUC generally above 0.80).
-
Lowest reported AUC:
- Hand/wrist (OMN): 0.73 (0.61-0.85)
- 50-59 (OAI): 0.77 (0.70-0.84)
- Hand/wrist (OAI): 0.77 (0.70-0.82)
- 70-79 (OAI): 0.78 (0.72-0.83)
- Males (OAI): 0.79 (0.75-0.84) | Not met for all datasets and some subgroups.
Original goal: Lower end of the two-sided 95% confidence interval of sensitivity >0.5 (for TN16 and OMN cohorts).
- Overall:
- TNI: 0.67 (0.66-0.69) - Met
- OMN: 0.45 (0.41-0.50) - Not Met
- OAI: 0.36 (0.31-0.41) - Not Met
- **Specific Subgroups with Sensitivity 0.775 and lower end of the two-sided 95% confidence interval of sensitivity >0.5 for the TN16 and OMN cohorts. However, the final device performance across all analysis groups did not meet those goals." It then immediately states, "The device met its performance goals of specificity and AUC in all three datasets." This implies the final accepted performance goals were a high specificity and meeting the AUC goal, despite not fully meeting the originally proposed sensitivity goal. The approval indicates the FDA found the benefit-risk acceptable given the high specificity and AUC, and the context of use where low sensitivity is mitigated because patients continue to receive standard of care.
Study Information:
-
A table of acceptance criteria and the reported device performance: (See table above)
-
Sample sizes used for the test set and the data provenance:
- Total Clinical Validation Set: 4842 cases (3729 TNI6 + 522 OMN + 591 OAI).
- Data Provenance:
- TNI6 (Clinical Validation Set): 3729 cases from six True North Imaging centers, Canada (retrospective).
- OMN (Clinical Validation Set): 522 US cases from OneMedNet (retrospective).
- OAI (Clinical Validation Set): 591 US cases from Osteoarthritis Initiative, a multicenter, longitudinal, prospective observational study of knee osteoarthritis. This data was "previously acquired," suggesting it was used retrospectively for this device validation.
-
Number of experts used to establish the ground truth for the test set and the qualifications of those experts:
- The text does not mention the use of experts for ground truth establishment.
- Ground truth was established by Dual Energy X-ray Absorptiometry (DXA).
-
Adjudication method (e.g., 2+1, 3+1, none) for the test set:
- Not applicable, as the ground truth was established by DXA, not by human experts requiring adjudication.
-
If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance:
- No MRMC comparative effectiveness study was done to assess human reader improvement with AI assistance. The study described is a standalone (algorithm only) performance study against a ground truth. The device is intended to "aid radiologists and/or physician interpreters" but its direct impact on human reader performance was not evaluated in this study.
-
If a standalone (i.e. algorithm only without human-in-the-loop performance) was done:
- Yes, a standalone performance study was done. The described clinical performance testing directly evaluates the algorithm's Sensitivity, Specificity, and AUC as a binary classifier (yes/no low BMD) against the DXA ground truth.
-
The type of ground truth used (expert consensus, pathology, outcomes data, etc.):
- Ground Truth: Dual Energy X-ray Absorptiometry (DXA).
- Definition of Low BMD: At least one of the femoral necks or L1-L4 vertebrae having a T-Score
N/A