(190 days)
The APPRAISE-HRI is a mobile health app intended to provide a means for military healthcare providers to screen U.S. Service members for hemorrhage risk after a physically traumatic event and stratify casualties who need immediate attention and emergency evacuation from those who are injured but may not be at risk for hemorrhage.
The APPRAISE-HRI is not intended to diagnose or direct treatment. Rather, it is intended to provide situational awareness and inform clinical management of potentially hemorrhagic casualties by identifying those at the greatest risk of hemorrhage.
The Automated Processing of the Physiological Registry for Assessment of Injury Severity (APPRAISE)-Hemorrhage Risk Index (HRI) is an Android application developed by the U.S. Army Medical Research and Development Command (USAMRDC) that uses vital-sign data collected from a trauma patient to provide a risk score that stratifies the patient's risk of hemorrhage. When the application runs, heart rate (HR), systolic blood pressure (SBP), and diastolic blood pressure (DBP) vital-sign data are collected from an external monitor [currently only the ZOLL Propaq M (K202375) is supported]. The ZOLL Propaq M monitor connects via Bluetooth with the Android platform running the APPRAISE-HRI application. The APPRAISE-HRI application continuously pulls data from the ZOLL Propaq M monitor to gather the required vital signs (HR, SBP, and DBP) and then runs the hemorrhage risk stratification algorithm to generate an output every one minute.
Once the vital-sign data from the ZOLL Propaq M monitor are transferred, the APPRAISE-HRI provides a stratification of hemorrhage into three Risk Level categories: Low (I), Average (II), or High (II). The APPRAISE-HRI application continuously displays both the input vital signs (HR, SBP, and DBP) and the Risk Level score to the caregiver to provide situational awareness and to inform clinical management of potentially hemorrhagic casualties by identifying those at the greatest risk of hemorrhage.
Here's a summary of the acceptance criteria and study details for the APPRAISE-HRI device, based on the provided document:
Acceptance Criteria and Device Performance
| Acceptance Criterion | Reported Device Performance (Likelihood Ratio [95% CI]) | Interpretation |
|---|---|---|
| HRI Level I Effectiveness | $\leq$ 0.20 (at least 5 times less likely for hemorrhage patients) | 0.18 [0.12, 0.26] |
| HRI Level II Effectiveness | Value close to 1 (almost as likely for hemorrhage patients as for control patients) | 0.70 [0.63, 0.76] |
| HRI Level III Effectiveness | $\geq$ 5.00 (at least 5 times more likely for hemorrhage patients) | 6.88 [6.04, 7.84] |
Note: The acceptance criteria were success criteria focusing on the device effectively and consistently differentiating between HRI Levels I, II, and III, and the ground truth for trauma patients tracking consistently with the algorithm results. The specific numerical targets for the Likelihood Ratios are derived from the interpretation in the document where "at least five times less likely," "almost as likely," and "nearly seven times more likely" were stated.
Study Details
| Feature | Description |
|---|---|
| Test Set Sample Size | 5,895 patients (543 Hemorrhage Patients, 5,352 Control Patients) |
| Test Set Data Provenance | Country of origin: Not explicitly stated, but collected from eight sites during patient transport and one hospital's Emergency Department within the context of a U.S. Army Medical Research and Development Command (USAMRDC) study. This implies a U.S. military-affiliated or broader U.S. patient population. |
| Retrospective/Prospective: "prospectively, retrospectively" design, meaning it prospectively validated the device using retrospectively collected vital-sign data. | |
| Number of Experts for Ground Truth | Not specified. |
| Qualifications of Experts | Not specified. |
| Adjudication Method | Not specified. |
| MRMC Comparative Effectiveness Study | No. The study did not involve human readers comparing performance with and without AI assistance. Instead, it evaluated the standalone performance of the algorithm against established ground truth. |
| Standalone Performance | Yes. The clinical utility study directly evaluated the performance of the APPRAISE-HRI algorithm (standalone) in stratifying hemorrhage risk. |
| Type of Ground Truth | Clinical Trauma Registry Data: Information included blood transfusion details, documented hemorrhagic injuries, clinical notes, International Classification of Disease 10th Revision (ICD-10) codes, and Abbreviated Injury Scale (AIS) codes. Patients were labeled as "Hemorrhage" or "Control" based on these data. |
| Training Set Sample Size | Not specified in the provided document. |
| Training Set Ground Truth Establishment | Not specified in the provided document. |
§ 870.2220 Adjunctive hemodynamic indicator with decision point.
(a)
Identification. An adjunctive hemodynamic indicator with decision point is a device that identifies and monitors hemodynamic condition(s) of interest and provides notifications at a clinically meaningful decision point. This device is intended to be used adjunctively along with other monitoring and patient information.(b)
Classification. Class II (special controls). The special controls for this device are:(1) Software description, verification, and validation based on comprehensive hazard analysis and risk assessment must be provided, including:
(i) Full characterization of technical parameters of the software, including algorithm(s);
(ii) Description of the expected impact of all applicable sensor acquisition hardware characteristics on performance and any associated hardware specifications;
(iii) Specification of acceptable incoming sensor data quality control measures;
(iv) Mitigation of impact of user error or failure of any subsystem components (signal detection and analysis, data display, and storage) on output accuracy; and
(v) The sensitivity, specificity, positive predictive value, and negative predictive value in both percentage and number form for clinically meaningful pre-specified time windows consistent with the device output.
(2) Scientific justification for the validity of the hemodynamic indicator algorithm(s) must be provided. Verification of algorithm calculations and validation testing of the algorithm must use an independent data set.
(3) Usability assessment must be provided to demonstrate that risk of misinterpretation of the status indicator is appropriately mitigated.
(4) Clinical data must support the intended use and include the following:
(i) The assessment must include a summary of the clinical data used, including source, patient demographics, and any techniques used for annotating and separating the data;
(ii) Output measure(s) must be compared to an acceptable reference method to demonstrate that the output represents the measure(s) that the device provides in an accurate and reproducible manner;
(iii) The data set must be representative of the intended use population for the device. Any selection criteria or limitations of the samples must be fully described and justified;
(iv) Where continuous measurement variables are displayed, agreement of the output with the reference measure(s) must be assessed across the full measurement range; and
(v) Data must be provided within the clinical validation study or using equivalent datasets to demonstrate the consistency of the output and be representative of the range of data sources and data quality likely to be encountered in the intended use population and relevant use conditions in the intended use environment.
(5) Labeling must include the following:
(i) The type of sensor data used, including specification of compatible sensors for data acquisition, and a clear description of what the device measures and outputs to the user;
(ii) Warnings identifying factors that may impact output results;
(iii) Guidance for interpretation of the outputs, including warning(s) specifying adjunctive use of the measurements;
(iv) Key assumptions made in the calculation and determination of measurements; and
(v) A summary of the clinical validation data, including details of the patient population studied (
e.g., age, gender, race/ethnicity), clinical study protocols, and device performance with confidence intervals for all intended use populations.