Predicate Devices

Not Found

Reference Devices

Not Found

AI/ML (Artificial Intelligence / Machine Learning)

Yes
The document explicitly states multiple times that the device is an "Artificial Intelligence/Machine Learning (AI/ML)-Based Software" and uses an "artificial intelligence/machine learning (AI/ML) based algorithm". It also mentions a "fixed machine learning model (probability random forest model)".

Therapeutic

No.
The device is an AI/ML-based software that aids in risk assessment for sepsis; it does not provide any therapy or treatment.

Diagnostic

Yes

The device aids in the risk assessment for the presence of or progression to sepsis, and performance studies mention its "diagnostic and predictive capability."

SaMD (Software as a Medical Device)

Yes

The device description explicitly states "The Sepsis ImmunoScore device is a software as a medical device". It processes data from an EMR and outputs a risk score and category, without mentioning any associated hardware components that are part of the regulated device.

IVD (In Vitro Diagnostic)

Based on the provided information, this device is not an In Vitro Diagnostic (IVD). Here's why:

Definition of IVD: An IVD is a medical device that is used to perform tests on samples taken from the human body, such as blood, urine, or tissue, to provide information about a person's health. The tests are performed in vitro (outside the body).
Device Function: The Sepsis ImmunoScore is a software device that uses pre-existing data from a patient's electronic health record (EMR). It does not perform any tests on biological samples. It analyzes existing data to generate a risk score.
Intended Use: The intended use is to aid in the risk assessment for sepsis using existing patient data, not to perform a diagnostic test on a biological sample.
Inputs: The inputs are data points from the EMR (demographics, vitals, labs, sepsis biomarkers), not results from a test performed on a biological sample by this device.

While the device uses laboratory findings as inputs, it does not perform the laboratory tests itself. It's a software tool that processes existing clinical and laboratory data.

PCCP (Predetermined Change Control Plan) Authorized

No
While the letter states "Control Plan Authorized" and lists "SPECIAL CONTROLS," it does not explicitly state that the FDA has reviewed, approved, or cleared a Predetermined Change Control Plan (PCCP) for this specific device. The provided text outlines specific controls for the device but does not use the explicit PCCP authorization language.

Overview

Intended Use / Indications for Use

The Sepsis ImmunoScore is an Artificial Intelligence/Machine Learning (AI/ML)-Based Software that identifies patients at risk for having or developing sepsis.

The Sepsis ImmunoScore uses up to 22 predetermined inputs from the patient's electronic health record to generate a risk score and to assign the patient to one of four discrete risk stratification categories, based on the increasing risk of sepsis.

The Sepsis ImmunoScore is intended to be used in conjunction with other laboratory findings and clinical assessments to aid in the risk assessment for presence of or progression to sepsis within 24 hours of patient assessment. It is intended to be used for patients admitted to the Emergency Department or hospital for whom sepsis is suspected, and a blood culture was ordered as part of the evaluation for sepsis. It should not be used as the sole basis to determine the presence of sepsis or risk of developing sepsis within 24 hours.

Product codes

SAK

Device Description

The Sepsis ImmunoScore device is a software as a medical device intended to aid in the risk assessment for progression to sepsis for patients, 18 and older, in an emergency department or hospital. The device is intended to identify patients, who have a blood culture ordered as part of their evaluation for sepsis and who are at risk of having or developing sepsis within the next 24 hours. The software uses 22 parameters from the hospital's electronic medical record (EMR). including demographics, vitals, labs, and sepsis biomarkers, and outputs the Sepsis Patient View.

The Sepsis Patient View can be viewed in the EMR system or the through a web interface and it displays both a sepsis risk score and a risk stratification category as well as other supplemental information. There are four risk stratification categories (Low, Medium, High, or Very High). The device uses an artificial intelligence/machine learning (AI/ML) based algorithm that is locked to compute the risk score and place the patient in a risk category. The Sepsis ImmunoScore is intended to be used in conjunction with other laboratory findings and clinical assessments.

Mentions image processing

Not Found

Mentions AI, DNN, or ML

The Sepsis ImmunoScore is an Artificial Intelligence/Machine Learning (AI/ML)-Based Software that identifies patients at risk for having or developing sepsis.

The device uses an artificial intelligence/machine learning (AI/ML) based algorithm that is locked to compute the risk score and place the patient in a risk category.

The core of the algorithm is a fixed machine learning model (probability random forest model) trained to identify sepsis in patients.

Due to the nature of AI/ML based software devices, there is a risk of algorithm performance difference or deterioration over time.

Input Imaging Modality

Not Found

Anatomical Site

Not Found

Indicated Patient Age Range

18 and older

Intended User / Care Setting

licensed care providers in a hospital environment. These care providers include physicians, physician extenders or advance practice providers (such as nurse practitioners and physician assistants), and nurses.

Description of the training set, sample size, data source, and annotation protocol

A total of 2,366 patients from three different sites in the NOSIS dataset were used to design and train the algorithm. Inclusion criteria included those 18 years or older, presented to the emergency department or hospital setting, had a blood culture order, and had a biobank sample ± 3 hours from the first order of a blood culture. Two methods were used during algorithm development to determine the presence of a sepsis event; a medical record analysis using a software encoded version of the Sepsis-3 criteria, and a retrospective chart review done by a team of three physicians that reviewed the medical chart to determine the presence of a sepsis event. Those conducting the chart review were blinded to the ImmunoScore results.

Description of the test set, sample size, data source, and annotation protocol

A retrospective study with prospectively collected data from a subset of the NOSIS dataset and biobank was conducted to demonstrate the diagnostic and predictive capability of the ImmunoScore algorithm. Patients were recruited sequentially based on the inclusion criteria from three sites: Beth Israel Deaconess Medical Center (356 patients), Jesse Brown VA - Chicago, IL (65 patients), and Beaumont - Royal Oak, MI (277 patients). The study population included all patients admitted to the emergency department or hospital for whom sepsis was suspected, as defined by the order of a blood culture as part of the evaluation for sepsis. Patients 18 and older were included. Any patients that did not have a qualifying plasma sample available in the NOSIS biobank originating from blood drawn within 3 hours of the first order of a blood culture were excluded. The ground truth comparison for the study was determined by using physician adjudication. The entirety of the patient's record was sent to an adjudication committee of three physicians. Physicians used a Retrospective Chart Diagnosis (RCD) Determination, to determine the presence of sepsis or lack thereof and timing of a Sepsis Event, if any. As per the Sepsis-3 definition, sepsis was adjudicated by determining three primary components: presence of infection, occurrence of organ dysfunction, and causality of organ dysfunction due to infection. The onset time of sepsis was adjudicated based on the timing of onset of organ disfunction caused by an infection, defined as the time that the Sequential Organ Failure Assessment (SOFA) score for a patient increased by at least 2 points consequent to the infection. The physicians were blinded to the results of the ImmunoScore and each subject was randomized for adjudication by physicians working at the healthcare institution from which the subject received care.

Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)

Study Type: Retrospective study with prospectively collected data.
Sample Size: N = 746 (Overall) with specific breakdowns for each clinical site: BIDMC - Boston, MA (370; 49.6%), Jesse Brown VA - Chicago, IL (73; 9.8%), Beaumont - Royal Oak, MI (303; 40.6%). This includes 735 patients for Forced Majority analysis and 523 patients for Forced Unanimous analysis.

AUC:

Adjudicated Forced Majority: 0.81 [0.76, 0.86]
Adjudicated Forced Unanimous: 0.84 [0.78, 0.90]

Standalone Performance: Not explicitly stated as standalone performance, but the device provides a risk score and stratification category.
Key Results:

Primary endpoint: Monotonic increase in the sepsis diagnostic predictive value and risk stratification category with an increase in severity and non-overlapping predictive value (95% confidence intervals) between the low and high and medium and very high risk stratification categories. This was met for both Forced Majority and Forced Unanimous adjudication schemes.
Secondary endpoints: Assessed in-hospital mortality, ICU admission, mechanical ventilation usage, vasopressor usage within 24 hours of patient assessment and median length of stay. All secondary objectives met the acceptance criteria except for mechanical ventilation within 24 hours (monotonic increase in PV, but insufficient power to demonstrate non-overlapping PV CIs). Overall, secondary endpoints support that as the likelihood of sepsis increases and risk categories increase, the likelihood of secondary outcomes occurring also increases.
Verification Bias Study: Overall results met acceptability criteria (lower bound of 95% CI no less than 80% agreement).
Diagnostic and Predictive Claim Subgroup Analysis: Both diagnostic and predictive breakdowns showed that both the primary and secondary endpoints were met for increasing predictive values and non-overlapping stratum specific likelihood ratios for the low/high and medium/very high risk categories.
Fresh vs. Frozen Plasma Samples for CRP and PCT Testing: High positive correlation (>0.99) observed for ImmunoScore output when frozen samples were replaced with fresh samples.
Precision/Sensitivity and Reproducibility Analysis: ImmunoScore is robust to input parameter perturbations.
Feature Imputation Study: Primary endpoint criteria were still met even in extreme imputation scenarios.
Risk Score Monotonicity: Analysis showed an increase in risk score with each risk category.
Reproducibility of SHAP Values: For the top 10 features, ICC was above 0.90, and for almost all remaining features, ICC was above 0.75, supporting adequate reproducibility.

Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)

Predictive Value (PV) and Stratum Specific Likelihood Ratio (SSLR) by Risk Category (Forced Majority - N=735):

Low: PV 3.02% [1.22%, 6.12%], SSLR 0.11 [0.05, 0.23]
Medium: PV 12.74% [7.96%, 18.99%], SSLR 0.53 [0.34, 0.82]
High: PV 36.59% [30.90%, 42.58%], SSLR 2.09 [1.77, 2.47]
Very High: PV 69.70% [51.29%, 84.41%], SSLR 8.33 [4.05, 17.12]

Predictive Value (PV) and Stratum Specific Likelihood Ratio (SSLR) by Risk Category (Forced Unanimous - N=523):

Low: PV 2.44% [0.80%, 5.60%], SSLR 0.13 [0.06, 0.31]
Medium: PV 8.40% [4.10%, 14.91%], SSLR 0.49 [0.27, 0.89]
High: PV 28.42% [22.01%, 35.54%], SSLR 2.11 [1.69, 2.63]
Very High: PV 73.91% [51.59%, 89.77%], SSLR 15.04 [6.11, 37.04]

Secondary Endpoint PV and SSLR (Forced Majority analysis on 735 patients):

ICU Transfer within 24 Hrs:
- Low: PV 4.74% [2.39%, 8.33%], SSLR 0.24 [0.13, 0.43]
- Medium: PV 12.74% [7.96%, 18.99%], SSLR 0.7 [0.45, 1.1]
- High: PV 25.72% [20.67%, 31.31%], SSLR 1.67 [1.32, 2.11]
- Very High: PV 54.55% [36.35%, 71.89%], SSLR 5.78 [2.95, 11.32]
In-Hospital Mortality:
- Low: PV 0.00% [0.00%, 1.58%], SSLR 0 [0, NaN]
- Medium: PV 1.91% [0.40%, 5.48%], SSLR 0.39 [0.13, 1.22]
- High: PV 8.70% [5.65%, 12.66%], SSLR 1.92 [1.29, 2.85]
- Very High: PV 18.18% [6.98%, 35.46%], SSLR 4.48 [1.87, 10.74]
Mechanical Ventilation within 24 Hrs:
- Low: PV 2.59% [0.95%, 5.54%], SSLR 0.53 [0.24, 1.19]
- Medium: PV 3.82% [1.42%, 8.13%], SSLR 0.8 [0.36, 1.79]
- High: PV 6.52% [3.91%, 10.11%], SSLR 1.41 [0.89, 2.22]
- Very High: PV 9.09% [1.92%, 24.33%], SSLR 2.02 [0.62, 6.55]
Vasopressor within 24 Hrs:
- Low: PV 0.86% [0.10%, 3.08%], SSLR 0.11 [0.03, 0.45]
- Medium: PV 1.91% [0.40%, 5.48%], SSLR 0.25 [0.08, 0.79]
- High: PV 11.59% [8.07%, 15.97%], SSLR 1.7 [1.21, 2.4]
- Very High: PV 39.39% [22.91%, 57.86%], SSLR 8.42 [4.24, 16.72]

Predicate Device(s)

Not Found

Reference Device(s)

Not Found

Predetermined Change Control Plan (PCCP) - All Relevant Information

Not Found

Regulation Number and Section

N/A

Summary

0

DE NOVO CLASSIFICATION REQUEST FOR SEPSIS IMMUNOSCORE

REGULATORY INFORMATION

FDA identifies this generic type of device as:

Software device to aid in the prediction or diagnosis of sepsis. A software device to aid in the prediction or diagnosis of sepsis uses advanced algorithms to analyze patient specific data to aid health care providers in the prediction and/or diagnosis of sepsis. The device is intended for adjunctive use and is not intended to be used as the sole determining factor in assessing a patient's sepsis status. The device may contain alarms that alert the care provider of the patient's status. The device is not intended to monitor response to treatment in patients being treated for sepsis.

NEW REGULATION NUMBER: 21 CFR 880.6316

CLASSIFICATION: Class II

PRODUCT CODE: SAK

BACKGROUND

DEVICE NAME: Sepsis ImmunoScore

SUBMISSION NUMBER: DEN230036

DATE DE NOVO RECEIVED: May 5, 2023

SPONSOR INFORMATION:

Prenosis, Inc. % Proxima Clinical Research 2450 Holcombe Blvd Houston. Texas 77021

INDICATIONS FOR USE

The Sepsis ImmunoScore is indicated as follows:

The Sepsis ImmunoScore is an Artificial Intelligence/Machine Learning (AI/ML)-Based Software that identifies patients at risk for having or developing sepsis.

The Sepsis ImmunoScore uses up to 22 predetermined inputs from the patient's electronic health record to generate a risk score and to assign the patient to one of four discrete risk stratification categories, based on the increasing risk of sepsis.

1

The Sepsis ImmunoScore is intended to be used in conjunction with other laboratory findings and clinical assessments to aid in the risk assessment for presence of or progression to sepsis within 24 hours of patient assessment. It is intended to be used for patients admitted to the Emergency Department or hospital for whom sepsis is suspected, and a blood culture was ordered as part of the evaluation for sepsis. It should not be used as the sole basis to determine the presence of sepsis or risk of developing sepsis within 24 hours.

LIMITATIONS

The sale, distribution, and use of the ImmunoScore are restricted to prescription use in accordance with 21 CFR 801.109.

The safety and effectiveness of the ImmunoScore device was not evaluated in subjects younger than 18 years of age.

The ImmunoScore has not been validated for use in specific inpatient settings such as ICU or Labor and Delivery units.

The device is not intended to be used as the sole basis to determine the presence of sepsis or risk of developing sepsis within 24 hours.

The ImmunoScore is positively correlated with the risk of having or developing sepsis within 24 hours. The score should not be interpreted as the probability, i.e., a patient with a risk score of 20 should not be interpreted as having a 20% probability or chance of developing or having sepsis within 24 hours.

The ImmunoScore is not intended to be used as a continuous monitoring or alert system, or to monitor response to treatment in patients being treated for sepsis. It is intended to simulate a diagnostic test, where an order for the test is placed and a set of outputs is provided as a onetime result.

PLEASE REFER TO THE LABELING FOR A COMPLETE LIST OF WARNINGS. PRECAUTIONS AND CONTRAINDICATIONS.

DEVICE DESCRIPTION

The Sepsis ImmunoScore device is a software as a medical device intended to aid in the risk assessment for progression to sepsis for patients, 18 and older, in an emergency department or hospital. The device is intended to identify patients, who have a blood culture ordered as part of their evaluation for sepsis and who are at risk of having or developing sepsis within the next 24 hours. The software uses 22 parameters from the hospital's electronic medical record (EMR). including demographics, vitals, labs, and sepsis biomarkers, and outputs the Sepsis Patient View.

2

The Sepsis Patient View can be viewed in the EMR system or the through a web interface and it displays both a sepsis risk score and a risk stratification category as well as other supplemental information. There are four risk stratification categories (Low, Medium, High, or Very High). The device uses an artificial intelligence/machine learning (AI/ML) based algorithm that is locked to compute the risk score and place the patient in a risk category. The Sepsis ImmunoScore is intended to be used in conjunction with other laboratory findings and clinical assessments.

Algorithm Description

The algorithm is a cloud-based system that uses a set of values measured in real time to generate the sepsis risk score and its auxiliary components, which are then stored. The core inputs to the algorithm include up to 22 parameters and the outputs are the risk score, risk stratification category, input features value, imputed (true/false) and feature Shapley (SHAP) value.

The core of the algorithm is a fixed machine learning model (probability random forest model) trained to identify sepsis in patients. A probability random forest calculates the mean predicted class probabilities from multiple simple models. Probability random forest performs bagging, a method of sampling a dataset with replacement. An individual simple model is trained on this sampled dataset. This sampling with replacement followed by training is performed many times to generate an ensemble, or forest, of simple models. The probability random forest used for the development of the ImmunoScore algorithm used 1000 decision trees as the base model to generate the forest. The hyperparameters used were a minimum node size of 13, the number of variables to randomly sample as candidates at each split of 8, and a split rule of extremely randomized trees. The output of the probability random forest model was then calibrated by performing a Platt calibration. Platt calibration was created by training a logistic regression model with the uncalibrated probability random forest output to predict the sepsis training label.

The trigger logic receives streaming data from the patient for each measurement type and determines when the algorithm has sufficient data to produce a result and which measurements to select for use in the algorithm. Some parameters are required for the ImmunoScore to generate a result and some are optional (see details below). Optional parameters are imputed based on bag imputation using an imputation template. Bag imputation is a statistical method that builds a random forest model for each inout feature in the Sepsis ImmunoScore Algorithm. Each random forest model uses the remaining observed input features to generate an imputed value.

In addition to the risk score and risk stratification category, SHapley Additive exPlanations (SHAP) were generated to explain predictions of the model by computing the individual contribution of each feature to the prediction. The sum of SHAP values and the baseline value, which is the mean sepsis risk score from the training dataset, equals the final prediction. Positive SHAP values are indicative of positive contributions to the Sepsis ImmunoScore, while negative SHAP values are indicative of negative contributions. SHAP values apply a game-theoretic approach to identify the contribution of features to the prediction for an observation. The SHAP values use the training data to estimate the feature contribution in the training dataset. Due to the computational complexity of calculating SHAP values, the software estimates a SHAP value using Monte-Carlo simulations with 100 rounds to estimate the feature contribution of the training data object with a fixed seed. This

3

estimate may be applied to a new observation to report the contribution of each feature to a prediction for that observation.

Input Parameters

The 22 patient parameters utilized include demographic, vital signs, and blood tests (hematology laboratory values, chemistry laboratory values, and sepsis biomarker concentrations). The selected parameters have either been cited in published literature as sepsis biomarkers, are part of the Sepsis-3 definition, or are well-known to correlate with a patient's chance of deterioration. Twelve of the parameters are required for calculating the ImmunoScore; an ImmunoScore will not be generated if any of those twelve values is missing. The 10 parameters listed in the table below as imputable can be missing and the device will generate an ImmunoScore by imputing values based on the training dataset.

	Parameter	Data Source	Example Device	Imputable
1	Age	Triage	-	Yes
2	Systolic Blood Pressure	Triage Vitals	Blood Pressure Monitor	No
3	Diastolic Blood Pressure	Triage Vitals	Blood Pressure Monitor	No
4	Temperature	Triage Vitals	Oral or Rectal
Thermometer	No
5	Respiratory Rate	Triage Vitals	Manual Measurement	No
6	Heart Rate	Triage Vitals	Pulse Monitor	No
7	Blood Oxygen Saturation	Triage Vitals	Pulse Oximeter	No
8	White Blood Cell Count	CBC Panel	Sysmex XN-9100	No
9	Lymphocyte Count	CBC Panel	Sysmex XN-9100	Yes
10	Neutrophil Count	CBC Panel	Sysmex XN-9100	Yes
11	Platelet Count	CBC Panel	Sysmex XN-9100	No
12	Blood Urea Nitrogen	BMP or CMP Panel	Siemens Atellica CH 930	No
13	Creatinine	BMP or CMP Panel	Siemens Atellica CH 930	No
14	Potassium	BMP or CMP Panel	Siemens Atellica CH 930	Yes
15	Chloride	BMP or CMP Panel	Siemens Atellica CH 930	Yes
16	Total Carbon Dioxide	BMP or CMP Panel	Siemens Atellica CH 930	Yes
17	Sodium	BMP or CMP Panel	Siemens Atellica CH 930	Yes
18	Albumin	CMP Panel	Siemens Atellica CH 930	Yes
19	Bilirubin	CMP Panel	Siemens Atellica CH 930	Yes
20	Procalcitonin	Stand-alone Test	Roche Cobas e411	No
21	C-Reactive Protein	Stand-alone Test	Roche Cobas e411	No
22	Lactate	Stand-alone Test	Siemens Atellica CH 930	Yes

Table 1. List of algorithm inputs

Algorithm Outputs

4

The main outputs include the ImmunoScore risk score and the risk stratification category. The risk score can range from 0 to 100 and denotes the risk of the patient meeting the Sepsis-3 criteria within 24 hours of the testing being ordered. The risk categories are stratified as low, medium, high, or very high risk and they are separated from one another using fixed thresholds.

Output	Possible	User Interpretation
Sepsis Risk
Score	0 - 100	Risk of having or developing
sepsis within 24 hours of the
Sepsis ImmunoScore being
ordered
Risk
Stratification
Category	Low
Medium
High
Very High	Each Risk Category has associated
diagnostic performance and
associated average predictive
metrics

Figure 1. Device Outputs

| Risk Stratification
Category | Diagnostic
Interpretation | Sepsis Risk Score Range |
|---------------------------------|------------------------------|-------------------------|
| Low | Sepsis unlikely | 0 - 12.2 |
| Medium | Sepsis possible | 12.2 - 30.6 |
| High | Sepsis likely | 30.6 - 87.2 |
| Very High | Sepsis very likely | 87.2 - 100 |

Figure 2. Risk Stratification Categories

5

| 92
score for sepsis

within 24 hours	Very High Risk Category
	Order Time
01/20/2023 22:48		Result Time
01/20/2023 22:18
	LOW	MEDIUM		HIGH	VERY HIGH
Parameters Increasing Risk of Sepsis			Parameter	Value	Collection Time
			Resp Rate	+ 63 breaths/min	01/20/2023 18:33
			Systolic BP	+ 77 mm Hg	01/20/2023 22:47
			PCT	+ 5.47 ng/ml	01/20/2023 22:43
			Sodium	+ 150 mmol/L	01/20/2023 15:04
			Temperature	+ 39.92 °C	01/20/2023 22:47
			CRP	+ 216.77 mg/L	01/20/2023 22:47
			Chloride	+ 117 mmol/L	01/20/2023 15:04
Parameters Decreasing Risk of Sepsis			Parameter	Value	Collection Time
			Platelets	423 10^9/L	01/20/2023 11:48
			Creatinine	0.85 mg/dl	01/20/2023 15:04
			Age	56 y	01/20/2023 22:47
			WBC	+ 11.5 10^9/L	01/20/2023 11:48
Parameters Unavailable at Result Time			Parameter	Value
			Albumin	Was unavailable

*Note: When device is deployed for real-world use, the "Non-clinical Use" button on the top of the screen will not be present. This will only appear if the device is used in a non-clinical setting (e.g., take device offline for maintenance or updates)

Figure 3. Sepsis ImmunoScore output screen

The system also identifies the contribution of each input parameter feature to the overall estimated probability via SHAP (Shapley) values. A positive SHAP value indicates the feature increased the estimated probability of Sepsis-3 while a negative one indicates the opposite. The greater the magnitude of the value, the stronger the contribution. It is important to note the relationship between features and estimated probability may be complex. In some cases, clinically abnormal values may have small contributions to the estimate due to greater contributions from other features.

The output screen also includes a weblink for "How does it work and what does it mean?". Clicking on that link brings up information regarding the algorithm development, clinical validation, and additional context regarding interpretation of the output,

6

Workflow

When an ImmunoScore is first ordered for a patient, the status of the score is displayed as pending. This time is used to collect any parameters needed for the algorithm. The software can inform the user of the orders that need to be placed and their status on the pending screen. While the necessary parameters are gathered, the risk score and category are displayed as shown in the screenshot below. If after three hours and thirty minutes the necessary parameters are not obtained, a "No Result" will appear on the screen and a score will not be calculated for this order of an ImmunoScore.

Image: question mark	Result Pending
score for sepsis
within 24 hours	Order Time
01/20/2023 22:47	Result Time

```
       | Image: speech bubble
```

Result Pending Screen:

Figure 4. Results Pending Screen

7

No Result Screen:

| No Result
score for sepsis
within 24 hours | Wait Time Has Expired
Required parameters have not resulted | | Sectionnines and
How does it work and what does it mean? |
|--------------------------------------------------|----------------------------------------------------------------|---------------------------------|----------------------------------------------------------------------------------------------------------------------------------------|
| | Order Time
05/17/2022 17:41 | Result Time
05/17/2022 21:11 | |

| Wait Time

Has Expired	BUN	⚠ Test has not resulted
	Creatinine	⚠ Test has not resulted

Figure 5. No Results Screen

Algorithm Development

The NOSIS Dataset and Biobank is from a consortium of clinical sites that contribute prospectively collected clinical data (Electronic Medical Records (EMR) data), time-series biological samples, and sample biomarker measurements to generate a unified database. A subset of the NOSIS Dataset and Biobank was used for algorithm development and for clinical validation of the algorithm. All data required by the ImmunoScore software is included in the NOSIS dataset. For vitals, laboratory parameters, and assessment, the associated order times and result times were retrieved from the NOSIS dataset. Clinical sites do not routinely measure concentrations of Procalcitonin and C-Reactive Protein. For this reason, values for these measurements used as inputs into the device were obtained from frozen samples available in the NOSIS Biobank, using the closest patient sample to the evaluation time and drawn within 3 hours of the suspicion of sepsis, as defined by the first order of a blood culture. Procalcitonin and C-Reactive Protein concentrations were measured by a reference laboratory.

A total of 2,366 patients from three different sites in the NOSIS dataset were used to design and train the algorithm. Inclusion criteria included those 18 years or older, presented to the emergency department or hospital setting, had a blood culture order, and had a biobank sample ± 3 hours from the first order of a blood culture. Two methods were used during algorithm development to determine the presence of a sepsis event; a medical record analysis using a software encoded version of the Sepsis-3 criteria, and a retrospective chart review done by a team of three physicians that reviewed the medical chart to determine the presence of a sepsis event. Those conducting the chart review were blinded to the ImmunoScore results.

To develop thresholds used to define the boundaries between risk stratification categories, a receiver operating characteristic curve (AUROC) was generated using the training data. Three points on the AUROC were selected using the following criteria to define the four risk stratification categories:

The threshold between the low and medium risk stratification categories was set to . achieve a high sensitivity to the detection of a sepsis event within 24 hours of the order of

8

the ImmunoScore (ordered concurrently with a blood culture) while maintaining a falsepositive rate of 50%. The 50% false-positive rate is based on the number of non-septic patients that receive antibiotics within three hours of a blood culture in a multi-site prospectively enrolled dataset and simulates a level of over-prescription of antibiotics representative of the current standard of care.

The threshold between the medium and high risk stratification categories was set to . simultaneously optimize both sensitivity and specificity of the device for identifying a sepsis event within 24 hours of the order of a blood culture.
. The threshold between the high and very high risk stratification categories was set so that patients in the top 5th percentile of sepsis probability were placed into a very high risk category.

| Demographic Information | Training Dataset
(N = 2366) |
|---------------------------------------------------|--------------------------------|
| Clinical Site (%) | |
| Beth Israel Deaconess Medical Center - Boston, MA | 0 (0.0) |
| OSF - Peoria, IL | 712 (30.1) |
| Jesse Brown VA - Chicago, IL | 0 (0.0) |
| Mercy Health - St. Louis, MO | 1061 (44.8) |
| Beaumont - Royal Oak, MI | 0 (0.0) |
| Carle Foundation Hospital - Urbana, IL | 593 (25.1) |
| Age (mean (SD)) | 64.20 (16.59) |
| Gender (%) | |
| Male | 1195 (50.5) |
| Female | 1171 (49.5) |
| Race (%) | |
| American Indian or Alaska Native | 1 (0.0) |
| Asian | 12 (0.5) |
| Black or African American | 315 (13.3) |
| Native Hawaiian or Other Pacific Islander | 0 (0.0) |
| Unknown | 85 (3.6) |
| White | 1953 (82.5) |
| Ethnicity (%) | |
| Hispanic or Latino | 26 (1.1) |
| Demographic Information | Training Dataset
(N = 2366) |
| Not Hispanic or Latino | 1725 (72.9) |
| Unknown | 615 (26.0) |
| High-Risk Comorbidities | |
| Acute Myocardial Infarction (%) | 97 (4.1) |
| History of Myocardial Infarction (%) | 101 (4.3) |
| Congestive Heart Failure (%) | 583 (24.6) |
| Peripheral Vascular Disease (%) | 225 (9.5) |
| Cerebrovascular Disease (%) | 130 (5.5) |
| Chronic Obstructive Pulmonary Disease (%) | 606 (25.6) |
| Dementia (%) | 167 (7.1) |
| Paralysis (%) | 68 (2.9) |
| Diabetes (%) | 630 (26.6) |
| Diabetes with Complications (%) | 423 (17.9) |
| Renal Disease (%) | 659 (27.9) |
| Mild Liver Disease (%) | 118 (5.0) |
| Moderate and Severe Liver Disease (%) | 45 (1.9) |
| Peptic Ulcer Disease (%) | 45 (1.9) |
| Rheumatologic Disease (%) | 105 (4.4) |
| AIDS (%) | 17 (0.7) |
| Immunocompromised (%) | 470 (19.9) |
| COVID-19 (%) | 189 (8.0) |

Demographics of the training dataset are:

9

Table 2. Demographics of Training Dataset

A separate tuning dataset was used to serve as a hold-out test set, to verify algorithm performance and determine the need for additional training of the algorithm. The training and tuning process for algorithm performance could be an iterative process, as shown in the figure below describing algorithm development:

10

Figure 6. Algorithm Development Process

(DX4)

| Site Name and Location | Site Used in
Training | Number of
Training
Patients | Site Used in
Tuning | Number of
Tuning Patients |
|-------------------------------------------|--------------------------|-----------------------------------|------------------------|------------------------------|
| OSF- Peoria, IL | Yes | 712 | Yes | 50 |
| Mercy Health - St. Louis, MO | Yes | 1061 | Yes | 136 |
| Jesse Brown VA - Chicago,
IL | No | 0 | Yes | 33 |
| Beaumont Royal Oaks, MI | No | 0 | Yes | 147 |
| Carle Foundation Hospital -
Urbana, IL | Yes | 593 | No | 0 |
| Total | | 2366 | | 366 |

The following table describes the sites used in the training and tuning phases.

Table 3. Sites Used in Training and Tuning Phases of Algorithm Development

Algorithm performance in the tuning dataset was assessed via the area under the receiver operating characteristic curve (AUROC), Following acceptable performance of the tuning dataset, the algorithm was locked.

SUMMARY OF CLINICAL INFORMATION

A retrospective study with prospectively collected data from a subset of the NOSIS dataset and biobank was conducted to demonstrate the diagnostic and predictive capability of the ImmunoScore algorithm.

CLINICAL SITES AND PATIENT DEMOGRAPHICS

Patients were recruited sequentially based on the inclusion criteria from three sites:

11

Hospital sites	Number of patients
Beth Israel Deaconess Medical Center	356
Jesse Brown VA - Chicago, IL	65
Beaumont - Royal Oak, MI	277

Table 4. Summary of the number of patients from each hospital site used for validating the ImmunoScore device.

Use of these three clinical validation sites provided data that was independent of the algorithm training and tuning sites, geographic diversity, and diversity in the type of electronic health record system utilized at the institution.

The study population included all patients admitted to the emergency department or hospital for whom sepsis was suspected, as defined by the order of a blood culture as part of the evaluation for sepsis. Patients 18 and older were included. Any patients that did not have a qualifying plasma sample available in the NOSIS biobank originating from blood drawn within 3 hours of the first order of a blood culture were excluded. The primary endpoint for the study was a monotonic increase in the sepsis diagnostic predictive value and risk stratification category with an increase in severity and non-overlapping predictive value (95% confidence intervals) between the low and high and medium and very high risk stratification categories. Secondary endpoints for the study assessed in-hospital mortality, ICU admission, mechanical ventilation usage, vasopressor usage within 24 hours of patient assessment and median length of stay. The acceptance criteria for the secondary endpoints were the same as those for the primary endpoints.

The following table provides details on the patient demographics for the study population:
Demographic Information
	Overall (N = 746)

Demographic Information	Overall (N = 746)
Clinical Site (%)
BIDMC - Boston, MA	370 (49.6)
Jesse Brown VA - Chicago, IL	73 (9.8)
Beaumont - Royal Oak, MI	303 (40.6)
Age (median [IQR])	66 [54, 77]
Sex (%)
Male	420 (56.3)
Female	326 (43.7)
Race (%)
American Indian or Alaska Native	2 (0.3)
Asian	16 (2.1)
Black or African American	169 (22.7)
Native Hawaiian or Other Pacific Islander	1 (0.1)
Unknown	128 (17.2)
White	430 (57.6)

12

Ethnicity (%)
Hispanic or Latino	100 (13.4)
Not Hispanic or Latino	604 (81.0)
Unknown	42 (5.6)
High-Risk Comorbidities
Acute Myocardial Infarction (%)	50 (6.7)
History of Myocardial Infarction (%)	62 (8.3)
Congestive Heart Failure (%)	187 (25.1)
Peripheral Vascular Disease (%)	76 (10.2)
Cerebrovascular Disease (%)	72 (9.7)
Chronic Obstructive Pulmonary Disease (%)	184 (24.7)
Dementia (%)	74 (9.9)
Paralysis (%)	25 (3.4)
Diabetes (%)	166 (22.3)
Diabetes with Complications (%)	167 (22.4)
Renal Disease (%)	233 (31.2)
Mild Liver Disease (%)	98 (13.1)
Moderate and Severe Liver Disease (%)	55 (7.4)
Peptic Ulcer Disease (%)	14 (1.9)
Rheumatologic Disease (%)	37 (5.0)
AIDS (%)	6 (0.8)
Immunocompromised (%)	202 (27.1)
COVID-19 (%)	79 (10.6)

Table 5. Demographics of Clinical Validation dataset

STUDY DESIGN AND PHYSICIAN ADJUDICATION

The following risk category thresholds were established prior to initiation of the clinical validation study:

Risk Category	ImmunoScore Range
Low	[0-12.2)
Medium	[12.2 - 30.6)
High	[30.6-87.2)
Very High	[87.2-100)

	Table 6. Risk Categories and thresholds

13

The ground truth comparison for the study was determined by using physician adjudication. The following is a summary of the adjudication process:

Image /page/13/Figure/1 description: This image is a flowchart that describes the process of determining whether an organ dysfunction is septic or not. The flowchart starts with "Organ Dysfunction" and branches out to "Infection Possible", "Infection Probable", and "Infection Definite". From there, the flowchart asks "Was the organ dysfunction caused by primary infection?" and branches out to "Yes", "No", and "Indefinite". The flowchart ends with "Non-Septic", "Indeterminate", "Septic", and "Forced Adjudication".

Figure 7. Physician Adjudication Process

The entirety of the patient's record was sent to an adjudication committee of three physicians. Physicians used a Retrospective Chart Diagnosis (RCD) Determination, to determine the presence of sepsis or lack thereof and timing of a Sepsis Event, if any. As per the Sepsis-3 definition, sepsis was adjudicated by determining three primary components: presence of infection, occurrence of organ dysfunction, and causality of organ dysfunction due to infection, The onset time of sepsis was adjudicated based on the timing of onset of organ disfunction caused by an infection, defined as the time that the Sequential Organ Failure Assessment (SOFA) score for a patient increased by at least 2 points consequent to the infection. If it was unclear whether the infection was the cause of organ dysfunction, the adjudicator was instructed to answer "Indefinite," and the patient's Sepsis status was labeled as "Indeterminate." If the infection did not cause the organ dysfunction event, the subject was recorded as "Non-Septic." and an alternate cause of organ dysfunction was recorded. If the infection was identified as "Probable" or "Definite," then the adjudicator deemed the patient as "Septic" if it was determined that the infection caused the organ dysfunction. In addition to providing the "Septic." "Non-Septic," or "Indeterminate" label for each subject, each adjudicator was also asked to also provide a "forced decision" in "Indeterminate" cases. This led to two groups for analysis, the adjudicated forced majority group and the adjudicated forced unanimous - the majority group was all patients that received adjudication and their Sepsis 3 determination was defined by the majority rule of diagnosis by physicians and the unanimous was where all physicians agreed on the diagnosis.

The physicians were blinded to the results of the ImmunoScore and each subject was randomized for adjudication by physicians working at the healthcare institution from which the subject received care. FDA recommends adjudication by independent physicians at separate institutions to minimize bias in the adjudication process. To assess the impact of the bias potentially

14

introduced by using same site adjudicators a verification bias study was conducted (discussed in more detail below), which demonstrated acceptable results.

RESULTS

The results of the clinical validation study included reporting of the metrics for primary and secondary endpoints and the AUROC:

An estimate of the AUROC for 95% confidence intervals was calculated for both the forced majority and forced unanimous adjudication schemes. There was a pre-specified performance goal of 0.75, which was achieved for both schemes:

Group	ImmunoScore [95% CI]
Adjudicated Forced Majority	0.81 [0.76,0.86]
Adjudicated Forced Unanimous	0.84 [0.78, 0.90]

Table 7. AUROC (95% CI) for ImmunoScore

Both the predictive vales and stratum specific likelihood ratios (SSLR) were calculated to assess the likelihood of sepsis in each risk category using the 95% CI:

| Sepsis
Group | Risk
Category | Total
Patients
(N) | Septic
Patients
(N) | PV [95% CI] | SSLR [95% CI] | Cochran
Armitage
Test (p-
value) |
|----------------------------------|------------------|--------------------------|---------------------------|-------------------------|------------------------|-------------------------------------------|
| Forced
Majority
(N = 735) | Low | 232 | 7 | 3.02% [1.22%, 6.12%] | 0.11 [0.05, 0.23] | 0.99) indicting that the use of frozen samples in the clinical validation did not impact the final ImmonoScore output.

There was a positive agreement of 95% with a 95% CI lower bound above 90% for both the fresh PCT only (0.97 [0.92. 0.99]) and fresh CRP only (1.00 [0.9. 1.00]) groups, but not for the CRP & PCT group (1.00 [0.88, 1.00]), despite perfect agreement. This is likely attributed to the limited sample size of n=28.

Image /page/21/Figure/4 description: The image is a scatter plot titled "Fresh vs Frozen C-reactive Protein Sepsis ImmunoScore results". The x-axis is labeled "Frozen C-reactive Protein Sepsis Risk Score", and the y-axis is labeled "EMR C-reactive Protein Sepsis Risk Score". The data points are clustered tightly around a dashed diagonal line, indicating a strong positive correlation. The text "R = 1, p