K Number

Device Name

eCARTv5 Clinical Deterioration Suite (eCART)

Manufacturer

AgileMD, Inc.

Date Cleared

2024-06-21

(267 days)

Product Code

QNL ONL

Regulation Number

870.2210

Intended Use

eCART is a software product that provides automated risk stratification and early warning for impending patient deterioration, signified as the composite outcome of death or ICU transfer. It is intended to be used on hospitalized ward patients 18 years of age or older by trained medical professionals. As a clinical decision support device, eCART's risk score and trend analysis is intended to aid clinical teams in identifying which patients are most likely to clinically deteriorate. eCART provides additional information and does not replace the standard of care or clinical judgment. eCART scoring is initiated by the documentation of any vital sign on an adult ward patient. The device calculates risk only from validated EHR data, such as vitals that have been confirmed by a registered nurse (RN); unvalidated data streaming from monitors/devices will not be used until confirmed by a healthcare professional. The product predictions are for reference only and no therapeutic decisions should be made based solely on the eCART scores.

Device Description

The AgileMD eCARTv5 Clinical Deterioration Suite ("eCART") is a cloud-based software device that is integrated into the electronic health record ("EHR") in order to anticipate clinical deterioration in adult ward patients, which is signified as either of the following two predicted outcomes: (1) death or (2) ICU transfer. The tool synthesizes routine vital signs, laboratory data, and patient demographics into a single value that can be used to flag patients at-risk of the composite outcome of clinical deterioration for additional evaluation and monitoring. eCARTv5 requires the healthcare system within which it will be used, to provide an EHR connection and data interfaces through which the patient data necessary to run the software will be transmitted. The primary functions of the system are imparted by the Gradient Boosted Machine ("GBM") learning algorithm that takes input directly from the EHR, in real time, to provide an assessment of patients and displays its outputs in an intuitive user interface which drives providers to follow standardized clinical workflows (established by their institutions) for elevated-risk patients. eCARTv5's end users include med-surg nursing staff, physicians and other providers caring for these patients. The eCARTv5 composite score is determined from the model output (predicted probability of deterioration) scaled from 0-100, based on the specificity (true negative rate). The observed rate of deterioration at each eCART score threshold, displayed as the odds of deterioration in the next 24 hours, is presented to the user along with the scaled score. Default thresholds are set to an eCART of 93 and 97, respectively, for moderate and high risk categorization.

More Information

Contact

Type

Center

Panel

Date Received

Product Code

Subsequent Product Codes

Applicant Name (Manufacturer)

Device Name

Decision

Street 1

Street 2

City

State

Country Code

ZIP

Postal Code

Class Advise Committee

SSP Indicator

Third Party Reviewed

Expedited Review

Predicate Devices

CLEWICU (K200717)

Reference Devices

PeraServer and PeraTrend (K172959)

AI/ML (Artificial Intelligence / Machine Learning)

Yes
The device description explicitly states that the primary functions are imparted by a "Gradient Boosted Machine ('GBM') learning algorithm" and mentions a "risk-predictive output generated using a machine-learning algorithm."

Therapeutic

No
The device is described as a clinical decision support system that provides risk stratification and early warning but does not replace standard of care or clinical judgment, nor does it make therapeutic decisions.

Diagnostic

Yes

Explanation: The "Intended Use / Indications for Use" section states that eCART is "intended to aid clinical teams in identifying which patients are most likely to clinically deteriorate." This aligns with the definition of a diagnostic device as it helps to identify a patient's condition or risk.

SaMD (Software as a Medical Device)

Yes

The device description explicitly states it is a "cloud-based software device" and relies on integration with the EHR for data input and display, with no mention of accompanying hardware components.

IVD (In Vitro Diagnostic)

Based on the provided information, this device is not an IVD (In Vitro Diagnostic).

Here's why:

IVD Definition: In Vitro Diagnostics are tests performed on samples taken from the human body, such as blood, urine, or tissue, to detect diseases, conditions, or infections.
eCART's Function: eCART is a software product that analyzes existing patient data from the Electronic Health Record (EHR), such as vital signs, laboratory data, and demographics. It does not perform any tests on biological samples.
Intended Use: The intended use is to provide automated risk stratification and early warning for patient deterioration based on existing clinical data, not to diagnose a condition through laboratory testing.

Therefore, eCART falls under the category of clinical decision support software, not an In Vitro Diagnostic device.

PCCP (Predetermined Change Control Plan) Authorized

No
The letter does not explicitly state that the FDA has reviewed and approved or cleared a PCCP for this specific device. The section for PCCP states "Not Found".

Overview

Intended Use / Indications for Use

As a clinical decision support device, eCART's risk score and trend analysis is intended to aid clinical teams in identifying which patients are most likely to clinically deteriorate. eCART provides additional information and does not replace the standard of care or clinical judgment.

eCART scoring is initiated by the documentation of any vital sign on an adult ward patient. The device calculates risk only from validated EHR data, such as vitals that have been confirmed by a registered nurse (RN); unvalidated data streaming from monitors/devices will not be used until confirmed by a healthcare professional. The product predictions are for reference only and no therapeutic decisions should be made based solely on the eCART scores.

Product codes (comma separated list FDA assigned to the subject device)

ONL

Device Description

The primary functions of the system are imparted by the Gradient Boosted Machine ("GBM") learning algorithm that takes input directly from the EHR, in real time, to provide an assessment of patients and displays its outputs in an intuitive user interface which drives providers to follow standardized clinical workflows (established by their institutions) for elevated-risk patients.

eCARTv5's end users include med-surg nursing staff, physicians and other providers caring for these patients. The eCARTv5 composite score is determined from the model output (predicted probability of deterioration) scaled from 0-100, based on the specificity (true negative rate). The observed rate of deterioration at each eCART score threshold, displayed as the odds of deterioration in the next 24 hours, is presented to the user along with the scaled score. Default thresholds are set to an eCART of 93 and 97, respectively, for moderate and high risk categorization.

Mentions image processing

Not Found

Mentions AI, DNN, or ML

The primary functions of the system are imparted by the Gradient Boosted Machine ("GBM") learning algorithm that takes input directly from the EHR, in real time, to provide an assessment of patients and displays its outputs in an intuitive user interface which drives providers to follow standardized clinical workflows (established by their institutions) for elevated-risk patients.
A risk-predictive output generated using a machine-learning algorithm.

Input Imaging Modality

Not Found

Anatomical Site

Not Found

Indicated Patient Age Range

18 years of age or older

Intended User / Care Setting

Used on hospitalized ward patients.
Trained medical professionals including med-surg nursing staff, physicians and other providers caring for these patients.

Description of the training set, sample size, data source, and annotation protocol

Not Found. (The document mentions the algorithm was trained, but does not provide details of the training set.)

Description of the test set, sample size, data source, and annotation protocol

Retrospective analysis included admissions between 2009 and 2023 in three health systems.
Prospective validation was performed in the same three healthcare systems where the retrospective analysis was undertaken. This analysis included non-overlapping admissions between 2023 and 2024.
Deterioration is defined as death or ward to ICU transfer within 24 hours following a score.
The mortality outcome is defined as death within 24 hours following a score.
Retrospective cohort: N=1,769,461 encounters (unique hospitalizations), n=132,873,833 observations (eCART scores), representing 934,454 unique patients.
Prospective cohort: N=205,946 encounters, n=21,516,964 observations, representing 151,233 unique patients.

Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)

eCART was assessed in retrospective and prospective validation studies with adult ward patients from three geographically distinct health systems.

Retrospective Study:

Sample Size (Encounters): N=1,769,461
Sample Size (Observations): n=132,873,833
AUROC (Deterioration): 0.835 (0.834, 0.835)
AUROC (Mortality): 0.923 (0.923, 0.924)

Prospective Study:

Sample Size (Encounters): N=205,946
Sample Size (Observations): n=21,516,964
AUROC (Deterioration): 0.828 (0.827, 0.829)
AUROC (Mortality): 0.913 (0.911, 0.914)

Key results (Deterioration - Retrospective):

Outcome Prevalence: 1.3%
Moderate-risk threshold (eCART≥93):
- Positivity Rate: 7.5%
- Sensitivity: 51.8% (51.7%, 51.8%)
- Specificity: 93.1% (93.1%, 93.1%)
- PPV: 9.0% (9.0%, 9.1%)
- NPV: 99.3% (99.3%, 99.3%)
High-risk threshold (eCART≥97):
- Positivity Rate: 3.6%
- Sensitivity: 38.6% (38.5%, 38.7%)
- Specificity: 96.9% (96.9%, 96.9%)
- PPV: 14.2% (14.1%, 14.2%)
- NPV: 99.2% (99.2%, 99.2%)

Key results (Deterioration - Prospective):

Outcome Prevalence: 1.3%
Moderate-risk threshold (eCART≥93):
- Positivity Rate: 7.3%
- Sensitivity: 48.8% (48.7%, 49.0%)
- Specificity: 93.3% (93.3%, 93.3%)
- PPV: 8.9% (8.8%, 8.9%)
- NPV: 99.3% (99.3%, 99.3%)
High-risk threshold (eCART≥97):
- Positivity Rate: 3.1%
- Sensitivity: 33.7% (33.6%, 33.9%)
- Specificity: 97.3% (97.3%, 97.3%)
- PPV: 14.2% (14.1%, 14.3%)
- NPV: 99.1% (99.1%, 99.1%)

Key results (Mortality - Retrospective):

Outcome Prevalence: 0.2%
Moderate-risk threshold (eCART≥93):
- Positivity Rate: 7.5%
- Sensitivity: 75.8% (75.6%, 75.9%)
- Specificity: 92.6% (92.6%, 92.6%)
- PPV: 1.9% (1.9%, 1.9%)
- NPV: 100.0% (100.0%, 100.0%)
High-risk threshold (eCART≥97):
- Positivity Rate: 3.6%
- Sensitivity: 64.7% (64.5%, 64.9%)
- Specificity: 96.5% (96.5%, 96.5%)
- PPV: 3.4% (3.4%, 3.4%)
- NPV: 99.2% (99.2%, 99.2%)

Key results (Mortality - Prospective):

Outcome Prevalence: 0.2%
Moderate-risk threshold (eCART≥93):
- Positivity Rate: 7.3%
- Sensitivity: 71.7% (71.3%, 72.1%)
- Specificity: 92.9% (92.8%, 92.9%)
- PPV: 1.8% (1.8%, 1.8%)
- NPV: 99.9% (99.9%, 99.9%)
High-risk threshold (eCART≥97):
- Positivity Rate: 3.1%
- Sensitivity: 58.1% (57.6%, 58.6%)
- Specificity: 97.0% (97.0%, 97.0%)
- PPV: 3.4% (3.4%, 3.5%)
- NPV: 99.9% (99.9%, 99.9%)

Subgroup analyses for select comorbidities (Congestive Heart Failure, COVID-19, Chronic Pulmonary Disease, Sepsis) and race were presented, showing comparable performance.

Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)

Deterioration in Retrospective Cohort:

Moderate-risk threshold (eCART≥93):
- Sensitivity: 51.8% (51.7%, 51.8%)
- Specificity: 93.1% (93.1%, 93.1%)
- PPV: 9.0% (9.0%, 9.1%)
- NPV: 99.3% (99.3%, 99.3%)
High-risk threshold (eCART≥97):
- Sensitivity: 38.6% (38.5%, 38.7%)
- Specificity: 96.9% (96.9%, 96.9%)
- PPV: 14.2% (14.1%, 14.2%)
- NPV: 99.2% (99.2%, 99.2%)

Deterioration in Prospective Cohort:

Moderate-risk threshold (eCART≥93):
- Sensitivity: 48.8% (48.7%, 49.0%)
- Specificity: 93.3% (93.3%, 93.3%)
- PPV: 8.9% (8.8%, 8.9%)
- NPV: 99.3% (99.3%, 99.3%)
High-risk threshold (eCART≥97):
- Sensitivity: 33.7% (33.6%, 33.9%)
- Specificity: 97.3% (97.3%, 97.3%)
- PPV: 14.2% (14.1%, 14.3%)
- NPV: 99.1% (99.1%, 99.1%)

Mortality in Retrospective Cohort:

Moderate-risk threshold (eCART≥93):
- Sensitivity: 75.8% (75.6%, 75.9%)
- Specificity: 92.6% (92.6%, 92.6%)
- PPV: 1.9% (1.9%, 1.9%)
- NPV: 100.0% (100.0%, 100.0%)
High-risk threshold (eCART≥97):
- Sensitivity: 64.7% (64.5%, 64.9%)
- Specificity: 96.5% (96.5%, 96.5%)
- PPV: 3.4% (3.4%, 3.4%)
- NPV: 99.2% (99.2%, 99.2%)

Mortality in Prospective Cohort:

Moderate-risk threshold (eCART≥93):
- Sensitivity: 71.7% (71.3%, 72.1%)
- Specificity: 92.9% (92.8%, 92.9%)
- PPV: 1.8% (1.8%, 1.8%)
- NPV: 99.9% (99.9%, 99.9%)
High-risk threshold (eCART≥97):
- Sensitivity: 58.1% (57.6%, 58.6%)
- Specificity: 97.0% (97.0%, 97.0%)
- PPV: 3.4% (3.4%, 3.5%)
- NPV: 99.9% (99.9%, 99.9%)

Predicate Device(s): If the device was cleared using the 510(k) pathway, identify the Predicate Device(s) K/DEN number used to claim substantial equivalence and list them here in a comma separated list exactly as they appear in the text. List the primary predicate first in the list.

CLEWICU (K200717)

Reference Device(s): Identify the Reference Device(s) K/DEN number and list them here in a comma separated list exactly as they appear in the text.

PeraServer and PeraTrend (K172959)

Predetermined Change Control Plan (PCCP) - All Relevant Information for the subject device only (e.g. presence / absence, what scope was granted / cleared under the PCCP, any restrictions, etc).

Not Found

Regulation Number and Section

§ 870.2210 Adjunctive predictive cardiovascular indicator.

(a)
Identification. The adjunctive predictive cardiovascular indicator is a prescription device that uses software algorithms to analyze cardiovascular vital signs and predict future cardiovascular status or events. This device is intended for adjunctive use with other physical vital sign parameters and patient information and is not intended to independently direct therapy.(b)
Classification. Class II (special controls). The special controls for this device are:(1) A software description and the results of verification and validation testing based on a comprehensive hazard analysis and risk assessment must be provided, including:
(i) A full characterization of the software technical parameters, including algorithms;
(ii) A description of the expected impact of all applicable sensor acquisition hardware characteristics and associated hardware specifications;
(iii) A description of sensor data quality control measures;
(iv) A description of all mitigations for user error or failure of any subsystem components (including signal detection, signal analysis, data display, and storage) on output accuracy;
(v) A description of the expected time to patient status or clinical event for all expected outputs, accounting for differences in patient condition and environment; and
(vi) The sensitivity, specificity, positive predictive value, and negative predictive value in both percentage and number form.
(2) A scientific justification for the validity of the predictive cardiovascular indicator algorithm(s) must be provided. This justification must include verification of the algorithm calculations and validation using an independent data set.
(3) A human factors and usability engineering assessment must be provided that evaluates the risk of misinterpretation of device output.
(4) A clinical data assessment must be provided. This assessment must fulfill the following:
(i) The assessment must include a summary of the clinical data used, including source, patient demographics, and any techniques used for annotating and separating the data.
(ii) The clinical data must be representative of the intended use population for the device. Any selection criteria or sample limitations must be fully described and justified.
(iii) The assessment must demonstrate output consistency using the expected range of data sources and data quality encountered in the intended use population and environment.
(iv) The assessment must evaluate how the device output correlates with the predicted event or status.
(5) Labeling must include:
(i) A description of what the device measures and outputs to the user;
(ii) Warnings identifying sensor acquisition factors that may impact measurement results;
(iii) Guidance for interpretation of the measurements, including a statement that the output is adjunctive to other physical vital sign parameters and patient information;
(iv) A specific time or a range of times before the predicted patient status or clinical event occurs, accounting for differences in patient condition and environment;
(v) Key assumptions made during calculation of the output;
(vi) The type(s) of sensor data used, including specification of compatible sensors for data acquisition;
(vii) The expected performance of the device for all intended use populations and environments; and
(viii) Relevant characteristics of the patients studied in the clinical validation (including age, gender, race or ethnicity, and patient condition) and a summary of validation results.

Summary

Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.

June 21, 2024

AgileMD, Inc. % Kelliann Payne Partner Hogan Lovells US LLP 1735 Market Street 23rd Floor Philadelphia, Pennsylvania 19103

Re: K233253

Trade/Device Name: eCARTy5 Clinical Deterioration Suite ("eCART") Regulation Number: 21 CFR 870.2210 Regulation Name: Adjunctive Predictive Cardiovascular Indicator Regulatory Class: Class II Product Code: ONL Dated: May 17, 2024 Received: May 17, 2024

Dear Kelliann Payne:

We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).

Your device is also subject to, among other requirements, the Quality System (OS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely,

Robert T. Kazmierski -S

for

d. F

Page 2

LCDR Stephen Browning Assistant Director

Division of Cardiac Electrophysiology, Diagnostics, and Monitoring Devices Office of Cardiovascular Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

510(k) Number (if known) K233253

Device Name

eCART

Indications for Use (Describe)

区 Prescription Use (Part 21 CFR 801 Subpart D) □ Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Druq Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number.

510(k) SUMMARY

eCARTv5 Clinical Deterioration Suite

Submitter

AgileMD, Inc. 2261 Market Street #4378 San Francisco CA, 94114 Phone: 415-650-0522 Contact Person: Borna Safabakhsh

Date Prepared: June 20, 2024

Name of Device: eCARTv5 Clinical Deterioration Suite ("eCART")

Common or Usual Name: Clinical Monitor

Classification Regulation: 21 CFR 870.2210 (Adjunctive Predictive Cardiovascular Indicator)

Regulatory Class: II

Product Code: QNL

Predicate Device: CLEWICU (K200717)

Reference Device: PeraServer and PeraTrend (K172959)

Device Description

Intended Use / Indications for Use

The differences in indications for use from the predicate device are not critical to the intended use of eCARTv5, nor do they raise different questions of safety or effectiveness when the subject device is used as labeled. The minor differences are supported by adequate performance testing that show the subject device is substantially equivalent for the proposed indications for use. Both devices are intended to provide trained healthcare providers with a patient status score that reflects the underlying patient condition to supplement standard of care and informed decision making.

Summary of Technological Characteristics

At a high level, the subject and predicate devices are based on the following same technological elements:

· A risk-predictive output generated using a machine-learning algorithm.
Inputs to the software device include vital signs, assessments, and laboratory data collected . from the hospital EHR system.
. The output is a color-defined (red-yellow), real-time risk categorization to provide users information regarding potential risk of patient deterioration.
Embedded workflow features enable users to acknowledge the risk prediction and record next . steps for medical management, based on their independent medical judgment.

The primary technological differences between the devices are that eCART's algorithm was trained on ward patients to predict the probability of ICU transfer or death, whereas the predicate is trained in ICU patients to predict respiratory failure and/or hemodynamic instability. Additionally, the predicate device does not provide a risk score (but rather, a risk category). The minor technological differences between the devices raise no different questions of safety or effectiveness. With respect to the output, the reference device (PeraServer and PeraTrend (K172959) provides a risk score (the Rothman Index), which was validated on the outcome of mortality within 24 hours and included training and testing on data from adult ward patients (similar to eCART). Performance data for eCART, including retrospective and prospective clinical testing as well as human factors validation, demonstrates that the subject device is substantially equivalent to the predicate device.

Performance Data

eCART was assessed in retrospective and prospective validation studies with adult ward patients from three geographically distinct health systems. Test characteristics for both cohorts are shown in

Table A below. The retrospective analysis included admissions between 2009 and 2023 in three health systems. Deterioration is defined as death or ward to ICU transfer within 24 hours following a score. The mortality outcome is defined as death within 24 hours following a score. The prospective validation was performed in the same three healthcare systems where the retrospective analysis was undertaken. This analysis included non-overlapping admissions between 2023 and 2024.

Encounters (N) are defined as unique hospitalizations for a single patients may have more than one encounter in the data set if they were admitted and discharged from a study hospital more than once during the study period. Each encounter includes all eCART scores (observations, n) generated during that hospitalization.

		DETERIORATION		MORTALITY
		Retrospective	Prospective	Retrospective	Prospective
		N=1,769,461	N=205,946	N=1,769,461	N=205,946
AUROC		0.835 (0.834, 0.835)	0.828 (0.827, 0.829)	0.923 (0.923, 0.924)	0.913 (0.911, 0.914)
		n=132,873,833	n=21,516,964	n=132,873,833	n=21,516,964
Outcome Prevalence		1.3%	1.3%	0.2%	0.2%
		1,744,044/132,873,833	284,678/21,516,964	249,636/132,873,833	39,609/21,516,964
Moderate-
risk
threshold	Positivity
Rate	7.5%	7.3%	7.5%	7.3%
		9,982,577/132,873,833	1,563,648/21,516,964	9,982,577/132,873,833	1,563,648/21,516,964
	Sensitivity	51.8% (51.7%, 51.8%)	48.8% (48.7%, 49.0%)	75.8% (75.6%, 75.9%)	71.7% (71.3%, 72.1%)
		902,951/1,744,044	139,043/284,678	189,171/249,636	28,399/39,609
	Specificity	93.1% (93.1%, 93.1%)	93.3% (93.3%, 93.3%)	92.6% (92.6%, 92.6%)	92.9% (92.8%, 92.9%)
		122,050,163/131,129,789	19,807,681/21,232,286	122,830,791/132,624,197	19,942,106/21,477,355
eCART≥93	PPV	9.0% (9.0%, 9.1%)	8.9% (8.8%, 8.9%)	1.9% (1.9%, 1.9%)	1.8% (1.8%, 1.8%)
		902,951/9,982,577	139,043/1,563,648	189,171/9,982,577	28,399/1,563,648
	NPV	99.3% (99.3%, 99.3%)	99.3% (99.3%, 99.3%)	100.0% (100.0%, 100.0%)	99.9% (99.9%, 99.9%)
		122,050,163/122,891,256	19,807,681/19,953,316	122,830,791/122,891,256	19,942,106/19,953,316
	Risk Diff	8.4% (8.3%, 8.4%)	8.2% (8.1%, 8.2%)	1.8% (1.8%, 1.9%)	1.8% (1.7%, 1.8%)
High-risk
threshold
eCART≥97	Positivity
Rate	3.6%	3.1%	3.6%	3.1%
		4,753,332/132,873,833	675,279/21,516,964	4,753,332/132,873,833	675,279/21,516,964
	Sensitivity	38.6% (38.5%, 38.7%)	33.7% (33.6%, 33.9%)	64.7% (64.5%, 64.9%)	58.1% (57.6%, 58.6%)
		673,539/1,744,044	96,051/284,678	161,531/249,636	23,025/39,609
	Specificity	96.9% (96.9%, 96.9%)	97.3% (97.3%, 97.3%)	96.5% (96.5%, 96.5%)	97.0% (97.0%, 97.0%)
		127,049,996/131,129,789	20,653,058/21,232,286	128,032,396/132,624,197	20,825,101/21,477,355
	PPV	14.2% (14.1%, 14.2%)	14.2% (14.1%, 14.3%)	3.4% (3.4%, 3.4%)	3.4% (3.4%, 3.5%)
		673,539/4,753,332	96,051/675,279	161,531/4,753,332	23,025/675,279
	NPV	99.2% (99.2%, 99.2%)	99.1% (99.1%, 99.1%)	99.2% (99.2%, 99.2%)	99.9% (99.9%, 99.9%)
		127,049,996/128,120,501	20,653,058/20,841,685	127,049,996/128,120,501	20,825,101/20,841,685
	Risk Diff	13.3% (13.3%, 13.4%)	13.3% (13.2%, 13.4%)	3.3% (3.3%, 3.3%)	3.3% (3.3%, 3.4%)

Table A. eCART Prediction of Deterioration and Mortality in the Full External Retrospective and Prospective Cohorts

Note: The encounter and observation data above represents 934,454 and 151,233 unique patients in the retrospective and prospective cohorts, respectively. 95% confidence intervals were calculated using the Clopper-Pearson method.

Subgroup analyses for select comorbidities are presented in Tables B-C below for deterioration and mortality, respectively, and show comparable performance across conditions. Sequencing data for the SARS-CoV-2 variants included in the COVID subgroup analyses was not available, but the sample likely included the common variants circulating in the US between 2020 and the early part of 2024 (see https://cov-lineages.org/lineage list.html). The comparative effectiveness of the model between existing SARS-CoV-2 strains or to future strains is unknown.

		Comorbidity
	DETERIORATION	Congestive Heart
Failure
N = 306,140	COVID-19
N = 49,834	Chronic Pulmonary
Disease
N = 443,263	Sepsis
N = 639,802
	AUROC	0.810 (0.809, 0.810)	0.858 (0.857, 0.859)	0.824 (0.823, 0.824)	0.836 (0.836, 0.836)
		n = 31,445,409	n = 5,961,084	n = 38,638,180	n = 72,055,421
	Outcome Prevalence	2.2%	2.3%	1.7%	1.8%
		695,781/31,445,409	136,077/5,961,084	646,095/38,638,180	1,295,361/72,055,421
	Sensitivity	53.9% (53.8%, 54.1%)	69.6% (69.4%, 69.9%)	53.9% (53.8%, 54.0%)	56.4% (56.3%, 56.5%)
		375,328/695,781	94,726/136,077	348,271/646,095	730,846/1,295,361
Moderate-	Specificity	89.8% (89.7%, 89.8%)	84.6% (84.6%, 84.7%)	91.1% (91.0%, 91.1%)	90.8% (90.8%, 90.8%)
risk		27,600,037/30,749,628	4,929,294/5,825,007	34,593,348/37,992,085	64,243,126/70,760,060
threshold	PPV	10.6% (10.6%, 10.7%)	9.6% (9.5%, 9.6%)	9.3% (9.3%, 9.3%)	10.1% (10.1%, 10.1%)
(≥93)		375,328/3,524,919	94,726/990,439	348,271/3,747,008	730,846/7,247,780
	NPV	98.9% (98.8%, 98.9%)	99.2% (99.2%, 99.2%)	99.1% (99.1%, 99.1%)	99.1% (99.1%, 99.1%)
		27,600,037/27,920,490	4,929,294/4,970,645	34,593,348/34,891,172	64,243,126/64,807,641
	Sensitivity	40.2% (40.1%, 40.3%)	55.9% (55.6%, 56.2%)	40.1% (40.0%, 40.3%)	42.6% (42.6%, 42.7%)
		279,748/695,781	76,086/136,077	259,354/646,095	552,417/1,295,361
	Specificity	95.2% (95.1%, 95.2%)	90.7% (90.6%, 90.7%)	95.8% (95.8%, 95.9%)	95.6% (95.6%, 95.6%)
High-risk		29,259,307/30,749,628	5,280,458/5,825,007	36,413,561/37,992,085	67,648,075/70,760,060
threshold
(≥97)	PPV	15.8% (15.8%, 15.9%)	12.3% (12.2%, 12.3%)	14.1% (14.1%, 14.2%)	15.1% (15.1%, 15.1%)
		279,748/1,770,069	76,086/620,635	259,354/1,837,878	552,417/3,664,402
	NPV	98.6% (98.6%, 98.6%)	98.9% (98.9%, 98.9%)	98.9% (98.9%, 99.0%)	98.9% (98.9%, 98.9%)
		29,259,307/29,675,340	5,280,458/5,340,449	36,413,561/36,800,302	67,648,075/68,391,019

Table B. eCART Prediction of Clinical Deterioration in Select Comorbidities in the Full External Retrospective Cohort

Note: The encounter and observation data above represents 136,197 unique patients with congestive heart failure, 44,231 with chronic obstructive pulmonary disease, and 385,997 with sepsis were calculated using the Cloper-Pearson method.

Table C. eCART Prediction of Mortality in Select Comorbidities in the Full External Retrospective Cohort

MORTALITY	Comorbidity
	Congestive Heart
Failure	COVID-19
N = 49,834	Chronic Pulmonary
Disease
N = 443,263	Sepsis
N = 639,802
	N = 306,140

AUROC	0.893 (0.892, 0.894)	0.924 (0.923, 0.926)	0.912 (0.911, 0.913)	0.914 (0.914, 0.915)
	n = 31,445,409	n = 5,961,084	n = 38,638,180	n = 72,055,421
Outcome Prevalence	0.3%
103,194/31,445,09	0.5%
30,714/5,961,084	0.2%
85,582/38,638,180	0.2%
160,348/72,055,421
Moderate-
risk
threshold
(≥93)	Sensitivity
73.6% (73.3%, 73.9%)
75,961/103,194	Sensitivity
86.7% (86.3%, 87.1%)
26,639/30,714	Sensitivity
76.1% (75.8%, 76.4%)
65,111/85,582	Sensitivity
77.% (76.8%, 77.2%)
123,467/160,348
	Specificity
89.0% (89.0%, 89.0%)
27,893,257/31,342,215	Specificity
83.7% (83.7%, 83.8%)
4,966,570/5,930,370	Specificity
90.4% (90.4%, 90.5%)
34,870,701/38,552,598	Specificity
90.1% (90.1%, 90.1%)
64,770,760/71,895,073
PPV
2.2% (2.1%, 2.2%)
75,961/3,524,919	PPV
2.7% (2.7%, 2.7%)
26,639/990,439	PPV
1.7% (1.7%, 1.8%)
65,111/3,747,008	PPV
1.7% (1.7%, 1.7%)
123,467/7,247,780
NPV
99.9% (99.9%, 99.9%)
27,893,257/27,920,490	NPV
99.9% (99.9%, 99.9%)
4,966,570/4,970,645	NPV
99.9% (99.9%, 99.9%)
34,870,701/34,891,172	NPV
99.9% (99.9%, 99.9%)
64,770,760/64,807,641
High-risk
threshold
(≥97)	Sensitivity
61.7% (61.4%, 62.0%)
63,697/103,194	Sensitivity
79.2% (78.7%, 79.6%)
24,315/30,714	Sensitivity
64.9% (64.5%, 65.2%)
55,507/85,582	Sensitivity
66.1% (65.8%, 66.3%)
105,915/160,348
	Specificity
94.6% (94.5%, 94.6%)
29,635,843/31,342,215	Specificity
89.9% (89.9%, 90.0%)
5,334,050/5,930,370	Specificity
95.4% (95.4%, 95.4%)
36,770,227/38,552,598	Specificity
95.1% (95.%, 95.1%)
68,336,586/71,895,073
PPV
3.6% (3.6%, 3.6%)
63,697/1,770,069	PPV
3.9% (3.9%, 4.0%)
24,315/620,635	PPV
3.0% (3.0%, 3.0%)
55,507/1,837,878	PPV
2.9% (2.9%, 2.9%)
105,915/3,664,402
NPV
99.9% (99.9%, 99.9%)
29,635,843/29,675,340	NPV
99.9% (99.9%, 99.9%)
5,334,050/5,340,449	NPV
99.9% (99.9%, 99.9%)
36,770,227/36,800,302	NPV
99.9% (99.9%, 99.9%)
68,336,586/68,391,019

Note: The encounter and observation data above represents 136,197 unique patients with congestive heart failure, 44,8231 with chronic obstructive pulmonary disease, and 385,997 with sepss. 95% confidence intervals were calculated using the Cloper-Pearson method.

Subgroup analyses for race are presented in Table D, below, for deterioration. All five groups met the performance thresholds for AUROC and sensitivity.

			Table D. eCART Retrospective Validation Test Characteristics for the Outcomes of Clinical
	Deterioration by Race
DETERIORATION	Race	American Indian or
Alaska Native
N = 6,468	Asian/Mideast Indian
N = 26,681	Black/African-
American
N = 252,982	Native
Hawaiian/Other
Pacific Islander
N = 2,496	White/Caucasian
N = 1,384,075
AUROC		0.814 (0.808, 0.820)
486,615	0.847 (0.844, 0.850)
1,694,681	0.831 (0.830, 0.832)
20,057,696	0.862 (0.854, 0.870)
177,513	0.834 (0.833, 0.834)
104,522,118
Outcome Prevalence		1.2%
6,026/486,615	1.2%
20,736/1,694,681	1.2%
234,979/20,057,696	1.3%
2,395/177,513	1.3%
1,403,867/104,522,118
Moderate-
risk
threshold
(≥93)	Pos
Rate	6.7%
32,455/486,615	7.2%
121,532/1,694,681	6.5%
1,301,626/20,057,696	7.2%
12,738/177,513	7.7%
8,085,868/104,522,118
	Sensitivity	46.6% (45.3%, 47.9%)
2,809/6,026	52.9% (52.2%, 53.6%)
10,968/20,736	48.1% (47.9%, 48.3%)
112,939/234,979	56.5% (54.5%, 58.5%)
1,353/2,395	52.3% (52.2%, 52.3%)
733,566/1,403,867
	Specificity	93.8% (93.8%, 93.9%)
450,943/480,589	93.4% (93.4%, 93.4%)
1,563,381/1,673,945	94.0% (94.0%, 94.0%)
18,634,030/19,822,717	93.5% (93.4%, 93.6%)
163,733/175,118	92.9% (92.9%, 92.9%)
95,765,949/103,118,251
	PPV	8.7% (8.4%, 9.0%)
2,809/32,455	9.0% (8.9%, 9.2%)
10,968/121,532	8.7% (8.6%, 8.7%)
112,939/1,301,626	10.6% (10.1%, 11.2%)
1,353/12,738	9.1% (9.1%, 9.1%)
733,566/8,085,868
	NPV	99.3% (99.3%, 99.3%)
450,943/454,160	99.4% (99.4%, 99.4%)
1,563,381/1,573,149	99.3% (99.3%, 99.4%)
18,634,030/18,756,070	99.4% (99.3%, 99.4%)
163,733/164,775	99.3% (99.3%, 99.3%)
95,765,949/96,436,250
	Risk Diff	7.9% (7.6%, 8.3%)	8.4% (8.2%, 8.6%)	8.0% (8.0%, 8.1%)	10.0% (9.5%, 10.5%)	8.4% (8.4%, 8.4%)
High-risk
threshold
(≥97)	Pos
Rate	3.1%
14,933/486,615	3.6%
60,178/1,694,681	3.1%
611,210/20,057,696	3.5%
6,174/177,513	3.7%
3,831,413/104,522,118
	Sensitivity	35.2% (34.0%, 36.4%)
2,122/6,026	40.9% (40.3%, 41.6%)
8,491/20,736	34.9% (34.7%, 35.1%)
81,982/234,979	40.0% (38.0%, 42.0%)
957/2,395	39.1% (39.0%, 39.1%)
548,404/1,403,867
	Specificity	97.3% (97.3%, 97.4%)
467,778/480,589	96.9% (96.9%, 96.9%)
1,622,258/1,673,945	97.3% (97.3%, 97.3%)
19,293,489/19,822,717	97.0% (96.9%, 97.1%)
169,901/175,118	96.8% (96.8%, 96.8%)
99,835,242/103,118,251
	PPV	14.2% (13.7%, 14.8%)
2,122/14,933	14.1% (13.8%, 14.4%)
8,491/60,178	13.4% (13.3%, 13.5%)
81,982/611,210	15.5% (14.6%, 16.4%)
957/6,174	14.3% (14.3%, 14.3%)
548,404/3,831,413
	NPV	99.2% (99.1%, 99.2%)
467,778/471,682	99.3% (99.2%, 99.3%)
1,622,258/1,634,503	99.2% (99.2%, 99.2%)
19,293,489/19,446,486	99.2% (99.1%, 99.2%)
169,901/171,339	99.2% (99.1%, 99.2%)
99,835,242/100,690,705
	Risk Diff	13.4% (12.8%, 13.9%)	13.4% (13.1%, 13.6%)	12.6% (12.5%, 12.7%)	14.7% (13.8%, 15.6%)	13.5% (13.4%, 13.5%)

Note: The encounter and observation data above represents 3,282 unique American Indian or Alaska Native patients, 17,657 Asian/Mideast Asian patients, 116,642 Black/African-American patients, 1,427 Native Hawaiian/Other Pacific Islander patients, 95% confidence intervals were calculated using the Clopper-Pearson method.

Conclusion

eCARTv5 has the same intended use and similar indications for use, technological characteristics, and principles of operation as its predicate device, CLEWICU. The minor differences in indications for use do not alter the intended fundamental clinical purpose of the device, nor affect its safety and effectiveness when used as labeled. In addition, the minor technological differences between eCARTv5 and its predicate device raise no different questions of safety or effectiveness and are further supported by performance data. Thus, the device is substantially equivalent.