(107 days)
Not Found
Yes
The device description explicitly states that it "uses models derived from machine learning".
No
The device provides predictive information for clinicians, but explicitly states, "no therapeutic decisions should be made based solely on the CLEWICU predictions." This indicates it is for informational use, not direct treatment.
Yes
The device is described as providing "physiological insight into a patient's likelihood of future hemodynamic instability" and giving "additional information regarding the patient's predicted future risk for clinical deterioration, as well as identifying patients at low risk for deterioration." This aligns with the definition of a diagnostic device, which helps in identifying or predicting a medical condition or risk.
Yes
The device description explicitly states that "ClewICUServer and ClewICUnitor are software-only devices that are run on a user-provided server or cloud-infrastructure." It also describes the system as a "stand-alone analytical software product."
Based on the provided information, this device is not an In Vitro Diagnostic (IVD).
Here's why:
- IVDs analyze biological samples: In Vitro Diagnostics are designed to examine specimens taken from the human body, such as blood, urine, or tissue, to provide information about a person's health.
- This device analyzes physiological data: The CLEWICU System imports patient data from sources like Electronic Health Records (EHR) and patient monitoring devices. This data represents physiological measurements and clinical information, not biological samples analyzed in a laboratory setting.
- The intended use is for predicting future risk based on existing data: The device uses machine learning models to predict the likelihood of future hemodynamic instability based on the patient's current and historical physiological data. This is distinct from diagnosing a condition or measuring a substance within a biological sample.
Therefore, while this device is a medical device providing valuable clinical information, it does not fit the definition of an In Vitro Diagnostic.
Yes
Explanation:
The "Predetermined Change Control Plan (PCCP) - All Relevant Information" section explicitly states: "This 510(k) implements a PCCP to allow the CLEW models to be trained and validated for new input data sets following the same validation protocol and meeting the same performance criteria as were used to validate the CLEWICU models described in this 510(k)." This directly indicates that the device has an authorized PCCP.
Intended Use / Indications for Use
CLEWICU provides the clinician with physiological insight into a patient's likelihood of future hemodynamic instability. CLEWICU is intended for use in hospital care settings for patients 18 years and over. CLEWICU is considered to provide additional information regarding the patient's predicted future risk for clinical deterioration, as well as identifying patients at low risk for deterioration. The productions are for reference only and no therapeutic decisions should be made based solely on the CLEWICU predictions.
Product codes
QNL
Device Description
The CLEWICU System is a stand-alone analytical software product that includes the ClewICUServer and the ClewICUnitor. ClewICUServer and ClewICUnitor are software-only devices that are run on a user-provided server or cloud-infrastructure.
The ClewICUServer is a backend software platform that imports patient data from various sources including Electronic Health Record (EHR) data and patient monitoring devices through an HL7 data connection. The data are then used by models operating within the ClewICUServer to compute and store the CLEWHI index (likelihood of hemodynamic instability requiring vasopressor / inotrope support), and CLEWLR (indication that the patient is at "low risk" for deterioration).
The ClewICUnitor is the web-based user interface displaying CLEWHI, and CLEWLR associated notifications and related measures, as well as presenting the overall unit status.
Mentions image processing
Not Found
Mentions AI, DNN, or ML
It uses models derived from machine learning to calculate the likelihood of occurrence of certain clinically significant events for patients in hospital critical care settings including:
Input Imaging Modality
Not Found
Anatomical Site
Not Found
Indicated Patient Age Range
18 years and over
Intended User / Care Setting
clinician, hospital care settings for patients
Description of the training set, sample size, data source, and annotation protocol
New model development will always follow the strict process that requires one dataset for training and a different, completely independent, dataset for testing.
Description of the test set, sample size, data source, and annotation protocol
This was a retrospective cohort study that involved two separate health care systems, each evaluated independently. Data for patient stays came from the University of Massachusetts elCU dataset ("UMass", containing 6534 unique patient stays) and from the MIMIC-III dataset ("Mimic", containing 5069 unique patient stays). The models were exposed to the data from each patient stay in a temporal fashion from the start of each patient's stay through discharge. The models could generate notifications using only data that would have been available up to each timepoint under consideration; they could not see future data. Each notification was recorded with its timestamp.
Validation of new models developed under this PCCP will use patient data from at least three geographically diverse sites, and no one site's data will contribute more than 50% to the total validation dataset.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Model validation testing: A model validation study was conducted to demonstrate that the CLEWICU models continue to provide clinically acceptable performance after being re-trained using a reduced set of features, i.e., input dataset. This was a retrospective cohort study that involved two separate health care systems, each evaluated independently. Data for patient stays came from the University of Massachusetts elCU dataset ("UMass", containing 6534 unique patient stays) and from the MIMIC-III dataset ("Mimic", containing 5069 unique patient stays).
The pre-defined performance criteria were the same as for the initial CLEWICU System:
Primary endpoint:
The target point estimate for sensitivity/TPR for the CLEWHI model is 60%. The target point estimate for the specificity/TNR for the CLEWLR model is 90%.
Secondary endpoints:
The target point estimate positive predictive value (PPV) for the CLEWHI model is 10%. The target point estimate sensitivity/TPR for the CLEWLR model is 25%.
Results:
CLEWHI Model:
UMASS: Sensitivity was 63% (95% Cl: 59-67%), Specificity was 93% (95%- 94%), and PPPV was 12% (95% CI: 11-14%)
MIMIC: Sensitivity was 69% (95% Cl: 66 73%), Specificity was 87% (95% Cl: 87%- 88%), and PPV was 10% (95% Cl: 9 – 11%)
CLEWLR Model:
UMASS: Sensitivity was 47% (95% Cl: 46.8 47.2) and Specificity was 90.5% (95% Cl: . 89.6 – 91.4)
MIMIC: Sensitivity was 35.5% (95% CI 35.3 35.7) and Specificity was 90% (95% CI . 89.1 - 80.9)
The model validation test results demonstrate that the clinical performance of the CLEWICU models continue to meet the pre-defined acceptance criteria after being trained with the reduced input data set.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Primary endpoint:
The target point estimate for sensitivity/TPR for the CLEWHI model is 60%. The target point estimate for the specificity/TNR for the CLEWLR model is 90%.
Secondary endpoints:
The target point estimate positive predictive value (PPV) for the CLEWHI model is 10%. The target point estimate sensitivity/TPR for the CLEWLR model is 25%.
CLEWHI Model:
UMASS: Sensitivity was 63% (95% Cl: 59-67%), Specificity was 93% (95%- 94%), and PPPV was 12% (95% CI: 11-14%)
MIMIC: Sensitivity was 69% (95% Cl: 66 73%), Specificity was 87% (95% Cl: 87%- 88%), and PPV was 10% (95% Cl: 9 – 11%)
CLEWLR Model:
UMASS: Sensitivity was 47% (95% Cl: 46.8 47.2) and Specificity was 90.5% (95% Cl: . 89.6 – 91.4)
MIMIC: Sensitivity was 35.5% (95% CI 35.3 35.7) and Specificity was 90% (95% CI . 89.1 - 80.9)
Predicate Device(s)
K200717 CLEWICU System
Reference Device(s)
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information
This 510(k) implements a PCCP to allow the CLEW models to be trained and validated for new input data sets following the same validation protocol and meeting the same performance criteria as were used to validate the CLEWICU models described in this 510(k). Due to differences in US hospital electronic medical record (EMR) systems and critical care patient monitoring protocols, CLEW anticipates that the CLEWICU models may need to be trained for hospitals that present: 1) reduced input data types or reduced frequency of data availability: or 2) additional input data types as compared to the models cleared under this 510(k). In addition, the PCCP allows the CLEWICU models to be trained with new datasets, as they become available, to increase the models' sensitivity while maintaining or improving the models' performance specifications.
New model development will follow the same CLEW protocols and procedures reviewed by FDA under this 510(k) for both in-house training and on-site system implementation and validation. New model development will always follow the strict process that requires one dataset for training and a different, completely independent, dataset for testing. Validation of new models developed under this PCCP will use patient data from at least three geographically diverse sites, and no one site's data will contribute more than 50% to the total validation dataset.
The validated new models must meet the same minimum required performance specifications as the cleared CLEWICU models:
For CLEWHI the required minimum sensitivity is 0.6 and PPV is 0.1. For CLEWLR the required minimum sensitivity is 0.25 fand SPC is 0.9.
New CLEWICU models may demonstrate improvement in the PPV/SPC, while the sensitivity remains within previous performance level and the lower bound of the 95% confidence interval of that measure to ensure that one or both improve after retraining. The PPV is dependent on the event prevalence, and its target will be adjusted accordingly.
The modifications allowed under this PCCP will not change the device indications for use or how the CLEWICU operates once the models are deployed at a hospital, including the user interface and the criteria for raising a notification of a potential hemodynamic event or determining that a patient is at low risk for a hemodynamic event, within the next 8 hours.
§ 870.2210 Adjunctive predictive cardiovascular indicator.
(a)
Identification. The adjunctive predictive cardiovascular indicator is a prescription device that uses software algorithms to analyze cardiovascular vital signs and predict future cardiovascular status or events. This device is intended for adjunctive use with other physical vital sign parameters and patient information and is not intended to independently direct therapy.(b)
Classification. Class II (special controls). The special controls for this device are:(1) A software description and the results of verification and validation testing based on a comprehensive hazard analysis and risk assessment must be provided, including:
(i) A full characterization of the software technical parameters, including algorithms;
(ii) A description of the expected impact of all applicable sensor acquisition hardware characteristics and associated hardware specifications;
(iii) A description of sensor data quality control measures;
(iv) A description of all mitigations for user error or failure of any subsystem components (including signal detection, signal analysis, data display, and storage) on output accuracy;
(v) A description of the expected time to patient status or clinical event for all expected outputs, accounting for differences in patient condition and environment; and
(vi) The sensitivity, specificity, positive predictive value, and negative predictive value in both percentage and number form.
(2) A scientific justification for the validity of the predictive cardiovascular indicator algorithm(s) must be provided. This justification must include verification of the algorithm calculations and validation using an independent data set.
(3) A human factors and usability engineering assessment must be provided that evaluates the risk of misinterpretation of device output.
(4) A clinical data assessment must be provided. This assessment must fulfill the following:
(i) The assessment must include a summary of the clinical data used, including source, patient demographics, and any techniques used for annotating and separating the data.
(ii) The clinical data must be representative of the intended use population for the device. Any selection criteria or sample limitations must be fully described and justified.
(iii) The assessment must demonstrate output consistency using the expected range of data sources and data quality encountered in the intended use population and environment.
(iv) The assessment must evaluate how the device output correlates with the predicted event or status.
(5) Labeling must include:
(i) A description of what the device measures and outputs to the user;
(ii) Warnings identifying sensor acquisition factors that may impact measurement results;
(iii) Guidance for interpretation of the measurements, including a statement that the output is adjunctive to other physical vital sign parameters and patient information;
(iv) A specific time or a range of times before the predicted patient status or clinical event occurs, accounting for differences in patient condition and environment;
(v) Key assumptions made during calculation of the output;
(vi) The type(s) of sensor data used, including specification of compatible sensors for data acquisition;
(vii) The expected performance of the device for all intended use populations and environments; and
(viii) Relevant characteristics of the patients studied in the clinical validation (including age, gender, race or ethnicity, and patient condition) and a summary of validation results.
0
Image /page/0/Picture/0 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). The logo consists of two parts: the Department of Health & Human Services logo on the left and the FDA logo on the right. The FDA logo is in blue and includes the letters "FDA" followed by the words "U.S. Food & Drug Administration".
January 13, 2024
Clew Medical Ltd. % Sheila Hemeon-Heyer President Heyer Regulatory Solutions 125 Cherry Lane Amherst, Massachusetts 01002
Re: K233216
Trade/Device Name: CLEWICU System Regulation Number: 21 CFR 870.2210 Regulation Name: Adjunctive Predictive Cardiovascular Indicator Regulatory Class: Class II Product Code: QNL Dated: December 14, 2023 Received: December 14, 2023
Dear Sheila Hemeon-Heyer:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
FDA's substantial equivalence determination also included the review and clearance of your Predetermined Change Control Plan (PCCP). Under section 515C(b)(1) of the Act, a new premarket notification is not required for a change to a device cleared under section 510(k) of the Act, if such change is consistent with an
1
established PCCP granted pursuant to section 515C(b)(2) of the Act. Under 21 CFR 807.81(a)(3), a new premarket notification is required if there is a major change or modification in the intended use of a device, or if there is a change or modification in a device that could significantly affect the safety or effectiveness of the device, e.g., a significant change or modification in design, material, chemical composition, energy source, or manufacturing process. Accordingly, if deviations from the established PCCP result in a major change or modification in the intended use of the device, or result in a change or modification in the device that could significantly affect the safety or effectiveness of the a new premarket notification would be required consistent with section 515C(b)(1) of the Act and 21 CFR 807.81(a)(3). Failure to submit such a premarket submission would constitute adulteration and misbranding under sections 501(f)(1)(B) and 502(o) of the Act, respectively.
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (OS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatory
2
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Stephen C. Browning -S
LCDR Stephen Browning Assistant Director Division of Cardiac Electrophysiology, Diagnostics, and Monitoring Devices Office of Cardiovascular Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
3
Indications for Use
510(k) Number (if known) K233216
Device Name
CLEWICU System
Indications for Use (Describe)
CLEWICU provides the clinician with physiological insight into a patient's likelihood of future hemodynamic instability. CLEWICU is intended for use in hospital care settings for patients 18 years and over. CLEWICU is considered to provide additional information regarding the patient's predicted future risk for clinical deterioration, as well as identifying patients at low risk for deterioration. The productions are for reference only and no therapeutic decisions should be made based solely on the CLEWICU predictions.
| Type of Use (Select one or both, as applicable) | ||
|---|---|---|
| ☒ Prescription Use (Part 21 CFR 801 Subpart D) ☐ Over-The-Counter Use (21 CFR 801 Subpart C) | ☒ Prescription Use (Part 21 CFR 801 Subpart D) | ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
| ☒ Prescription Use (Part 21 CFR 801 Subpart D) | ☐ Over-The-Counter Use (21 CFR 801 Subpart C) |
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
4
K233216
510(k) Summary
This summary of 510(k) safety and effectiveness information is being submitted in accordance with the requirements of 21 CFR 807.92.
-
510(k) Applicant: A. CLEW Medical, Ltd. 5 Hamelacha St. Netanya Israel 4206002
Contact: Avigdor Faians Phone: +972-9-779-5995 Email: avi.f(@clewmed.com -
B. Date Prepared: January 12, 2024
D. Device Name and Classification Information:
| Trade name: | CLEWICU System |
|---|---|
| Common name: | Future Health Condition Prediction Software |
| Classification name: | Adjunctive Predictive Cardiovascular Indicator |
| Classification regulation: | 21 CFR 870.2210 |
| Product Code: | QNL |
| Class: | II |
- E. Predicate Device(s): K200717 CLEWICU System
Device Description: ட
The CLEWICU System is a stand-alone analytical software product that includes the ClewICUServer and the ClewICUnitor. It uses models derived from machine learning to calculate the likelihood of occurrence of certain clinically significant events for patients in hospital critical care settings including:
- Intensive Care Unit (ICU) .
- . Emergency Department's (ED) Critical Care or Resuscitation area
- Post-Anesthesia Care Unit (PACU) .
- . Step-Down Unit
- Post-Surgical Recovery Unit .
- . Other Specialized Care Units (e.g., Cardiac Care Unit (CCU), Neurocritical Care Unit (NCU), High-dependency Care Unit (HDU)
5
K233216
ClewICUServer and ClewICUnitor are software-only devices that are run on a user-provided server or cloud-infrastructure.
The ClewICUServer is a backend software platform that imports patient data from various sources including Electronic Health Record (EHR) data and patient monitoring devices through an HL7 data connection. The data are then used by models operating within the ClewICUServer to compute and store the CLEWHI index (likelihood of hemodynamic instability requiring vasopressor / inotrope support), and CLEWLR (indication that the patient is at "low risk" for deterioration).
The ClewICUnitor is the web-based user interface displaying CLEWHI, and CLEWLR associated notifications and related measures, as well as presenting the overall unit status.
G. Indications for Use Statement:
CLEWICU provides the clinician with physiological insight into a patient's likelihood of future hemodynamic instability. CLEWICU is intended for use in hospital critical care settings for patients 18 years and over. CLEWICU is considered to provide additional information regarding the patient's predicted future risk for clinical deterioration, as well as identifying patients at low risk for deterioration. The product predictions are for reference only and no therapeutic decisions should be made based solely on the CLEWICU predictions.
| Parameter | Predicate Device
K200717 | Proposed Device | Comparison |
|----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
| Device Trade Name | CLEWICU System | CLEWICU System | Same |
| Device Manufacturer | CLEW Medical Ltd. | CLEW Medical Ltd. | Same |
| Classification
regulation | 21 CFR 870.2210 | 21 CFR 870.2210 | Same |
| Product Code | QNL | QNL | Same |
| Indications for Use
Statement | CLEWICU provides the
clinician with physiological
insight into a patient's
likelihood of future
hemodynamic instability.
CLEWICU is intended for
use with intensive care unit
(ICU) patients 18 years and
over. CLEWICU is
considered to provide
additional information
regarding the patient's
predicted future risk for
clinical deterioration, as well | CLEWICU provides the
clinician with physiological
insight into a patient's
likelihood of future
hemodynamic instability.
CLEWICU is intended for
use in hospital critical care
settings for patients 18 years
and over. CLEWICU is
considered to provide
additional information
regarding the patient's
predicted future risk for
clinical deterioration, as well | Different
See note a |
H. Comparison with Predicate Device:
6
| Parameter | Predicate Device
K200717 | Proposed Device | Comparison |
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|
| | as identifying patients at low
risk for deterioration. The
product predictions are for
reference only and no
therapeutic decisions should
be made based solely on the
CLEWICU predictions. | as identifying patients at low
risk for deterioration. The
product predictions are for
reference only and no
therapeutic decisions should
be made based solely on the
CLEWICU predictions. | |
| Intended use
population | ICU patients ≥ 18 years | Critical care patients ≥ 18
years | Different, see
note a |
| Rx or OTC | Rx | Rx | Same |
| Product design | Analytical software product
that runs on user-provided
hardware. | Analytical software product
that runs on user-provided
hardware. | Same |
| Model features | Original set of 80 features | Reduced set of 50 features | Different
See note b |
Discussion of Differences .
Note a: During the initial installations of CLEWICU, hospitals have requested to be able to use the software in other critical care areas of the hospital in addition to the ICU. Therefore, the intended use environment of the indications for use statement is being broadened to allow use in all critical care areas of the hospital. This change does not impact the safety or effectiveness of CLEWICU because; 1) CLEWICU is housed on a networked server and is accessible in multiple areas of the hospital; 2) the level of training and experience of clinical staff in all critical care areas of the hospital is expected to be similar; and 3) the installation and training activities for CLEWICU will be the same regardless of where the system will be used in the hospital.
Note b: The purpose of this 510(k) is to demonstrate that the clinical performance of the CLEWICU system meets the same statistical criteria established under the prior 510(k) (K200717) after training the CLEWHI and CLEWLR models on a reduced set of input data (features). See below for the data demonstrating that CLEWICU with the reduced input dataset meets the established performance criteria and, therefore, does not impact the device safety or effectiveness.
J. Summary of Data Submitted to Support Substantial Equivalence
The following testing was submitted to support this 510(k):
Software validation testing: Software development and testing was conducted in accordance with IEC 62304:2015-06 Medical device software - Software life cycle processes.
Model validation testing: A model validation study was conducted to demonstrate that the CLEWICU models continue to provide clinically acceptable performance after being re-trained
7
using a reduced set of features, i.e., input dataset. This was a retrospective cohort study that involved two separate health care systems, each evaluated independently. Data for patient stays came from the University of Massachusetts elCU dataset ("UMass", containing 6534 unique patient stays) and from the MIMIC-III dataset ("Mimic", containing 5069 unique patient stays). The models were exposed to the data from each patient stay in a temporal fashion from the start of each patient's stay through discharge. The models could generate notifications using only data that would have been available up to each timepoint under consideration; they could not see future data. Each notification was recorded with its timestamp.
The pre-defined performance criteria were the same as for the initial CLEWICU System:
Primary endpoint:
The target point estimate for sensitivity/TPR for the CLEWHI model is 60%. The target point estimate for the specificity/TNR for the CLEWLR model is 90%.
Secondary endpoints:
The target point estimate positive predictive value (PPV) for the CLEWHI model is 10%. The target point estimate sensitivity/TPR for the CLEWLR model is 25%.
Results:
CLEWHI Model:
- UMASS: Sensitivity was 63% (95% Cl: 59-67%), Specificity was 93% (95%-. 94%), and PPPV was 12% (95% CI: 11-14%)
- MIMIC: Sensitivity was 69% (95% Cl: 66 73%), Specificity was 87% (95% Cl: 87%-. 88%), and PPV was 10% (95% Cl: 9 – 11%)
CLEWLR Model:
- UMASS: Sensitivity was 47% (95% Cl: 46.8 47.2) and Specificity was 90.5% (95% Cl: . 89.6 – 91.4)
- MIMIC: Sensitivity was 35.5% (95% CI 35.3 35.7) and Specificity was 90% (95% CI . 89.1 - 80.9)
The model validation test results demonstrate that the clinical performance of the CLEWICU models continue to meet the pre-defined acceptance criteria after being trained with the reduced input data set.
K. Predetermined Change Control Plan (PCCP)
This 510(k) implements a PCCP to allow the CLEW models to be trained and validated for new input data sets following the same validation protocol and meeting the same performance criteria as were used to validate the CLEWICU models described in this 510(k). Due to differences in US hospital electronic medical record (EMR) systems and critical care patient monitoring protocols, CLEW anticipates that the CLEWICU models may need to be trained for
8
K233216
hospitals that present: 1) reduced input data types or reduced frequency of data availability: or 2) additional input data types as compared to the models cleared under this 510(k). In addition, the PCCP allows the CLEWICU models to be trained with new datasets, as they become available, to increase the models' sensitivity while maintaining or improving the models' performance specifications.
New model development will follow the same CLEW protocols and procedures reviewed by FDA under this 510(k) for both in-house training and on-site system implementation and validation. New model development will always follow the strict process that requires one dataset for training and a different, completely independent, dataset for testing. Validation of new models developed under this PCCP will use patient data from at least three geographically diverse sites, and no one site's data will contribute more than 50% to the total validation dataset.
The validated new models must meet the same minimum required performance specifications as the cleared CLEWICU models:
For CLEWHI the required minimum sensitivity is 0.6 and PPV is 0.1. For CLEWLR the required minimum sensitivity is 0.25 fand SPC is 0.9.
New CLEWICU models may demonstrate improvement in the PPV/SPC, while the sensitivity remains within previous performance level and the lower bound of the 95% confidence interval of that measure to ensure that one or both improve after retraining. The PPV is dependent on the event prevalence, and its target will be adjusted accordingly.
The modifications allowed under this PCCP will not change the device indications for use or how the CLEWICU operates once the models are deployed at a hospital, including the user interface and the criteria for raising a notification of a potential hemodynamic event or determining that a patient is at low risk for a hemodynamic event, within the next 8 hours.
L. Conclusion
The differences between the predicate and the subject device do not raise any new or different questions of safety or effectiveness. The information and testing presented in this 510(k) demonstrate that the CLEWICU System is substantially equivalent to the predicate device cleared under K200717.