Search Results

CLEWICU provides the clinician with physiological insight into a patient's likelihood of future hemodynamic instability. CLEWICU is intended for use in hospital critical care settings for patients 18 years and over. CLEWICU is considered to provide additional information regarding the patient's predicted future risk for clinical deterioration, as well as identifying patients at low risk for deterioration. The product predictions are for reference only and no therapeutic decisions should be made based solely on the CLEWICU predictions.

The CLEWICU System is a stand-alone analytical software product that includes the ClewICUServer and the ClewICUnitor. It uses models derived from machine learning to calculate the likelihood of occurrence of certain clinically significant events for patients in hospital critical care settings including:

• Intensive Care Unit (ICU) .
• . Emergency Department's (ED) Critical Care or Resuscitation area
• Post-Anesthesia Care Unit (PACU) .
• . Step-Down Unit
• Post-Surgical Recovery Unit .
• . Other Specialized Care Units (e.g., Cardiac Care Unit (CCU), Neurocritical Care Unit (NCU), High-dependency Care Unit (HDU)
  ClewICUServer and ClewICUnitor are software-only devices that are run on a user-provided server or cloud-infrastructure.
  The ClewICUServer is a backend software platform that imports patient data from various sources including Electronic Health Record (EHR) data and patient monitoring devices through an HL7 data connection. The data are then used by models operating within the ClewICUServer to compute and store the CLEWHI index (likelihood of hemodynamic instability requiring vasopressor / inotrope support), and CLEWLR (indication that the patient is at "low risk" for deterioration).
  The ClewICUnitor is the web-based user interface displaying CLEWHI, and CLEWLR associated notifications and related measures, as well as presenting the overall unit status.

Here is an analysis of the acceptance criteria and study proving the device meets them, based on the provided text:

1. Table of Acceptance Criteria and Reported Device Performance

The CLEWICU System includes two models: CLEWHI (predicts hemodynamic instability) and CLEWLR (identifies low risk for deterioration).

Note on CLEWLR Specificity (MIMIC): The provided 95% CI for MIMIC Specificity for CLEWLR is stated as (89.1 - 80.9). This appears to be a typo, as the lower bound (89.1%) is higher than the upper bound (80.9%). Assuming the intent was 89.1-90.9 or similar, and given the point estimate is 90% (meeting the target), it is considered to have met the criteria. The text explicitly states "The model validation test results demonstrate that the clinical performance of the CLEWICU models continue to meet the pre-defined acceptance criteria."

Minimum Required Performance Specifications for PCCP (Post-clearance models):

2. Sample Size Used for the Test Set and Data Provenance

• Sample Sizes:
  • UMass dataset: 6534 unique patient stays
  • MIMIC-III dataset: 5069 unique patient stays
• Data Provenance: Retrospective cohort study. The text explicitly states "This was a retrospective cohort study that involved two separate health care systems, each evaluated independently."
  • UMass dataset: From the University of Massachusetts elCU dataset.
  • MIMIC-III dataset: From the MIMIC-III dataset (general knowledge indicates this is a publicly available dataset primarily from Beth Israel Deaconess Medical Center, USA). The country of origin for both is implicitly the USA, as these are US-based datasets/institutions.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

The document does not specify the number or qualifications of experts used to establish the ground truth for the test set. It describes the models predicting "hemodynamic instability requiring vasopressor / inotrope support" and "low risk for deterioration," but it doesn't detail how these ground truth labels were derived.

4. Adjudication Method for the Test Set

The document does not describe any adjudication method for establishing the ground truth for the test set.

5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study was Done, and Effect Size of Human Improvement with AI vs Without AI Assistance

No, a Multi-Reader Multi-Case (MRMC) comparative effectiveness study was not done. The study focuses purely on the standalone performance of the algorithm. There is no mention of human readers or AI assistance for human readers, nor any effect size for human improvement.

6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) was Done

Yes, a standalone study was done. The entire study described focuses on the direct performance of the CLEWHI and CLEWLR models against predefined criteria. The device is a "stand-alone analytical software product."

7. The Type of Ground Truth Used (expert consensus, pathology, outcomes data, etc.)

The ground truth used appears to be outcomes data based on clinical events. The CLEWHI model predicts "likelihood of occurrence of certain clinically significant events... including hemodynamic instability requiring vasopressor / inotrope support." The CLEWLR model identifies patients at "low risk for deterioration." These are objective clinical outcomes that can be derived from EHRs and patient monitoring data, which are the sources for "patient data from various sources including Electronic Health Record (EHR) data and patient monitoring devices." The document does not mention expert consensus or pathology for ground truth.

8. The Sample Size for the Training Set

The document states that the models were "re-trained using a reduced set of features." However, it does not explicitly state the sample size of the training set(s) used for this re-training. It only provides the sample sizes for the independent test sets (UMass and MIMIC-III). The PCCP section mentions "one dataset for training and a different, completely independent, dataset for testing." This implies a separate training dataset was used, but its size is not given.

9. How the Ground Truth for the Training Set was Established

The document does not explicitly describe how the ground truth for the training set was established. Given the nature of the ground truth for the test sets (clinical outcomes like hemodynamic instability or low risk of deterioration), it can be inferred that the training set's ground truth was established by similar objective clinical event definitions derived from historical patient data (EHR, monitoring devices).

Ask a specific question about this device

CLEWICU provides the clinician with physiological insight into a patient's likelihood of future hemodynamic instability. CLEWICU is intended for use with intensive care unit (ICU) patients 18 years and over. CLEWICU is considered to provide additional information regarding the patient's predicted future risk for clinical deterioration, as well as identifying patients at low risk for deterioration. The product predictions are for reference only and no therapeutic decisions should be made based solely on the CLEWICU predictions.

The CLEWICU System is a stand-alone analytical software product that includes the ClewICUServer and the ClewICUnitor. It uses models derived from machine learning to calculate the likelihood of occurrence of certain clinically significant events for patients in the intensive care unit (ICU). ClewICUServer and ClewICUnitor are software-only devices that are installed on user-provided hardware. The ClewICUServer is a backend software platform that imports patient data from various sources including Electronic Health Record (EHR) data and medical device data. The data are then used by models operating within the ClewICUServer to compute and store the CLEWHI index (likelihood of hemodynamic instability requiring vasopressor / inotrope support), and CLEWLR (indication that the patient is at "low risk" for deterioration). The ClewICUnitor is the web-based user interface displaying CLEWHI, and CLEWLR associated notifications and related measures, as well as presenting the overall unit status.

Here's a breakdown of the acceptance criteria and the study details for the CLEWICU System, based on the provided text:

1. Acceptance Criteria and Reported Device Performance

The acceptance criteria are implied by the reported performance metrics of the device, particularly the ranges for the 95% Confidence Intervals (CI). The study aimed to demonstrate acceptable performance for predicting hemodynamic instability (CLEWHI) and identifying low-risk patients (CLEWLR).

2. Sample Size Used for the Test Set and Data Provenance

• Sample Size (Test Set): Not explicitly stated as a single number. The study utilized a dataset from the WakeMed Health System, including patient stays in 7 intensive care units across 2 hospitals. The number of patients or patient stays for the retrospective clinical validation is not precisely quantified, but it was used for both training and validation.
• Data Provenance:
  • Country of Origin: United States (WakeMed Health System indicated).
  • Retrospective or Prospective: Retrospective.

3. Number of Experts Used to Establish the Ground Truth for the Test Set and Qualifications of Those Experts

• Number of Experts: Not explicitly stated as a specific number. The text mentions a "tagging system was developed and validated (against human physician readers as ground truth)." This implies multiple physician readers were involved in establishing or validating the ground truth for selected cases.
• Qualifications of Experts: "Human physician readers." No further specific qualifications (e.g., years of experience, subspecialty) are provided.

4. Adjudication Method for the Test Set

• Adjudication Method: Not explicitly detailed. The text states "a tagging system was developed and validated (against human physician readers as ground truth)." This suggests an indirect method where the tagging system learned from or was compared against physician assessments, rather than direct physician adjudication of every case in the test set.

5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study

• MRMC Study: No, an MRMC comparative effectiveness study was not explicitly mentioned or described. The study focused on the performance of the standalone AI system.
• Effect Size with AI vs. Without AI Assistance: Not applicable, as an MRMC study was not performed.

6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study

• Standalone Study: Yes, the described clinical validation is a standalone performance study. The reported metrics (Sensitivity, PPV, Specificity, Lead-time) directly reflect the algorithm's performance without explicit human intervention or assistance during the evaluation. The device is described as "a stand-alone analytical software product."

7. Type of Ground Truth Used

• Type of Ground Truth: The ground truth was established by "a tagging system... validated (against human physician readers as ground truth)." This suggests a hybrid approach where an automated tagging system, verified by human expert consensus (physician readers), was used to create the labels for the "events of interest" (hemodynamic instability, low risk).

8. Sample Size for the Training Set

• Sample Size (Training Set): Not explicitly stated as a specific number. The text mentions "the WakeMed dataset included patient stays in 7 intensive care units across 2 hospitals between 5 November 2019 and 30 June 2020." This dataset was used for "training of the CLEWICU predictive models," but the specific portion or number of cases allocated for training is not provided.

9. How the Ground Truth for the Training Set Was Established

• Ground Truth for Training Set: "Once validated, the tagging system was used to generate the clinical truth labels needed, both for training of the CLEWICU predictive models and for validation of the clinical performance of those models." This indicates that the same "tagging system" (validated against human physician readers) was used to establish the ground truth for the training set.

Ask a specific question about this device