(623 days)
The PMD-200 with the Nociception Level (NOL) Index is indicated for use in a clinical setting that requires assessment of changes in nociception levels in adult patients under general anesthesia receiving opioid or opioid-sparing analgesia as part of their care.
The PMD-200 should be used as an adjunct to clinical judgment. Clinical judgment should always be used when interpreting the NOL index in conjunction with other available clinical and vital signs.
The PMD-200 (device) is a nociception monitoring system which assists assessment of anaesthetized patient's physiological response to noxious stimuli and helps tailor analgesic administration to the patient's actual response to painful stimuli. The device provides quantification of the patient's response to noxious stimuli with the nociception level index (NOL).
The PMD-200 system consists of 4 main components:
- Monitor acquisition and display of physiological raw data signals and NOL . index and trend graph.
- . Software includes two main components:
- Calculation of the NOL index based on physiological parameters extracted from raw data signals.
- O User interface displays physiological raw data signals, NOL index and trend graph.
- Finger probe -includes the following sensors: .
- Photoplethysmogram (PPG or Pleth) o
- · Thermistor for peripheral temperature (TMP)
- · A 3-axis accelerometer for movement (ACC or Movement).
- Connectors for the Galvanic skin response sensor (GSR or Conductance)
- Single use sensor (connected to the finger probe) includes the following sensor: . o Bio-impedance Galvanic skin response sensor (GSR or Conductance)
The NOL index is a relative, non-linear measure with a range of 0-100, where 0 represents no pain /nociceptive response and 100 represents extreme pain/nociceptive response. The NOL index and trend are intended to support clinical decisions concerning the administration of analgesic medications.
Here's a breakdown of the acceptance criteria and study information for the Medasense Biometrics Ltd. PMD-200, based on the provided text:
PMD-200 Acceptance Criteria and Performance Studies
1. Table of Acceptance Criteria and Reported Device Performance
The acceptance criteria for the algorithm's performance are primarily detailed in the "Algorithm Verification" and "Algorithm Validation" sections. The device met all specified acceptance criteria.
Algorithm Verification Results (Table 1 from text):
| Test description | Test metrics | Acceptance criteria | Results |
|---|---|---|---|
| Algorithm overall predictability | Estimated vs reference CISA: mean +/- 95% CI | 95% CI 0.85 | Pass |
| Characterization of NOL values at times of no noxious stimulus | Mean NOL value during periods where no nociception is reported | 97.7% of NOL samples should be below 25 | Pass |
Algorithm Validation Phase I Results (Table 2 from text):
| Test description | Test metrics | Acceptance criteria | Result |
|---|---|---|---|
| Algorithm overall predictability | Estimated vs reference CISA: mean +/- 95% CI | 95% CI mean {NOL(TP2)} > mean {NOL(TNP)} (Wilcoxon signed-rank test applied) | Pass |
| NOL algorithm decision accuracy (binary testing) | Youden Index for NOL threshold of 25 | 25 +/- 3 NOL units | Pass |
| AUC | 0.85 (0.8 for low 95% CI) | Pass | |
| Specificity | 0.85 (0.75 for low 95% CI) | Pass | |
| Sensitivity | 0.85 (0.75 for low 95% CI) | Pass | |
| Accuracy | > 0.85 | Pass | |
| Clinical relevance score | > 0.8 | Pass | |
| Characterization of NOL values at times of no noxious stimulus | Mean NOL value during periods where no nociception is reported | 97.7% of NOL samples should be below 25 | Pass |
Algorithm Validation Phase II (Sub-group Analysis) (Table 3 from text):
All subgroups (Female, Male, Age 65, BMI 18-25, BMI 25-30, BMI > 30, Remifentanil, Fentanyl, Desflurane, Sevoflurane, Propofol, ASA I, ASA II, ASA III) passed all 7 tests.
2. Sample Size Used for the Test Set and Data Provenance
The document describes three main types of studies for evaluating the PMD-200.
- Algorithm Verification: A multinational, multi-center dataset was used. The specific number of patients or data points for this dataset is not explicitly stated, but it covered a wide range of patient demographics and anesthetic drugs.
- Algorithm Validation: A separate multinational, multi-center dataset was used, distinct from the verification and training datasets.
- Phase I Validation: The exact number of patients or data points for Phase I validation is not explicitly stated for all tests; however, the sub-group analysis (Phase II) indicates that each sub-group contained at least 40 patients.
- Phase II Validation (Sub-group Analysis): The dataset used for Phase II combined both the verification and validation datasets.
- Female: 75 patients (retrospective study mentions 483 data points)
- Male: 75 patients (retrospective study mentions 219 data points)
- Age 65: 48 patients (retrospective study mentions 217 data points)
- BMI 18-25: 39 patients
- BMI 25-30: 45 patients
- BMI > 30: 45 patients
- Remifentanil: 55 patients (retrospective study mentions 265 data points)
- Fentanyl: 67 patients (retrospective study mentions 395 data points)
- Desflurane: 155 data points (from retrospective study)
- Sevoflurane: 354 data points (from retrospective study)
- Propofol: 173 data points (from retrospective study)
- ASA I: 112 data points (from retrospective study)
- ASA II: 418 data points (from retrospective study)
- ASA III: 91 data points (from retrospective study)
- NOL Retrospective Clinical Validation Study: This study used a database comprising 500 adult patients with 702 data points total across all subgroups (summing N data points in the table), collected retrospectively between January 2015 to December 2020.
- Provenance: This multi-center dataset included data from:
- HMR, Montreal, Canada (datasets: CINAAMON, NOLGYN, Remi-Trach)
- Shaare Zedek Medical Center, Jerusalem, Israel (dataset: Abdomi-NOL)
- Leiden University Medical Center, Leiden, the Netherlands (datasets: NOLA, SOLAR)
- University of Vermont Medical Center, Larner College of Medicine, Burlington, Vermont (Observational study)
- Cleveland Clinic Foundation, Cleveland, USA (Fentanyl pilot study)
- Provenance: This multi-center dataset included data from:
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document does not explicitly state the number of experts used to establish ground truth for the algorithm verification or validation datasets, nor does it specify their qualifications. The ground truth appears to be based on observed physiological responses to stimuli during anesthesia (e.g., intubation, incision) as recorded during surgical procedures.
For the human factors testing, the following was mentioned:
- Usability validation study in Israel: 18 participants representative of intended users.
- Usability validation study in Boulder, Colorado: Fifteen (15) US licensed anesthesiologists and CRNAs. Both are considered professional users with extensive clinical training.
4. Adjudication Method for the Test Set
The document does not describe a formal expert adjudication method (e.g., 2+1, 3+1) for establishing the ground truth of the physiological responses. The ground truth appears to be derived from the timing and nature of clinical events (e.g., intubation, incision) which are considered noxious stimuli. The retrospective study mentions "manual and automated data review" for excluding patients with incomplete records, which suggests some level of data quality assessment but not necessarily expert consensus on event labeling.
5. If a Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study Was Done, and Effect Size of Human Readers Improvement with AI vs. without AI Assistance
The document does not describe an MRMC study comparing human readers with and without AI assistance. Instead, it presents two clinical outcome studies (Abdomi-NOL and SOLAR) that compare NOL-guided analgesia (which implicitly means human clinicians using the PMD-200 for guidance) against standard of care monitoring (human clinicians using traditional methods like blood pressure and heart rate).
Effect Size (from Abdomi-NOL and SOLAR studies comparing NOL-guided vs. SOC):
- Abdomi-NOL Study:
- Median postoperative pain scores (NRS scale) at 60 minutes in PACU: 3 (NOL-guided) vs. 5 (SOC). Actual difference: 1.6 (95% CI 0.5-2.7). Corrected for sex distribution: 1.9 (95% CI 0.7-3.0).
- Median postoperative pain scores (NRS scale) at 90 minutes in PACU: 3 (NOL-guided) vs. 5 (SOC). Actual difference: 1.3 (95% CI 0.3-2.3). Corrected for sex distribution: 1.5 (95% CI 0.5-2.6).
- These reductions represent 30-33% improvement on the 11-point NRS scale.
- SOLAR Study:
- Median postoperative pain scores (NRS scale) during the first 90 minutes of PACU stay: 3.2 (NOL-guided) vs. 4.8 (control group). Actual difference: 1.6 (95% CI 0.5-2.7).
- Increased ACTH and cortisol levels (stress hormones) were 1.5 to 2-fold greater in the standard care group compared to the NOL-guided group.
These studies demonstrate an improvement in patient outcomes (reduced postoperative pain) when clinicians use the NOL index as an adjunct, but they do not isolate the improvement of human reader performance itself.
6. If a Standalone (i.e., algorithm only without human-in-the-loop performance) Was Done
Yes, standalone performance (algorithm only) was assessed.
The "Algorithm Verification," "Algorithm Validation" (Phase I and II), and "NOL Retrospective Clinical Validation Study" sections describe the standalone performance of the PMD-200 algorithm. These studies evaluate the algorithm's ability to correctly identify and predict nociception based on its input physiological data, independent of direct clinician intervention. The metrics like AUC, sensitivity, specificity, and accuracy are measures of the algorithm's standalone diagnostic performance.
7. The Type of Ground Truth Used
The ground truth for both the algorithm verification, validation activities, and the retrospective study primarily relies on the occurrence of specific noxious stimuli during surgical procedures.
Examples of ground truth events indicating nociception include:
- Intubation (TP1)
- Incision/trocar placement (TP2)
- Periods of "no noxious stimulus" or "no nociception reported" (TNP)
These events represent discrete points in time during surgery where a nociceptive response is expected (or not expected), allowing for the classification of the PMD-200's NOL index against these clinical events.
8. The Sample Size for the Training Set
The document explicitly states that the recordings used for verification were not used in the training dataset, and similarly, validation data was unique, i.e., not used for training the algorithm or for verification purposes.
However, the specific sample size for the training set is not provided in the given text.
9. How the Ground Truth for the Training Set Was Established
While the document states that the system has a training dataset, it does not explicitly describe how the ground truth for the training set was established. It can be inferred that it would follow a similar methodology to the verification and validation datasets, likely based on recorded physiological responses to known noxious and non-noxious stimuli during general anesthesia, as observed in surgical procedures.
N/A