(266 days)
The PanopticAl Vital Signs device is intended for noninvasive spot measurement of pulse rate when the subject is still. It is software for assessing facial video stream captured from a specified smartphone or tablet camera.
The PanopticAl Vital Signs device is intended for use by healthcare professionals. The device is only intended to be used in healthy subjects.
The PanopticAl Vital Signs device is indicated for use on humans 18 to 60 years of age who do not require critical care or continuous monitoring.
The PanopticAl Vital Signs device is not intended to be the sole method to assess a subject's physical health condition. The pulse rate measurements it provides should complement, not replace, professional medical care and/or medication.
PanopticAl Vital Signs is a medical software device that uses remote photoplethysmography (rPPG) to measure a person's pulse rate. The app utilizes the surrounding light as the light source and works by capturing and measuring the subtle color changes on the skin caused by light absorption and reflection by the blood vessels beneath the skin. The app uses the front camera of an iPhone or iPad to capture videos of the subject. Then, the algorithm in the app detects and tracks the subject's face to capture the subtle light changes reflected in the changes in RGB pixel values. This information is sent to PanopticAl's cloud server for further processing to calculate the pulse rate. The pulse rate value is then returned to the PanopticAl Vital Signs app, and the result is displayed on the app.
Here's a breakdown of the acceptance criteria and study details for the PanopticAI Vital Signs device, based on the provided text:
1. Table of Acceptance Criteria and Reported Device Performance
| Acceptance Criteria (Endpoint) | Target | Reported Device Performance (iPhone) | Reported Device Performance (iPad) |
|---|---|---|---|
| Accuracy (overall RMSE) | Below 3 BPM | Demonstrated to be substantially equivalent to predicate (RMSE) | Demonstrated to be substantially equivalent to predicate (RMSE) |
| Overall Mean Bias | Below 3 BPM | Below 3 BPM | Below 3 BPM |
| 95% CI of Upper Limit of Agreement [LoA] | Within ± 5 BPM (inclusive) | (2.3094 to 3.0645) - Within criteria | (2.5285 to 3.4571) - Within criteria |
| 95% CI of Lower Limit of Agreement [LoA] | Within ± 5 BPM (inclusive) | (-2.1954 to -1.4402) - Within criteria | (-3.0104 to -2.0818) - Within criteria |
| Correlation coefficient | Not explicitly stated as a numerical target, but "met" | "Met" | "Met" |
| Intercept of zero within 95% CI | Contained within 95% CIs | "Met" | "Met" |
| Slope of one within 95% CI | Contained within 95% CIs | "Met" | "Met" |
| Subgroup analysis (Bias, LoA within ±5 BPM) | Expected to be met across subgroups | Majority of 95% CI of LoA within ±5 BPM | Majority of 95% CI of LoA within ±5 BPM (exceptions for Heart Shape and History of Hypertension in iPad subjects, but absolute bias still within 3 BPM) |
| Performance with glasses | No significant difference | No significant difference | No significant difference |
| Performance with extreme heart rate (50-60 BPM) | No significant difference | No significant difference | No significant difference |
| Performance with extreme heart rate (100-130 BPM) | No significant difference | No significant difference | No significant difference |
| Performance with make-up | No significant difference | No significant difference | No significant difference |
| Performance at distance (0.4 m and 0.6 m) | No significant difference | No significant difference | No significant difference |
| Performance with facial hair | No significant difference | No significant difference | No significant difference |
| Performance at luminosity (100 lux and 500 lux) | No significant difference | No significant difference | No significant difference |
2. Sample Size Used for the Test Set and Data Provenance
- Sample Size (Clinical Validation Study): N=107
- Data Provenance: The document does not explicitly state the country of origin. It describes participants based on gender, age, BMI, facial shape, history of hypertension, race/ethnicity (Asian, Black, Hispanic, White), and Fitzpatrick Skin Type Scale. The study appears to be prospective since it's a clinical validation study aiming to assess agreement between the device and a "standard pulse rate measurement by a clinician."
3. Number of Experts Used to Establish the Ground Truth for the Test Set and Their Qualifications
The document mentions "standard pulse rate measurement by a clinician" as the ground truth. It does not specify the number of clinicians or their specific qualifications (e.g., years of experience).
4. Adjudication Method for the Test Set
The document does not describe an adjudication method for establishing ground truth. The ground truth was based on "standard pulse rate measurement by a clinician."
5. Multi-Reader Multi-Case (MRMC) Comparative Effectiveness Study
No. The document describes a clinical validation study comparing the device's performance to a clinician's standard measurement. It does not mention an MRMC study or the effect size of human readers improving with AI assistance. The device is a standalone measurement tool.
6. Standalone (Algorithm Only Without Human-in-the-Loop Performance) Study
Yes, a standalone study was performed. The clinical validation study directly assesses the PanopticAI Vital Signs app's accuracy against a clinical standard, demonstrating its performance without human interpretation of its outputs beyond what is displayed by the app itself. The device is intended for "non-invasive spot measurement of pulse rate," implying a direct output from the algorithm.
7. Type of Ground Truth Used
The ground truth used was expert consensus / clinical measurement from a "clinician" performing "standard pulse rate measurement."
8. Sample Size for the Training Set
The document does not provide information regarding the sample size for the training set. It focuses solely on the clinical validation (test) set.
9. How the Ground Truth for the Training Set Was Established
Since the training set sample size is not provided, how its ground truth was established is also not described in this document.
§ 870.2785 Software for optical camera-based measurement of pulse rate, heart rate, breathing rate, and/or respiratory rate.
(a)
Identification. The device uses software algorithms to analyze video signal and estimate pulse rate, heart rate, breathing rate, and/or respiratory rate. This device is not intended to independently direct therapy.(b)
Classification. Class II (special controls). The special controls for this device are:(1) A software description and the results of verification and validation testing based on a comprehensive hazard analysis and risk assessment must include:
(i) A full characterization of the software technical parameters, including algorithms;
(ii) If required image acquisition hardware is not included with the device, full specifications of the hardware requirements and testing to demonstrate the specified hardware ensures adequate data for validated and accurate measurements;
(iii) A description of the expected impact of all applicable sensor acquisition hardware characteristics and associated hardware specifications;
(iv) A description of all mitigations for user error or failure of any subsystem components (including signal detection, signal analysis, data display, and storage) on output accuracy; and
(v) Software documentation must include a cybersecurity vulnerability and management process to assure software functionality.
(2) Clinical data must be provided. This assessment must fulfill the following:
(i) The clinical data must be representative of the intended use population for the device. Any selection criteria or sample limitations must be fully described and justified.
(ii) The assessment must demonstrate output consistency using the expected range of data sources and data quality encountered in the intended use population and environment.
(iii) The assessment must compare device output with a clinically accurate patient-contacting relevant comparator device in an accurate and reproducible manner.
(3) A human factors and usability engineering assessment must be provided that evaluates the risk of improper measurement.
(4) Labeling must include:
(i) A description of what the device measures and outputs to the user;
(ii) Warnings identifying sensor acquisition factors or subject conditions or characteristics (garment types/textures, motion, etc.) that may impact measurement results;
(iii) Guidance for interpretation of the measurements, including a statement that the output is adjunctive to other physical vital sign parameters and patient information;
(iv) The expected performance of the device for all intended use populations and environments; and
(v) Robust instructions to ensure correct system setup.