(344 days)
The DermaSensor device is indicated for use to evaluate skin lesions suggestive of melanoma, basal cell carcinoma, and/or squamous cell carcinoma in patients aged 40 and above to assist in the decision regarding referral of the patient to a dermatologist. The DermaSensor device should be used in conjunction with the totality of clinically relevant information from the clinical assessment, including visual analysis of the lesion, by physicians who are not dermatologists. The device should be used on lesions already assessed as suspicious for skin cancer and not as a screening tool. The device should not be used as the sole diagnostic criterion nor to confirm clinical diagnosis of skin cancer.
The DermaSensor device (hereinafter referred to as 'DermaSensor', or the 'DermaSensor device'; Figure 1) utilizes optical spectroscopy and an artificial intelligence/machine learning (AI/ML) based software algorithm to analyze an intact skin lesion to which the device is non-invasively applied.
The device is a combination of a handheld unit and a base unit. The handheld unit contains a xenon arc lamp and a fiber-optic probe tip which together transmit broadband white light to a lesion surface. Samples of the backscattered light from the tissue are collected by an adjacent detection optical fiber, also within the probe tip, and are conveyed to a microspectrometer, vielding Elastic Scattering Spectroscopy (ESS) spectral recordings. The handheld unit is operated using a touchscreen interface with step-by-step guidance. The small fiber-optic tip is the only component that contacts the patient. The handheld unit remains in the base when not actively being applied to a lesion and its battery is recharged by the base's wireless charging mechanism. The base unit also contains calibration material that is accessible to the handheld unit.
In the DermaSensor device, analysis of the optical recordings of backscattered light over the range of wavelengths is carried out using a proprietary ML-derived classifier algorithm. The spectrum of scattered intensity vs. wavelength is a pattern, which is analyzed by a proprietary classifier algorithm in the device's built-in microprocessor to assess for the potential presence of melanoma. squamous cell carcinoma, or basal cell carcinoma. An internal microprocessor and classifier algorithm analyze the recording and provide results to the user at the point of care, Results are provided as "Monitor," for a negative result, or "Investigate Further" for a positive result. For positive output ("Investigate Further") the DermaSensor additionally displays a Similarity Score of 1-10, with higher scores representing greater similarity to signals seen in malignant lesions.
Here's a breakdown of the acceptance criteria and study proving device performance for the DermaSensor, based on the provided text:
Acceptance Criteria and Reported Device Performance
Note: The document outlines the special controls (acceptance criteria) that the device must meet. The "Reported Device Performance" column reflects the results presented in the CLINICAL PERFORMANCE TESTING section, specifically from the DERM-SUCCESS pivotal study and the reader studies.
| Acceptance Criteria Category (Special Control) | Specific Acceptance Criteria | Reported Device Performance |
|---|---|---|
| 1. Clinical Performance Validation Testing (Premarket & Post-market) | (i) Data must demonstrate superior accuracy of device-aided users' diagnostic characterization of the indicated lesions compared to the accuracy of unaided users. (iii) Standalone device performance testing must demonstrate the accuracy of the device output relative to ground truth, including: (A) Testing must demonstrate at least 90% sensitivity of the device output for lesions with high metastatic potential, or an alternative clinical consideration must be provided to justify lower sensitivity. Clinical justification must be provided for the reported specificity. | Device-Aided User Performance (MRMC Studies): - DERM-SUCCESS initial reader study (all skin cancers): Device-aided PCPs had a sensitivity of 91.4% (vs. unaided 82.0%; p=0.0027). Sensitivity + specificity > 1. AUROC increased 5.4% (from 0.708 to 0.762). - DERM-SUCCESS pigmented lesion reader study (pigmented lesions): Aided PCP AUROC > unaided PCP AUROC by 1.5% (p<0.001 for non-inferiority). Aided sensitivity for all skin cancers increased from 80.5% to 86.3% (p<0.001 for superiority). Aided sensitivity for melanoma increased from 68.8% to 75.4% (p<0.001 for non-inferiority). Ratio of correctly changed refer decisions to incorrectly changed non-refer decisions for melanoma was 248:49 (factor of 5 benefit). - DERM-ASSESS III reader study (melanoma-suggestive lesions): Aided AUROC > unaided AUROC by 4.1% (p<0.001 for non-inferiority, p=0.036 for superiority). Aided sensitivity increased from 73.7% to 81.8% (p<0.001 for superiority). Aided sensitivity for melanoma increased from 70.2% to 79.1% (p<0.001 for superiority). Standalone Device Performance (DERM-SUCCESS pivotal study): - Overall Sensitivity (High-Risk Lesions): 95.5% (91.7%-97.6%) (for melanoma, SCC, BCC) for all ages/phototypes. - Overall Specificity (Benign Lesions): 20.7% (18.5%-23.1%). - Met primary endpoint of sensitivity + specificity > 1 (95.5% + 20.7% > 1). - Met secondary endpoint of overall sensitivity > 90% (p<0.001). - Sensitivity by Lesion Type (All participants): - Melanoma (and severely atypical nevi): 87.5% - Melanoma: 82.8% - Highly Atypical Nevus: 94.7% - Squamous Cell Carcinoma (SCC): 97.7% - Basal Cell Carcinoma (BCC): 97.8% - Sensitivity by Lesion Type (Age 40+): - High Risk Lesions: 96.3% - Melanoma (and severely atypical nevi): 90.2% - Squamous Cell Carcinoma (SCC): 97.7% - Basal Cell Carcinoma (BCC): 97.8% - Clinical Justification for Specificity: Lower specificity (20.7%) is clinically acceptable due to high sensitivity (95.5%) and the significantly greater risk of a false negative, particularly for melanoma, than a false positive. It is mitigated by its adjunctive use. |
| 2. Non-clinical Performance Testing | Must demonstrate that the device performs as intended under anticipated conditions of use, including compatibility testing of the device software with specific signal or image acquisition hardware. | Bench testing performed (Dimensional, Battery, Lamp Reliability, Optical/Lamp Performance, WiFi connectivity, Calibrator Configuration, Wireless Charging, Label Validation, Useful Life, PCBA functional verification, System Packaging/Distribution, Manual Cleaning/Disinfection/Sensitization/Cytotoxicity/Irritation). All tests passed, demonstrating the device performs as expected. |
| 3. Precision Testing | Must demonstrate device precision, including repeatability and reproducibility of device performance, across operators and challenging use conditions. | Test performed with two study sites, three PCP investigators. - Repeatability (device scans): 84.2% overall; 97.1% for high-risk lesions. PPA 94.8%, NPA 46.0%. - Reproducibility (between physicians): 87.8% overall; 97.1% for high-risk lesions. APA 91.6%, ANA 44.8%. - Low repeatability/reproducibility for negative binary decisions and spectral similarity scores, but high (>90%) for positive binary decisions. Deemed acceptable within clinical performance context. |
| 4. Electrical Safety, Mechanical Safety, Thermal Safety, EMC | Performance testing must demonstrate the electrical safety, mechanical safety, thermal safety, and electromagnetic compatibility of any electrical components of the device. | Passed all relevant testing according to ETSI EN 301 489-17, IEC 60601-1-2, IEC 60601-1, IEC 62133-2, IEC 62471, IEC 60601-1-6, and IEC 60601-2-57. |
| 5. Reprocessing Instructions Validation | Performance testing must validate reprocessing instructions for reusable components of the device. | Cleaning and disinfection validations conducted per FDA guidance, AAMI TIR 12:2010, and AAMI TIR30:2011 (R2016). All testing and results adequate. |
| 6. Biocompatibility | The patient-contacting components of the device must be demonstrated to be biocompatible. | Passed cytotoxicity (No evidence of cell lysis or toxicity), sensitization (met requirements at 24, 48, 72 hrs), and irritation (No erythema or edema observed at 24, 48, 72 hrs) tests according to ISO 10993-1, -5, -10, -12 and FDA guidance. |
| 7. Software Verification, Validation, and Hazard Analysis | Must be performed. | Software developed and tested according to FDA guidance (2005, 2014, 2023), IEC 62304, and ISO 14971. Documentation and testing (including cybersecurity) demonstrate proper operation, address potential hazards (malfunction, errors, hardware failures, unauthorized access) with satisfactory results. Adequate for reasonable assurance of specified operation and protection from cyber vulnerability. |
| 8. Human Factors Assessment | Must demonstrate that the device can be safely and correctly used by intended users. | Summative human factors validation testing conducted with two user groups (15 PCPs, 15 mid-level practitioners). All 30 participants identified appropriate indications/contraindications and acknowledged the device is not for sole diagnosis. No unmitigated use difficulties, close calls, or use errors. Results demonstrated safe and effective operation by intended users. |
| 9. Labeling | (i) Summary of standalone and clinical performance, including sensitivity, specificity, and CIs for relevant subgroups. (ii) Description of patient population used in algorithm development/training. (iii) Device limitations or subpopulations where performance may differ/not validated. (iv) Interpretation information, including risks of misinterpretation. (v) Warnings for energy-emitting components. (vi) Statement: "not intended for standalone diagnostic." (vii) Maintenance/reprocessing instructions. | Labeling confirmed to include: performance measures (sensitivity, specificity, CI), performance for subgroups, patient population for algorithm training/development, limitations (e.g., increased risk patients, lesion types/locations, Fitzpatrick IV-VI limitations), specific warnings, statement that it is not intended for standalone diagnostic, and maintenance/reprocessing instructions. Post-market surveillance will update labeling. |
Study Proving Device Meets Acceptance Criteria
The primary study proving the device meets the acceptance criteria is the DERM-SUCCESS pivotal clinical study, supplemented by three Multi-Reader, Multi-Case (MRMC) comparative effectiveness studies.
1. A table of acceptance criteria and the reported device performance
(See table above)
2. Sample size used for the test set and the data provenance
Test Set (DERM-SUCCESS pivotal study):
- Sample Size: 1,005 participants with 1,579 lesions (from an initial enrolment of 1,021 participants with 1,598 lesions).
- Data Provenance:
- Country of Origin: International, multicenter study conducted at 22 study sites.
- 18 locations in the United States (including Arizona, California, Florida, Kansas, Minnesota, Pennsylvania, Rhode Island, Tennessee, Texas, Utah, Virginia),
- 4 locations in Australia.
- Retrospective or Prospective: Prospective, blinded clinical study.
- Country of Origin: International, multicenter study conducted at 22 study sites.
Test Set (MRMC Reader Studies):
- DERM-SUCCESS initial reader study: 25 malignant and 25 benign lesions (total 50 lesions).
- DERM-SUCCESS pigmented lesion reader study: 69 malignant lesions (36% melanoma, 26% SCC, 36% BCC) and 67 benign lesions (total 136 lesions).
- DERM-ASSESS III reader study: 50 malignant lesions (68% melanoma, 16% SCC, 16% BCC) and 50 benign lesions (total 100 lesions).
3. Number of experts used to establish the ground truth for the test set and the qualifications of those experts
DERM-SUCCESS pivotal study (Standalone Performance):
- Number of Experts: 2 to 5 central study dermatopathologists.
- Qualifications of Experts: Dermatopathologists. (Specific experience level like "10 years of experience" is not mentioned, but the term implies specialized expertise in skin pathology).
MRMC Reader Studies (Reader Performance Evaluation):
- Number of Experts: Not directly stated for ground truth, but the lesions were derived from the DERM-SUCCESS pivotal study, implying the same ground truth establishment method.
- For Reader Participants:
- DERM-SUCCESS initial reader study: 108 readers trained in internal medicine or family practice (PCPs).
- DERM-SUCCESS pigmented lesion reader study: 77 PCP readers.
- DERM-ASSESS III reader study: 118 PCP readers.
4. Adjudication method for the test set
DERM-SUCCESS pivotal study:
- Pathology findings were validated by two to five central study dermatopathologists depending on the histological severity and discordance for the diagnoses. This implies a consensus or majority vote approach for adjudication, with more experts involved as complexity or disagreement increased.
5. If a multi reader multi case (MRMC) comparative effectiveness study was done, If so, what was the effect size of how much human readers improve with AI vs without AI assistance
Yes, three MRMC comparative effectiveness studies were done:
-
DERM-SUCCESS initial reader study (all skin cancers):
- Effect Size: Device-aided PCPs had a higher sensitivity of 91.4% compared with unaided PCP sensitivity of 82.0% (an increase of 9.4% points).
- The AUROC (overall performance) increased 5.4% (from 0.708 to 0.762).
-
DERM-SUCCESS pigmented lesion reader study:
- Effect Size (AUROC): Aided PCP AUROC was greater than unaided PCP AUROC by 1.5%.
- Effect Size (Sensitivity, all skin cancers): Aided PCP sensitivity increased from 80.5% to 86.3% (an increase of 5.8% points).
- Effect Size (Sensitivity, melanoma): Aided PCP sensitivity for melanoma increased from 68.8% to 75.4% (an increase of 6.6% points).
-
DERM-ASSESS III reader study (lesions suggestive of melanoma):
- Effect Size (AUROC): Aided AUROC was greater than unaided AUROC by 4.1%.
- Effect Size (Sensitivity): Aided sensitivity increased from 73.7% to 81.8% (an increase of 8.1% points).
- Effect Size (Sensitivity, melanoma): Aided sensitivity for melanoma increased from 70.2% to 79.1% (an increase of 8.9% points).
6. If a standalone (i.e. algorithm only without human-in-the loop performance) was done
Yes, a standalone performance evaluation was done in the DERM-SUCCESS pivotal clinical study.
- Overall Sensitivity (High-Risk Lesions): 95.5% (91.7%-97.6%) for identification of melanoma, SCC, or BCC.
- Overall Specificity (Benign Lesions): 20.7% (18.5%-23.1%).
- AUROC: 0.7896.
7. The type of ground truth used
The ground truth used was pathology findings (histological diagnosis). These findings were validated by multiple central study dermatopathologists.
8. The sample size for the training set
The training dataset initially contained 950 lesions contributing approximately 4,200 spectra. This study dataset was later used entirely for algorithm training during the initial development phases.
A separate table provides detailed demographics for a "Training Dataset" and a "Tuning Dataset":
- Training Dataset (Table 2): 1067 lesions (sum of age groups, excluding unknowns)
- Tuning Dataset (Table 2): 438 lesions (sum of age groups)
This suggests a larger training and tuning dataset was used for the final algorithm, totaling 1505 lesions.
9. How the ground truth for the training set was established
The text describes initial algorithm development using a "blinded, multicenter clinical study through an initial ML retrospective algorithm." The ground truth for both the training and testing datasets in this earlier study was established by evaluating "the ability of ESS to differentiate malignant from benign melanocytic lesions." While not explicitly stated for this training set, it is highly probable that the ground truth for the training set was established through histological diagnosis (biopsy results), consistent with how the test set ground truth was established, given the nature of skin cancer detection. The subsequent pivotal study (DERM-SUCCESS), which used biopsy as ground truth, built upon this prior development.
{0}------------------------------------------------
DE NOVO CLASSIFICATION REQUEST FOR DERMASENSOR
REGULATORY INFORMATION
FDA identifies this generic type of device as:
Software-aided adjunctive diagnostic device for use by physicians on lesions suspicious for skin cancer. A software-aided adjunctive diagnostic device for use by physicians on lesions suspicious for skin cancer is a prescription device that uses a software algorithm to analyze optical or other physical properties of a skin lesion and returns a classification of the skin lesion. The device is intended for use by a physician not trained in the clinical diagnosis and management of skin cancer as an adjunctive second-read device following identification of a suspicious skin lesion. It is not for use as a standalone diagnostic and is not for use to confirm a clinical diagnosis.
NEW REGULATION NUMBER: 21 CFR 878.1830
CLASSIFICATION: Class II
PRODUCT CODE: QZS
BACKGROUND
DEVICE NAME: DermaSensor
SUBMISSION NUMBER: DEN230008
DATE DE NOVO RECEIVED: February 2, 2023
SPONSOR INFORMATION: DermaSensor, Inc. 80 SW 8th Street #2000 Miami, Florida 33130 USA
INDICATIONS FOR USE
The DermaSensor device is indicated for use to evaluate skin lesions suggestive of melanoma, basal cell carcinoma, and/or squamous cell carcinoma in patients aged 40 and above to assist in the decision regarding referral of the patient to a dermatologist. The DermaSensor device should be used in conjunction with the totality of clinically relevant information from the clinical assessment, including visual analysis of the lesion, by physicians who are not dermatologists. The device should be used on lesions already assessed as suspicious for skin cancer and not as a screening tool. The device should not be used as the sole diagnostic criterion nor to confirm clinical diagnosis of skin cancer.
{1}------------------------------------------------
LIMITATIONS
The performance of the device has not been specifically evaluated in patients with increased risk for skin cancer, e.g., inherited or drug-induced photosensitivity; genetic predisposition to melanoma or basal cell carcinoma (BCC): immune compromise: or other medical conditions that increase the risk of skin cancer or its metastasis.
The device is intended to assist in clinical decisions related only to the skin malignancies melanoma (including severely atypical nevi), squamous cell carcinoma (SCC), and BCC. It has been tested on each of these three common skin cancer types but has not been tested on rare skin cancer types; thus, it should not be used for lesions that are suggestive of malignancies other than melanoma, BCC and/or SCC.
The device is intended for use on primary lesions only and has not been tested on lesions that are previously biopsied, recurrent, or metastatic; on scars, tattoos, sunburned skin, or within a hairy area; or which are located on palms, soles, mucosal surfaces, genitals, ears, within 1 cm of the eye, or under nails.
Consistent with the lower prevalence of skin cancer in Fitzpatrick skin phototypes IV-VI. less data is available for sensitivity of the DermaSensor device for melanoma in these patients. The decision to refer patients with suspicious pigmented lesions in this group should be primarily based on clinical concern.
The sale, distribution, and use of the DermaSensor are restricted to prescription use in accordance with 21 CFR 801.109.
The device is not intended to be used as a stand-alone diagnostic device.
The device is not intended to replace biopsy.
The device is not intended to replace clinical decision making.
PLEASE REFER TO THE LABELING FOR A MORE COMPLETE LIST OF CONTRAINDICATIONS, WARNINGS AND PRECAUTIONS.
DEVICE DESCRIPTION
The DermaSensor device (hereinafter referred to as 'DermaSensor', or the 'DermaSensor device'; Figure 1) utilizes optical spectroscopy and an artificial intelligence/machine learning (AI/ML) based software algorithm to analyze an intact skin lesion to which the device is non-invasively applied.
{2}------------------------------------------------
Figure 1: DermaSensor Device
Image /page/2/Picture/1 description: This image shows a black electronic device and its base. The device is sleek and modern, with a glossy finish. The base is cylindrical and has a similar glossy finish. The device appears to be designed to fit into the base, possibly for charging or storage.
The device is a combination of a handheld unit and a base unit. The handheld unit contains a xenon arc lamp and a fiber-optic probe tip which together transmit broadband white light to a lesion surface. Samples of the backscattered light from the tissue are collected by an adjacent detection optical fiber, also within the probe tip, and are conveyed to a microspectrometer, vielding Elastic Scattering Spectroscopy (ESS) spectral recordings. The handheld unit is operated using a touchscreen interface with step-by-step guidance. The small fiber-optic tip is the only component that contacts the patient. The handheld unit remains in the base when not actively being applied to a lesion and its battery is recharged by the base's wireless charging mechanism. The base unit also contains calibration material that is accessible to the handheld unit.
In the DermaSensor device, analysis of the optical recordings of backscattered light over the range of wavelengths is carried out using a proprietary ML-derived classifier algorithm. The spectrum of scattered intensity vs. wavelength is a pattern, which is analyzed by a proprietary classifier algorithm in the device's built-in microprocessor to assess for the potential presence of melanoma. squamous cell carcinoma, or basal cell carcinoma. An internal microprocessor and classifier algorithm analyze the recording and provide results to the user at the point of care, Results are provided as "Monitor," for a negative result, or "Investigate Further" for a positive result. For positive output ("Investigate Further") the DermaSensor additionally displays a Similarity Score of 1-10, with higher scores representing greater similarity to signals seen in malignant lesions.
SUMMARY OF NONCLINICAL/BENCH STUDIES
BIOCOMPATIBILITY/MATERIALS
DermaSensor does not contain substances that are intended to be introduced into the human body nor absorbed by or locally dispersed in the human body. The patient contacting materials are comprised of a stainless-steel tip encapsulating silica core fiber optics and synthetic polymer. The device tip is a surface-contacting device intended to contact intact skin for less than 3 minutes, and the user remains gloved during operation; therefore, meeting the requirements of <24 hours for only limited (e.g., transient) use. The risk assessment of the materials, manufacturing process, nature of contact and duration determined that only cytotoxicity, sensitization, and irritation testing were applicable to be performed. These tests were conducted according to the following applicable standards and guidance:
{3}------------------------------------------------
- · ISO 10993-1 Fifth edition 2018-08 Biological evaluation of medical devices -- Part 1: Evaluation and testing within a risk management process.
- · ISO 10993-5: 2009/revised 2014, Biological evaluation of medical devices Part 5: Tests for in vitro cytotoxicity.
- · ISO 10993-10: 2010/revised 2014, Biological evaluation of medical devices Part 10: Tests for irritation and skin sensitization.
- ISO 10993-12: 2012, Biological evaluation of medical devices Part 12: Sample . preparation and reference materials.
- · FDA guidance: Use of International Standard ISO 10993-1: 2016, "Biological evaluation of medical devices - Part I: Evaluation and testing within a risk management process" - Guidance for Industry and Food and Drug Administration Staff
The results are provided in Table 1. The device passed the tests and the results demonstrate mitigation of the risk of adverse tissue reactions and infections for the patient.
| Endpoint | Test Method | Findings |
|---|---|---|
| Cytotoxicity | MEM Elution Assay Method(ISO 10993-5:2009) | No evidence of cell lysis or toxicity |
| Sensitization | Guinea Pig MaximizationSensitization Test(ISO 10993-10:2010) | Test article met the requirements at24, 48 and 72 hrs. |
| Irritation | Intracutaneous Irritation Test(ISO 10993-10:2010) | No erythema or edema observed at24, 48 or 72 hrs. |
Table 1: Summary of biocompatibility test methods and results
CLEANING AND DISINFECTION
Cleaning and disinfection validations were conducted to validate the cleaning and disinfecting instructions for the reusable device components of the DermaSensor device. Cleaning and disinfection vas performed in accordance with FDA guidance Reprocessing Medical Devices in Health Care Settings: Validation Methods and Labeling - Guidance for Industry and Food and Drug Administration Staff (March 17, 2015), AAMI TIR 12:2010, and AAMI TIR30:2011 (R2016). The cleaning procedure per the instructions for use was performed and the bio load reduction and cytotoxicity (MEM Elution) were evaluated. All testing and results were considered to be adequate and met the above standards.
{4}------------------------------------------------
SHELF LIFE/STERILITY
The DermaSensor device is provided non-sterile and is not intended for sterilization by the end user. The use life of the DermaSensor device is a minimum of three years. Product uselife testing was performed for the device and the results were considered to be adequate.
ELECTROMAGNETIC COMPATIBILITY (EMC) & ELECTRICAL SAFETY
The following Electrical Safety and Electromagnetic Compatibility testing has been performed:
- ETSI EN 301 489 -17 V3.2.2 (2019-12) and V3.2.3 (2019-11): ElectroMagnetic . Compatibility (EMC) standard for radio equipment and services; Part 17: Specific conditions for Broadband Data Transmission Systems; Harmonized Standard for ElectroMagnetic Compatibility
- . IEC 60601-1-2:2014 Medical Electrical Equipment - Part 1-2: General requirements for basic safety and essential performance - Collateral Standard: Electromagnetic disturbances - Requirements and tests
- . IEC 60601-1:2005 + A1:2012 Medical electrical equipment - Part 1: General requirements for basic safety and essential performance
- . IEC 62133-2:2017 Secondary cells and batteries containing alkaline or other non-acid electrolytes - requirements for portable sealed secondary cells, and for batteries made from them, for use in portable applications-Part 2: Lithium systems
- . IEC 62471:2006 Photobiological safety of lamps and lamp systems. According to the standard, the pulsed light lamp has been considered as a non-general lighting lamp (non-GLS)
- IEC 60601-1-6: 2010 + A1:2013 Medical electrical equipment Part 1-6: General . requirements for basic safety and essential performance - Collateral standard: Usability
- . IEC 60601-2-57:2011 Medical electrical equipment - Part 2-57: Particular requirements for the basic safety and essential performance of non-laser light source equipment intended for therapeutic, diagnostic, monitoring and cosmetic/aesthetic use
DermaSensor passed all relevant portions of the testing, and the testing and results were considered to be adequate.
{5}------------------------------------------------
SOFTWARE/CYBERSECURITY
All components of the device are controlled/monitored by software, which is responsible for the functionality, user interface, safety checks and performance accuracy. The software was developed and tested according to the following FDA guidance documents and standards:
- . FDA guidance. Guidance for the Content of Premarket Submission for Software Contained in Medical Devices (May 11, 2005)
- FDA guidance, Content of Premarket Submissions for Management of Cybersecurity . in Medical Devices (October 2, 2014)
- . FDA guidance, Cybersecurity in Medical Devices: Ouality System Considerations and Content of Premarket Submissions (September 2023)
- IEC 62304: 2006 Medical device software Software life-cycle processes .
- · ISO 14971:2019 Medical devices Application of risk management to medical devices
Software documentation and testing, including cybersecurity information , demonstrates that the software will operate in a manner described in the specifications. The hazard analysis characterized software and cybersecurity risks, including device malfunction, measurement-related errors, sensor, cable and other hardware failures, and unauthorized access by malicious end users. The submission describes verification and validation testing to address the potential hazards with satisfactory results. The cybersecurity documentation included a cybersecurity hazard analysis and mitigation information, an upgrade and maintenance plan. other information for safeguarding the device during manufacturing and upon commercial distribution, and warning and precaution information in the product labeling.
Overall, the software documentation contains sufficient detail to provide reasonable assurance that the software will operate in a manner described in the specifications. Software security information demonstrated the device is protected from cyber vulnerability threats. All testing and results were considered to be adequate and met the above standards.
Algorithm Development
Initial work on the ESS technology identified the optimal wavelength range and spectral features relevant to skin cancer detection. Clinical development began with a blinded, multicenter clinical study through an initial ML retrospective algorithm, for which the algorithm was developed and tested on the same data set to evaluate the ability of ESS to differentiate malignant from benign melanocytic lesions. A subsequent blinded, multicenter clinical study investigated the performance of an algorithm using ESS and ML for the most common types of skin cancers (Rodriguez-Diaz et al. Photochem, Photobiol.
{6}------------------------------------------------
2019:95(6):1441-1445). The training dataset contained 950 lesions contributing ~4,200 spectra while the testing dataset included 361 lesions, for a total of 1,311 skin lesions. The training and testing datasets were independent and were chosen randomly. This study dataset was later used entirely for algorithm training.
The algorithm was trained in the spectral range of 360 to 810 nm, selected due to the low signal-to-noise ratio in this range. Forty-seven wavelengths were chosen within this range to reduce dimensionality. Tuning sensitivity estimates were 96.5% (221/229). Specificity for physician-biopsied lesions was 20.6% (28/136). The estimated specificity for physician-selected un-biopsied lesions was 33.0% (128/388). When pooling lesions across the tuning datasets, there was high sensitivity (i.e., at or above 90%) for each of the three individual cancers. The high sensitivity observed in tuning provided the basis for further clinical studies. The associated specificity of 20.6% was clinically accepted because it was paired with the high sensitivity and was supported by the greater benefit of identifying skin cancer than the risk of excess referrals.
The patient demographics for the lesions used to train and tune the dataset are summarized in Table 2. The training and tuning data included a majority of older patients (age 60-79), Fitzpatrick phototype I-III, and individuals who self-identified as White. This pattern reflects the greater incidence of skin cancer in these populations, particularly considering that lesions on the palms, soles, and under the nails were specifically excluded from the study.
| Training Dataset | Tuning Dataset | |||
|---|---|---|---|---|
| Age | Sample Size | Percent (%) | Sample Size | Percent (%) |
| 20-39 | 81 | 7.6% | 62 | 14.2% |
| 40-59 | 87 | 8.2% | 87 | 19.9% |
| 60-79 | 604 | 56.6% | 216 | 49.3% |
| 80+ | 45 | 4.2% | 73 | 16.7% |
| Unknown | 250 | 23.4% | 0 | 0.0% |
| Fitzpatrick Phototype | Sample Size | Percent (%) | Sample Size | Percent (%) |
| I | 67 | 6.3% | 64 | 14.6% |
| II | 332 | 31.1% | 160 | 36.5% |
| III | 244 | 22.9% | 91 | 20.8% |
| IV | 46 | 4.3% | 50 | 11.4% |
| V | 8 | 0.7% | 54 | 12.3% |
| VI | 3 | 0.3% | 16 | 3.7% |
| Unknown | 367 | 34.4% | 3 | 0.7% |
| Ethnicity/Race | Sample Size | Percent (%) | Sample Size | Percent (%) |
| White | 784 | 73.5% | 402 | 91.8% |
| Black or AfricanAmerican | 5 | 0.5% | 15 | 3.4% |
| Asian | 9 | 0.8% | 5 | 1.1% |
Table 2: Summary of patient demographics in algorithm training and tuning datasets
{7}------------------------------------------------
| Multiracial | 2 | 0.2% | 1 | 0.2% |
|---|---|---|---|---|
| Native Hawaiian orOther | 17 | 1.6% | 15 | 3.4% |
| Unknown | 250 | 23.4% | 0 | 0.0% |
| Gender | Sample Size | Percent (%) | Sample Size | Percent (%) |
| Male | 424 | 39.7% | 169 | 38.6% |
| Female | 265 | 24.8% | 195 | 44.5% |
| Unknown | 378 | 35.4% | 74 | 16.9% |
The anatomical locations of the lesions used to train and tune the dataset are summarized in Table 3. Lesions on the head and neck constituted the majority of lesions in the training and tuning data set. This pattern is consistent with the relatively higher rate of skin cancer on the head and neck, particularly of SCC and BCC, due to greater sun exposure.
Table 3: Summary of anatomical location of lesions in algorithm training and tuning datasets
| Training Dataset | Tuning Dataset | ||||
|---|---|---|---|---|---|
| Body site | Sample Size | Percent (%) | Sample Size | Percent (%) | |
| Sun-exposed | |||||
| Head/neck | 136 | 5.9% | 192 | 15.6% | |
| Chest/Upper back | 75 | 3.3% | 140 | 11.4% | |
| Forearms/hands | 80 | 3.5% | 120 | 9.7% | |
| Lower legs | 33 | 1.4% | 44 | 3.6% | |
| Other | 25 | 1.1% | 41 | 3.3% | |
| Unexposed | |||||
| Buttocks/hips | 2 | 0.1% | 5 | 0.4% | |
| Upper legs | 25 | 1.1% | 40 | 3.2% | |
| Lower trunk | 16 | 0.7% | 39 | 3.2% | |
| Other | 1 | 0.0% | 2 | 0.2% | |
| Unknown | 1900 | 82.9% | 610 | 49.5% |
The diagnoses for the lesions used to train and tune the dataset are summarized in Table 4. The greatest number of lesions were BCC, followed by SCC, and then melanoma (including highly atypical nevi). This pattern is consistent with the relatively higher rate of BCC than SCC than atypical pigmented lesions in the US population. Additional lesions were included as mimics of melanoma, SCC, and BCC to support sensitivity and specificity assessments.
Table 4: Summary of lesion diagnoses in algorithm training and tuning datasets
| Training Dataset | Tuning Dataset | |
|---|---|---|
| -- | ------------------ | ---------------- |
{8}------------------------------------------------
| Diagnosis Distribution | Sample Size | Percent (%) | Sample Size | Percent (%) |
|---|---|---|---|---|
| Melanoma and severelydysplastic nevi | 187 | 8.2% | 38 | 3.1% |
| SCC | 207 | 9.0% | 83 | 6.7% |
| BCC | 294 | 12.8% | 107 | 8.7% |
| Mixed Malignant | 2 | 0.1% | 1 | 0.1% |
| Melanoma mimic | ||||
| SK (brown) | 158 | 6.9% | 209 | 17.0% |
| Benign Nevi | 631 | 27.5% | 367 | 29.8% |
| Spitz Nevi | 0 | 0.0% | 0 | 0.0% |
| Lentigo | 98 | 4.3% | 41 | 3.3% |
| Angioma | 15 | 0.7% | 12 | 1.0% |
| Dermatofibroma | 4 | 0.2% | 9 | 0.7% |
| Tattoo/foreign objects | 0 | 0.0% | 0 | 0.0% |
| SCC Mimics | ||||
| AK | 172 | 7.5% | 145 | 11.8% |
| Wart | 14 | 0.6% | 4 | 0.3% |
| Dermatitis | 0 | 0.0% | 0 | 0.0% |
| SK (nonbrown) | 75 | 3.3% | 104 | 8.4% |
| BCC Mimics | ||||
| Acne/rosacea | 0 | 0.0% | 0 | 0.0% |
| Sebaceous hyperplasia/cyst | 3 | 0.1% | 4 | 0.3% |
| Other benign tumors | 0 | 0.0% | 0 | 0.0% |
| Insect bite | 0 | 0.0% | 0 | 0.0% |
| Xanthoma/ xanthelasma | 0 | 0.0% | 0 | 0.0% |
| Other malignant lesions | 6 | 0.3% | 1 | 0.1% |
| Other benign lesions | 427 | 18.6% | 109 | 8.8% |
NON-CLINICAL PERFORMANCE TESTING - BENCH
Bench testing was conducted to demonstrate that the DermaSensor device performs as expected under the anticipated conditions of use. Table 5 describes the bench testing conducted to demonstrate the device performance characteristics.
| Test | Description/Acceptance Criteria | |
|---|---|---|
| Dimensional | The tolerance stack for 1) optical design components and the2) wireless charging circuit is acceptable. | |
| Battery Performance | Testing was conducted to IEC 62133, IEC 62471, ETSI EN301 489-17, IEC 60601-1-2, IEC 60601-1, IEC 60601-2-57,and 60601-1-6 and passed. All batteries tested surpassed the | |
| use life target and the review of data showed the batteries werenot adversely impacted by the stress of the testing. | ||
| Lamp Reliability | All the test articles registered sample counts and demonstratedstability within the range specified in the acceptance criteria,indicating that the lamp life is stable over the simulated use lifedays. | |
| Optical and LampPerformance | Lamp engine testing was conducted to IEC 62471 and found tobe exempt. Testing was conducted to ETSI EN 301 489-17,IEC 60601-1-2, IEC 60601-1, IEC 60601-2-57, and 60601-1-6and passed. The optical configuration provided the maximumcaptured light from the lamp back to the microspectrometer. | |
| WiFi connectivity | Testing was conducted to ETSI EN 301 489-17, IEC 60601-1-2, IEC 60601-1, IEC 60601-2-57, and 60601-1-6 and passed.IP address assigned demonstrating that the device connectionis successful. | |
| Calibrator Configuration | The peak signal of a calibration recording achieved thenecessary dynamic range and demonstrated that the calibrationis successful. | |
| Wireless Charging | Testing was conducted to ETSI EN 301 489-17, IEC 60601-1-2, IEC 60601-1, IEC 60601-2-57, and 60601-1-6 and passed.The Handheld Unit charged while placed in the Base at 120vand 240v. The Handheld Unit remained functional whileplaced in and removed from the Base. | |
| Label Validation | All text content is legible. | |
| Estimation of Useful Life | The device functioned beyond useful life and simulated usedays. | |
| PCBA functionalverification | Testing was conducted to ETSI EN 301 489-17, IEC 60601-1-2, IEC 60601-1, IEC 60601-2-57, and 60601-1-6 and passed.Testing of the device operations performed as intendedindicating the PCBAs function as intended. | |
| System Packaging andDistribution | Testing was conducted to ASTM D4169 and passed. | |
| Manual Cleaning,Disinfection, Sensitization,Cytotoxicity, Irritation | Testing was conducted to AAMI TIR12, AAMI TIR30, ENISO 10993-1, EN ISO 10993-5, EN ISO 10993-10, EN ISO10993-12 and results passed. |
{9}------------------------------------------------
HUMAN FACTORS/USABILITY
Human factors testing was successfully completed with formative and summative validation studies, summarized below (Table 6).
Table 6: Summary of Human Factors Testing
{10}------------------------------------------------
| Study | Summary |
|---|---|
| Human FactorsValidation Testing | After formative testing, summative human factors validation testingevaluated user tasks through simulated-use to assess the use-relatedhazards of the device. Key attributes of the intended users, uses, and useenvironments were considered and sufficiently simulated to demonstratesafe and effective operation of the device under actual conditions of use.Two (2) user groups participated in the testing: 15 primary carephysicians (PCPs) representing a range of clinical experiences workingin primary care, and 15 mid-level practitioners that included PhysicianAssistants (PAs), Nurse Practitioners (NPs), and Registered Nurses(RNs). Participant performance during the simulated use scenarios,responses to knowledge/comprehension questions, and any participantcomments on performed user tasks and the device-user interfaces werereviewed.All 30 participants were able to identify the indications for use and allcontraindications for use, and that this device should not be used for thesole diagnosis of skin cancer. Use difficulties, close calls, and use errorswere analyzed to determine their likely root cause(s) and wereadequately mitigated. The results demonstrated that participating PCPswere able to independently and appropriately take recordings from mocklesions with acceptable levels of residual risk. |
REPEATABILITY/REPRODUCIBILITY
A repeatability study was conducted across two study sites, each with three PCP investigators. At least two investigators completed the full recording process four times for each enrolled lesion, twice with one device (Device A) and twice with a second device (Device B). Analyses included Intraclass Correlation (ICC) and Fleiss's Kappa (reproducibility with multiple raters); Positive Percent Agreement (PPA) between two runs and Negative Percent Agreement (NPA) for repeatability; and Average Positive Agreement (APA) between three physicians with Device A and Average Negative Agreement (ANA) between three physicians with Device A for reproducibility.
Secondary endpoints included a repeat of the primary analysis, separately for all malignant and all high-risk lesions using biopsy results as the gold standard. The malignant lesion analysis included all malignant lesions but did not include highly atypical melanocytic nevi since this pathology was not considered malignant and since there is a high rate of discordance for it among dermatopathologists.
The device agreement between device scans (i.e., repeatability) for all 64 lesions was 84.2% (95% CI 78.2-89.1%) and between physicians (i.e., reproducibility) was 87.8% (95% CI: 82.2-92.1%). For lesions that were high risk, the agreement was 97.1% (95% CI: 89.9-99.6%) for repeatability and 97.1% (95% CI: 89.9- 99.6%) for reproducibility. The PPA and NPA for repeatability were 94.8% (95% CI: 92.1-97.1%) and 46.0% (95% CI:
{11}------------------------------------------------
32.1-60.0%) respectively. The APA and ANA for reproducibility were 91.6% (95% CI: 86.8-95.7%) and 44.8% (95% CI: 20.8-63.3%) respectively.
The device agreement of the spectral score (1-10) between device scans (i.e., repeatability) was assessed. The repeatability percentage of scores 1-8 is below 37.5%, score 9 is 50.0%. and score 10 is 62.5%. The device agreement of the underlying continuous model output between device scans (repeatability) was also assessed.
Overall, the device shows low repeatability and reproducibility of negative binary decisions (NPA and ANA), low repeatability of the reported spectral similarity score, and high variability of the underlying continuous model output. The device shows repeatability and reproducibility of positive binary decisions (PPA and APA) greater than 90%. The repeatability and reproducibility information was considered acceptable within the context of the clinical performance testing and is included in the labeling.
CLINICAL PERFORMANCE TESTING
Eight clinical studies were conducted to assess performance of the DermaSensor device and algorithm during development. The final version of the device and algorithm were tested with lesions from the intended lesion types in four of these studies. The results of those four studies provide the basis for determination of safety and effectiveness of the device when used as intended to evaluate skin lesions that a primary care physician has identified as suggestive of melanoma, squamous cell carcinoma (SCC), and/or basal cell carcinoma (BCC).
Standalone Algorithm Performance Evaluation
The standalone performance of the device was assessed in the DERM-SUCCESS pivotal clinical study. The study was designed to identify the sensitivity and specificity of the device output compared to ground truth diagnosis (biopsy). The study also assessed the sensitivity and specificity of primary care physicians (PCPs) on the same lesion set. The study was an international, multicenter, prospective, blinded clinical study conducted at 22 study sites including 18 locations in the United States, with a range of sun exposure patterns (Arizona, California, Florida, Kansas, Minnesota, Pennsylvania, Rhode Island, Tennessee, Texas, Utah, Virginia), and four in Australia. The investigators were 30 primary care physicians, and the study enrolled a total of 1,005 participants with 1,579 lesions. The study enrolled participants with skin lesions suggestive of melanoma. BCC, and/or SCC. Each enrolled lesion was scanned with the device. Investigators and subjects were blinded to the device results, and clinical management was provided per standard of care. All enrolled lesions were biopsied. DermaSensor device output (binary classification of a negative result, "Monitor." or a positive result, "Investigate Further") was compared to pathology findings that were validated by two to five central study dermatopathologists depending on the histological severity and discordance for the diagnoses.
Patient inclusion/exclusion
The study enrolled men or women of any ethnic group aged 22 and older with a primary skin lesion(s) suggestive of melanoma, BCC, or SCC. Participants were recruited in
{12}------------------------------------------------
primary care offices where physicians performed their own clinical assessments, used the DermaSensor device, and performed biopsies of the lesions.
Patients were excluded from enrollment in the study if they met any of the following exclusion criteria:
- Lesion < 2.5mm in diameter or > 15mm in diameter -
- -Lesion surface not accessible (e.g. inside ears, under nails, completely covered by a crust or scale)
- Lesion on area of crust, psoriasis, eczema, or similar skin condition 」
- Lesion has erosion and/or ulceration with no area >2.5mm intact -
- -Lesion has foreign matter (e.g. tattoo, splinter, dermoscopy oils, or other medicated or non-medicated topical solutions)
- -Lesion in which the device tip cannot be placed entirely within the border of the targeted area
- Lesion located on acral skin (e.g. sole or palms) -
- Lesion located within 1 cm of the eye -
- -Lesion on or adjacent to scars, areas previously biopsied, or areas subjected to any past surgical intervention
- . Lesion located on mucosal surfaces (e.g. genitals, lips)
- -Lesion located on acute sunburn
- Six (6) or more lesions suggestive of melanoma, basal cell carcinoma, and/or ! squamous cell carcinoma requiring biopsy to assess risk of malignancy
See Table 7 and Figure 2, for additional summary information regarding the number of participants and inclusion/exclusion of participants in this study.
Table 7: Participant accountability
| Participantsn (%) | Lesionsn (%) | |
|---|---|---|
| Enrolled | 1,021 | 1,598 |
| ITT Safety Population | 1,013 (99.2%) | 1,591 (99.6%) |
| Excluded from ITT Population | 8 (0.8%) | 7 (0.4%) |
| mITT Effectiveness Population | 1,005 (98.4%) | 1,579 (98.8%) |
| Excluded from mITT Population | 8 (0.8%) | 12 (0.8%) |
{13}------------------------------------------------
Image /page/13/Figure/0 description: This image shows a flow diagram of a clinical trial. The trial started with 22 sites, 18 in the US and 4 in Australia. 1,028 subjects with 1,598 lesions were recruited, and then 1,021 subjects with 1,598 lesions were enrolled. The diagram also shows the number of subjects and lesions that were excluded from the trial at each stage, as well as the reasons for exclusion.
Figure 2: Inclusion/Exclusion diagram for all recruited participants and lesions
Table 8 summarizes the demographics of participants in the study. The participants in the DERM-SUCCESS pivotal study included a majority of older patients (age 60-79), Fitzpatrick phototype I-III, and individuals who self-identified as White. This pattern reflects the greater incidence of skin cancer in these populations, particularly considering that lesions on the palms, soles, and under the nails were specifically excluded from the study.
Table 8: Participant characteristics in DERM-SUCCESS pivotal study (mITT, n=1,005)
| Characteristics | n (%) |
|---|---|
| Sex | |
| Male | 487 (48.5%) |
| Female | 518 (51.5%) |
| Age | |
| Mean (STD) | 58.5 (15.1) |
| Median (Q1-Q3) | 60.0 (49.0-69.0) |
| Min-Max | 22-95 |
| Ethnicity | |
| Hispanic or Latino | 79 (7.9%) |
| Not Hispanic or Latino | 913 (90.8%) |
| Unknown | 13 (1.3%) |
| Race | |
| White | 976 (97.1%) |
| Native Hawaiian or Other Pacific Islander | 3 (0.3%) |
| Asian | 9 (0.9%) |
| Black or African American | 7 (0.7%) |
| Other/Multiracial | 10 (1.0%) |
| Fitzpatrick Phototype | |
| I - Always burns, never tans | 99 (9.9%) |
| II - Always burns, tans minimally | 278 (27.7%) |
| III - Sometimes mild burn, tans uniformly | 352 (35.0%) |
| IV - Burns minimally, always tans well | 148 (14.7%) |
| V - Very rarely burns, tans very easily | 110 (10.9%) |
| VI - Never burns | 18 (1.8%) |
| Risk Factors | |
| Ultraviolet light exposure (natural or tanning bed) | 455 (45.3%) |
| Weakened immune system | 32 (3.2%) |
| Family history of skin cancer | 332 (33.0%) |
| Xeroderma Pigmentosum | 1 (0.1%) |
| Fair skin; freckling; light hair | 362 (36.0%) |
| Many moles and/or dysplastic nevi | 331 (32.9%) |
| Personal history of skin cancer | 233 (23.2%) |
| New or changing lesion(s) | 725 (72.1%) |
| None of the above | 29 (2.9%) |
| Number of Eligible Lesions per subject Enrolled | |
| 1 | 657 (65.4%) |
| 2 | 207 (20.6%) |
| 3 | 78 (7.8%) |
| 4 | 41 (4.1%) |
| 5 | 22 (2.2%) |
{14}------------------------------------------------
{15}------------------------------------------------
Table 9 summarizes the lesion characteristics of patients in the study. In the DERM-SUCCESS pivotal study, lesions on the trunk constituted the majority of lesions, followed by lesions on the arms and head.
| Anatomic Location | n | % Anatomic Location | % Total |
|---|---|---|---|
| Head | 247 | 15.6% | |
| Scalp | 50 | 20.2% | 3.2% |
| Forehead | 59 | 23.9% | 3.7% |
| Cheek or nose | 85 | 34.4% | 5.4% |
| Chin | 8 | 3.2% | 0.5% |
| Other | 45 | 18.2% | 2.8% |
| Arm | 299 | 18.9% | |
| Upper Arm | 108 | 36.1% | 6.8% |
| Elbow | 11 | 3.7% | 0.7% |
| Forearm | 121 | 40.5% | 7.7% |
| Hand | 26 | 8.7% | 1.6% |
| Other | 33 | 11.0% | 2.1% |
| Leg | 207 | 13.1% | |
| Upper Leg | 66 | 31.9% | 4.2% |
| Lower Leg | 116 | 56.0% | 7.3% |
| Knee | 10 | 4.8% | 0.6% |
| Foot | 7 | 3.4% | 0.4% |
| Other | 8 | 3.9% | 0.5% |
| Trunk | 826 | 52.3% | |
| Neck | 63 | 7.6% | 4.0% |
| Chest | 128 | 15.5% | 8.1% |
| Upper Back | 361 | 43.7% | 22.9% |
| Lower Back | 149 | 18.0% | 9.4% |
| Abdomen | 64 | 7.7% | 4.1% |
| Pubic, Inguinal | 5 | 0.6% | 0.3% |
| Buttocks | 9 | 1.1% | 0.6% |
| Other | 47 | 5.7% | 3.0% |
Table 9: Lesion characteristics in DERM-SUCCESS pivotal study (mITT, n=1,579)
Table 10 summarizes the diagnoses of the lesions included in the study according to dermatopathology. The final pathology results were "high risk lesions" (high risk melanocytic lesions (melanoma and highly atypical nevi), SCC, and BCC). Consistent with the known occurrence rate, BCC was the most common lesion, with a slightly smaller number of SCC, and approximately one half the number of high risk melanocytic lesions.
{16}------------------------------------------------
| Parent Pathology or Diagnosis | n (%) |
|---|---|
| High Risk Lesions | 224 (14.2%) |
| Melanoma and severely atypical nevi | 48 (3.0%) |
| Squamous Cell Carcinoma (SCC) | 86 (5.4%) |
| Basal Cell Carcinoma (BCC) | 90 (5.7%) |
| Low Risk Lesions | 1355 (85.8%) |
| Benign melanocytic nevus | 500 (36.9%) |
| Seborrheic keratosis | 490 (36.2%) |
| Actinic Keratosis (AK) | 71 (5.2%) |
| Lentigo | 65 (4.8%) |
| Other | 53 (3.9%) |
| Verruca / Wart | 48 (3.5%) |
| Dermatofibroma | 42 (3.1%) |
| Lichenoid keratosis | 29 (2.1%) |
| Angioma or vascular lesion | 20 (1.5%) |
| Categories with <20 lesions | 37 (2.3%) |
| Sebaceous hyperplasia | 9 (0.7%) |
| Acrochordon | 8 (0.6%) |
| Scar | 6 (0.4%) |
| Epidermal cyst | 5 (0.4%) |
| Folliculitis | 3 (0.2%) |
| Angiofibroma | 2 (0.1%) |
| Rosacea | 1 (0.1%) |
| Excoriation | 1 (0.1%) |
| Solar elastosis | 1 (0.1%) |
| Angiokeratoma | 1 (0.1%) |
Table 10: Final dermatopathology diagnoses in DERM-SUCCESS pivotal study (mITT, n=1,579)
Study endpoints
The co-primary endpoints of the DERM-SUCCESS study were: DermaSensor sensitivity compared to that of the study physicians (primary care physicians); and a sensitivity + specificity > 1. The endpoint of sensitivity compared to the primary care physicians was selected to determine if the device provides superior sensitivity compared to primary care physicians. The second co-primary endpoint was selected to confirm that the performance was non-random. The secondary endpoint of the DERM-SUCCESS study was DermaSensor sensitivity compared to a performance goal of 90%. This endpoint was selected to compare the sensitivity of the DermaSensor device to that of the current gold standard for skin lesion evaluation, i.e., dermatologist evaluation, which literature review indicated is approximately 90%.
Results of DERM-SUCCESS pivotal study
The DERM-SUCCESS pivotal study demonstrated an overall sensitivity of 95.5% when assessed over all ages and Fitzpatrick skin phototypes for identification of melanoma. SCC,
{17}------------------------------------------------
or BCC. The first co-primary endpoint was met, as the device sensitivity for high-risk lesions, defined as the three target malignancies of melanoma (including highly atypical nevi), SCC, and BCC, across all ages and Fitzpatrick skin phototypes was 95.5%, which was superior to the sensitivity of PCPs for the same population (83.0%, p-value <0.0001) as shown in Table 11. Specificity for all high-risk lesions was 20.7% (18.5%-23.1%). lower than the PCP specificity of 54.2% (50.7-57.6%). The second co-primary endpoint was met with sensitivity (95.5%) + specificity (20.7%) > 1. The secondary endpoint was met, with overall sensitivity >90% (<0.001). The specificity of 20.7% was deemed clinically acceptable when paired with the high overall sensitivity (95.5%) due to the significantly greater risk of a false negative, particularly for melanoma, than a false positive. The low specificity is also mitigated by the intended use of the device as one component/adjunctive information in the decision of the provider.
| Sensitivity (%) and 95% Confidence Interval1 | ||||
|---|---|---|---|---|
| # Lesions | Device | PCP | P-value2 | |
| All malignant lesions -sensitivity | 224 | 95.5%(91.7%-97.6%) | 83.0%(77.7%-87.3%) | <0.001 |
| Benign lesions - specificity | 1355 | 20.7%(18.5%-23.1%) | 54.2%(50.7-57.6%) | N/A |
Table 11: Summary of DERM-SUCCESS study sensitivity and specificity results
1 95% Confidence Interval calculated accounting for within-subject correlations using Wilson method 2 P-value obtained using the method of moments for clustered paired data
The performance of the DermaSensor device across age groupings is presented in Table 12. Table 12: Device performance across age groupings
| AgeGroup | DeviceSensitivity | DeviceSpecificity | PCP UnaidedSensitivity | PCP AidedSensitivity | Difference |
|---|---|---|---|---|---|
| 40-49* | 100% (9/9) | 22.9% (39/170) | 83.1%* | 91.8%* | 8.7%* |
| 50-59 | 95.6% (43/45) | 24.1% (72/299) | 77.7-83.6% | 90.3-94.4% | 6.8-16.8% |
| 60-69 | 93.9% (62/66) | 22.0% (89/405) | 72.9-84.9% | 82.0-91.0% | 6.2-9.2% |
| 70-79 | 98.3% (59/60) | 12.6% (27/214) | 84.6-90.7% | 87.4-98.6% | 2.8-7.9% |
| 80-89 | 97.3% (36/37) | 10.8% (7/67) | 79.6-93.9% | 98.9-99.1% | 5.0-19.4% |
- Only DERM-SUCCESS Pigmented Reader Study results are available.
The area under the receiver operating characteristic (ROC) curve (AUROC) for the device was 0.7896. This was greater than the AUROC for PCPs, which was 0.7404 for all lesions. Device AUROC was also greater than the AUROC for the PCPs when limited to study lesions for which they reported low confidence in their clinical diagnosis, which was 0.5555. When using the ROC curves to interpolate the PCPs' specificity at the same level of sensitivity as the device (i.e., 95.5%), the corresponding specificity of the device is 20.7% while the PCP specificity is 9.0%.
{18}------------------------------------------------
Spectral similarity score evaluation
For positive output ("Investigate Further") the DermaSensor displays a Similarity Score of 1-10, with higher scores representing greater similarity to signals seen in malignant lesions. Standalone performance testing demonstrated that higher scores were associated with a higher positive predictive value (PPV), as calculated relative to the frequency of the score among all spectral scores displayed (total 1,288 scores; Table 13). The PPVs for each score, when assessed alone, are impacted by prevalence (frequency) of each score within the lesions of the pivotal study, which varied from 4.8% to 16.5%. Prevalence of skin cancer in lesions assessed for skin cancer may be different in real world use, and this would change the PPV. Therefore, this assessment was intended to show a correlation between higher scores and greater likelihood of malignancy, and not to predict PPV of the score in clinical use outside of the DERM-SUCCESS pivotal study.
| Spectral Score | Positive Total | PPV (Frequency) | Lower and upper bounds |
|---|---|---|---|
| 1 | 13 / 202 | 6.4% (15.7%) | 3.7 - 10.9% |
| 2 | 7 / 213 | 3.3% (16.5%) | 1.5 - 7% |
| 3 | 15 / 177 | 8.5% (13.7%) | 4.9 - 14.2% |
| 4 | 14 / 132 | 10.6% (10.2%) | 6.2 - 17.5% |
| 5 | 12 / 103 | 11.7% (8%) | 6.6 - 19.7% |
| 6 | 25 / 104 | 24% (8.1%) | 16.7 - 33.2% |
| 7 | 33 / 117 | 28.2% (9.1%) | 20.5 - 37.5% |
| 8 | 26 / 101 | 25.7% (7.8%) | 18.1 - 35.2% |
| 9 | 31 / 77 | 40.3% (6%) | 30.3 - 51.1% |
| 10 | 38 / 62 | 61.3% (4.8%) | 48.3 - 72.9% |
Table 13: Summary of positive values by spectral similarity score output
Standalone Algorithm Performance Testing Conclusions
The DERM-SUCCESS pivotal study demonstrated an overall sensitivity for malignancy of 95.5%. The first co-primary endpoint was met, as the device sensitivity for all malignancies was superior to the sensitivity of PCPs, 83.0%, for the same population. Specificity for benign lesions was 20.7%, meeting the second co-primary endpoint of sensitivity + specificity > 1. The secondary endpoint was met, with overall sensitivity >90%. A correlation was observed between the magnitude of the similarity score and positive predictive value. thereby meeting the second co-primary endpoint.
Safety
No adverse events were observed during the DERM-SUCCESS pivotal study. False negative and false positive device outcomes were not considered adverse events as part of the study due to the non-interventional design. However, false negatives would be considered adverse events during real world use of the device.
Adjunctive Use Performance Testing
Three studies were completed to assess performance of the device as an aid to PCPs. The first study was conducted with all lesions suggestive of skin cancer, per the indications for
{19}------------------------------------------------
use, to demonstrate superior performance of the device-aided PCP compared to unaided PCPs. However, melanoma poses the highest risk of the three lesion types assessed due to a higher metastatic potential. higher morbidity, and higher mortality compared with SCC or BCC. Therefore, two additional reader studies were performed to ensure the device does not negatively impact the clinical decision of readers specifically in case sets enriched for pigmented lesions. Both studies were performed in a sequential read design to mimic the intended use of the device as a second read device.
DERM-SUCCESS initial reader study: "A Multi-Reader, Multi-Case (MRMC) Companion Study to the DERM-SUCCESS Clinical Study"
The DERM-SUCCESS initial reader study was intended to assess superiority of deviceaided PCPs over unaided PCPs across all skin cancers (melanoma, SCC, and BCC). The study included 108 readers trained in internal medicine or family practice who evaluated 25 malignant (20% melanoma, 40% SCC, 40% BCC) and 25 benign lesions. Readers were shown images of the lesions either with or without the device output in random order. All lesions were viewed twice during the study, once without device output and once with device output. After each lesion, the readers were asked to provide a management decision of "Refer" or "Do not refer." The co-primary endpoints were aided sensitivity superiority to unaided sensitivity, and aided sensitivity + specificity >1. The study demonstrated that:
- device-aided PCPs had a higher sensitivity of 91.4% (95% CI: 85.7-97.1%) . compared with unaided PCP sensitivity 82.0% (95% CI: 76.4-87.6%; p=0.0027); and
- sensitivity + specificity of PCPs aided by the device was greater than 1 (Odds Ratio . 6.8. CI: 4.7 to 9.8. p<0.001).
The co-primary endpoints were therefore met.
Additional analyses showed that:
- device-aided PCPs had a lower specificity of 32.4% (95% CI: 20.7-44.1%) . compared with unaided PCP specificity 44.2% (95% CI: 36.0-52.4%); and
- when assessed for overall performance of aided vs. unaided reader performance, . the area under the receiver operating characteristic curve (AUROC) increased 5.4% from 0.708 to 0.762.
The data in this study, in which DermaSensor output was provided concurrently with the image of the lesion, demonstrated that PCPs performed better when viewing images with the device output than when viewing the image alone. However, the increased sensitivity was associated with reduced specificity. The specificity of the device-aided PCP was higher in this study than the specificity of the device alone, as quantified in the standalone performance testing, indicating that adjunctive use by PCPs may, to some extent, mitigate the risk associated with the low specificity of the device alone. Nonetheless, use of the device is still expected to result in higher referral of both true positives and false positives. The demonstrated increase in AUROC, which reflects both sensitivity and specificity, indicates that overall accuracy was greater in aided reads than in unaided reads.
{20}------------------------------------------------
DERM-SUCCESS pigmented lesion reader study: "A Multi-Reader, Multi-Case (MRMC) Study for Evaluating the Impact of the DermaSensor Device on Primary Care Physicians' Assessment of Pigmented Lesions"
The DERM-SUCCESS pigmented lesion reader study was intended to demonstrate noninferiority of PCPs aided by the device compared to unaided PCPs for pigmented lesions suggestive of skin cancer. Seventy-seven PCP readers were shown an image of a lesion and entered an initial management decision of "Refer" or "Do not refer." The device output was then revealed, and the readers entered a second management decision. The lesions included 69 malignant lesions (36% melanoma, 26% SCC, 36% BCC) and 67 benign lesions (59 pigmented, 10 non-pigmented). The primary endpoint was aided AUROC noninferiority for all skin cancers. When the device result was shown the aided PCP AUROC was found to be greater than the unaided PCP AUROC by 1.5%, with statistically significant non-inferiority (p<0.001). The primary endpoint was therefore met.
Secondary endpoints were aided reader sensitivity non-inferiority for all skin cancers. aided reader sensitivity non-inferiority for melanoma, and specificity non-inferiority margin of 20%. Analysis of secondary endpoints showed:
- . aided PCP sensitivity for all skin cancers increased from 80.5% (95% C1: 75.1-85.9%) to 86.3% (95% CI: 65.3-85.6%) with statistically significant superiority (p<0.001);
- aided PCP sensitivity for melanoma increased from 68.8% (95% CI: 59.6-78.1%) . to 75.4% (95% CI: 65.3-85.6%) with statistically significant non-inferiority (p<0.001); and
- aided PCP specificity for all skin cancers decreased from 45.7% (95% CI: 39.2-. 52.2%) to 35.1% (95% CI: 27.4-42.9%) with statistically significant non-inferiority (p=0.003).
This second-read design study demonstrated that PCPs improved their initial management decision when provided with the device output and were able to revise the management decision. Use of the device as a second read improved the sensitivity of the study PCPs for skin cancers overall and did not decrease sensitivity for lesions with the highest metastatic potential (i.e., melanoma). However, the increased sensitivity was associated with reduced specificity, which may result in higher referral of both true positives and false positives. AUROC, which reflects both sensitivity and specificity, indicated that the device led to improvement in overall accuracy based on the point-estimate; the statistically significant non-inferiority of AUROC supports that in a study enriched with pigmented lesions, PCP accuracy does not decrease for lesions with high metastatic potential, mitigating the risk of false negatives in this high-risk patient population.
The results are summarized in Table 14.
Table 14: PCP Management AUROC and Sensitivity Without and With Device
| LesionAssessments | Estimate (95% Confidence Interval) | |||
|---|---|---|---|---|
| Without Device | With Device | P-value |
{21}------------------------------------------------
| AUROC | n=20,944 | 0.702(0.653-0.751) | 0.717(0.661-0.773) | <0.001 |
|---|---|---|---|---|
| Sensitivity | n=10,626 | 80.5%(75.1-85.9%) | 86.3%(81.1-91.5%) | <0.001* |
| Sensitivity forMelanoma | n=3,850 | 68.8%(59.6-78.1%) | 75.4%(65.3-85.6%) | 0.029* |
| Specificity | n=10,318 | 45.7%(39.2-52.2%) | 35.1%(27.4-42.9%) | 0.003 |
*p-value for superiority, otherwise p-value is for non-inferiority
DERM-ASSESS III reader study: "A Multi-Reader, Multi-Case (MRMC) Study for Evaluating the Impact of the DermaSensor Device on Primary Care Physicians' Assessment of Lesions Suggestive of Melanoma"
The DERM-ASSESS III reader study for lesions suggestive of melanoma was intended to demonstrate non-inferiority of PCPs aided by the device compared to unaided PCPs, for lesions suggestive of melanoma. The readers, 118 PCPs, were shown an image of a lesion and entered an initial management decision of "Refer" or "Do not refer." The device output was then revealed, and the readers entered a second management decision. The lesions included 50 malignant lesions (68% melanoma, 16% SCC, 16% BCC) and 50 benign lesions. The primary endpoint was aided AUROC non-inferiority for all skin cancers. When the device result was shown the aided read AUROC was found to be greater than the unaided AUROC by 4.1%, with statistically significant non-inferiority (p<0.001) and superiority (p=0.036). The primary endpoint was therefore met. Secondary endpoints aided reader sensitivity non-inferiority for all skin cancers, aided sensitivity non-inferiority for melanoma, and specificity non-inferiority margin of 20%, were also met. The results are summarized in Table 15.
This study also demonstrated that use of the device as a second read source of information improved the sensitivity of the PCPs in the study. The increase in sensitivity was associated with lower specificity. This is anticipated to result in higher referral of both true positives and false positives. AUROC, which reflects both sensitivity and specificity, was improved when the device output was provided, indicating that the device led to improvement in overall accuracy based on the point-estimate; the statistically significant non-inferiority of AUROC supports that in a study enriched with pigmented lesions, PCP accuracy does not decrease for lesions with high metastatic potential, mitigating the risk of false negatives in this high-risk patient population.
| LesionAssessments | Estimate (95% Confidence Interval) | |||
|---|---|---|---|---|
| Without Device | With Device | P-value | ||
| AUROC | n=11,800 | 0.630(0.582-0.678) | 0.671(0.611-0.732) | 0.036* |
| Sensitivity | n=5,900 | 73.7%(67.7-79.6%) | 81.8%(76.0-87.6%) | <0.001* |
| Table 15: PCP Management AUROC and Sensitivity Without and With Device | |
|---|---|
{22}------------------------------------------------
| Sensitivity forMelanoma | n=3,850 | 70.2%(62.9-77.6%) | 79.1%(72.4-85.7%) | <0.001* |
|---|---|---|---|---|
| Specificity | n=5,900 | 44.2%(38.1-50.2%) | 38.6%(30.9-46.2%) | <0.001 |
*p-value for superiority, otherwise p-value is for non-inferiority
Clinical Performance Results Analysis by Skin Cancer Type
Melanoma, SCC, and BCC are all skin cancers, but they are three distinct diseases, with different natural histories, risk of metastasis, urgency of diagnosis and treatment, and mortality rates. Device standalone performance was assessed for each lesion type and is summarized below (Table 16). Device standalone performance and the performance of the device in aiding PCP performance was further assessed by patient age and Fitzpatrick skin phototypes; this is also summarized below by each lesion type.
Table 16 (a, b): Standalone device sensitivity and specificity by skin cancer subtype (based on binary output correlation with final histological diagnosis)
a. All participants
| Final Diagnosis | n | True Positives orTrue Negatives¹ | Sensitivity orSpecificity (%)² |
|---|---|---|---|
| High risk lesions | 224 | 214 | 95.5% (91.7%-97.6%) |
| Melanoma and severely atypical nevi | 48 | 42 | 87.5% (76.4%-93.8%) |
| Melanoma | 29 | 24 | 82.8% (64.2%-94.2%) |
| Highly Atypical Nevus | 19 | 18 | 94.7% (91.9%-96.6%) |
| Squamous Cell Carcinoma (SCC) | 86 | 84 | 97.7% (91.1%-99.4%) |
| Basal Cell Carcinoma (BCC) | 90 | 88 | 97.8% (91.3%-99.5%) |
| Low risk lesions | 1355 | 281 | 20.7% (18.5%-23.1%) |
b. Participants age 40+ (Intended use population)
| Final Diagnosis | n | True Positives orTrue Negatives1 | Sensitivity orSpecificity (%)2 |
|---|---|---|---|
| High risk lesions | 217 | 209 | 96.3% (92.9-98.4%) |
| Melanoma and severely atypical nevi | 41 | 37 | 90.2% (76.9-97.3%) |
| Squamous Cell Carcinoma (SCC) | 86 | 84 | 97.7% (91.1%-99.4%) |
| Basal Cell Carcinoma (BCC) | 90 | 88 | 97.8% (91.3%-99.5%) |
| Low risk lesions | 1155 | 234 | 20.3% (18.0-22.7%) |
1 True positives for high risk lesions and true negatives for low risk lesions
295% CI calculated accounting for within-subject correlations using Wilson method.
Melanoma (including severely/highly atypical nevi):
{23}------------------------------------------------
Standalone device sensitivity for melanoma was highest in Fitzpatrick skin phototypes I and II (100% and 91.7%, respectively for the full population; 100% and 90.9%, for age 40+). The study included Fitzpatrick skin phototypes VI, V, and VI with a relatively small number of lesions per phototype (5, 6, and 1). consistent with the lower incidence of melanoma in Fitzpatrick skin phototypes IV-VI. Standalone device sensitivity was high in participants aged 40-59 (92.9%) and 60-79 (91.7%) but relatively lower in participants under age 40 (71.4%) and participants aged over 80 (66.7%).
When used as a second read device for adjunctive use in clinical decision support, the DermaSensor device improved performance of PCPs for participants ages 40 and older and with Fitzpatrick skin phototypes I-VI. Table 17 and Table 18 below summarize device standalone sensitivity and specificity and degree of effect on readers by subgroup by Fitzpatrick skin phototype and age group. We note that melanoma patients with Fitzpatrick skin phototypes IV-VI, were less represented in the studies consistent with their lower incidence in the US population. To assess safety in this patient subgroup, the performance of the device was compared to the PCP performance for the 12 melanomas enrolled in this subgroup in the DERM-SUCCESS standalone performance test study. The unaided PCP melanoma sensitivity was 66.7% (8/12), compared to the device sensitivity of 83.3% (10/12) for the same lesions. Thus, the data are suggestive of a margin of safety between the device sensitivity and PCP sensitivity for melanoma patients in this subgroup.
| Table 17 (a.b): Summary of Device Melanoma Performance and PCP Melanoma |
|---|
| Performance Based on Fitzpatrick Skin Phototypes |
| FitzpatrickPhototype | DERM-SUCCESSStandalone device performance | DERM-SUCCESSPigmented lesion reader study - sensitivity | ||||
|---|---|---|---|---|---|---|
| Sensitivity | Specificity | No.Lesions | PCPUnaided | PCPAided | % Changefor Aided | |
| Overall | 87.5% (42/48) | 23.7% (216/913) | 25 | 68.8% | 75.4% | 6.6% |
| I | 100.0% (10/10) | 19.3% (16/83) | 4 | 74.0% | 81.2% | 7.1% |
| II | 91.7% (11/12) | 24.4% (50/205) | 9 | 80.7% | 86.1% | 5.5% |
| III | 78.6% (11/14) | 21.4% (65/304) | 7 | 59.6% | 75.1% | 15.6% |
| IV | 80.0% (4/5) | 26.2% (43/164) | 2 | 72.7% | 57.8% | -14.9% |
| V | 83.3% (5/6) | 26.1% (37/142) | 3 | 45.5% | 48.1% | 2.6% |
| VI | 100.0% (1/1) | 33.3% (5/15) | 0 | N/A | N/A | N/A |
a. All participants
b. Participants age 40+ (Intended use population)
| DERM-SUCCESSStandalone device performance | DERM-SUCCESSPigmented lesion reader study - sensitivity | |||||
|---|---|---|---|---|---|---|
| FitzpatrickPhototype | Sensitivity | Specificity | No.Lesions | PCPUnaided | PCPAided | % Changefor Aided |
| Overall | 90.2% (37/41) | 23.0% (171/742) | 21 | 69.9% | 78.4% | 8.5% |
| I | 100% (9/9) | 20.9% (14/67) | 3 | 72.7% | 82.7% | 10.0% |
{24}------------------------------------------------
| II | 90.9% (10/11) | 25.3% (45/178) | 8 | 79.4% | 86.4% | 7.0% |
|---|---|---|---|---|---|---|
| III | 81.8% (9/11) | 20.0% (48/240) | 6 | 59.1% | 76.4% | 17.3% |
| IV | 80.0% (4/5) | 26.2% (43/164) | 2 | 72.7% | 57.8% | -14.9% |
| V | 100% (4/4) | 26.7% (31/116) | 2 | 57.1% | 66.2% | 9.1% |
| VI | 100.0% (1/1) | 33.3% (5/15) | 0 | N/A | N/A | N/A |
| Table 18: Summary of Device Melanoma Performance and PCP Melanoma Performance | ||
|---|---|---|
| Based on Age Group |
| DERM-SUCCESSStandalone device performance | DERM-SUCCESSPigmented Lesion reader study - sensitivity | |||||
|---|---|---|---|---|---|---|
| Age Group | DeviceSensitivity | DeviceSpecificity | No.Lesions | PCPUnaided | PCP Aided | % Changefor Aided |
| Age 20-40 | 71.4% (5/7) | 25.3% (46/182) | 4 | 63.3% | 60.1% | -3.20% |
| Age 41-60 | 92.9% (13/14) | 27.4% (57/208) | 8 | 75.8% | 79.2% | 3.40% |
| Age 61-80 | 91.7% (22/24) | 28.3% (30/106) | 13 | 66.2% | 77.8% | 11.60% |
| Age 81+ | 66.7% (2/3) | 0.0% (0/4) | 0 | NA | NA | NA |
| Overall | 87.5% (42/48) | 23.7% (216/913) | 25 | 68.8% | 75.4% | 6.6% |
| Ages 40+ | 90.2% (37/41) | 23.0% (171/742) | 21 | 69.9% | 78.4% | 8.5% |
The DERM-SUCCESS pigmented lesion reader study was analyzed to determine whether the improvement in sensitivity in the context of reduced specificity had a net positive change in referral decision. The analysis demonstrated that the ratio of readers' management decisions that were correctly changed from unaided "Do not refer" to aided "Refer" (resulting in correction of a potential missed diagnosis), compared to management decisions that were incorrectly changed from unaided "Refer" to aided "Do not refer (resulting in a missed diagnosis) was 248:49, indicating that the device was more likely to correct user management decision than to harm management decision by a factor of 5, indicating an overall benefit as an aid in second reading of pigmented lesions suspicious for melanoma (Table 19).
| Table 19: Summary of converted reader decisions. | |
|---|---|
| -------------------------------------------------- | -- |
| Unaided reads | Aided reads converted | |
|---|---|---|
| Total melanomareads1925 | False negative600 | After device true positive248 (43% conversion rate) |
| True positive1325 | After device false negative49 (3.7% conversion rate) |
Number of reads = number of melanomas x number of readers
Squamous Cell Carcinoma (SCC):
Overall standalone device sensitivity for SCC detection was 97.7%, with specificity of 17.3% for non-melanocytic lesions. There was insufficient support (number of lesions studied) for participants aged below 40 to confirm that benefit outweighed risk. The table
{25}------------------------------------------------
below summarizes performance by clinically-relevant subgroups from the DERM-SUCCESS standalone performance study and DERM-SUCCESS pigmented lesion reader study for SCC lesions.
When used as a second read device for adjunctive use in clinical decision support, the DermaSensor device improved performance of PCPs for participants aged 40 and over. Table 20 and Table 21 below summarize device standalone sensitivity and specificity by Fitzpatrick skin phototype and age group.
| Table 20: Summary of Device SCC Performance for All Patients Based on Fitzpatrick Skin | ||||
|---|---|---|---|---|
| Phototypes |
| DERM-SUCCESSStandalone device performance | ||
|---|---|---|
| FitzpatrickPhototype | Sensitivity | Specificity |
| I | 90.0% (9/10) | 16.2% (11/68) |
| II | 100% (28/28) | 19.5% (46/236) |
| III | 100% (19/19) | 10.8% (29/269) |
| IV | 100% (18/18) | 20.4% (31/152) |
| V | 90.0% (9/10) | 23.3% (27/116) |
| VI | 100% (1/1) | 28.6% (4/14) |
| Table 21: Summary of Device SCC Performance and PCP SCC Performance for All | |
|---|---|
| Patients Based on Age Group |
| Age(years) | DERM-SUCCESSStandalone device performance | DERM-SUCCESSPigmented Lesion reader study -sensitivity | |||
|---|---|---|---|---|---|
| Sensitivity | Specificity | UnaidedReader | AidedReader | % Changefor Aided | |
| 20-40 | 100% (1/1) | 22.0% (9/41) | 84.40% | 93.50% | 9.10% |
| 41-60 | 100% (10/10) | 21.1% (57/270) | 88.30% | 95.70% | 7.40% |
| 61-80 | 96.7% (58/60) | 15.4% (77/500) | 76.80% | 81.70% | 4.90% |
| 81+ | 100% (15/15) | 11.4% (5/44) | 94.50% | 99.70% | 5.20% |
1A total of 77 PCP readers performed evaluations of these cases.
Basal Cell Carcinoma (BCC):
Overall standalone device sensitivity for BCC detection was 97.8% with specificity of 17.3% for non-melanocytic lesions. Table 22 and Table 23 below summarize performance by clinically-relevant subgroups from the DERM-SUCCESS clinical study and DERM-SUCCESS pigmented reader study for BCC lesions.
When used as a second read device for adjunctive use in clinical decision support, the DermaSensor device improved performance of PCPs for patients aged 40 and older. The
{26}------------------------------------------------
tabulated results below summarize device standalone sensitivity and specificity and degree of effect on readers by subgroup by Fitzpatrick skin phototype and age group.
| DERM-SUCCESSStandalone device performance | ||
|---|---|---|
| FitzpatrickPhototype | Sensitivity | Specificity |
| I | 100% (7/7) | 16.2% (11/68) |
| II | 97.4% (37/38) | 19.5% (46/236) |
| III | 100% (35/35) | 10.8% (29/269) |
| IV | 100% (2/2) | 20.4% (31/152) |
| V | 85.7% (6/7) | 23.3% (27/116) |
| VI | 100% (1/1) | 28.6% (4/14) |
Table 22: Summary of Device BCC Performance for All Patients Based on Fitzpatrick Skin Phototypes
| Table 23: Summary of Device BCC Performance and PCP BCC Performance for All | |
|---|---|
| Patients Based on Age Group |
| Age(years) | DERM-SUCCESSStandalone device performance | DERM-SUCCESSPigmented Lesion reader study -sensitivity | |||
|---|---|---|---|---|---|
| Sensitivity | Specificity | UnaidedReader | AidedReader | % Changefor Aided | |
| 20-40 | 100% (1/1) | 22% (9/41) | NA | NA | NA |
| 41-60 | 94.4% (34/36) | 21.1% (57/270) | 87.20% | 92.60% | 5.40% |
| 61-80 | 100% (39/39) | 15.4% (77/500) | 92.60% | 97.80% | 5.20% |
| 81+ | 100% (14/14) | 11.4% (5/44) | 92.90% | 97.40% | 4.50% |
Summary of Clinical Information
Overall, the data presented demonstrate increased sensitivity of melanoma, SCC, and BCC detection when PCPs use the DermaSensor device in the intended use population as a second-read device for adjunctive use, and that adjunctive use, to some extent, mitigates the risk of increased referrals for benign lesions associated with a relatively low specificity of the device.
POSTMARKET SURVEILLANCE
In order to satisfy special control (1) below. DermaSensor must collect and report post-market surveillance data acquired under anticipated conditions of use to demonstrate that the device performs as intended when used to analyze data from the intended patient population. Specifically, the sponsor must conduct post-market clinical validation performance testing of the DermaSensor device in patients from demographic groups representative of the U.S. population, to include populations for whom uncertainty remains in the data provided due to lower representation in the mITT population (i.e. Fitzpatrick skin phototypes with lower prevalence of skin cancer).
{27}------------------------------------------------
FDA expects that the post-market clinical validation performance testing will include a statistically justified study sample size to confirm that performance of the device in post-market use is not inferior to the performance observed in the pre-market studied subgroup(s) or the overall population. The study should enroll a representative range of subjects with overrepresentation of patients who have Fitzpatrick skin phototypes with lower prevalence of skin cancer (e.g., Fitzpatrick skin phototypes IV. V. and VI). The study should record age, sex, socioeconomic status, race, and risk factors for skin cancer development and progression, and assess their impact on the device's effectiveness as measured by sensitivity, specificity, and/or improvement of the user's management decision.
LABELING
Device labeling includes the product labels on the device and instructions for use labeling. The instructions for use include a description of the device components and outputs, device technical parameters, instructions for use of the device, electromagnetic compatibility information, shelf life, and device reprocessing and disposal instructions for use also contain relevant information about the patients in the algorithm development and training datasets, and findings from the standalone and adjunctive use clinical studies with the performance characteristics of the device when used as intended. The document also states the shelf life for any sterile components as well as the necessary measures to properly dispose of any single use items and clean the reusable components of the device. The sale, distribution, and use of DermaSensor device are restricted to prescription use in accordance with 21 CFR 801.109.
Labeling also includes the following:
- A description of performance measures, including sensitivity and specificity, and . statistical confidence intervals, as well as performance of the device for all clinically relevant subgroups within the intended use population
- . A statement that the device is not intended for use as a standalone diagnostic.
- . A statement that performance of the device has not been specifically evaluated in patients with increased risk for skin cancer, e.g., inherited or drug-induced photosensitivity; genetic predisposition to melanoma or basal cell carcinoma (BCC); immune compromise; or other medical conditions that increase the risk of skin cancer or its metastasis.
- . A statement that the device should not be used for lesions that are suggestive of malignancies other than melanoma, BCC and/or SCC.
- . A statement that the device is intended for use on primary lesions only and has not been tested on lesions that are previously biopsied, recurrent, or metastatic; on scars, tattoos, sunburned skin, or within a hairy area; or which are located on palms, soles, mucosal surfaces, genitals, ears, within 1 cm of the eve, or under nails.
- A statement that, consistent with the lower prevalence of skin cancer in Fitzpatrick skin . phototypes IV-VI. less data is available for sensitivity of the DermaSensor device for melanoma in these patients. The decision to refer patients with suspicious pigmented lesions in this group should be primarily based on clinical concern.
Labeling will be updated in accordance with data collected via post-market surveillance to provide updated clinical performance data and effectiveness data of the device.
{28}------------------------------------------------
RISKS TO HEALTH
The table below identifies the risks to health that may be associated with use of a software-aided adjunctive diagnostic device for use by physicians on lesions suspicious for skin cancer and the measures necessary to mitigate these risks.
| Identified Risks to Health | Mitigation Measures |
|---|---|
| False negative results, leading tofailure to treat cancer and cancerprogression, or false positive results,leading to unnecessary referrals and/ormedical procedures | Clinical performance testing |
| Postmarket surveillance | |
| Non-clinical performance testing | |
| Labeling | |
| False results or failure to generate aresult due to use error or improperdevice use | Precision testing |
| Human factors testing | |
| Labeling | |
| False results or failure to generate aresult due to device failure ormalfunction | Non-clinical performance testing |
| Precision testing | |
| Software verification, validation, and hazard analysis | |
| Labeling | |
| Electrical, thermal, mechanical, orlight exposure-related injury | Electrical, mechanical, and thermal safety testing |
| Software verification, validation, and hazard analysis | |
| Labeling | |
| Interference with other devices | Electromagnetic compatibility testing |
| Adverse tissue reaction | Biocompatibility evaluation |
| Infection and cross contamination | Cleaning and disinfection validation |
| Labeling |
SPECIAL CONTROLS
In combination with the general controls of the FD&C Act. the software-aided adjunctive diagnostic device for use by physicians on lesions suspicious for skin cancer are subject to the following special controls:
- (1) Data obtained from premarket clinical performance validation testing and post-market surveillance acquired under anticipated conditions of use must demonstrate that the device performs as intended in the intended patient population, unless FDA determines based on the totality of the information provided for premarket review that data from post-market surveillance is not required.
- Data must demonstrate superior accuracy of device-aided users' diagnostic (i) characterization of the indicated lesions compared to the accuracy of un-aided users.
- (ii) Clinical testing must evaluate patients across a range of skin phototypes, risk factors, and anatomic areas that represents the intended use population.
- (iii) Standalone device performance testing must demonstrate the accuracy of the device output relative to ground truth, including the following:
{29}------------------------------------------------
- (A) Testing must demonstrate at least 90% sensitivity of the device output for lesions with high metastatic potential, or an alternative clinical consideration must be provided to justify lower sensitivity. Clinical justification must be provided for the reported specificity.
- (B) Lesions must be selected by representative users and include a justified quantity and range of mimic lesions per diagnosis.
- (C) Justification must be provided for the determination of ground truth.
- (D) Testing must include a representative range of individuals with diverse risk factors (including age, body site, and skin phototype, and other clinical factors), and analysis of standalone performance must include subgroup analysis by relevant risk factors.
- (2) Non-clinical performance testing must demonstrate that the device performs as intended under anticipated conditions of use, including compatibility testing of the device software with specific signal or image acquisition hardware. Testing must include a description of compatible hardware and processes, pre-specified compatibility testing protocols and dataset(s).
- Performance testing must demonstrate device precision, including repeatability and (3) reproducibility of device performance, across operators and challenging use conditions.
- (4) Performance testing must demonstrate the electrical safety, mechanical safety, thermal safety, and electromagnetic compatibility of any electrical components of the device.
- (5) Performance testing must validate reprocessing instructions for reusable components of the device.
- (6) The patient-contacting components of the device must be demonstrated to be biocompatible.
- Software verification, validation, and hazard analysis must be performed. (7)
- (8) Human factors assessment must demonstrate that the device can be safely and correctly used by intended users.
- (9) Labeling must include:
- A summary of standalone and clinical performance testing conducted with the (i) device. The summary must describe performance measures, including sensitivity and specificity, and statistical confidence intervals, as well as performance of the device for all clinically relevant subgroups within the intended use population;
- (ii) A description of the patient population that was used in development or training of the device algorithm;
- Device limitations or subpopulations for which the device may not perform as (iii) expected or for whom the device has not been validated;
{30}------------------------------------------------
- (iv) Information for interpretation of the device outputs detailing the risks associated with misinterpretation of the device outputs;
- Warnings to avoid unsafe exposure to any energy-emitting components of the (v) device, e.g., excluding lesions close to the eye;
- (vi) A statement that the device is not intended for use as a standalone diagnostic: and
- (vii) Instructions for device maintenance and validated methods and instructions for reprocessing of any reusable components.
BENEFIT-RISK DETERMINATION
Risks and Other Factors
The risks of the device are based on nonclinical laboratory studies as well as data collected in the clinical studies described above.
No device or procedure related adverse events (AEs), serious adverse events (SAEs), or unanticipated adverse device effects (UADEs) were observed in the DERM-SUCCESS pivotal study.
The main risks of the DermaSensor device are false negative output and false positive output. False negative output may cause a provider to monitor rather than refer a lesion, which could result in delayed diagnosis of melanoma, SCC, or BCC. False positive output may result in referral of a benign lesion for dermatology evaluation. This may result in patient anxiety and potentially increase the probability of biopsy. These risks are mitigated by the high standalone device sensitivity in the intended use population (>90%); by the PCP's use of the device output as a second read and in addition to all available clinical information and examination; and by the final management decision by the dermatologist to whom the patient is referred.
Benefits
The probable benefits of the device are based on nonclinical laboratory studies as well as data collected in the clinical studies described above.
The main benefit of the DermaSensor device is the demonstrated ability to improve the management decision of PCPs in the intended use population. In participants aged 40 and older with melanoma (including severely/highly atypical nevi), SCC, or BCC lesion(s), this benefit was indicated by high standalone device sensitivity (>90%), in a pivotal study that included a large number of participants and lesions (>1000 each) across 22 clinical sites and Fitzpatrick phototypes I-VI. Three reader studies demonstrated that use of the DermaSensor resulted in superiority of the aided PCP sensitivity over the unaided sensitivity of the same PCP and that the device led to a greater increase in true positive aided referral decisions than false negative aided referral decisions.
The clinical performance testing, including standalone device sensitivity and demonstrated aid in reader studies, supports the benefit of the DermaSensor device as a second-read adjunctive aid to
{31}------------------------------------------------
primary care physicians in correct referral decisions for individuals aged 40 and over with melanoma (including highly atypical nevi), SCC, and BCC.
Uncertainty
The performance of the device has not been specifically evaluated in patients with increased risk for skin cancer, e.g., inherited or drug-induced photosensitivity; genetic predisposition to melanoma or BCC; immune compromise; or other medical conditions that increase the risk of skin cancer or its metastasis. Device-aided user performance may differ compared with the premarket studies provided, due to, for example, differences in reliance on the device in the real world compared to in the retrospective studies provided. Sensitivity of the DermaSensor device in participants under the age of 40 was less than 90%. Algorithm training and clinical testing had limited representation of patients with Fitzpatrick skin phototypes IV, V, and VI, reflective of the relative incidence of skin cancer in these patient subpopulations. This uncertainty is mitigated by the post-market surveillance special control and labeling which includes a precaution stating that limited data is available regarding the safety and effectiveness of the DermaSensor device for patients with Fitzpatrick skin phototypes IV, V, and VI and those at increased risk for skin cancer, e.g., inherited or drug-induced photosensitivity; genetic predisposition to melanoma or BCC; immune compromise; or other medical conditions that increase the risk of skin cancer or its metastasis.
Patient Perspectives
This submission did not include specific information on patient perspectives for this device.
Benefit/Risk Conclusion
In conclusion, given the available information above, the data support that for the following indication statement:
The DermaSensor device is indicated for use to evaluate skin lesions suggestive of melanoma, basal cell carcinoma, and/or squamous cell carcinoma in patients aged 40 and above to assist in the decision regarding referral of the patient to a dermatologist. The DermaSensor device should be used in conjunction with the totality of clinically relevant information from the clinical assessment, including visual analysis of the lesion, by physicians who are not dermatologists. The device should be used on lesions already assessed as suspicious for skin cancer and not as a screening tool. The device should not be used as the sole diagnostic criterion nor to confirm clinical diagnosis of skin cancer.
The probable benefits outweigh the probable risks for the DermaSensor device. The device provides benefits, and the risks can be mitigated by the use of general controls and the identified special controls.
CONCLUSION
{32}------------------------------------------------
The De Novo request for the DermaSensor is granted, and the device is classified as follows:
Product Code: QZS Device Type: Software-aided adjunctive diagnostic device for use by physicians on lesions suspicious for skin cancer. Regulation Number: 21 CFR 878.1830 Class: II
N/A