(126 days)
No reference devices have been used in this submission.
Yes
The device description explicitly states: "autoSCORE is an AI model that has been trained with standard deep learning principles using a large training dataset." and "Deep Learning is a subset of the Artificial Intelligence and Machine Learning methodologies, which uses artificial neural networks for data analysis."
No
The device aids in assessment and analysis of EEG recordings by flagging potential abnormalities for review by a qualified medical practitioner. It explicitly states it "does not provide any diagnostic conclusion about the patient's condition" nor "treatment options," indicating it's a diagnostic aid, not a therapeutic device.
No.
The "Intended Use / Indications for Use" section explicitly states: "This device does not provide any diagnostic conclusion about the patient's condition to the user." and "autoSCORE does not provide any diagnostic conclusion about the patient's condition nor treatment options to the user, and does not replace visual assessment of the EEG by the user." Instead, it is intended to "aid neurologists in the assessment of EEG" and "assist qualified clinical practitioners in the assessment of EEG traces" by marking sections that may correspond to abnormalities.
Yes
The device explicitly states "autoSCORE is a software only device." It performs analysis of EEG recordings and provides annotations to existing EEG reviewing software, without requiring any proprietary hardware.
No
The device processes previously acquired EEG recordings, which are physiological signals, not in-vitro samples (e.g., blood, tissue). Its output aids medical practitioners in assessing EEG data but does not perform diagnostics on biological specimens.
No
The clearance letter does not explicitly state that the FDA has reviewed and approved or cleared a Predetermined Change Control Plan (PCCP) for this specific device. The provided text only indicates "Not Found" for "Control Plan Authorized (PCCP) and relevant text."
Intended Use / Indications for Use
• autoSCORE is intended for the review, monitoring and analysis of EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes and to aid neurologists in the assessment of EEG. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
• The spike detection component of autoSCORE is intended to mark previously acquired sections of the patient's EEG recordings that may correspond to spikes, in order to assist qualified clinical practitioners in the assessment of EEG traces. The spike detection component is intended to be used in patients at least three months old for EEGs 4 hours. The autoSCORE component has not been assessed for intracranial recordings.
• autoSCORE is intended to assess the probability that previously acquired sections of EEG recordings contain abnormalities, and classifies these into pre-defined types of abnormalities, including epileptiform and non-epileptiform abnormalities. autoSCORE does not have a user interface. autoSCORE sends this information to the EEG reviewing software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE also provides the probability that EEG recordings include abnormalities and the type of abnormalities. The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.
• This device does not provide any diagnostic conclusion about the patient's condition to the user. The device is not intended to detect or classify seizures.
Product codes
OMB
Device Description
autoSCORE is a software only device.
autoSCORE is an AI model that has been trained with standard deep learning principles using a large training dataset. The model will be locked in the field, so it cannot learn from data to which it is exposed when in use. It can only be used with a compatible electroencephalogram (EEG) reviewing software, which acquires and displays the EEG. The model has no user interface. The form of the visualization of the annotations is determined and provided by the EEG reviewing software.
autoSCORE has been trained to identify and then indicate to the user sections of EEG which may include abnormalities and to provide the level of probability of the presence of an abnormality. The algorithm also provides categorization of identified areas of abnormality into the four predefined types of abnormalities, again including a probability of that predefined abnormality type. This is performed by identifying epileptiform abnormalities/spikes (Focal epileptiform and generalised epileptiform) as well identifying non-epileptiform abnormalities (Focal non-epileptiform and Diffuse Non-Epileptiform).
This data is then provided by the algorithm to the EEG reviewing software, for it to display as part of the EEG output for the clinician to review. autoSCORE does not provide any diagnostic conclusion about the patient's condition nor treatment options to the user, and does not replace visual assessment of the EEG by the user. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
Mentions image processing
Not Found
Mentions AI, DNN, or ML
autoSCORE is an AI model that has been trained with standard deep learning principles using a large training dataset.
Deep Learning is a subset of the Artificial Intelligence and Machine Learning methodologies, which uses artificial neural networks for data analysis.
autoSCORE assesses the EEG using the autoSCORE AI model and automatically annotates the EEG where an abnormality is identified (including the type of abnormality and its probability).
Input Imaging Modality
Electroencephalogram (EEG) recordings made by electroencephalogram (EEG) devices using scalp electrodes.
Anatomical Site
Not Found
Indicated Patient Age Range
patients at least three months old for EEGs 4 hours.
Intended User / Care Setting
The intended user is a suitably trained professional who is qualified to clinically review EEG recordings.
autoSCORE can be used wherever EEG data must be evaluated. This includes in particular neurological wards, epilepsy monitoring units and neurological practices.
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
Study Population: 40 Long Term Monitoring EEGs (LTMs) and 40 Ambulatory EEGs (AEEGs) were included ensuring broad distribution of age, gender, patient setting (excluding ICU and neonatal recordings) and types of abnormalities. EEG recordings used in this validation were anonymized by the source hospital/organization. The anonymization included patient metadata, with exclusion of age and gender.
Reference Standard: A consensus of three HEs was used as the reference standard for all calculations. Each segment was prepared in two forms:
- Without any markers placed by autoSCORE v 2.0 for recording level validation
- With autoSCORE v2.0 markers and their assigned type of abnormality for marker level validation.
To prevent bias in HE assessment, no HE evaluated the same EEG segment in both recording-level (without markers) and marker-level (with markers) formats. Each HE was assigned a distinct set of EEG segments and was blinded to the autoSCORE output for their assigned recording level validation segments. EEG segments and markers were distributed to ensure a three-HE consensus per EEG. HEs were blinded to patient metadata, with the exception of age and gender, and to the outputs of autoSCORE.
While reviewing EEGs, HEs were permitted to change montages, filters, gain, and time resolution. For recording-level validation, HEs independently labelled each EEG segment using the same predefined abnormality types as autoSCORE and inserted markers into the EEG where abnormalities could be found.
For marker-level validation, HEs reviewed autoSCORE v 2.0 markers, retaining those where they agreed that the given abnormality type was present within the markers' boundaries and removing markers if the given abnormality type was absent.
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Study type: Retrospective non-interventional comprehensive clinical validation.
Sample size: 40 Long Term Monitoring EEGs (LTMs) and 40 Ambulatory EEGs (AEEGs), totaling 80 EEGs.
Standalone performance:
- Recording level:
- autoSCORE V2 for Abnormal: Accuracy 0.912, Sensitivity 0.926, Specificity 0.833, PPV 0.969, NPV 0.666
- autoSCORE V2 for Focal Epi: Accuracy 0.787, Sensitivity 0.765, Specificity 0.804, PPV 0.743, NPV 0.822
- autoSCORE V2 for Gen Epi: Accuracy 0.925, Sensitivity 0.964, Specificity 0.904, PPV 0.844, NPV 0.979
- autoSCORE V2 for Non-Epi Diff: Accuracy 0.850, Sensitivity 0.680, Specificity 0.927, PPV 0.809, NPV 0.864
- autoSCORE V2 for Non-Epi Focal: Accuracy 0.838, Sensitivity 0.667, Specificity 0.957, PPV 0.917, NPV 0.804
- autoSCORE V2 for IED: Accuracy 0.875, Sensitivity 0.939, Specificity 0.774, PPV 0.868, NPV 0.889
- Marker level:
- autoSCORE v2.0 Focal Epi: PPV 0.560 (Number of Samples: 807, FP: 355, TP: 452)
- autoSCORE v2.0 Gen Epi: PPV 0.446 (Number of Samples: 568, FP: 315, TP: 253)
- autoSCORE v2.0 Focal Non-Epi: PPV 0.823 (Number of Samples: 667, FP: 118, TP: 549)
- autoSCORE v2.0 Diff Non-Epi: PPV 0.849 (Number of Samples: 664, FP: 100, TP: 564)
- autoSCORE v2.0 IED: PPV 0.513 (Number of Samples: 1375, FP: 670, TP: 705)
Key results: In this clinical performance validation, autoSCORE demonstrated a higher PPV overall compared to the predicate device encevis and a similar PPV compared to autoSCORE v1.4. These results indicate that autoSCORE's output performance is similar to both encevis and autoSCORE v1.4.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
Recording level (autoSCORE V2):
- Abnormal: Accuracy 0.912, Sensitivity 0.926, Specificity 0.833, Precision (PPV) 0.969, NPV 0.666
- Focal Epi: Accuracy 0.787, Sensitivity 0.765, Specificity 0.804, Precision (PPV) 0.743, NPV 0.822
- Gen Epi: Accuracy 0.925, Sensitivity 0.964, Specificity 0.904, Precision (PPV) 0.844, NPV 0.979
- Non-Epi Diff: Accuracy 0.850, Sensitivity 0.680, Specificity 0.927, Precision (PPV) 0.809, NPV 0.864
- Non-Epi Focal: Accuracy 0.838, Sensitivity 0.667, Specificity 0.957, Precision (PPV) 0.917, NPV 0.804
- IED: Accuracy 0.875, Sensitivity 0.939, Specificity 0.774, Precision (PPV) 0.868, NPV 0.889
Marker level (autoSCORE v2.0 PPV):
- Focal Epi: 0.560
- Gen Epi: 0.446
- Focal Non-Epi: 0.823
- Diff Non-Epi: 0.849
- IED: 0.513
Predicate Device(s)
Reference Device(s)
No reference devices have been used in this submission.
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 882.1400 Electroencephalograph.
(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).
FDA 510(k) Clearance Letter - autoSCORE (V 2.0.0)
Page 1
April 9, 2025
Holberg EEG AS
Smriti Franklin
QARA Director
Fjøsangerveien 70A
Bergen, Bergen 5068
Norway
Re: K243743
Trade/Device Name: autoSCORE (V 2.0.0)
Regulation Number: 21 CFR 882.1400
Regulation Name: Electroencephalograph
Regulatory Class: Class II
Product Code: OMB
Dated: December 4, 2024
Received: March 10, 2025
Dear Smriti Franklin:
We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Page 2
K243743 - Smriti Franklin Page 2
Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).
Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review, the QS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting (reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reporting-combination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.
All medical devices, including Class I and unclassified devices and combination product device constituent parts are required to be in compliance with the final Unique Device Identification System rule ("UDI Rule"). The UDI Rule requires, among other things, that a device bear a unique device identifier (UDI) on its label and package (21 CFR 801.20(a)) unless an exception or alternative applies (21 CFR 801.20(b)) and that the dates on the device label be formatted in accordance with 21 CFR 801.18. The UDI Rule (21 CFR 830.300(a) and 830.320(b)) also requires that certain information be submitted to the Global Unique Device Identification Database (GUDID) (21 CFR Part 830 Subpart E). For additional information on these requirements, please see the UDI System webpage at https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/unique-device-identification-system-udi-system.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-devices/medical-device-safety/medical-device-reporting-mdr-how-report-medical-device-problems.
For comprehensive regulatory information about medical devices and radiation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-
Page 3
K243743 - Smriti Franklin Page 3
assistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
Jay R. Gupta -S
Jay Gupta
Assistant Director
DHT5A: Division of Neurosurgical,
Neurointerventional, and
Neurodiagnostic Devices
OHT5: Office of Neurological and
Physical Medicine Devices
Office of Product Evaluation and Quality
Center for Devices and Radiological Health
Enclosure
Page 4
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Food and Drug Administration
Form Approved: OMB No. 0910-0120
Expiration Date: 07/31/2026
See PRA Statement below.
Indications for Use
Submission Number (if known)
K243743
Device Name
autoSCORE (V 2.0.0)
Indications for Use (Describe)
• autoSCORE is intended for the review, monitoring and analysis of EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes and to aid neurologists in the assessment of EEG. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
• The spike detection component of autoSCORE is intended to mark previously acquired sections of the patient's EEG recordings that may correspond to spikes, in order to assist qualified clinical practitioners in the assessment of EEG traces. The spike detection component is intended to be used in patients at least three months old for EEGs 4 hours. The autoSCORE component has not been assessed for intracranial recordings.
• autoSCORE is intended to assess the probability that previously acquired sections of EEG recordings contain abnormalities, and classifies these into pre-defined types of abnormalities, including epileptiform and non-epileptiform abnormalities. autoSCORE does not have a user interface. autoSCORE sends this information to the EEG reviewing software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE also provides the probability that EEG recordings include abnormalities and the type of abnormalities. The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.
• This device does not provide any diagnostic conclusion about the patient's condition to the user. The device is not intended to detect or classify seizures.
Type of Use (Select one or both, as applicable)
☒ Prescription Use (Part 21 CFR 801 Subpart D) ☐ Over-The-Counter Use (21 CFR 801 Subpart C)
CONTINUE ON A SEPARATE PAGE IF NEEDED.
This section applies only to requirements of the Paperwork Reduction Act of 1995.
DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.
The burden time for this collection of information is estimated to average 79 hours per response, including the time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:
Department of Health and Human Services
Food and Drug Administration
Office of Chief Information Officer
Paperwork Reduction Act (PRA) Staff
PRAStaff@fda.hhs.gov
"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."
Page 5
510(k) Summary
1. 510K SUBMITTER
Holberg EEG AS
Fjøsangerveien 70A
5068 Bergen, Norway
Phone: +47 926 44 261
Contact Person: Smriti Franklin
Date Prepared: 26th November 2024
2. DEVICE IDENTIFICATION
Name of Device: autoSCORE V2.0
Common or Usual Name: autoSCORE
Classification Name and Regulation Number: Electroencephalograph, 21 CFR 882.1400
Regulatory Class: II
Product code: OMB
3. PREDICATE DEVICES
4.1 Primary Predicate Device
Trade/Device Name: encevis
Model Number: 1.6
510(K) Submitter/Holder: AIT Austrian Institute of Technology GmbH
510(K) Reference: K171720
4.2 Additional Predicate Device
Trade/Device Name: autoSCORE
Model Number: autoSCORE V1.4
510(K) Submitter/Holder: Holberg EEG AS
510(K) Reference: K231068
No reference devices have been used in this submission.
Page 1 of 19
Page 6
4. Device Description
autoSCORE is a software only device.
autoSCORE is an AI model that has been trained with standard deep learning principles using a large training dataset. The model will be locked in the field, so it cannot learn from data to which it is exposed when in use. It can only be used with a compatible electroencephalogram (EEG) reviewing software, which acquires and displays the EEG. The model has no user interface. The form of the visualization of the annotations is determined and provided by the EEG reviewing software.
autoSCORE has been trained to identify and then indicate to the user sections of EEG which may include abnormalities and to provide the level of probability of the presence of an abnormality. The algorithm also provides categorization of identified areas of abnormality into the four predefined types of abnormalities, again including a probability of that predefined abnormality type. This is performed by identifying epileptiform abnormalities/spikes (Focal epileptiform and generalised epileptiform) as well identifying non-epileptiform abnormalities (Focal non-epileptiform and Diffuse Non-Epileptiform).
This data is then provided by the algorithm to the EEG reviewing software, for it to display as part of the EEG output for the clinician to review. autoSCORE does not provide any diagnostic conclusion about the patient's condition nor treatment options to the user, and does not replace visual assessment of the EEG by the user. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
5.1 Intended Use of the Device
Detailed Intended Use
autoSCORE is a software-only decision support product intended to be used with compatible EEG software. It is intended to assist the user when reviewing EEG recordings by assessing the probability that the previously acquired sections of EEG recordings contain abnormalities and classifying these into predefined types of abnormality. autoSCORE sends this information to the EEG software to indicate where markers indicating abnormality are to be placed in the EEG.
autoSCORE also provides an overview of the probabilities that EEG recordings between 14 minutes and 4 hours include any abnormalities and the probabilities of specific predefined type of abnormalities they include. For EEG recordings of duration more than 4 hours, autoSCORE indicates the number of segments with duration of 2-4 hours that include any abnormalities and the total number of analyzed segments. The overview for EEG recordings of duration more than 4 hours also provides the number of segments that include specific pre-defined types of abnormalities and the total number of analyzed segments.
The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.
autoSCORE cannot detect or classify seizures. The recorded EEG activity is not altered by the information
Page 2 of 19
Page 7
provided by autoSCORE. autoSCORE is not intended to provide information for diagnosis but to assist clinical workflow when using the EEG software.
5.2 Intended Users
The intended user is a suitably trained professional who is qualified to clinically review EEG recordings.
5.3 Indications for Use
autoSCORE can be used wherever EEG data must be evaluated. This includes in particular neurological wards, epilepsy monitoring units and neurological practices.
5.3.1 Indications for use Statement –
-
autoSCORE is intended for the review, monitoring and analysis of EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes and to aid neurologists in the assessment of EEG. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
-
The spike detection component of autoSCORE is intended to mark previously acquired sections of the patient's EEG recordings that may correspond to spikes, in order to assist qualified clinical practitioners in the assessment of EEG traces. The spike detection component is intended to be used in patients at least three months old for EEGs 4 hours. The autoSCORE component has not been assessed for intracranial recordings.
-
autoSCORE is intended to assess the probability that previously acquired sections of EEG recordings contain abnormalities, and classifies these into pre-defined types of abnormalities. autoSCORE does not have a user interface. autoSCORE sends this information to the EEG reviewing software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE also provides the probability that EEG recordings include abnormalities, and the type of abnormalities. The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.
-
This device does not provide any diagnostic conclusion about the patient's condition to the user.
The Indications for Use statement for autoSCORE is identical to secondary predicate device, however, it's not identical to the primary predicate device as autoSCORE does not contain certain encevis features like seizure detection, burst suppression or calculates quantitative measures. Indications for use statement point 1, 2 and 4 are identical to the respective parts of primary predicate device's indications for use statement. However, Point 3 of the indications for use statement describes autoSCORE's technological characteristics that are different from the predicate device, same as autoSCORE V1.4. These differences do not alter the intended use of the device nor do they affect the safety and effectiveness of the device
Page 3 of 19
Page 8
relative to the primary predicate. Both the subject and predicate devices have the same intended use for analysing electroencephalograph data, detecting events like spike detection and output detected parameters for interpretation by a qualified user.
5.4 autoSCORE Software Technology
autoSCORE is a decision support software that assists trained healthcare professionals with the clinical reviewing of human scalp EEG recordings acquired from patients aged 3 months or older for EEG 4 hours. It is a locked algorithm using Deep Learning principles to assess the probability that previously acquired sections of EEG contain abnormalities. Deep Learning is a subset of the Artificial Intelligence and Machine Learning methodologies, which uses artificial neural networks for data analysis.
autoSCORE assesses epileptiform as well as non-epileptiform abnormalities in the patient's EEG. It categorizes the assessed abnormalities into predefined types including Focal Epileptiform, Generalized Epileptiform, Focal Non-Epileptiform and Diffuse Non-Epileptiform abnormalities. The probability of abnormality is assessed for each type of abnormality on the level of the EEG recording as well as for individual markers within the EEG recording.
autoSCORE cannot detect or classify seizures.
autoSCORE is designed to integrate with compatible EEG reviewing software through an integration layer. Users do not need to connect autoSCORE to the EEG Reviewing software and it cannot be purchased by an individual physician without an integration with the EEG reviewing software. autoSCORE shall be available as a feature in the compatible EEG reviewing software. autoSCORE receives EEG data and EEG metadata as input from the compatible EEG reviewing software, including the patient's age, gender, and the electrode sensor labels of the EEG recording.
autoSCORE assesses the EEG using the autoSCORE AI model and automatically annotates the EEG where an abnormality is identified (including the type of abnormality and its probability). This annotation, categorization and probability output is generated and sent to the compatible EEG reviewing software. The output is then presented in the electronic user interface of the compatible EEG reviewing software to a qualified medical professional for independent assessment. The recorded EEG activity and the EEG metadata used as input are not altered by the information provided by autoSCORE. autoSCORE does not store any input or output data. Input data are merely utilized by autoSCORE for the purpose of generating output data, which are then sent to the EEG reviewing software.
5. Device Comparison Table
The device comparison table outlines the differences and similarities between autoSCORE and the predicate devices including technological characteristics.
Page 4 of 19
Page 9
Table 1: Comparison of autoSCORE against predicate devices.
| encevis | autoSCORE v1.4 | autoSCORE V2 | Comments | |
|---|---|---|---|---|
| Device Description and Features | ||||
| Device Type | Software-only Device | Software-only Device | Software-only Device | Identical |
| General Device Description | EEG Review and Analysis Software | EEG Review and Analysis Software | EEG Review and Analysis Software | Identical |
| Identifies Spikes | Yes | Yes | Yes | Identical |
| Assessment and categorization of abnormalities including probability in previously acquired sections of EEG | No | Yes | Yes | Different for primary predicate. Same for secondary predicate device. |
| Device Operation | ||||
| Type of EEG | Scalp EEG | Scalp EEG | Scalp EEG | Identical |
| Population age | Adults (age > 18) | > 3 months | > 3 months for EEGs 2 years for EEGs >4 hours | Minimum patient age higher than secondary predicate device. |
| Design Input | Raw EEG signal | Raw EEG Signal | Raw EEG Signal | Identical |
| Design Input files | Calculation is based on EEG data recorded by external EEG systems. They are either read from the EEG file provided by the EEG system or can be send to encevis using the interface provided by AIT (AITInterfaceDLL) | Calculation is based on EEG data recorded by external EEG systems. They are read from the EEG data provided by the EEG system | Calculation is based on EEG data recorded by external EEG systems. They are read from the EEG data provided by the EEG system | Identical (No AIT interface) |
| Algorithm | Convolutional Neural Network | Convolutional Neural Network | Convolutional Neural Network | Identical |
| User-defined parameters | No parameters in spike detection algorithm can be changed by the user | No parameters in spike detection algorithm can be changed by the user | No parameters in spike detection algorithm can be changed by the user | |
| Type of EEG Analysis | Post-hoc analysis | Post-hoc analysis | Post-hoc analysis | Identical |
| Design Output | Spike Detection component makes the results available to the user in form of markers | Spike Detection component makes the results available to the user in form of markers | Spike Detection component makes the results available to the user in form of markers | Identical |
| Output Files | Results are stored in a database and/or is send over the interface AITInterfaceDLL to an external EEG system. User | Results are returned back to the host software after analysis. | Results are returned back to the host software after analysis. | Similar |
Page 5 of 19
Page 10
| encevis | autoSCORE v1.4 | autoSCORE V2 | Comments | |
|---|---|---|---|---|
| output is given by graphical user interfaces | ||||
| Diagnostic conclusion | This device does not provide any diagnostic conclusion about the patient's condition to the user. | This device does not provide any diagnostic conclusion about the patient's condition to the user. | This device does not provide any diagnostic conclusion about the patient's condition to the user. | Identical |
| User | This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information. | This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information. | This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information. | Identical |
| Compliance | No standard data format available in the industry | No standard data format available in the industry | No standard data format available in the industry | Identical |
| Compatible and interoperable Equipment and software | encevis can read and process EEG data from several EEG vendors. A list of compatible EEG systems can be found on http://www.encevis.com | autoSCORE can read and process EEG data from any compatible/interoperable EEG systems. https://www.holbergeeg.com/compatible-eeg-reviewing-software | autoSCORE can read and process EEG data from any compatible/interoperable EEG systems. https://www.holbergeeg.com/compatible-eeg-reviewing-software | Similar |
Colour Key
- Identical/Similar Characteristics
- Different or N/A Characteristics
There are Technological differences between the subject device (autoSCORE V2) and primary predicate device (encevis) that have been highlighted in Table 1 above. There are additional features in the primary predicate device like seizure detection, analysis of quantitative features, user interface, aEEG functionality etc that is outside the intended use of the subject device. These features are completely independent functions that do not impact the spike detection component. The absence of these features only makes the output given by subject device (autoSCORE V2) lower risk to the patient than the output provided by primary predicate device (encevis).
encevis and autoSCORE V2 detect spikes (epileptiform abnormalities.) In addition to the spike detection of epileptiform abnormalities, autoSCORE V2 and autoSCORE V1.4 also detect non-epileptiform
Page 6 of 19
Page 11
abnormalities. autoSCORE V2 and V1.4 also gives the probability of the detected abnormality being an epileptiform abnormality - Focal epileptiform, Generalized epileptiform or non-epileptiform abnormality - Focal non-epileptiform, Diffuse non-epileptiform. The identification of additional abnormalities and categorization of these abnormalities does not pose any additional risks to the information provided by predicate devices as evidenced through performance validation.
There are no technological features between the subject device (autoSCORE V2) and the secondary predicate device (autoSCORE V1). There are some minor design changes but all major technological characteristics including the AI model is the same for both autoSCORE V1.4 and V2.0
7. Performance Validation
autoSCORE Performance Validation was conducted to evaluate autoSCORE performance in two parts.
- Non-Clinical Validation – To validate autoSCORE outputs against defined autoSCORE Inputs and User requirements. Verification and validation activities established the safety and performance characteristics of the subject device with respect to the predicate device.
- Clinical Validation – To validate autoSCORE performance against Independent Human Experts and predicate devices.
These validations have been summarised below.
7.1 Non clinical Performance Validation
Software verification and validation testing was conducted and documented in accordance with FDA Guidance for Industry and FDA Staff, Guidance for the Content of Software Contained in Medical Devices. Product Design and Software Requirements Traceability has been documented and verified against verification and validation test results.
Verification and validation testing includes:
- Code Review
- Unit level testing
- System level testing
- Integration level testing
Page 7 of 19
Page 12
Verification and validation activities established the safety and performance characteristics of the subject device with respect to the predicate device. The following performance data have been provided in support of the substantial equivalence determination.
Table 2: Type of performance test per feature
| Verification Tests Performed | autoSCORE Features - Identification and categorization of following abnormalities | ||||
|---|---|---|---|---|---|
| Normal EEG | Spike Detection - epileptiform abnormalities | Non-epileptiform abnormalities | |||
| Focal epileptiform | Generalized epileptiform | Focal non-epileptiform | Diffuse non-epileptiform | ||
| Software Verification and Validation Testing | x | x | x | Not available in predicate | Not available in predicate |
7.2 Clinical Performance Validation
7.2.1 Clinical Performance Evaluation
A retrospective non-interventional comprehensive clinical validation was performed using de-identified data to evaluate the performance of all autoSCORE features against Human Experts (HEs) and predicate devices to establish substantial equivalence.
The following performance data have been provided in support of the substantial equivalence determination.
Table 3: Type of performance test per feature
| Validation Tests Performed | autoSCORE Features - Identification and Categorization of the Following Abnormalities | ||||
|---|---|---|---|---|---|
| Normal EEG | Spike Detection - Epileptiform Abnormalities | Non-Epileptiform Abnormalities | |||
| Focal Epileptiform | Generalized Epileptiform | Focal Non-Epileptiform | Diffuse Non-Epileptiform | ||
| Direct Comparison Against Predicate Device | x | x | x | autoSCORE V1.4 | autoSCORE V1.4 |
| Comparison with Human Expert Evaluation | x | x | x | x | x |
For performance evaluation of the autoSCORE spike detection device, the study was conducted to measure outputs of autoSCORE V2 against the spike detection from encevis and autoSCORE V1.4, using HE consensus as the reference standard.
Page 8 of 19
Page 13
7.2.2 Study Population
40 Long Term Monitoring EEGs (LTMs) and 40 Ambulatory EEGs (AEEGs) were included ensuring broad distribution of age, gender, patient setting (excluding ICU and neonatal recordings) and types of abnormalities. EEG recordings used in this validation were anonymized by the source hospital/organization. The anonymization included patient metadata, with exclusion of age and gender.
The following distribution of EEGs was used in this validation.
Figure 1: The figure shows the distribution of normal/abnormal (including abnormality types) EEGs in LTM and ambulatory settings for adult and pediatric EEGs. (NOTE - Each EEG may contain multiple abnormalities.)
| Normal | Focal Epi | Generalized Epi | Focal Non-Epi | Diffuse Non-Epi | |
|---|---|---|---|---|---|
| AEEG Adult | 4 | 8 | 4 | 3 | 6 |
| AEEG Paediatric | 4 | 10 | 7 | 4 | 4 |
| LTM Adult | 4 | 8 | 5 | 5 | 4 |
| LTM Paediatric | 4 | 7 | 8 | 7 | 4 |
For performance evaluation of the autoSCORE spike detection device, the study was conducted to measure outputs of autoSCORE V2 against the spike detection from encevis and autoSCORE V1.4, using HE consensus as the reference standard.
Page 9 of 19
Page 14
7.2.3 Reference Standard
A consensus of three HEs was used as the reference standard for all calculations. Each segment was prepared in two forms:
- Without any markers placed by autoSCORE v 2.0 for recording level validation
- With autoSCORE v2.0 markers and their assigned type of abnormality for marker level validation.
To prevent bias in HE assessment, no HE evaluated the same EEG segment in both recording-level (without markers) and marker-level (with markers) formats. Each HE was assigned a distinct set of EEG segments and was blinded to the autoSCORE output for their assigned recording level validation segments. EEG segments and markers were distributed to ensure a three-HE consensus per EEG. HEs were blinded to patient metadata, with the exception of age and gender, and to the outputs of autoSCORE.
While reviewing EEGs, HEs were permitted to change montages, filters, gain, and time resolution. For recording-level validation, HEs independently labelled each EEG segment using the same predefined abnormality types as autoSCORE and inserted markers into the EEG where abnormalities could be found.
For marker-level validation, HEs reviewed autoSCORE v 2.0 markers, retaining those where they agreed that the given abnormality type was present within the markers' boundaries and removing markers if the given abnormality type was absent.
Page 10 of 19
Page 15
7.2.4 Analytical Methods
The analytical methods employed in this validation are described in 7.2.4.1-7.2.4.5 below.
Figure 1 – This flowchart shows the hierarchical organization of the autoSCORE outputs, including the thresholds used to classify recordings into categories such as normal or abnormal, specific abnormality types, and associated output for recordings with duration of four hours and longer. The arrows indicate dependencies, for example: a marker of the type "Focal Epi" is only given if the corresponding segment-level output also exceeds the threshold for "Focal Epi"
Page 11 of 19
Page 16
7.2.4.1 Recording Level Validation
The binary metrics given in Table 4, in section 7.2.5, were computed independently for each feature (Normal/Abnormal, Focal Epi, Gen Epi, Focal Non-Epi, Diffuse Non-Epi) with 95% symmetric confidence intervals. The following definitions were used for the binary metrics for the recording segment level:
TP – HE consensus indicated that the condition is present and autoSCORE also indicates that the condition is present.
FP - HE consensus indicated that the condition is not present but autoSCORE indicates that the condition is present.
TN - HE consensus indicated that the condition is not present and autoSCORE also indicates that the condition is not present.
FN - HE consensus indicated that the condition is present but autoSCORE indicates that the condition is not present.
Values from the contingency tables were used to calculate the following performance metrics, with 95% confidence intervals computed using bootstrap resampling: Sensitivity (TPR), Specificity (TNR), Positive Predictive Value (PPV), Negative Predictive Value (NPV) and Accuracy.
7.2.4.2 Marker Level Validation
In the current study, marker-level validation was performed using multiple approaches to evaluate the performance of autoSCORE in detecting and annotating EEG abnormalities. Given the practical limitations of HEs in marking every abnormality within lengthy EEG recordings, different methods were employed to compute the relevant performance metrics: Positive Predictive Value (PPV), True Positive Rate (TPR), False Positive Rate (FPR), and Negative Predictive Value (NPV).
Positive Predictive Value (PPV):
PPV was calculated using the assessments of autoSCORE markers by a consensus of three HEs. Each marker placed by autoSCORE was reviewed by HEs who had not participated in the initial recording-level assessment of the same EEG. A marker was classified as a True Positive (TP) if at least two HEs agreed that it correctly the abnormality type. Conversely, if fewer than two HEs agreed, the marker was considered a False Positive (FP). This approach allowed us to compute the PPV as:
𝑃𝑃𝑉 = 𝑇𝑃/(𝑇𝑃 + 𝐹𝑃)
Page 12 of 19
Page 17
The resulting PPV values are given in table 5 of in section 7.2.5.
From a clinical perspective, avoiding false positives is generally considered more critical than avoiding false negatives. For this reason PPV was chosen as the acceptance criterion for evaluating autoSCORE's performance [1, 2]. The most robust method for calculating the PPV of autoSCORE markers involved presenting the markers to HEs, who were blinded to the recording-level autoSCORE outputs as outlined above.
Table 3 summarises the overlap between autoSCORE markers, encevis markers, HE-placed markers, and the HE consensus.
7.2.4.3 Validation of Probability Output
The probability outputs assigned to the markers by autoSCORE were validated by analyzing the relationship between these probabilities and the correctness of the markers. The validation process was conducted as follows:
- All autoSCORE markers were categorized into 5-percentage-points bins based on their assigned probabilities.
- For each bin, the average probability and the number of True Positives (TPs) were calculated.
- A Pearson correlation coefficient was computed to assess the relationship between the average probabilities and the number of TPs across all bins (See table 6).
The criterion for validation was a significant positive correlation (p-value