Predicate Devices

Reference Devices

AI/ML (Artificial Intelligence / Machine Learning)

Yes
The device description explicitly states that autoSCORE uses an algorithm trained with "standard deep learning principles" and mentions "Automated Interpretation of Clinical Electroencephalograms Using Artificial Intelligence."

Therapeutic

No
The device is a software-only decision support product intended to assist in the review and analysis of EEG recordings and to aid neurologists in the assessment of EEG. It does not provide any diagnostic conclusions or therapeutic interventions.

Diagnostic

No

The device description explicitly states: "This device does not provide any diagnostic conclusion about the patient's condition to the user. The device is not intended to detect or classify seizures... autoSCORE is not intended to provide information for diagnosis but to assist clinical workflow when using the EEG software."

SaMD (Software as a Medical Device)

Yes

The device description explicitly states "autoSCORE is a software-only decision support product". While it interacts with EEG review software and processes data from EEG devices, it is presented as a software component itself, not including the hardware for data acquisition.

IVD (In Vitro Diagnostic)

Based on the provided information, this device is not an In Vitro Diagnostic (IVD).

Here's why:

IVDs are used to examine specimens derived from the human body. The intended use and device description clearly state that autoSCORE analyzes EEG recordings made by electroencephalogram devices using scalp electrodes. These are recordings of electrical activity from the brain, not specimens derived from the body like blood, urine, or tissue.
The device's function is to aid in the assessment of EEG recordings. It provides information about the probability and type of abnormalities within the EEG data. It does not perform tests on biological samples to diagnose a condition.
The device explicitly states it does not provide a diagnostic conclusion. This is a key characteristic that differentiates it from many IVDs, which are often used to provide diagnostic information.

Therefore, while autoSCORE is a medical device used in a clinical setting, its function and the type of data it analyzes do not align with the definition of an In Vitro Diagnostic.

PCCP (Predetermined Change Control Plan) Authorized

No
The letter does not state that the FDA has reviewed and approved or cleared the PCCP for this specific device.

Overview

Intended Use / Indications for Use

autoSCORE is intended for the review, monitoring and analysis of EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes and to aid neurologists in the assessment of EEG. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
The spike detection component of autoSCORE is intended to mark previously acquired sections of the patient's EEG recordings that may correspond to spikes, in order to assist qualified clinical practitioners in the assessment of EEG traces. The spike detection component is intended to be used in patients at least three months old. The autoSCORE component has not been assessed for intracranial recordings.
autoSCORE is intended to assess the probability that previously acquired sections of EEG recordings contain abnormalities and classifies these into pre-defined types of abnormalities, including epileptiform and non-epileptiform abnormalities. autoSCORE does not have a user interface. autoSCORE sends this information to the EEG reviewing software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE also provides the probability that EEG recordings include abnormalities, and the type of abnormalities. The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.
This device does not provide any diagnostic conclusion about the patient's condition to the user. The device is not intended to detect or classify seizures.

Product codes

OMB

Device Description

autoSCORE is a software-only decision support product intended to be used with compatible electroencephalography (EEG) review software. It is intended to assist the user when reviewing EEG recordings, by assessing the probability that previously acquired sections of EEG recordings contain abnormalities, and classifying these into pre-defined types of abnormality. autoSCORE sends this information to the EEG software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE uses an algorithm that has been trained with standard deep learning principles using a large training dataset.

autoSCORE also provides an overview of the probability that EEG recordings and sections of EEG recordings include abnormalities, and which type(s) of abnormality they include. This is performed by identifying spikes of epileptiform abnormalities (Focal epileptiform and Generalized epileptiform) as well identifying non-epileptiform abnormalities (Focal Nonepileptiform and Diffuse Non-epileptiform).

The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.

autoSCORE cannot detect or classify seizures. The recorded EEG activity is not altered by the information provided by autoSCORE. autoSCORE is not intended to provide information for diagnosis but to assist clinical workflow when using the EEG software.

Mentions image processing

Not Found

Mentions AI, DNN, or ML

autoSCORE uses an algorithm that has been trained with standard deep learning principles using a large training dataset.

Input Imaging Modality

EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes.

Anatomical Site

Not Found (Implied: Brain)

Indicated Patient Age Range

patients at least three months old.

Intended User / Care Setting

autoSCORE is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.

autoSCORE is intended to be used in environments where clinical EEGs are acquired or reviewed by suitably trained and qualified professionals.

autoSCORE is intended to be used for the analysis of EEG that has been recorded in environments suitable for adult and pediatric routine EEG acquisition according to best clinical practice, excluding acquisition environments for ICU and neonatal recordings. autoSCORE is not validated for EEG recorded in a home/ambulatory environment or any non-hospital/EEG laboratory setting.

Description of the training set, sample size, data source, and annotation protocol

Not Found. The document states "None of the EEGs used in the validation were used in the development of the Al model." but does not describe the training set.

Description of the test set, sample size, data source, and annotation protocol

The clinical validation study was carried out in five parts to compare the performance of autoSCORE with the human experts as well as with the predicate devices:

1. Performance evaluation against human experts (single-Center): A single-center dataset of 4,850 EEGs assessed by 9 human experts assessing more than 1% of the EEGs each.
- Study 1 – The reference standard was based on 4850 EEGs described by multiple HEs, but a single HE reviewer per EEG. The HEs inserted markers in the EEGs defining if the EEG was abnormal or normal, and if abnormal, the abnormality categories, and served as reference standard both on recording level and marker level. The HE assessments were part of the routine EEG assessment in their respective hospitals, and the HEs had all relevant patient clinical information. Apart from age and gender, all clinical data was removed for this clinical validation to avoid any associated bias.
1. Performance evaluation against human experts (multi-center): A multi-center dataset of 100 EEGs were assessed by 11 independent human experts.
- Study 2 – The reference standard was based on HE consensus of 11 HEs reviewing 100 EEGs. The HEs assessed if the EEGs on recording level were normal or abnormal, and if abnormal if the EEGs contained one or more of the abnormality categories Focal Epi, Gen Epi, Focal Non-Epi, and Diffuse Non-Epi. The HEs were blinded to all patient data except age and gender.
1. Direct comparison against primary predicate device (encevis): The same dataset of 100 EEGs used in Part 2 were used to evaluate performance against the primary predicate device, encevis.
- Study 3 – This study uses the same dataset as for study 2, and thus also the same HE consensus reference standard.
1. Benchmarking against primary and secondary predicate device (encevis and Persyst 13): A dataset of 58 EEGs was used to benchmark performance of both the primary predicate device encevis, the predicate device Persyst 13, and autoSCORE against human expert consensus.
- Study 4 – The reference standard was obtained by visual assessment of 58 EEGs by 3 HEs. Marker time points for the IEDs were recorded for each EEG and each HE. The reference standard was the majority consensus scoring of the HEs. This served as reference standard both on recording level and marker level for the IEDs. HEs were blinded to all patient data except age and gender.
1. Performance evaluation against human experts (two centers): A hold-out dataset of 1315 EEGs not used for training of the Al model acquired from two centers were assessed by 15 human experts assessing more than 1% of the EEGs each.
- Study 5 – The reference standard was based on 1315 EEGs described by multiple HEs, but a single HE reviewer per EEG. The HEs inserted markers in the EEGs defining if the EEG was abnormal or normal and, if abnormal, the abnormality categories, and served as reference standard both on recording level and marker level. The HE assessments were part of the routine EEG assessment in their respective hospitals, and the HEs had all relevant patient clinical information. Apart from age and gender, all clinical data was removed for this clinical validation to avoid any associated bias.

Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)

Performance validation to evaluate autoSCORE performance was conducted in two parts:

Non-Clinical Validation – To validate autoSCORE outputs against defined autoSCORE design inputs and user requirements.
Clinical Validation – To validate autoSCORE performance against independent human experts and predicate devices.

Non-clinical Performance Validation:
Software verification and validation testing was conducted and documented in accordance with 2005 FDA Guidance, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices. Software verification and validation testing included: 1. Code Review, 2. Unit level testing, 3. System level testing, 4. Integration level testing. The software for this device is determined as a "moderate" level of concern. Software verification and validation activities demonstrated that the device software meets all software requirements.

Clinical Performance Validation:
A retrospective non-interventional comprehensive clinical validation was performed using de-identified data to evaluate performance of all autoSCORE features against Human Experts and predicate devices to establish substantial equivalence.

Study Type and Sample Sizes:

Part 1: Performance evaluation against human experts (single-Center). Sample Size: 4,850 EEGs. Reviewers: 9 HEs.
Part 2: Performance evaluation against human experts (multi-center). Sample Size: 100 EEGs. Reviewers: 11 HEs.
Part 3: Direct comparison against primary predicate device (encevis). Sample Size: 100 EEGs (same as Part 2). Reviewers: 11 HEs.
Part 4: Benchmarking against primary and secondary predicate device (encevis and Persyst 13). Sample Size: 58 EEGs. Reviewers: 3 HEs.
Part 5: Performance evaluation against human experts (two centers). Sample Size: 1315 EEGs. Reviewers: 15 HEs.

Key Results:

Comparison of performance with HEs (Recording-level, Abnormal EEG Classification):

Part 2 (n=100): Sensitivity: 100 [100, 100], Specificity: 88.4 [77.8, 97.4], PPV: 92.0 [84.5, 98.3], NPV: 100 [100.0, 100.0], Correlation coefficient: 0.96 (p

Regulation Number and Section

§ 882.1400 Electroencephalograph.

(a)
Identification. An electroencephalograph is a device used to measure and record the electrical activity of the patient's brain obtained by placing two or more electrodes on the head.(b)
Classification. Class II (performance standards).

Summary

0

January 7, 2024

Image /page/0/Picture/1 description: The image shows the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is a blue square with the letters "FDA" in white. To the right of the blue square is the text "U.S. FOOD & DRUG ADMINISTRATION" in blue.

Holberg EEG AS Smriti Franklin QA/RA Manager Fjøsangerveien 70A 5068 Bergen, Norway

Re: K231068

Trade/Device Name: autoSCORE Regulation Number: 21 CFR 882.1400 Regulation Name: Electroencephalograph Regulatory Class: Class II Product Code: OMB Dated: December 8, 2023 Received: December 8, 2023

Dear Smriti Franklin:

We have reviewed your section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (the Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.

If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.

Additional information about changes that may require a new premarket notification are provided in the FDA guidance documents entitled "Deciding When to Submit a 510(k) for a Change to an Existing Device" (https://www.fda.gov/media/99812/download) and "Deciding When to Submit a 510(k) for a Software Change to an Existing Device" (https://www.fda.gov/media/99785/download).

1

Your device is also subject to, among other requirements, the Quality System (QS) regulation (21 CFR Part 820), which includes, but is not limited to, 21 CFR 820.30, Design controls; 21 CFR 820.90, Nonconforming product; and 21 CFR 820.100, Corrective and preventive action. Please note that regardless of whether a change requires premarket review. the OS regulation requires device manufacturers to review and approve changes to device design and production (21 CFR 820.30 and 21 CFR 820.70) and document changes and approvals in the device master record (21 CFR 820.181).

Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR Part 803) for devices or postmarketing safety reporting (21 CFR Part 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (QS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR Part 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR Parts 1000-1050.

Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.

For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).

Sincerely,

Jay R. Gupta -S

Jay Gupta Assistant Director DHT5A: Division of Neurosurgical, Neurointerventional and Neurodiagnostic Devices OHT5: Office of Neurological and Physical Medicine Devices

2

Office of Product Evaluation and Quality Center for Devices and Radiological Health

Enclosure

3

Indications for Use

510(k) Number (if known) K231068

Device Name autoSCORE

Indications for Use (Describe)

autoSCORE is intended for the review, monitoring and analysis of EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes and to aid neurologists in the assessment of EEG. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
The spike detection component of autoSCORE is intended to mark previously acquired sections of the patient's EEG recordings that may correspond to spikes, in order to assist qualified clinical practitioners in the assessment of EEG traces. The spike detection component is intended to be used in patients at least three months old. The autoSCORE component has not been assessed for intracranial recordings.
autoSCORE is intended to assess the probability that previously acquired sections of EEG recordings contain abnormalities, and classifies these into pre-defined types of abnormalities, including epileptiform abnormalities. autoSCORE does not have a user interface. autoSCORE sends this information to the EEG reviewing software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE also provides the probability that EEG recordings include abnormalities and the type of abnormalities. The user is required to review the EEG and exercise their clinical judgendently make a conclusion supporting or not supporting brain disease.
This device does not provide any diagnostic conclusion about the patient's condition to the user. The device is not intended to detect or classify seizures.

Type of Use (Select one or both, as applicable)
☑ Prescription Use (Part 21 CFR 801 Subpart D) ☐ Over-The-Counter Use (21 CFR 801 Subpart C)	☑ Prescription Use (Part 21 CFR 801 Subpart D)	☐ Over-The-Counter Use (21 CFR 801 Subpart C)
☑ Prescription Use (Part 21 CFR 801 Subpart D)	☐ Over-The-Counter Use (21 CFR 801 Subpart C)

CONTINUE ON A SEPARATE PAGE IF NEEDED.

This section applies only to requirements of the Paperwork Reduction Act of 1995.

DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW.

The burden time for this collection of information is estimated to average 79 hours per response, including the

collection of information is estimated to average 79 hour time to review instructions, search existing data sources, gather and maintain the data needed and complete and review the collection of information. Send comments regarding this burden estimate or any other aspect of this information collection, including suggestions for reducing this burden, to:

Department of Health and Human Services Food and Drug Administration Office of Chief Information Officer Paperwork Reduction Act (PRA) Staff PRAStaff(@fda.hhs.gov

"An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB number."

4

Image /page/4/Picture/1 description: The image contains a logo for Holberg EEG. The logo consists of a purple circle with a stylized EEG waveform inside it. To the right of the circle, the text "HOLBERG EEG" is written in a simple, sans-serif font.

510(K) Summary

1. SUBMITTER

Holberg EEG AS Fjøsangerveien 70A 5068 Bergen, Norway Phone: +47 926 44 261 Contact Person: Smriti Franklin Date Prepared: March 23rd, 2023

2. DEVICE IDENTIFICATION

Trade Name: autoSCORE

Common Name: Automatic event detection software for full-montage electroencephalograph Classification Name and Regulation Number: Electroencephalograph, 21 CFR 882.1400 Regulatory Class: II Product Code: OMB

3. PREDICATE DEVICES

Primary Predicate Device

Trade/Device Name: encevis, K171720

Additional Predicate Device

Trade/Device Name: Persyst 13, K151929

5

Image /page/5/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in a simple, sans-serif font. The waveform graphic is meant to represent brain activity, which is related to the EEG acronym.

4. DEVICE DESCRIPTION

autoSCORE is a software-only decision support product intended to be used with compatible electroencephalography (EEG) review software. It is intended to assist the user when reviewing EEG recordings, by assessing the probability that previously acquired sections of EEG recordings contain abnormalities, and classifying these into pre-defined types of abnormality. autoSCORE sends this information to the EEG software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE uses an algorithm that has been trained with standard deep learning principles using a large training dataset.

autoSCORE also provides an overview of the probability that EEG recordings and sections of EEG recordings include abnormalities, and which type(s) of abnormality they include. This is performed by identifying spikes of epileptiform abnormalities (Focal epileptiform and Generalized epileptiform) as well identifying non-epileptiform abnormalities (Focal Nonepileptiform and Diffuse Non-epileptiform).

The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.

autoSCORE cannot detect or classify seizures. The recorded EEG activity is not altered by the information provided by autoSCORE. autoSCORE is not intended to provide information for diagnosis but to assist clinical workflow when using the EEG software.

5. INDICATIONS FOR USE/ INTENDED USE

5.1 INDICATIONS FOR USE STATEMENT

1. autoSCORE is intended for the review, monitoring and analysis of EEG recordings made by electroencephalogram (EEG) devices using scalp electrodes and to aid neurologists in the assessment of EEG. This device is intended to be used by qualified medical practitioners who will exercise professional judgment in using the information.
1. The spike detection component of autoSCORE is intended to mark previously acquired sections of the patient's EEG recordings that may correspond to spikes, in order to assist qualified clinical practitioners in the assessment of EEG traces. The spike detection component is intended to be used in patients at least three months old. The autoSCORE component has not been assessed for intracranial recordings.
1. autoSCORE is intended to assess the probability that previously acquired sections of EEG recordings contain abnormalities and classifies these into pre-defined types of

6

abnormalities, including epileptiform and non-epileptiform abnormalities. autoSCORE does not have a user interface. autoSCORE sends this information to the EEG reviewing software to indicate where markers indicating abnormality are to be placed in the EEG. autoSCORE also provides the probability that EEG recordings include abnormalities, and the type of abnormalities. The user is required to review the EEG and exercise their clinical judgement to independently make a conclusion supporting or not supporting brain disease.

1. This device does not provide any diagnostic conclusion about the patient's condition to the user. The device is not intended to detect or classify seizures.

5.2 INTENDED USE ENVIRONMENT

autoSCORE is intended to be used in environments where clinical EEGs are acquired or reviewed by suitably trained and qualified professionals.

autoSCORE is intended to be used for the analysis of EEG that has been recorded in environments suitable for adult and pediatric routine EEG acquisition according to best clinical practice, excluding acquisition environments for ICU and neonatal recordings. autoSCORE is not validated for EEG recorded in a home/ambulatory environment or any non-hospital/EEG laboratory setting.

5.3 INTENDED PATIENT POPULATION

autoSCORE use is restricted to EEG recordings from patients over 3 months of age. autoSCORE cannot be used for EEG recordings from neonatal patients. This restriction applies to all features of autoSCORE. There are no other restrictions regarding the patient population.

6. SUBSTANTIAL EQUIVALENCE DISCUSSION

The following table 1 compares autoSCORE to the predicate device with respect to intended use, technological characteristics and operating principles. The comments section provides further information on the determination of substantial equivalence.

		autoSCORE	encevis	Persyst 13	Comments
	510k Reference	Subject device	K171720	K151929	N/A
	Product Code	OMB	OMB	OMB	Identical
	Class	II	II	II	Identical
Device, regulation and sponsor details	Regulation
Number	21 CFR 882.1400	21 CFR 882.1400	21 CFR 882.1400	Identical
	Regulation
Name	Electroencephalograph	Electroencephalograph	Electroencephalograph	Identical
		autoSCORE	encevis	Persyst 13	Comments
Device Description and Features	Manufacturer	Holberg EEG AS	AIT Austrian Institute of
Technology GmbH	Persyst Development
Corporation	N/A
	Device Type	Software-only Device	Software-only Device	Software-only Device	Identical
	General Device
Description	EEG Review and Analysis
Software	EEG Review and Analysis
Software	EEG Review and
Analysis Software	Identical
	Identifies
Spikes	Yes	Yes	Yes	Identical
	Assessment
and
categorization
of
abnormalities
including
probability in
previously
acquired
sections of EEG	Yes	No	No	Different
	Type of EEG	Scalp EEG	Scalp EEG	Scalp EEG	Identical
	Intended Use
Environments	autoSCORE is intended to
be used in environments
where clinical EEGs are
acquired or reviewed by
suitably trained and
qualified professionals.

autoSCORE is intended to
be used for the analysis of
EEG that has been
recorded in environments
suitable for adult and
pediatric routine EEG
acquisition according to
best clinical practice,
excluding acquisition
environments for ICU and
neonatal recordings.
autoSCORE is not
validated for EEG
recorded in a
home/ambulatory
environment or any non-
hospital/EEG laboratory
setting. | encevis is intended to be
used in environments
where clinical EEGs are
acquired or reviewed by
suitably trained and
qualified professionals.

encevis Spike Detection
component is intended to
be used in adult patients
greater than or equal to
18 years. encevis Spike
Detection performance
has not been assessed for
intracranial recordings. | Persyst 13 is intended
to be used in
environments where
clinical EEGs are
acquired or reviewed by
suitably trained and
qualified professionals.

The Spike Detection
component is intended
to be used in patients at
least one month old.
Persyst 13 Spike
Detection performance
has not been assessed
for intracranial
recordings. | Similar. See
Patient age for
intended
population
comparison. |
| | Population age | > 3 months | Adults (age > 18 years) | > 1 month | Minimum
patient age
more than
the predicate
device. |
| | | autoSCORE | encevis | Persyst 13 | Comments |
| Device Operation | Design Input | Raw EEG Signal | Raw EEG signal | | Identical |
| | Design
Input files | Calculation is based on
EEG data recorded by
external EEG systems.
They are read from the
EEG data provided by
the EEG system | Calculation is based on
EEG data recorded by
external EEG systems.
They are either read
from the EEG file
provided by the EEG
system or can be sent
to encevis using the
interface provided by
AIT (AITInterfaceDLL) | Calculation is based
on EEG data
recorded by external
EEG systems. They
are read from the
EEG file provided by
the EEG system | Identical
(No AIT
interface) |
| | Algorithm | Convolutional Neural
Network | Convolutional Neural
Network | Neural Network | Identical |
| | User-defined
parameters | No parameters in spike
detection algorithm can
be changed by the user | No parameters in spike
detection algorithm
can be changed by the
user | | Similar |
| | Type of EEG-
Analysis | Post-hoc analysis | Post-hoc analysis | Post-hoc analysis | Identical |
| | Design Output | Spike Detection
component makes the
results available to the
user in form of
markers | Spike Detection
component makes the
results available to the
user in form of markers | Spike Detection
component makes the
results available to the
user in form of markers | Identical |
| | Device Outputs | Identification and
categorization of
epileptiform and non-
epileptiform
abnormalities
including probability
that EEG recordings
include abnormalities,
and the type of
abnormalities. These
outputs are given at
both recording Level
and marker Level. | Identification of
epileptiform
abnormalities (spikes).
These outputs are
given at marker level. | Identification of
epileptiform
abnormalities (spikes).
These outputs are
given at marker level. | Some predicate
device features
are not
included in
autoSCORE.
These include
seizure
detection,
burst
suppression,
aEEG, rhythmic
and periodic
patterns and
frequency
bands. |
| | Output Files | Results are returned
back to the host
software after analysis. | Results are stored in a
database and/or sent
over the interface
AITInterfaceDLL to an
external EEG system.
User output is given by
graphical user
interfaces. | Results are stored in
additional files in the
file system placed in
the same folder as the
EEG file. User output is
given by graphical
user interfaces. | Similar |
| | Diagnostic
conclusion | | | | Comments |
| | autoSCORE | encevis | Persyst 13 | | |
| User | This device is intended
to be used by qualified
medical practitioners
who will exercise
professional judgment
in using the
information. | This device is intended
to be used by qualified
medical practitioners
who will exercise
professional judgment
in using the
information. | This device is intended
to be used by qualified
medical practitioners
who will exercise
professional judgment
in using the
information. | Identical | |
| Compatible
and
interoperable
Equipment
and software | autoSCORE can read
and process EEG data
from Natus®
NeuroWorks®
software | encevis can read and
process EEG data from
several EEG vendors. A
list of compatible EEG
systems can be found
on
http://www.encevis.co
m | Persyst 13 can read
and process EEG data
from several EEG
vendors. A list of
compatible EEG
systems can be found
on
http://www.persyst.co
m/suppo
rt/supported-
formats/ | Similar | |

Table 1: Comparison of autoSCORE against predicate devices.

7

Image /page/7/Picture/1 description: The image contains the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in a bold, sans-serif font. The waveform graphic is also purple, matching the color of the circle.

8

Image /page/8/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in a simple, sans-serif font. The waveform graphic likely represents brain activity, which is relevant to the EEG (electroencephalogram) service offered by Holberg.

9

Image /page/9/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in a sans-serif font. The waveform graphic is meant to represent an electroencephalogram (EEG), which is a test that measures electrical activity in the brain.

Color Key
Identical/Similar Characteristics		Different or N/A Characteristics

Comparison of Intended Use/ Indications for Use

The Indications for Use statement for autoSCORE is similar to the predicate devices. However, autoSCORE does not contain certain predicate device features including seizure detection, burst suppression, and other quantitative measures. Indications for use statement point 1, 2, and 4 are identical to the respective parts of predicate devices indications for use statement. Point 3 of the indications for use statement describes autoSCORE's technological characteristics, including additional outputs that are different from the predicate devices.

These differences do not alter the intended use of the device, nor do they affect the safety and effectiveness of the device relative to the predicates. Both the subject and predicate devices have the same intended use for analyzing electroencephalograph data, identifying events including spike detection and producing outputs based on analysis of EEG for interpretation by a qualified user.

Comparison of Technological Characteristics

Technological differences between the subject and predicate devices have been highlighted in Table 1 above. There are additional features in the predicate devices, including seizure detection, analysis of additional quantitative features, and a different user interface, which Page 5 - 6

10

Image /page/10/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized EEG waveform inside it. To the right of the circle, the words "HOLBERG EEG" are written in black, sans-serif font.

are outside the intended use of the subject device. These features are completely independent functions and do not affect the abnormality detection features of the subject device.

Both the predicate devices and subject device detect features related to epileptiform abnormalities (e.g. spikes). In addition to detecting epileptiform abnormalities, the subject device also detects non-epileptiform abnormalities. The subject device also provides the probability of the detected abnormality being an epileptiform abnormality, such as a focal epileptiform or generalized epileptiform abnormality, or a non-epileptiform abnormality, such as a focal non-epileptiform or diffuse non-epileptiform abnormality. The identification of additional abnormalities and categorization of these abnormalities does not affect the intended use of the device and does not pose any additional risks as compared to the predicate devices as evidenced through performance validation.

7. Performance Validation

Performance validation to evaluate autoSCORE performance was conducted in two parts:

-Non-Clinical Validation – To validate autoSCORE outputs against defined autoSCORE design inputs and user requirements.
-Clinical Validation – To validate autoSCORE performance against independent human experts and predicate devices.

These validations have been summarized below.

7.1 Non-clinical Performance Validation

Software verification and validation testing was conducted and documented in accordance with 2005 FDA Guidance, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices.

Product Design and Software Requirements Traceability has been documented and verified against verification and validation test results.

Software verification and validation testing included:

1. Code Review
1. Unit level testing
1. System level testing
1. Integration level testing

The software for this device is determined as a "moderate" level of concern because a failure or latent flaw could indirectly result in minor injury to the patient or operator through incorrect information or through the action of a care provider.

Software verification and validation activities demonstrated that the device software meets

Page 5 - 7

11

Image /page/11/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized EEG waveform inside it. To the right of the circle, the text "HOLBERG EEG" is written in a simple, sans-serif font.

all software requirements.

7.2 Clinical Performance Validation

7.2.1 Clinical Performance Evaluation

A retrospective non-interventional comprehensive clinical validation was performed using de-identified data to evaluate performance of all autoSCORE features against Human Experts and predicate devices to establish substantial equivalence.

The following performance data have been provided in support of the substantial equivalence determination.

Table 2: Type of software performance test per feature. autoSCORE indicates the EEG as normal if it does not contain epileptiform or non-epileptiform abnormalities, and abnormal if it contains one or both of these abnormalities. Part 1, Part 2, and Part 5 of the clinical study show comparable results against Human experts where an EEG is marked as 'normal' by autoSCORE. Part 3 and 4 of the study include the assessment of presence and absence of epileptiform abnormalities in predicate devices and autoSCORE that feeds into the assessment of a normal or abnormal EEG.

| Validation
Tests
Performed | autoSCORE Features - Identification and categorization of following
abnormalities | | | | |
|---------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------------|-------------------------------|-----------------------------------|
| | | Spike Detection -
epileptiform abnormalities | | | Non-epileptiform
abnormalities |
| | Normal
EEG | Focal
epileptiform | Generalized
epileptiform | Focal non-
epileptiform | Diffuse non-
epileptiform |
| Direct
Comparison
against
predicate device | x | x | x | Not available
in predicate | Not available
in predicate |
| Benchmarking
against both
predicate
devices with
external gold
standard EEGs | x | x | x | Not available
in predicate | Not available
in predicate |
| Comparison
with Human
Expert
Evaluation | x | x | x | x | x |

12

Image /page/12/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in black. The waveform appears to represent brain activity, which is relevant to the company's focus on EEG technology.

Clinical Performance Evaluations

For the performance evaluation of the autoSCORE spike detection device, the study was conducted to measure outputs of autoSCORE against the assessments from independent human experts as well as the spike detection from the predicate devices – encevis and Persyst 13.

Further, for the autoSCORE performance evaluation of additional technological features (not in the predicate device), the study was conducted to measure autoSCORE results of nonepileptiform abnormalities detection and categorization of abnormalities against Human Experts (HE).

The clinical validation study was carried out in five parts to compare the performance of autoSCORE with the human experts as well as with the predicate devices:

1. Performance evaluation against human experts (single-Center): A single-center dataset of 4,850 EEGs assessed by 9 human experts assessing more than 1% of the EEGs each.
1. Performance evaluation against human experts (multi-center): A multi-center dataset of 100 EEGs were assessed by 11 independent human experts.

3. Direct comparison against primary predicate device (encevis): The same dataset of 100 EEGs used in Part 2 were used to evaluate performance against the primary predicate device, encevis.

4. Benchmarking against primary and secondary predicate device (encevis and Persyst 13):

A dataset of 58 EEGs was used to benchmark performance of both the primary predicate device encevis, the predicate device Persyst 13, and autoSCORE against human expert consensus.

1. Performance evaluation against human experts (two centers):
  A hold-out dataset of 1315 EEGs not used for training of the Al model acquired from two centers were assessed by 15 human experts assessing more than 1% of the EEGs each.

The validation study was performed across five separate datasets with the following characteristics:

Validation Parts	Number of sites	Sample size	Number of reviewers	Patient gender	EEG Duration min-max	Patient age min-max (median)	Pediatric (P) vs Adult (A)*
------------------	-----------------	-------------	---------------------	----------------	----------------------	------------------------------	-----------------------------

Table 3. Summary of study parts used for validation of autoSCORE.

13

Image /page/13/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized EEG waveform inside it. To the right of the circle are the words "HOLBERG EEG" in a simple, sans-serif font.

| 1. Performance
evaluation against
HE
a. Recording
level
b. Marker
Level | 1 | 4850 | 9 | 2527 (M)
2248 (F)
75
(unknown) | 14 - 120
minutes | 3 months

106
years (39
years) | P - 1490
A - 3360 |
|-------------------------------------------------------------------------------------------------------------------------------------------------|----|------|----|-----------------------------------------|---------------------|------------------------------------------|----------------------|
| 2. Performance
evaluation against
HE

Direct
comparison against
predicate device | 16 | 100 | 14 | 39 (M)
61 (F) | 20 - 240
minutes | 9 months

95
years
(26 years) | P - 43
A - 57 |
| 4. Benchmarking
against predicate
devices and marker
validation against
HE consensus
a. Recording
level
b. Marker
Level | 4 | 58 | 3 | 27 (M)
31 (F) | 20 - 30
minutes | 2 - 77
years (36
years) | P - 13
A - 45 |
| 5. Performance
evaluation against
HE
a. Recording
level
b. Marker | 2 | 1315 | 15 | 636 (M)
642 (F)
37
(unknown) | 14 - 240
minutes | 3 months
99
years
(38 years) | P - 467
A - 848 |

*Pediatric - 22 years

7.2.2 Study Population and Refence Standards

None of the EEGs used in the validation were used in the development of the Al model. The HEs providing the reference standards in the validation phase of Study 1, 2, 3, and 4 were different from those who participated in the development portion of the process.

Study 1 – The reference standard was based on 4850 EEGs described by multiple HEs, but a single HE reviewer per EEG. The HEs inserted markers in the EEGs defining if the EEG was abnormal or normal, and if abnormal, the abnormality categories, and served as reference standard both on recording level and marker level. The HE assessments were part of the routine EEG assessment in their respective hospitals, and the HEs had all relevant patient clinical information. Apart from age and gender, all clinical data was removed for this clinical validation to avoid any associated bias.

Study 2 – The reference standard was based on HE consensus of 11 HEs reviewing 100 EEGs. The HEs assessed if the EEGs on recording level were normal or abnormal, and if abnormal if the EEGs contained one or more of the abnormality categories Focal Epi, Gen Epi, Focal Non-Epi, and Diffuse Non-Epi. The HEs were blinded to all patient data except age and gender.

14

Image /page/14/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in a simple, sans-serif font. The waveform graphic suggests a connection to brain activity or neurological monitoring.

Study 3 – This study uses the same dataset as for study 2, and thus also the same HE consensus reference standard.

Study 4 – The reference standard was obtained by visual assessment of 58 EEGs by 3 HEs. Marker time points for the IEDs were recorded for each EEG and each HE. The reference standard was the majority consensus scoring of the HEs. This served as reference standard both on recording level and marker level for the IEDs. HEs were blinded to all patient data except age and gender.

Study 5 – The reference standard was based on 1315 EEGs described by multiple HEs, but a single HE reviewer per EEG. The HEs inserted markers in the EEGs defining if the EEG was abnormal or normal and, if abnormal, the abnormality categories, and served as reference standard both on recording level and marker level. The HE assessments were part of the routine EEG assessment in their respective hospitals, and the HEs had all relevant patient clinical information. Apart from age and gender, all clinical data was removed for this clinical validation to avoid any associated bias.

All relevant autoSCORE outputs were covered in the above studies.

7.2.3 Analytical Methods

The analytical methods used in this validation have been described in 7.2.3.1 and 7.2.3.2.

The figure 1 below shows a hierarchical representation of autoSCORE recording- and marker outputs and the conditions under which these outputs are presented in the Natus® NeuroWorks® user interface.

15

autoSCORE recording result		Recording probability	Recording output		Marker probability	Marker output
	Abnormal probability	0.0% - 11.5%	Normal	Focal Epi Probability	0.0% - 46.5%
		11.5% - 50.9%	Probable Normal		46.5% - 100.0%	Focal Epi Marker(s)
		50.9% - 73.5%	Probable Abnormal
		73.5% - 100.0%	Abnormal
	Focal Epi probability	0.0% - 53.4%		Gen Epi Probability	0.0% - 50.7%
		53.4% - 85.8%	Probable Focal Epi		50.7% - 100.0%	Gen Epi Marker(s)
		85.8% - 100.0%	Focal Epi
1	Gen Epi probability	0.0% - 49.2%		Focal Non-epi Probability	0.0% - 49.4%
		49.2% - 90.2%	Probable Gen Epi		49.4% - 100.0%	Focal Non-Epi Marker(s)
		90.2% - 100.0%	Gen Epi
	Focal Non-Epi probability	0.0% - 52.4%		Diffuse Non-epi Probability	0.0% - 47.7%
		52.4% - 75.1%	Probable Focal Non-Epi		47.7% - 100.0%	Diffuse Non-Epi Marker(s)
		75.1% - 100.0%	Focal Non-Epi
	Diffuse Non-Epi probability	0.0% - 48.9%
		48.9% - 88.5%	Probable Diffuse Non-Epi
		88.5% - 100.0%	Diffuse Non-Epi

Figure 1: Hierarchical representation of autoSCORE recording and marker level outputs.

7.2.3.1 Comparison of performance with HEs

Binary Metrics

The binary metrics given in Table 4, Table 6 (sensitivity, specificity, PPV and NPV) in results section were computed independently for each study part and each feature (Normal/Abnormal, Focal Epi, Gen Epi, Focal Non-Epi) with 95% symmetric confidence intervals obtained using bootstrap resampling (n ≥ 10000).

The following definitions were used for the binary metrics for the recording level outputs (where HE was used in study 1 and 5 and HE consensus in study 2):

TP – HE or HE consensus indicated that the condition is present and autoSCORE also indicates that the condition is present.

FP - HE or HE consensus indicated that the condition is not present but autoSCORE indicates that the condition is present.

TN - HE or HE consensus indicated that the condition is not present and autoSCORE also indicates that the condition is not present.

FN - HE or HE consensus indicated that the condition is present but autoSCORE indicates that the condition is not present.

For the marker level outputs the following definitions were used for study 1 and 5:

TP - A 16 second window where HE has indicated that an abnormality is present and autoSCORE also would place a marker for the same type of abnormality.

FP - A 16 second window where autoSCORE would place a marker for a specific type of abnormality and this window is either: 1) randomly extracted from an EEG that HE has

16

assessed as normal or 2) a 16 second window which the HE has assessed as not containing this type of abnormality.

FN- A 16 second window which HE has assessed as containing a specific type of abnormality where autoSCORE would not place a marker for this type of abnormality.

TN- A 16 second window where autoSCORE would not place a marker of a specific type of abnormality and this window is either: 1) randomly extracted from an EEG that the HE has assessed as normal or 2) a 16 second window which the HE has assessed as not containing this type of abnormality.

For the marker level outputs for study 4, validation was based on areas in the EEGs marked by autoSCORE and HE consensus markers. TP, TN, FP, and FN were derived from the resulting segmentation of the recording into areas marked only by autoSCORE, areas marked only by HE consensus and areas marked by both autoSCORE and HE consensus.

In the next step, values from the contingency matrices were used to calculate:

Sensitivity, also referred to as True Positive Rate or TPR = TP/(TP+FN) ●
Specificity, also referred to as True Negative Rate or TNR = TN/(TN + FP) ●
PPV = TP/(TP + FP) .
NPV = TN/(TN + FN)
Prevalence = (TP + FN)/(TP + FP + TN + FN) = (number of true condition positive)/(number of samples).

Probability

To validate the probability output given by autoSCORE, several HE outputs were averaged in order to obtain a probability reference for the HEs. This grouping was done in different ways for study part 1/5 and for study part 2:

Study part 1 and 5: The large number of EEG recordings allowed their grouping . depending on autoSCORE probability values, applicable both for recording level and marker level. The grouping was uniform with 10 bins from 0% to 100%, each of 10 percent-points.

. Study part 2: The large number of HEs involved in the study allowed grouping EEGs depending on probability based on the number of HEs "voting" for the presence of the respective abnormalities, applicable only for recording level. Since each EEG was rated by 11 HEs, the granularity of this grouping was 9 percent-points.

The correlation coefficients given in Table 4, Table 6 and associated p values are Pearsons Correlation calculated using the python scipy stats package with the mean autoSCORE output and mean HE assessments in each bin as described above. In study part 4, a similar discretization was performed on the markers placed. The ranges were here 100-90%, 90-80%, 80-70%, 70-60%, 60-50% and 50-0%. Only markers above threshold for user output were included, and a qualitative analysis of the number of overlapping markers with reference (HE consensus or primary predicate device) for each probability range.

Levels of abnormalities

Figure 2 below presents the strategy employed to validate categorical outputs. Categories in grey indicate areas that were not considered for determining TP, TN, FP and FN, and consequently, were not used for calculation performance parameters.

17

Image /page/17/Picture/1 description: The image shows the logo for Holberg EEG. The logo consists of a purple circle with a stylized waveform inside, followed by the text "HOLBERG EEG" in a simple, sans-serif font. The waveform graphic is also purple, matching the color of the circle.

Image /page/17/Figure/2 description: The image shows a comparison of high and low probability thresholds for different medical conditions. The left side of the image lists conditions such as 'Normal', 'not Epi Focal', 'not Epi Generalized', 'not Non-Epi Diffuse', and 'not Non-Epi Focal'. The right side of the image lists corresponding probable and abnormal conditions, such as 'Probable Abnormal', 'Abnormal', 'Probable Epi Focal', 'Epi Focal', 'Probable Epi Generalized', 'Epi Generalized', 'Probable Non-Epi Diffuse', 'Non-Epi Diffuse', 'Probable Non-Epi Focal', and 'Non-Epi Focal'.

Figure 2: Schematic representation of strategy employed to validate the levels of abnormalities.

7.2.3.2 Comparison of performance with predicate devices

Binary Metrics - recording level

The binary metrics given in Table 7 (accuracy, sensitivity, specificity, PPV and NPV) in the results section were computed in the same way as described above in the comparison with HE section. To allow comparison with predicate devices encevis (study part 3 and 4) and Persyst (study part 4), a number of assumptions and limitations had to be addressed:

-Only the epileptiform activity abnormalities can be validated against the predicate device. Focal Epi and Gen Epi parts of the autoSCORE output were merged.
Since predicate devices are not designed to give recording level output, rules of interpretation had to be applied. If at least one spike is generated by the predicate device, the EEG recording is classified as Abnormal.

Probability level of abnormality – marker level

Only Focal Epi and Gen Epi markers placed by autoSCORE could be compared with the placement of the encevis spikes in study part 4.

18

7.2.4 Results of Performance Evaluation

7.2.4.1 Comparison of performance with HEs

Summary results obtained by the abovementioned methods are presented in Table 4 below. Tveit et al. [1] describes in detail the agreement between HEs and autoSCORE in comparison to HE-HE agreement.

Table 4: Performance results for autoSCORE classification of Abnormal EEG at recording-level based on comparison with HE assessment. Reference standards for each study part are discussed in a previous section of this document.

| Age
Group | Study
Part | Sensitivity
(%) | Specificity (%) | PPV (%) | NPV (%) | Correlation
coefficient (p-value) |
|--------------|---------------------|--------------------------------------|----------------------|-----------------------|-------------------------|--------------------------------------|
| All ages | Part 2
(n=100) | 100 [100,
100] | 88.4 [77.8,
97.4] | 92.0
[84.5,98.3] | 100 [100.0,
100.0] | 0.96 (p