(326 days)
Ascom Mobile Monitoring Gateway (MMG), K111215
Not Found
No
The description focuses on message integration, forwarding, and filtering based on predefined rules, without mentioning any learning or adaptive capabilities characteristic of AI/ML. The performance testing also focuses on throughput and accuracy of message processing based on these rules.
No.
The device is described as an alarm interface or secondary alarm notification system, not a device that provides therapeutic treatment or intervention.
No
The "Intended Use / Indications for Use" section explicitly states, "The BoxView Smart Alarm Interface, Model SA-01 is not intended to be used for diagnostic purposes."
Yes
The device description explicitly states, "The BoxView Smart Alarm Interface (SAI), Model SA-01, is a Software Medical Device (SaMD) product..." and "The Box View Smart Alarm Interface, Model SA-01 is a software product and cannot come into physical contact with patients." The description focuses on its function as a message integrator and data formatter, running on a standard operating system, and interacting with existing hardware (patient monitors, display devices, network infrastructure) but not including or modifying that hardware itself.
Based on the provided information, the Box View Smart Alarm Interface (SAI), Model SA-01 is not an In Vitro Diagnostic (IVD) device.
Here's why:
- Intended Use: The intended use clearly states that the device is for providing an interface to forward alarm information from physiological patient monitoring systems to display devices. It explicitly states it is not intended to be used for diagnostic purposes.
- Device Description: The description reinforces its function as a message integrator and secondary alarm notification system. It focuses on forwarding existing alarm and data information, not on analyzing biological samples or providing diagnostic results.
- Lack of IVD Characteristics: The description does not mention any interaction with biological samples (blood, urine, tissue, etc.), reagents, or any process related to analyzing such samples for diagnostic purposes.
Therefore, the Box View Smart Alarm Interface (SAI), Model SA-01 falls outside the definition of an In Vitro Diagnostic device. It is a software medical device (SaMD) intended for alarm management and notification.
N/A
Intended Use / Indications for Use
The intended use of the Box View Smart Alarm Interface (SAI). Model SA-01 is to provide an interface with physiological patient monitoring systems to forward information associated to an alarm event to a designated display device(s). For medical, near real time alarms, the BoxView Smart Alarm Interface, Model SA-01 is intended to serve as a parallel, redundant, mechanism to inform healthcare professionals of particular medical alarm events. The BoxView Smart Alarm Interface, Model SA-01 does not alter the primary medical devices and associated alarm annunciations.
The BoxView Smart Alarm Interface. Model SA-01 is intended for use as a secondary alarm notification system. It does not replace the primary alarm function on the monitor.
The BoxView Smart Alarm Interface. Model SA-01 is not intended to be used for diagnostic purposes. The Box View Smart Alarm Interface, Model SA-01 is intended for use by professional clinical personnel and relies on proper use and operation of both the communication in place at the healthcare facility and the display devices used. The BoxView Smart Alarm Interface, Model SA-01 is a software product and cannot come into physical contact with patients.
Product codes
MSX
Device Description
The BoxView Smart Alarm Interface (SAI), Model SA-01, is a Software Medical Device (SaMD) product intended to be located on-site in the hospital, or pre-configured off site in the 'cloud' utilizing a standard Linux operating system. The primary purpose of the BoxView Smart Alarm Interface (SAI), Model SA-01 is to act as a message integrator to forward patient monitor status and alarm event information originating from a patient monitoring network. Users receive interactive, time-critical information from patient monitoring devices directly via their display devices as text, alarms or data. The BoxView Smart Alarm Interface (SAI), Model SA-01 allows caregivers to be informed of their patient's alarm conditions when they are not in the patient vicinity.
The BoxView Smart Alarm Interface (SAI), Model SA-01 is an open system that is compatible with most smart phones or computers. The BoxView Smart Alarm Interface (SAI), Model SA-01 connects to the information sources through wired Ethernet connections which are part of the customer's infrastructure. The BoxView Smart Alarm Interface (SAI), Model SA-01 software acquires patient data from patient monitoring devices and allows the user to configure the system to determine which information, including alarm notifications, is delivered to which users communicators. The BoxView Smart Alarm Interface (SAI), Model SA-01 then formats the data for delivery to the display devices.
The BoxView Smart Alarm Interface (SAI), Model SA-01 system is designed to forward alarm event information as the alarms are recognized by the patient monitoring network. The system is also capable of being configured to periodically forward a patient's physiological data as well. All messaging activities are recorded by the BoxView Smart Alarm Interface (SAI), Model SA-01 providing real-time activity logging for audit trail records and reporting.
The BoxView Smart Alarm Interface (SAI), Model SA-01, system is a secondary alarm notification system. It does not replace the primary alarm function of the bedside monitor or telemetry monitoring system.
Mentions image processing
Not Found
Mentions AI, DNN, or ML
Not Found
Input Imaging Modality
Not Found
Anatomical Site
Not Found
Indicated Patient Age Range
Not Found
Intended User / Care Setting
professional clinical personnel / healthcare facility
Description of the training set, sample size, data source, and annotation protocol
Not Found
Description of the test set, sample size, data source, and annotation protocol
Not Found
Summary of Performance Studies (study type, sample size, AUC, MRMC, standalone performance, key results)
Non-Clinical Bench Performance Testing Information
-
Test(s) Performed
The bench performance test for Smart Alarm Interface involves simulating alert/alarm messages from concurrent alert/alarm sources to validate that below and at server capacity the alarm messages filter correctly and that notifications send to caregiver devices when applicable. Another bench performance test will involve simulating alert/alarm messages over server capacity to identify when the application performance begins to degrade. -
Objective(s) of the Test(s)
The first objective of the test is to confirm that Smart Alarm Interface can process and respond to 250 alert/alarm messages per second from 25 concurrent alert/alarm sources, this results to each alert/alarm source sending 10 alerts/alarm signals a second.
The second objective is to determine the number of alert/alarm messages and concurrent alert/alarms sources in which the application performance starts to worsen. Performance test is also used to help identify any bottle necks or areas of the code that have the potential to be better optimized. These performance tests are to be executed before every major software release to confirm that the application is performing per requirements.
- Test Methods
a. Test Environment
i. XprezzNet Simulator Used to simulate alarm messages from Spacelabs patient monitors. (XprezzNet is the name Spacelabs has designated for their network communication)
ii. Amazon Web Services A cloud computing service that hosts the docker container(s) that run the Smart Alarm Interface application and Postgres database.
iii. Mobile Devices iOS and Android communicator devices.
- iPhone XR iOS 13.4.1
- iPhone 8 iOS 12.3.1
- iPhone XR iOS 13.5.1
- iPhone X iOS 13.1
- iPhone XS iOS 13.3.1
- Samsung Galaxy A10e Android 9
- Google Pixel 4 Android 10b
b. JMeter is used to execute requests with the XprezzNet simulate alarm sources that send alert/alarm messages to validate that Smart Alarm Interface filters alarm signals properly and creates or updates alarm signals and notifications. A set number of alarm signals will be sent from a set number of alarm sources over different intervals of time to simulate and validate the applications performance under different load conditions. The alarm signals are sent to the alarm source end point and injected into the application to begin the filtering process where corresponding alarm signals and/or notifications are created if they match predefined rules. If a notification is created, then a push notification is expected to send to a mobile device. The performance test is executed in a simulated hospital network environment.
-
Pass/Fail Criteria
The pass criteria of the performance test are that Smart Alarm Interface can process and respond to at least 250 alarm signals a second. If Smart Alarm Interface is not able to meet the pass criteria or if it processes an alarm signal incorrectly the test is considered a failure and the deviation will be corrected immediately. -
Results Summary
The acceptance criteria were met for multiple iterations of the tests performed. Smart Alarm Interface was able to correctly process and respond to 250 alarm signals a second over various time intervals. The results were validated by comparing the number of sent simulated alarm signals that match a rule in Smart Alarm Interface to the number of alarm signals created in the alarm database. Several simulated alert signals intentionally do not match a rule so we know Smart Alarm Interface can properly match alarm signals to rules under high use. During the simulation, the correct number of alarm signals were created in Smart Alarm Interface. For alarm signals that did match rules, start times were compared to validate the time periods that the alarm signals were received. Additionally, the results were validated by making sure that for each alarm signal created in Smart Alarm Interface had a corresponding notification that was created and that a push notification was sent by checking that the "sendon" column in the notification table was populated with a value that matched the expected time period.
At around 400 alarm signals a second, throughput started to decline slightly with alarm signal and notification timestamps started to be delayed slightly, but the alarm signal filtering and notification creation was still handling correctly with an error rate of 0.0%. It is unclear if the decline in throughput is due to the application performance or to the test setup.
Key Metrics (Sensitivity, Specificity, PPV, NPV, etc.)
error rate: 0.0%
Predicate Device(s)
Ascom Mobile Monitoring Gateway (MMG), K111215
Reference Device(s)
Not Found
Predetermined Change Control Plan (PCCP) - All Relevant Information
Not Found
§ 870.2300 Cardiac monitor (including cardiotachometer and rate alarm).
(a)
Identification. A cardiac monitor (including cardiotachometer and rate alarm) is a device used to measure the heart rate from an analog signal produced by an electrocardiograph, vectorcardiograph, or blood pressure monitor. This device may sound an alarm when the heart rate falls outside preset upper and lower limits.(b)
Classification. Class II (performance standards).
0
Image /page/0/Picture/0 description: The image contains the logo of the U.S. Food and Drug Administration (FDA). On the left is the Department of Health & Human Services logo. To the right of that is the FDA logo, which consists of the letters "FDA" in a blue square, followed by the words "U.S. FOOD & DRUG ADMINISTRATION" in blue text.
October 30, 2020
BoxView, LLC % Robert Steurer Principal Consultant Steurer Consulting Group 800 Blue Quail Rd. Keller, Texas 76248
Re: K193421
Trade/Device Name: Box View Smart Alarm Interface (SAI), Model SA-01 Regulation Number: 21 CFR 870.2300 Regulation Name: Cardiac Monitor (Including Cardiotachometer And Rate Alarm) Regulatory Class: Class II Product Code: MSX Dated: July 28, 2020 Received: July 31, 2020
Dear Robert Steurer:
We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading.
If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls. Existing major regulations affecting your device can be found in the Code of Federal Regulations, Title 21, Parts 800 to 898. In addition, FDA may publish further announcements concerning your device in the Federal Register.
Please be advised that FDA's issuance of a substantial equivalence determination does not mean that FDA has made a determination that your device complies with other requirements of the Act or any Federal statutes and regulations administered by other Federal agencies. You must comply with all the Act's
1
requirements, including, but not limited to: registration and listing (21 CFR Part 807); labeling (21 CFR Part 801); medical device reporting of medical device-related adverse events) (21 CFR 803) for devices or postmarketing safety reporting (21 CFR 4, Subpart B) for combination products (see https://www.fda.gov/combination-products/guidance-regulatory-information/postmarketing-safety-reportingcombination-products); good manufacturing practice requirements as set forth in the quality systems (OS) regulation (21 CFR Part 820) for devices or current good manufacturing practices (21 CFR 4, Subpart A) for combination products; and, if applicable, the electronic product radiation control provisions (Sections 531-542 of the Act); 21 CFR 1000-1050.
Also, please note the regulation entitled, "Misbranding by reference to premarket notification" (21 CFR Part 807.97). For questions regarding the reporting of adverse events under the MDR regulation (21 CFR Part 803), please go to https://www.fda.gov/medical-device-safety/medical-device-reportingmdr-how-report-medical-device-problems.
For comprehensive regulatory information about mediation-emitting products, including information about labeling regulations, please see Device Advice (https://www.fda.gov/medicaldevices/device-advice-comprehensive-regulatory-assistance) and CDRH Learn (https://www.fda.gov/training-and-continuing-education/cdrh-learn). Additionally, you may contact the Division of Industry and Consumer Education (DICE) to ask a question about a specific regulatory topic. See the DICE website (https://www.fda.gov/medical-device-advice-comprehensive-regulatoryassistance/contact-us-division-industry-and-consumer-education-dice) for more information or contact DICE by email (DICE@fda.hhs.gov) or phone (1-800-638-2041 or 301-796-7100).
Sincerely,
for
Jennifer Shih Kozen Assistant Director Division of Cardiac Electrophysiology, Diagnostics and Monitoring Devices Office of Cardiovascular Devices Office of Product Evaluation and Quality Center for Devices and Radiological Health
Enclosure
2
Indications for Use
510(k) Number (if known) K193421
Device Name
BoxView Smart Alarm Interface (SAI), Model SA-01
Indications for Use (Describe)
The intended use of the Box View Smart Alarm Interface (SAI). Model SA-01 is to provide an interface with physiological patient monitoring systems to forward information associated to an alarm event to a designated display device(s). For medical, near real time alarms, the BoxView Smart Alarm Interface, Model SA-01 is intended to serve as a parallel, redundant, mechanism to inform healthcare professionals of particular medical alarm events. The BoxView Smart Alarm Interface, Model SA-01 does not alter the primary medical devices and associated alarm annunciations.
The BoxView Smart Alarm Interface. Model SA-01 is intended for use as a secondary alarm notification system. It does not replace the primary alarm function on the monitor.
The BoxView Smart Alarm Interface. Model SA-01 is not intended to be used for diagnostic purposes. The Box View Smart Alarm Interface, Model SA-01 is intended for use by professional clinical personnel and relies on proper use and operation of both the communication in place at the healthcare facility and the display devices used. The BoxView Smart Alarm Interface, Model SA-01 is a software product and cannot come into physical contact with patients.
| Type of Use (Select one or both, as applicable) | ||||
|---|---|---|---|---|
| X Prescription Use (Part 21 CFR 801 Subpart D) | Over-The-Counter Use (21 CFR 801 Subpart C) | |||
| CONTINUE ON A SEPARATE PAGE IF NEEDED. | ||||
| This section applies only to requirements of the Paperwork Reduction Act of 1995. | ||||
| DO NOT SEND YOUR COMPLETED FORM TO THE PRA STAFF EMAIL ADDRESS BELOW. | ||||
| of this information collection, including suggestions for reducing this burden, to: | The burden time for this collection of information is estimated to average 79 hours per response, including the | |||
| time to review instructions, search existing data sources, gather and maintain the data needed and complete | ||||
| and review the collection of information. Send comments regarding this burden estimate or any other aspect | ||||
| Food and Drug Administration | ||||
| Office of Chief Information Officer | ||||
| Paperwork Reduction Act (PRA) Staff | ||||
| PRAStaff@fda.hhs.gov | Department of Health and Human Services | |||
| "An agency may not conduct or sponsor, and a person is not required to respond to. a collection of | ||||
| information unless it displays a currently valid OMB number." | ||||
| EORM EDA 3884 17/47) | Page 1 of 1 |
3
| Date Prepared: | October 29, 2020 |
|---|---|
| Submitter: | BoxView, LLC |
| 14001 McAuley Blvd., Suite 220 | |
| Oklahoma City, OK 73134 | |
| Submitter Contact: | BoxView, LLC |
| 14001 McAuley Blvd., Suite 220 | |
| Oklahoma City, OK 73134 | |
| Mr. Cody Guest | |
| BoxView, LLC | |
| Phone: +1 (405) 740-5785 | |
| Fax: | |
| Email: cody.guest@boxview.com | |
| Application | |
| Correspondent: | Bob Steurer |
| Principal Consultant | |
| Steurer Consulting Group | |
| 800 Blue Quail Rd. | |
| Keller, TX 76248 | |
| steurerbob@gmail.com | |
| +1 (425) 358-1072 | |
| Manufacturing Site: | BoxView, LLC |
| 14001 McAuley Blvd., Suite 220 | |
| Oklahoma City, OK 73134 | |
| Trade Name: | BoxView Smart Alarm Interface (SAI), Model SA-01 |
| Common Name: | Network and Communication Middleware |
| Classification Name: | System, Network and Communication, Physiological Monitors |
| Primary | |
| Classification | |
| Regulation: | 21 CFR §870.2300 |
| Primary Product | |
| Code: | MSX |
| Predicate Device: | Ascom Mobile Monitoring Gateway (MMG), K111215 |
4
| Substantially
Equivalent Devices: | Company | Predicate 510(k)
Number | Predicate
Manufacturer / Model |
|--------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------|------------------------------------------|
| | Ascom | K111215 | Ascom Mobile Monitoring
Gateway (MMG) |
| Device Description: | The BoxView Smart Alarm Interface (SAI), Model SA-01, is a Software
Medical Device (SaMD) product intended to be located on-site in the
hospital, or pre-configured off site in the 'cloud' utilizing a standard Linux
operating system. The primary purpose of the BoxView Smart Alarm
Interface (SAI), Model SA-01 is to act as a message integrator to forward
patient monitor status and alarm event information originating from a
patient monitoring network. Users receive interactive, time-critical
information from patient monitoring devices directly via their display
devices as text, alarms or data. The BoxView Smart Alarm Interface (SAI),
Model SA-01 allows caregivers to be informed of their patient's alarm
conditions when they are not in the patient vicinity.
The BoxView Smart Alarm Interface (SAI), Model SA-01 is an open system
that is compatible with most smart phones or computers. The BoxView
Smart Alarm Interface (SAI), Model SA-01 connects to the information
sources through wired Ethernet connections which are part of the
customer's infrastructure. The BoxView Smart Alarm Interface (SAI),
Model SA-01 software acquires patient data from patient monitoring
devices and allows the user to configure the system to determine which
information, including alarm notifications, is delivered to which users
communicators. The BoxView Smart Alarm Interface (SAI), Model SA-01
then formats the data for delivery to the display devices.
The BoxView Smart Alarm Interface (SAI), Model SA-01 system is designed to forward alarm event information as the alarms are recognized by the
patient monitoring network. The system is also capable of being
configured to periodically forward a patient's physiological data as well.
All messaging activities are recorded by the BoxView Smart Alarm
Interface (SAI), Model SA-01 providing real-time activity logging for audit
trail records and reporting.
The BoxView Smart Alarm Interface (SAI), Model SA-01, system is a
secondary alarm notification system. It does not replace the primary
alarm function of the bedside monitor or telemetry monitoring system. | | |
5
Indications/Intended The intended use of the BoxView Smart Alarm Interface (SAI), Model SA-Use: 01 is to provide an interface with physiological patient monitoring systems to forward information associated to an alarm event to a designated display device(s).
For medical, near real time alarms, the BoxView Smart Alarm Interface, Model SA-01 is intended to serve as a parallel, redundant, mechanism to inform healthcare professionals of particular medical alarm events. The BoxView Smart Alarm Interface, Model SA-01 does not alter the behavior of the primary medical devices and associated alarm annunciations.
The BoxView Smart Alarm Interface, Model SA-01 is intended for use as a secondary alarm notification system. It does not replace the primary alarm function on the monitor.
The BoxView Smart Alarm Interface, Model SA-01 is not intended to be used for diagnostic purposes. The BoxView Smart Alarm Interface, Model SA-01 is intended for use by professional clinical personnel and relies on proper use and operation of both the communication infrastructure in place at the healthcare facility and the display devices used. The BoxView Smart Alarm Interface, Model SA-01 is a software product and cannot come into physical contact with patients.
Summary of Substantial Equivalence:
The predicate device is intended to interface to GE Healthcare patient monitoring network to forward alarm signals from patient monitors and telemetry transmitters to specified caregivers which is the same as the BoxView SAI which is designed to interface to the Spacelabs patient monitoring network using protocol provided from Spacelabs. Both the predicate device and the BoxView SAI provide a secondary means of alarm annunciation to a mobile device or computer. Neither the predicate nor the BoxView SAI alters the context of the alarm signal, the behavior of the monitoring system, and neither is intended for diagnostic use. The differences in intended use are not critical to the safety and effectiveness of the device when used per labeling instructions.
The predicate device is configured on hardware provided by Ascom while the BoxView SAI can be configured on local hardware that meets minimum requirements specified by BoxView and can be configured as a cloud-based system. The hardware specifications from BoxView reflect newer technology than that offered by Ascom in 2011/2012. This does
6
not alter the intended use of the BoxView SAI but provides additional flexibility for users. This difference does not affect safety or effectiveness of the BoxView SAI when hardware is used as specified.
Technology Comparison:
The BoxView Smart Alarm Interface, Model SA-01 employs the same technological characteristics as the predicate device.
| Characteristic | Predicate Device –
Ascom MMG | Smart Alarm Interface
Model SA-01 |
|-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Alarm Interface | Transports alarm signals
to display devices
provided by Ascom or
third-party mobile
device companies. | Transports alarm signals
to display devices
provided by compatible
third-party mobile
device companies
including Ascom |
| Configurations | Web browser-based
application configured
to send alarm
notifications to specific
users | Web browser-based
application configured
to send alarm
notifications to specific
users |
| Hardware
Requirements | Server: (Elise Embedded
Linux Server) Appliance
with Microsoft® Internet
Explorer® 6.0 or later
browser.
Sun™ Java™ Runtime
Environment (JRE) 6 or
later; Network
10 baseT or 100 baseT
(10 MB or 100 MB) | Minimum Specs for
Server: 64-bit
(x64) processor, two
core; 8GB Ram; 40GB
Hard Drive; Linux OS -
CentOS 8 or Later
(CentOS is a version of
Linux); Google Chrome
54.0 or later browser;
Network - 100MB. |
| Communications
Operation | Wireless
communications
platform, WiFi, paging
technology, hospital
infrastructure. | Wireless
communications
platform, WiFi, paging
technology, hospital
infrastructure. |
| Administration | User ID and Password required to enter the configuration manager | User ID and Password required to enter the configuration manager |
| Connection to Network | Messaging integration to hardware on-site via wired Ethernet connections | Message integration to hardware on-site via wired Ethernet connections; or Cloud based application connected through site-to-site internet connection |
| Logging | Stores messages, alarms, faults, input/output activities. | Stores messages, alarms, faults, input/output activities. |
| User Interface | Accept or reject messages if busy – automatically redirect to another caregiver. | Accept or reject messages if busy – automatically redirect to another caregiver. |
| Alarm Configuration | Filter to provide specific alarm signals to specific caregivers | Filter to provide specific alarm signals to specific caregivers |
7
8
Performance Testing Summary
Non-Clinical Bench Performance Testing Information
Test Report Summaries
-
- Test(s) Performed
The bench performance test for Smart Alarm Interface involves simulating alert/alarm messages from concurrent alert/alarm sources to validate that below and at server capacity the alarm messages filter correctly and that notifications send to caregiver devices when applicable. Another bench performance test will involve simulating alert/alarm messages over server capacity to identify when the application performance begins to degrade.
- Test(s) Performed
-
- Objective(s) of the Test(s)
The first objective of the test is to confirm that Smart Alarm Interface can process and respond to 250 alert/alarm messages per second from 25 concurrent alert/alarm sources, this results to each alert/alarm source sending 10 alerts/alarm signals a second.
- Objective(s) of the Test(s)
The second objective is to determine the number of alert/alarm messages and concurrent alert/alarms sources in which the application performance starts to worsen. Performance test is also used to help identify any bottle necks or areas of the code that have the potential to be better optimized. These performance tests are to be executed before every major software release to confirm that the application is performing per requirements.
-
- Test Methods
- a. Test Environment
- a. XprezzNet Simulator Used to simulate alarm messages from Spacelabs patient monitors. (XprezzNet is the name Spacelabs has designated for their network communication)
- b. Amazon Web Services A cloud computing service that hosts the docker container(s) that run the Smart Alarm Interface application and Postgres database.
- c. Mobile Devices iOS and Android communicator devices.
- i. iPhone XR iOS 13.4.1
- ii. iPhone 8 iOS 12.3.1
- iii. iPhone XR iOS 13.5.1
- iv. iPhone X iOS 13.1
- iPhone XS iOS 13.3.1 V.
- vi. Samsung Galaxy A10e Android 9
- vii. Google Pixel 4 Android 10b
- b. JMeter is used to execute requests with the XprezzNet simulate alarm sources that send alert/alarm messages to validate that Smart Alarm Interface filters alarm signals properly and creates or updates alarm signals and notifications. A set number of alarm signals will be sent from a set number of alarm sources over different intervals of time to simulate and validate the applications performance under different load conditions. The alarm signals are sent to the alarm source end point and injected into the application to begin the filtering process where corresponding alarm signals and/or notifications are created if they match predefined rules. If a notification is created, then a push notification is expected to send to a mobile device. The performance test is executed in a simulated hospital network environment.
9
4. Pass/Fail Criteria
The pass criteria of the performance test are that Smart Alarm Interface can process and respond to at least 250 alarm signals a second. If Smart Alarm Interface is not able to meet the pass criteria or if it processes an alarm signal incorrectly the test is considered a failure and the deviation will be corrected immediately.
5. Results Summary
The acceptance criteria were met for multiple iterations of the tests performed. Smart Alarm Interface was able to correctly process and respond to 250 alarm signals a second over various time intervals. The results were validated by comparing the number of sent simulated alarm signals that match a rule in Smart Alarm Interface to the number of alarm signals created in the alarm database. Several simulated alert signals intentionally do not match a rule so we know Smart Alarm Interface can properly match alarm signals to rules under high use. During the simulation, the correct number of alarm signals were created in Smart Alarm Interface. For alarm signals that did match rules, start times were compared to validate the time periods that the alarm signals were received. Additionally, the results were validated by making sure that for each alarm signal created in Smart Alarm Interface had a corresponding notification that was created and that a push notification was sent by checking that the "sendon" column in the notification table was populated with a value that matched the expected time period.
At around 400 alarm signals a second, throughput started to decline slightly with alarm signal and notification timestamps started to be delayed slightly, but the alarm signal filtering and notification creation was still handling correctly with an error rate of 0.0%. It is unclear if the decline in throughput is due to the application performance or to the test setup.
6. Discussion/Conclusions
The performance test used simulated alarm signals from alarm sources where the alarm signals were injected directly into the Smart Alarm Interface application. It is plausible that results may vary in production with physical alarm source devices. The objective of this test was to validate the speed at which Smart Alarm Interface can process alarm signals and create/push corresponding notifications. This test is not intended to validate the performance of the alarm source or the notification messaging system. Based on the test results Smart Alarm Interface can successfully process the recommended server capacity of 250 alarm signals per second from 25 concurrent alarm signal sources and potential to process above that under extremely high usage.
10
| Sterilization and
Shelf-life | The BoxView Smart Alarm Interface (SAI) is a Software as a Medical Device
(SaMD) product and is not provided sterile. The BoxView Smart Alarm
Interface (SAI) does not have a shelf-life.
Therefore, this section is not applicable |
|---------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Biocompatibility | The BoxView Smart Alarm Interface (SAI) is a Software as a Medical Device
(SaMD) product and does not directly nor indirectly contact the patient.
Therefore, this section is not applicable. |
| Software Testing | The BoxView Smart Alarm Interface (SAI) was designed and developed
according to the BoxView internal software development process. The
BoxView Smart Alarm Interface (SAI) was tested using verification and
validation methods, the results of which indicate the Smart Alarm
Interface (SAI) complies with its specifications. |
| Electrical Safety | The BoxView Smart Alarm Interface (SAI) is a Software as a Medical Device
(SaMD) product. Therefore, this section is not applicable. |
| Electromagnetic
Compatibility
Testing | The BoxView Smart Alarm Interface (SAI) is a Software as a Medical Device
(SaMD) product designed to perform its intended purpose without being
part of a hardware medical device. Therefore, this section is not
applicable. |
| Performance Testing
– Bench | The BoxView Smart Alarm Interface (SAI) was tested for performance in
accordance with its predetermined specifications. |
| | The results of the performance testing show the BoxView Smart Alarm
Interface (SAI) complies with these specifications. |
| Conclusion | A comparison of the predicate device and a review of the testing results
shows that the BoxView Smart Alarm Interface (SAI) is substantially
equivalent to the predicate device. |