There’s no doubt that the internet of things (IoT) creates a new technological ecosystem in healthcare, helping organizations manage bottlenecks in care and bridging the gap between patients and providers. The list of applications for connected devices in hospitals keeps growing from real-time staff tracking across facilities to ensuring preventive equipment maintenance and 24/7 patient vitals monitoring in ER and resuscitation departments.
But care delivery doesn’t stop after discharge. A range of smart consumer-facing medical devices can ensure that the patient’s health status will be kept under control between office visits. This will allow providers to grasp a multitude of subjective and objective symptoms, analyze them, build patterns and observe them in dynamics.
Additionally, the patient becomes more informed and invested in their own health management, whether they have a chronic condition or not. With valuable insights about regular processes or sudden sensations, such as mood, sleep, temperature, pain, cough and more, patients gain a better understanding of how their own body works.
Experts acknowledge the growing role of IoT in healthcare, too. For instance, Deloitte anticipates the healthcare IoT market volume to reach $158.1B by 2022.
However, with changes come challenges. As many connected medical devices gather patient vitals in chronic and acute cases, assuring their performance becomes the vendors’ top priority. Any performance interruptions bear potential risks for patient safety and health outcomes, with varying severity depending on a particular device’s intended purpose.
Therefore, QA specialists are under increased pressure when carrying performance and compatibility testing of medical IoT devices. If the team is unaware of domain specifics and isn’t ready to mitigate arising challenges, the resulting product may fall short of the high requirements for medical software or even fail the FDA approval.
Challenge #1. Internet Disruptions Can Kill
When checking the performance of medical IoT software, testing specialists deal with the load, network bandwidth, latency, and other metrics both for mobile and web applications. While it is undesirable for a website to crash under an unexpected load surge, in healthcare IoT crashes are unacceptable. Especially not for smart medical devices directly involved in patient care, such as a continuous patient monitoring system or a smart insulin pump.
Still, force majeures do happen, and it is critical to prepare the product accordingly. When testing consumer-facing IoT devices, some QA teams need to take all the important variables into account. For example, use conditions can be highly diverse, especially when the connected device is designed for use in rural areas.
From smart thermometers to blood coagulation meters and inhalers, the stability of the user’s internet connection can twist the test results or even undermine them if the receiving specialists weren’t aware of the actual network bandwidth. To try and compensate for such pitfalls, both developers and QA specialists have to ensure that service disruptions won’t affect data transmission. For example, all sensitive PHI data can be backed up prior to transferring, so that each data package could be erased only after the sending attempt is successful.
Additionally, performance engineers need to keep in mind that IoT devices don’t operate with planned downtimes, so any patches and updates have to be integrated seamlessly into the device operation. It’s highly recommended to test the process at least with the first patch in order to polish it and ensure that new updates won’t disrupt user experience or tamper with sensitive data.
Challenge #2. Diversity of Protocols Calls for Specialized Tools
As connected medical devices are entering the mainstream, they are creating a whole new niche in digital care delivery. With this comes another performance testing challenge: devices frequently need new protocols to manage communication with the servers. The healthcare IoT ecosystem is already complex since it lacks standardization of IoT protocols currently in use. While the MQTT protocol is one of the most common because it handles low bandwidth networks and low-memory devices, there are also HTTP, CoAP, XMPP, DDS, and many others.
The diversity of available protocols and the devices they support puts an additional strain on performance engineers as they have to find a fitting load test tool covering this particular combination. But that’s not the only difficulty in choosing the right toolset.
Challenge #3. Not All Load Testing Tools Were Made Equal
There’s no better way for QA specialists to hop into a clinician’s shoes than deciding on the testing toolkit for a particular connected device. They need to pick the right instruments to arrive at a relevant diagnosis as well as treatment suggestions. Healthcare IoT architectures are complex and comprise unique combinations of devices, sensors, and actuators communicating with disparate systems via distinct protocols, generating and transferring enormous amounts of data. Not to forget that patient data is highly sensitive, so each transaction should be secure.
To find a fitting load testing tool, performance engineers can break down the infrastructure in question and range the parameters according to their load testing priorities. For example, if both continuous data sharing and scaling to multiple devices are required, it’s worth compromising only when there are thousands or millions of devices. Continuous data sharing, however, should be prioritized in any toolkit.
Nailing Healthcare IoT Performance Testing
While testing the performance of a connected medical device might seem similar to that of a mobile app or a website, the complexity of IoT architectures creates new challenges for QA teams. These challenges can overwhelm even experienced specialists because they have to evaluate the system while always keeping in mind patient safety and data integrity.
Here are a few suggestions on how to effectively deal with the challenges described above:
– Emphasize atypical use cases: whether you are evaluating a bedside monitor or an implantable smart device, users can and will get creative with the system.
– Don’t try to find a perfect load testing product supporting all protocols within your healthcare IoT stack. Pick the tool with an SDK to start building tests right away.
– Narrow down your test scenarios by defining the most popular device/software version pairings to ensure more effective coverage. Then, test all new devices that communicate with the network and ensure that data will stay safe in case of internet network disruptions.
– Make sure you monitor the IoT platform performance between builds.
Maxim Chernyak is Head of Test Automation and Performance Testing Lab at A1QA, an expert in test automation methodologies and tools for functional and nonfunctional testing. Accountable for the education and adoption of state-of-the-art quality engineering practices by QA teams.
