Note: This is post #1 of a series which focuses on product development best practices for software-enabled medical device teams that want to ship faster.

Imagine being in the middle of verification & validation testing with a submission timeline approaching fast in the order of weeks. To your team's surprise, user testing discovers the need to implement an unexpected design change.

We meet these two types of teams: those who take one month to ship such a product update and need to push back their timelines, and those who take less than one week. Being set up as the latter requires deep intention in the processes and tools that are leveraged. 

Zooming out, medical devices are experiencing a revolution in innovation, with more devices integrating software and AI to improve patient care. But companies developing these software-intensive, connected devices face novel challenges in their development processes. 

They need to manage greater product complexity, cybersecurity, interoperability with other systems, and increasing amounts of third party software. And they often need to do this with distributed product teams (distributed over multiple locations and time zones) and in an environment of increasing regulatory scrutiny.

How do you balance speed with quality? 

In this post, Ultralight has collaborated with Aaron Joseph–principal consultant with Sunstone Pilot and a seasoned expert in best practices for software-enabled devices–to compare two approaches to product development of software-enabled devices: 1) traditional, document-based, and 2) modern, platform-based.

A Hypothetical That's All Too Common

Let's begin with a hypothetical medical device company developing a hypothetical diagnostic device. This device consists of a wearable and mobile app which sends data to the cloud where proprietary algorithms process the data and send the results to the patient's physician. This IOT architecture is common with many new medical devices but presents considerably more complexity then older devices with a limited amount of firmware running on a single microcontroller.

The company has been developing the product for a year and a half, and they are now in the V&V testing phase and they've discovered a problem with the user interface.

Usability testing showed that 25% of users misinterpreted the LED patterns on the wearable, which has triggered a change to the user interface and the steps the user performs to set up the device and sync it with their smartphone.

The product team, under intense time pressure, needs to make multiple changes to the software and the related documentation. They need to change both the firmware in the wearable and the mobile app software. They also need to update the user instructions which includes both the instructions visible in the mobile app and a printed quick reference guide. They need to update software requirements and software design documentation and software test documentation, retest the software, and rerun the usability testing. And they need to do all of this rapidly without missing any details.

Method 1: Traditional, document-based approach

How does the product team manage this urgent change? The Project Manager first fills out a change request form (Word Document) and emails it to all the stakeholders. After some back and forth they finally settle on a description of the necessary changes and release the form. The software team makes changes to the firmware and the mobile app and does some initial testing. The project manager emails a description of the user interface changes to a human factors consulting firm who is managing the usability testing. The consulting firm prepares to repeat some of the usability testing to confirm that the new user interface will work and be interpreted properly by the users. The software testers are waiting until updated software requirements and software design documents are released but those documents won't be updated until the software team is finished with the code changes.

What is the result? The software testers eventually test the modified firmware and mobile app but have to repeat the mobile app testing because some of the software test cases that were affected were not updated. The usability testing is delayed twice: once because the information about the change that was emailed to the firm was incomplete and a second time because the quick reference guide had not been updated with the changes to the user interface. Finally, after a full month, the team has re-released the software and completed the change.

However, the Quality Lead discovers three weeks later that they forgot to update the Usability Risk Analysis document with the changes to the user workflow. And then the Quality Lead has to write a long DHF memo trying to explain why this was done after the change was implemented and after the usability testing was performed.

This is a common situation we’ve seen at many medical device companies. A relatively small change to the product software results in excessive time to fully test and document the change. In this scenario we see many of the typical problems with the traditional approach: a slow, error-prone, and stressful experience. (Did we mention that the software testers had to spend one weekend testing to make up for lost time?)

Method 2: Modern, platform approach

Let’s look at what this same design change scenario would look like if the team was using the collaboration and documentation automation features in a modern, integrated platform that we’ll refer to as “The Hub” (a set of software tools configured for rapid medical device development).

First, The Hub visualizes everything affected by the product design change (upstream and downstream traceability are easily understood, and a thorough change impact assessment is possible). This is described in detail in the digital change order which is immediately visible to everyone while it’s being drafted and after it is approved.

The team quickly updates the related usability risk analysis, software requirements, and software design documentation in The Hub to capture the change. Then everyone on the distributed product team can see the same information about the change (from a “single source of truth”) and can act quickly in sync. The developers update the firmware and mobile app, the software testers revise their test protocols, the contracted graphic artist updates the Quick Reference Guide, and the HF consultants prepare for another round of usability testing with all of the changes. Once the software testers re-test the updated software, the usability testing is quickly performed and the results are automatically reflected in The Hub’s dedicated design control module. Throughout this process, The Hub automatically re-generates new revisions of all the affected DHF documents as well as generating new DHF documents (15 documents in total).

The result: working on this collaboration platform the product team was able to make all the changes, repeat the testing, release all the new and revised DHF documentation, and release the software all in just one week without any errors!

Conclusion

Our example is all too common in the market. Perhaps counterintuitively, the teams that are able to move the fastest have the highest quality and most valuable products and companies, with the best patient experiences.

But in order to move faster without breaking things, a modern approach to product development requires a modern toolset. In our next post, we’ll discuss how a modern tool for product lifecycle management can dramatically accelerate release cycles for software-enabled devices.

Aaron Joseph, principal consultant with Sunstone Pilot, is a biomedical engineer based in Waltham, Mass. With over 20 years of experience across a broad range of medical devices from surgical robotics to medical imaging to IOT and SaMD products, he helps clients efficiently tackle risk management and design controls for new product development.

Ultralight Labs develops software to streamline product development and documentation for the most innovative medical device teams. Book an Ultralight demo here.