Planning Usability Testing During COVID-19

The recent Covid-19 pandemic has changed how we are able to interact socially, and it has changed our ability to adapt.

We have seen businesses close, travel halted, and lives lost. While the world works towards administering vaccines, it is important to remain as safe as possible when interacting with people outside of your immediate household. We have seen work process adjustments made by pharmaceutical, combination device, and medical device manufacturers, as well as the vendors that support these companies. While most companies have shifted their employees to working from home, it is not always possible to complete certain activities remotely, such as human factors usability testing.

Despite this pandemic, the medical industry strives to continue designing medical devices that are capable of supporting the clinicians serving Covid-19 and other patients around the globe. Whether standalone devices such as ventilators or combination devices such as autoinjectors and pre-filled syringes, human factors specialists work to gather information from the user that can inform the usability of these devices. If you find yourself in need of conducting a human factors usability formative or summative study during this pandemic, here are 5 important considerations.

 Commit more time during the planning phase.

The best studies have always been those that have received plenty of thought. Given the new health environment of Covid-19, planning becomes even more important when attempting to complete a usability study. You must now even put more thought into what, how, when, and where you are testing.

WHAT type of device you are testing will determine your options for HOW to complete the usability testing. There are some devices or software that may not require in person usability testing and can be tested remotely. Online studies can provide valuable user feedback during formative studies. However, it has not been recommended to use online usability testing for summative/validation studies; and completing remote usability testing presents its own unique challenges for accurately reporting performance and maintaining confidentiality of proprietary information.

If you must complete the usability testing in person, it is best to think about WHERE you are planning to complete the usability testing and to do thorough research about that location’s current restrictions on completing in-person usability testing. Every state, city, and facility may have different rules about completion of usability testing. This means that WHEN you are planning to complete usability testing will play a role because different countries, states, and cities are all at different phases of reopening. Furthermore, depending on the intended user population, it may be difficult to recruit participants, so it is recommended to increase the participant recruitment period to ensure that the study can be fully recruited.

Be prepared for increased study costs.

When completing usability testing in a Covid-19 health environment, it is important to be prepared for higher study costs. Completing usability testing in person during Covid-19 can increase the cost of the study due to an increase in study length, participant number, and/or participant compensation.

Increased Study Length

If usability testing is going to be completed in person, it is imperative that every step is taken to ensure the safety of everyone involved. One critical step is to ensure that the rooms and equipment are properly sterilized between participants. To properly clean the testing room and equipment it is recommended to build extra time in between the sessions.  Without proper time, safety precautions may fall between the cracks or get rushed, which could increase the risk of virus transmission.

Increased Participant Number

During the current pandemic, it is possible to have more unexpected no-shows or cancellations than usual. With the possibility of a greater participant attrition rate, it may be necessary to recruit more participants over and beyond what would normally be considered sufficient to ensure the minimum number of participants is achieved. If a study fails to fill the minimum required number of participants, this could lead to additional facility costs and a delay in study completion and potentially delayed product submissions.

Increased Participant Compensation

With many people feeling hesitant to leave their home, it may be more difficult to recruit study participants. This may present a challenge for products that cannot be tested in a virtual format such as Zoom or Skype. To ease the recruitment process, you may need to accept that participant costs (recruitment/compensation) may be higher than originally budgeted. For safety reasons you do not want to risk a participant feeling obligated to come in for their session if they are under the weather just to receive the compensation. This means it may also be appropriate to provide compensation to participants who cancel their session.

Follow social distancing guidelines.

To decrease the risk of exposure of staff and participants to Covid-19 it is imperative to adhere to social distancing guidelines. Social distancing can be incorporated into the study by organizing the training or test room to accommodate for the minimum 6 feet between the test staff and the participants or even to conduct the study with the participants and test staff in different rooms. The efficacy of social distancing can be further enhanced by wearing personal preventative equipment (PPE). Depending on the facility and product being evaluated, it is possible that the study can be completed with the experimenter and participant in the same building, just different rooms (i.e., the experimenter in the observation room and the participant in the testing room) in order to increase social distancing. At times, social distancing may not be possible. If social distancing guidelines must be overridden at any point during the study, it is recommended to inform the participant ahead of time under which circumstances this may happen. While it is best to limit the number of times this occurs, participants and staff should be provided the appropriate PPE to limit the risk of transmission.

Check with research facilities.

Many, if not all, research facilities have begun to reopen and accept studies while complying to state or federal laws. It is important to verify that your research facility partners have completed all the necessary changes to their physical space and necessary documentation prior to entering an agreement, such as their ability to contact participants and study team members in the event they are notified of a possible exposure. While this may create a slightly different environment than what has been experienced in the past with facility amenities, remember that the facilities are working hard to ensure everyone’s safety in addition to providing the best customer experience in this new environment. Common new procedural requirements include temperature checks, wearing personal preventative equipment (PPE), and established flow patterns to limit the number of people that are allowed in each room at a given time. Some facilities have also added plexiglass to the tables in the study rooms to increase safety for both the test staff and the participants. In addition to additional physical health precautions, it is possible that each facility may have additional paperwork to help ensure that the participants entering the facility will not increase the exposure risk per their state and city guidelines.

 Factoring high risk participants.

Depending on the type of product you are looking to investigate, it is possible that the primary user population currently engages with Covid-19 positive patients during their normal work hours. While it is recommended to avoid using participants that are currently treating Covid-19 positive patients, depending on the product, having health care professionals who treat Covid-19 patients may be unavoidable. If this is the case, then adherence to cleanliness and social distancing become more important. Even if the participants were negative on a recent Covid-19 test, that does not mean that they have not been exposed following their last test.

It is important to note…

Even if a usability test CAN be done, that does not necessarily mean that the test SHOULD be done. Given the current health environment with Covid-19, the number one concern for all organizations should be the safety of those people who would be involved with the study – the research facility staff, human factors team, and study participants. As an organization, it is important to make the internal decision about which projects need to proceed to testing and which projects can be postponed to a later date.

Want to take advantage of the best practices Suttons Creek has developed in usability studies and their administration both in-person and virtually?  We would be happy to share our knowledge, support your Human Factors efforts and ensure your products get to market with the safest and most effective route possible.

Matthew Nare, Consultant – Matthew Nare is a junior consultant on the human factors team for Suttons Creek, Inc. His previous work as a physical therapy aide provided an opportunity to gain clinical experience in inpatient and outpatient rehabilitative care settings. He came to Suttons Creek, Inc. following an internship completing clinical observational research with MedStar Health’s National Center for Human Factors in Healthcare. Additionally, he spent over a year completing accessibility evaluations at Cal State Long Beach’s Center for Usability in Design and Accessibility. This combination of work experience and his educational background provides him with a unique perspective on how the medical devices and their users function. LinkedIn: Matthew Nare