The doctor(s) are in: Planning and conducting moderated usability testing

Our team decided on conducting moderated usability tests to collect data, as observing users completing tasks relevant to their professional interests would give the most valuable insights on their behavior, thought process, and opinions on the site’s flow.

We recruited nine participants consisting of both contacts from Medscape and our own personal networks. To ensure these users matched up with Decision Point’s intended audience, we screened for working healthcare professionals that used desktop interfaces in their free time to look up medical news and information.

We chose tasks for the moderated user tests that would evaluate the ability of the Decision Point to handle realistic inquiries that working healthcare professionals would use the site for. 

1. Please browse the site and give us your thoughts.
2. You have a family member that is at risk for kidney disease and you want to educate them about best prevention and monitoring methods. Please use Decision Point to find videos related to this topic.
3. You’ve watched the video, and are interested in finding more content related to this topic. Find a related video about this topic.
4. A patient of yours has the heart condition HFpEF (heart failure with preserved ejection fraction), and you want to know if you should treat them with Sacubitril/Valsartan? Find content on Decision Point that will assist with this issue.

We also asked participants to fill out a pre-test questionnaire, a post-test questionnaire, and a System Usability Scale (SUS) to supplement the test data with participants’ opinions and a set of quantitative data we could concretely evaluate.

Getting to the (Decision) Point: Findings and Recommendations

Finding: Users had trouble returning to the homepage after finding content

During the tests, 3 out of 9 participants experienced frustration when trying to return to the Decision Point homepage. When on pages with video content, users would click on the header logo expecting to return to the homepage, but the site would instead take them to the category page, limiting the freedom in exploring the site. This is a factor that likely contributed to a lack of engagement with the Decision Point’s non-video pages because it limited the user’s control and ability to explore the site.

Recommendation: Provide an easier way to return to Decision Point homepage

We recommend linking the header logo to the homepage to bring the user flow more in line with user’s expectations and moving the category name to the middle of the header to support the usability heuristic of recognition over recall for users knowing what section of the flow they are in.

Finding: Homepage categories don’t support typical use cases

When landing on the homepage, 5 out of 9 participants mentioned that they were unsatisfied with the categories options that were given at the start of the user flow. We found that users viewed the categories shown as too rigid and preferred a more precise method of filtering content directly from the homepage.

Recommendation: Add search functionality to Decision Point homepage

‍

In order to accommodate users who prefer a more direct route to finding content, we recommend adding a clear, visible search bar on the Decision Point Homepage. This function was previously available on the category page, and making it available to users landing on the homepage will provide a more cohesive experience and give confidence that Decision Point’s interface is a reliable source in immediately finding answers they need.

Finding: Category page introduces friction in user flow

Users felt that the category page, which bridges the homepage and Decision Tree filter pages, was confusing because it was tough to find the next step in the user flow and did not add any value in their goal of accomplishing the tasks during the test. In fact, 3 of 9 users arrived at the category page for the first time and scrolled to the bottom searching for content before scrolling back to the top of the page, because it was not clear what to interact with. Some quotes from users were as follow:

“The expert contributors take up too much space.”

“The start button is hidden in the banner, it could be more obvious.”

“Do I use ‘Start’ or ‘Search’ to find videos?”

Recommendation: Remove Category page

We recommended removing the category page and linking the homepage directly to the Decision Tree. This change is an example of addition through subtraction to provide a more streamlined flow for users by lessening the amount of clicks the user needs to get to their desired page.

Finding: Video filters and results are visually unbalanced 

While users described the Decision tree video filters as “functionally usable”, many felt the hierarchy and balance of the page was skewed in the wrong way. The filters occupy a disproportionate amount of screen space and the results area appeared too small, with one user describing that it “takes more brain power and time to figure out where I am looking at” 

Recommendation: Prioritize video results over filters

We recommended adjusting the balance between the filters and video results on the Decision Tree page so users could better accomplish their goal: finding content relevant to their professional interests. The Decision Point search function has a results page that better reflects a video focused balance, and introducing this already existing component on the page is a small change with easy implementation rather than a larger project.

Delivering the insights and Conclusion

After 8 weeks of hard work, we detailed our findings to the client via Zoom, emphasizing what recommendations would be most effective based on our usability testing. Our report was received positively, with the Medscape team noting that our research validated a lot of the assumptions their internal team had about the product and referred to it as a great starting point for their upcoming redesign.

Our team’s collaboration and communication skills improved dramatically over the course of the research study as we grew accustomed to each other's strengths and overcame obstacles in the process. In future projects it will be important to keep the following in mind:

1. Improvisation during usability testing - Throughout our testing sessions, we encountered situations that forced us to adjust our script, such as a user’s screen share not working. As a researcher you will often have to think on your feet and figure out how to get the best data when circumstances don’t allow you to directly follow the script.
2. Standardize note taking - When you have a group of people with differing processes on how they conduct research and format data, it is difficult to compare and contrast notes because everyone has a different idea on what is the most important. If we agree on a concrete outline for note-taking on moderated usability testing, it will make the insight gathering much more efficient and streamlined.