Engaging Users in Research without Opening Pandora’s Box

Engaging a product’s users early in the process enables the team to gather information about user needs and make user-driven decisions out of the gate. But some software developers might worry, “What if users ask for features that we can’t deliver right away, or worse, can’t deliver at all?”

It can be a tricky line to walk, but the results are well worthwhile. Here are some methods I employ to keep user discussions focused, while still giving people lots of opportunities to tell us how software could work better for them.

1. Get to know your SME (subject matter expert).

Everyone is an expert on something. Take time to get to know the person you’re meeting, what a day in their work looks like, what started them on this career path, and what motivates them from day to day. Not only does this help everyone relax a bit, but it can also help guide later questions.

For example, If I were interviewing a nurse about a patient scheduling application, I might ask:

“You mentioned you were on your feet a lot going from room to room, rarely at your desk…what information do you need when you’re transitioning from one patient to the next?”

I’m focusing the discussion on a moment in time and what is most critical, while demonstrating that I heard them when we were just getting to know each other. I’m also asking the expert to underline what is most important to them. From there, we can follow several possible paths to a solution.

2. Tell them your goal and where you are in the process.

This can be tricky, so I’ll put a caution on this to use your best judgment about how much to share. In my experience though, when I’m upfront with users about what I’m trying to learn and where we are in the process (i.e. “these are early sketches; we have room to make changes”), the SME can align their thinking to what you need right now.

This is especially helpful when you know the first version of your software will lack a lot of desired features, or you’re planning a big shift in the interface or the way people find what they need. I find that people tend to zero in on the best and worst aspects of what they’re testing, so I can explore what makes it great or where the current solution is missing the mark.

3. Provide a scenario or two.

Open-ended questions can be difficult. For example, it’s hard to answer, “What’s the most important aspect of your job?”

Setting up scenarios offers a better way to the same insights. Providing a scenario goes a little like this:

We’re testing the process of booking a hotel. Imagine you’ve decided to book a weekend getaway to New York City and you want to find a time with a good deal (you’re flexible on the dates). You’ve already navigated to the home page of the website. Show me what you would do next.

As the facilitator, you provide the end goal, but you don’t tell them how to get there. The scenario keeps the workflow, the questions, and the observations focused. As the user walks through the scenario, you can capture what works well and where they’re getting hung up. Then you can discuss it in detail after they complete the task.

4. Let your expert prioritize.

I’ve been on projects where it is nearly impossible to decide which functions should be in the menu bar. They’re all important, but they won’t fit. Instead of trying to make those decisions alone, why not ask the product’s users?

Card sorting is a great method for this. You can ask your experts to arrange all of the options from most to least important, or you can provide a plethora of features and ask them to pick which ones belong in a limited number of slots. I recommend doing this activity with at least half a dozen people (possibly more) so that some patterns emerge.

I like to end these sessions by restating my goal or purpose for the interview and asking, “Is there anything else I didn’t cover?”

This is a broad question–the one that could open Pandora’s box. Here’s why I still ask it: I’ve just facilitated a very focused interview. I could have missed something really important by designing such a narrow user testing plan. This question allows the expert to show me anything I’ve missed that is critical to them.

Secondly, if users have any questions about the process, the product, or what to expect, this is a great time for them to ask. I may not have the answers, and I’ll tell them as much. But just knowing their questions or concerns is helpful to our team as we communicate expectations and begin promoting the new launch or next release.