Recruiting Unbiased Candidates for Your User Interviews

Recruiting Unbiased Candidates for Your User Interviews

Last updated: 

August 2, 2022

Monica Fidele

Monica Fidele

You might be tempted to pick the first people you can find for your user research interviews — anyone who’s available, willing, and able to talk for more than 5 minutes without getting distracted. But the data you collect during this period of customer discovery will enable product-market fit only if it comes from an unbiased source.

The right interviewees are people who will be honest, and are representative of what the market truly wants. Without those people, even a carefully planned research interview can fall apart in seconds — or collect a bunch of data that leads to bad decisions, and no product-market fit.

Here’s how you can identify and recruit unbiased users for your customer discovery interviews, and how we handled this selection, vetting, and interview process while building our own product, Grain.

Keeping a Clean Lab

Who you pick matters, but keeping a clean lab matters more.

When we started doing user interviews prior to building Grain, everybody on the team read Rob Fitzpatrick’s The Mom Test. The book basically says that if users know they’re talking to you about your company or your product they’ll lie -- at least a little bit -- to preserve your ego.

All of these people are trying to be nice, but in a user interview, they’re harming the process (and your eventual product). You need to collect clean, unbiased data to understand what product to build. If you don’t get that data, you’re likely to make decisions based solely on praise, only to go to market and fail spectacularly.

Nice interviewees won’t help you find your market. Honest ones will.

But you’re busy, and running user interviews is probably not the only thing you have to do this week. How do you choose people who aren’t going to have some level of bias?

What we found is, before you start building your product, who you choose has less of an impact on the data you collect than how you actually conduct your interviews. If you give your research candidates even an inkling of what you’re planning to build or who your target is, the data you collect is probably going to be garbage to some extent. But if they don’t have any clue what you’re doing or why you’re interviewing them in the first place, they’re much more likely to be honest and give you the data you need to make smart decisions.

We call this “keeping a clean laboratory.” Just as a scientist needs to have a clean environment to safely do experiments, your research environment should be free from your branding. It shouldn’t give clues about your product or services in any way.

How to Recruit Your User Interview Candidates

Once you’ve decided what you want to build and who your target users of this product are, you have to start your customer discovery by recruiting some interviewees.

The candidates you choose in the generative research period should reflect your target users. So if you’re planning to build a chat tool for businesses, your goal should be to talk to people already using chat tools at work. You get the idea. Interviewing people that look like your future users is the most accurate way to predict whether or not you’re planning to build something they want or need.

An important thing to remember, as you’re choosing these interviewees, is that you’re not trying to confirm that your product idea has merit. You’re doing this research and talking to all these people to confirm what the users in this target market are doing to solve their problems. In short: it’s not about you, it’s about them.

As long as you know who you need to talk to, finding interviewees isn’t that hard. Time-consuming, yes, but it’s worth it to get the data so you don’t build a product that flops. When we were doing our initial user interviews for Grain, we relied on a couple of specific channels for finding candidates:

  • — This site was an amazing place to start. Since they already have a huge database of people to tap into, we saved a ton of time on cold outreach. All we had to do was decide who we wanted to talk to (using filters like their job title or education) and then set up the interview.
  • Cold outreach — We still did some cold outreach to find additional folks in our target user group. This took the form of our core team shooting emails to people he looked up on LinkedIn to ask if they could participate in a quick interview.
  • Social media — This took place in the later stages of our research, once we had an initial prototype of Grain, but I still wanted to mention it because it has the potential to be useful. I’d occasionally respond to tweets (like this one) directly asking them to try our product. After they emailed me for beta access, I’d follow up to set up an interview. This could still work for generative research as long as you’re reaching out from a personal account and aren’t yet publicly talking about what you’re planning to build.

Here's a comprehensive guide on recruiting participants for user research interviews.

Using Surveys to Narrow Down Your Candidate Pool

Earlier I said that who you interview isn’t quite as important as keeping a sterile interview environment. But you still can’t just choose the first 50 people who show up and volunteer. There’s got to be some filtering involved in the candidate selection process.

The people you really want to talk to are the ones already doing the prerequisite actions for your theoretical product to be useful. So going back to our chat example, let’s say the product you want to build will allow incredibly fast communication between remote teams of 500 or more. So the people you need to talk to:

  • Have to be working remotely
  • Have to be working at a company with more than 500 employees
  • Have to already be using some sort of live chat tool (Slack, Skype, Mattermost, etc.) for their work communication
  • Consider that tool a critical part of their workplace communication

Knowing this criteria, you can take steps to narrow down your candidate pool. This should take the form of a survey you send to the people who have volunteered or you’ve done outreach to. Your survey should include questions specifically to narrow the field and ensure that your eventual interviewees fit the criteria you’ve established.

So in this case you’d ask questions like:

  • Do you work remotely?
  • How many employees does your company have?
  • Do you use any of these tools?
  • How important, on a scale of 1 to 10, are the above tools to your success at work?

Once you get the responses you can start cutting people who don’t fit. Anyone who doesn’t work remotely is out, as is anyone who works at a small company. People who use a chat tool but don’t consider it mission critical? They’re out too. You only want to keep the users who check all four of these boxes.

When you’ve got the users left who check those boxes, what you have is a set of users representative of your target audience. And those are the people you should set up interviews with.

Our survey asked if the people participating used video calls for user research and what tools they currently used (we were looking mainly for Zoom users to start). If the answers were good, we reached out to these people to schedule an interview. If they weren’t what we were looking for, we still reached out to thank them for their time.

And that was pretty much it. We ended up running about 100 interviews and then built an initial prototype of Grain based on the data we collected in these generative interviews. After some evaluative testing, we moved on to usability interviews that helped us narrow down the scope of the product and launch the version you can try for yourself. But that’s a story for another post.

Your Ideal Interview Candidates are Your Future Users

When you’re building a product, the biggest struggle is closing the gap between what you’re building and what the market needs. The only way you can close this gap is by getting reliable data.

As we found, the most reliable data is going to come from people who are your future users. If you run your interviews with anyone else, you run the risk of tainting your data and jeopardizing that product-market fit. Or, to put it a little more bluntly: garbage in, garbage out.

What’s next? Once you have your group of future users ready, learn how to conduct user research interviews to understand what questions to ask, how to ask them, and how to keep your your data free of bias.

Try Grain for FREE with unlimited users and recordings for 30 days

Securely record, transcribe, highlight, and share every video call.

Try Grain for Free