The Founder’s Guide to Actually Understanding Users

When building any technology product, one of the most common pieces of advice is “talk to your users.”

But the default way most of us talk to customers and prospects is unscientific and fraught with confirmation bias, putting us in danger of being lied to and wasting months building something nobody wants.

I learned this truth the hard way over the past decade founding multiple companies — but it wasn’t until I was working on my third startup that I came to understand a better way to actually understand users. I'll share how a founder can lead an effective customer discovery process, resulting in a product that your users want.

“The greatest teacher, failure is”

When I first started building software 10 years ago, I never anticipated how applicable Yoda’s wisdom would be as a founder.

10 years ago I was a complete startup newb. I had an idea, turned it into a pretty UI, and then hired developers to build the actual product. After nearly a year and tens of thousands of dollars, we launched.

Ghost town. Crickets. Nobody wanted what we’d built.

I determined that the missing piece of the puzzle was my lack of technical ability, so I enrolled as one of the first dozen students at a now-famous coding bootcamp and actually got a job as a software engineer at a real company.

So surely when I started my next start-up, this time things would be different. This time we talked to dozens of potential users and industry experts before we built anything. This time it worked. Sort of.

Our mission was compelling as we launched to fanfare and raised $11.5M within 10 months of founding the company. However, after just two years we faced challenges scaling the business, were acqui-hired, and the product was shut down.

With hindsight, I could look back to my original research notes and see I had ignored several fatal warnings. I listened what they said — exactly as they said it, but I did not realize until much later I failed to actually understand what they meant.

So if I wanted to avoid failing a third time, I needed to figure out what I was missing about how to really understand users.

Why Customer Discovery is Important?

Marty Cagan, Silicon Valley Product Group founder and former PM at early eBay, says there are “two inconvenient truths about product.”

Truth #1: At least half of our ideas are just not going to work:

Truth #2: Even the good ideas take several iterations to become viable.

My experience has also been that there’s simply no escaping these inconvenient truths — I only wish I would have learned about them sooner.

It doesn’t matter how smart or experienced we may be, statistically speaking, most of our ideas are simply not going to work. And the successful ones take time and hard work to turn into a real product that gets widely adopted by a market.

Your ideas are not nearly as important as your process — and the best process starts with understanding what the customers you wish to serve already do to solve their problems today and even more importantly, understanding why.

Dialogue before development

Yet, even as a 3rd time founder, I fell into the trap of ignoring the two inconvenient truths again.

Confirmation bias is a hell of a drug.

When we started Grain two years ago, we began with a specific product solution in mind, built prototypes, and got feedback from users. They told us they’d love to use it but after months turning prototypes into a product, few actually did. So we started over from scratch, but this time with a different approach!

Our Customer Discovery Process

Our customer discovery process started with two clear goals:

1. Focus on a very specific user type with a very specific job to be done.

2. Interview dozens of them only to understand how and why they solve their problem today.

We weren't trying to validate whether the merit of a specific solution but to observe existing customer behaviors and desires as a means of generating new ideas for potential product solutions.

This is what is known as generative research.

Step one: generative user research

As you listen to your target market describe what they do today to solve their problems, you can better understand potential customers’ existing incentives, behaviors, and desires in anticipation for how they’d react to a new solution.

Rob Fitzpatrick has famously coined this generative research phase “The Mom Test,” which is a set of simple rules to ask good questions so that even your Mom can’t lie to you in her answers to protect your ego.

Generative research questions are focused on understanding existing behavior. For example, here are some questions from an interview guide we used at Grain to understand how our prospective users already document and share information from live meetings:

- What’s your current process to document and share information from a video meeting?

- How important is it that the information you document and share is accurate?

- What measures do you currently take to ensure accuracy of captured information?

- What can happen if your documentation is inaccurate?

- How often are you in conversations where you don’t need to document or share anything?

- Which types of conversations are the most important for you to document and share?

Be sure to avoid hypothetical questions about what people might do. Don’t try to validate your future product with questions that begin with “would you use this” or “what do you think about the possibility of” — that’s what we call leading the witness, and it will inevitably bias your data and waste your time building the wrong thing. Ensure you're conducting effective user interviews that get true insights. Put simply, at this stage, you simply need to observe what users are already doing, not what they might theoretically do.

I recently connected with Rob where he shared an updated model of 3 ways where Users will lie to you if you’re not careful:

1. Asking the wrong questions

2. Remembering the wrong thing

3. Making the wrong decision “justified” by what you think you heard

Rob and most other researchers suggest asking for permission from their interviewees to record these interviews and take time-annotated notes that will help them to accurately remember and codify behavioral patterns that could eventually help to define “jobs to be done” that product, engineering, and design teams can build for with confidence.

Step two: eVALUEative user testing

After gaining insights about the problems your target market faces in generative research, you may be confident enough to test out a specific product solution to see if these users would actually value it.

This is the concept behind evaluative testing.

At this early stage, you want to put an ultra-lightweight implementation of a product solution in front of your target users to see how they react. While the closer to reality your prototype is the better, it doesn’t need to be a fully functional product yet: designs on paper, prototypes, mock-ups-anything like that will work.

Your goal at this stage is to get clear qualitative signals that users:

1. understand the proposed product solution

2. express unmistakable excitement about the prospect of the product as a superior solution to the status quo

Unfortunately, all too many product teams speed through this testing or skip it all together and simply march ahead to engineering and delivery. Depending on the complexity of the market and the problem you’re trying to solve, this stage could take months or, in some cases, years.

That might sound discouraging and time-consuming, but I know this for certain: the success of your product will be directly proportional to the quality of work done in this initial customer discovery phase. It’s worth doing it, and it’s certainly worth doing it well.

If you're looking to recruit users to conduct generative research and evaluative user testing, here's the best place to start.

Step three: usability testing

Even if your team creates something that people want, if customers can’t figure out how to use it, the product is dead in the water. This is why product teams conduct usability testing throughout the build process.

The traditional approach to usability interviews is to set up a test environment, where we watch as a user navigates the product. An interviewer encourages a user to explain what they see, think, and observe. The interviewer also offers prompts for what the user might consider next if they get stuck using the product. Usability issues in the product become self-evident in most of these cases.

My friend Behzod Sirjani, has created a framework for conducting usability testing interviews where he recommends asking the participant about their:

1. Expectation (about what will happen)

2. Reaction (to what happens)

3. Reflection (on the difference between 1 and 2)

A less scientific and more agile approach to identifying lower-hanging usability issues is concierge onboarding. In concierge onboarding, someone from your team guides — via video call is best — new users through setting up the product and answers the questions in real-time. Concierge onboarding helps the team member understand the steps users are asked to take and the ways those steps directly lead to value.

In a recent Zoom call with Behzod, he told me how at Slack it was essential to turn usability interviews into video highlights of moments of user struggle to help his team form a shared understanding of the problem and gain alignment around solutions that will actually work.

Step four: continuous discovery

The best product teams never stop this work of generative and evaluative testing for new features. Even as their initial research and testing turns into a real product, they know the importance of creating a customer discovery and product delivery engine that never stops learning and growing.

It’s much more common for product teams to continually learn and discover from their existing users than it is for them to gather insights from completely unbiased non-users. But a balance between the two groups — existing and new — is ideal. New users can give you a better understanding of your initial product experience, and existing “power users” can offer you insights that come from living with a product for weeks or months.

Great product teams develop long-standing relationships of trust with their most active users. You’ll often see the people on these teams setting up recurring feedback sessions to gain insight and listen to users’ concerns and ideas. The point of these interviews is to find out what’s delightful and what’s frustrating-what’s there and working well, and what’s still missing.

In the early days of building your product, a standing weekly or biweekly check-in is common for these feedback sessions. These can shift to monthly or even quarterly as time goes on and the product gets closer to reaching market fit.

Making data driven decisions as a team

At every phase of the discovery process, you should be using what you learn to drive decisions. For inexperienced product teams, these decisions are often based on one perspective or option with little data to back it up. The members of the team also often evaluate this information separately to draw their own conclusions-and those conclusions don’t align, resulting in a poorly designed product.

For the sake of your team-the audience of your research-prioritize making your data easily shareable. For your team to ultimately present a case for or against a decision, they need to be able to access the same information as the person who conducted the interview in the first place.

It’s 2020, and with the high quality of today’s video conferencing, you can do this work from everywhere. Because it’s already a digital setting, there’s no excuse not to record it. After hundreds of times asking “Do you mind if I record this?” I’ve yet to hear any answer except for yes.

Save the audio and video, mine them for insights, and look for patterns that can be shared with your team. The easier the notes are to relate back to the relevant part of the recording, the easier it is to get multiple unbiased perspectives and recognize true behavioral patterns together.

Successful products take time

Whether we like it, the two inconvenient truths of start-ups mean that if we are to build great products that people love, we have no choice but to do the hard work of talking to our users in the right way and the even harder work to gain consensus as a product team about their actual jobs to be done.

I wanted people to agree with me, and prioritized my fear of rejection over the truth. Eventually I found a better way, and if you follow this guide I’m confident you will, too.