Knowing What Matters, Part I (Case)

The Pivot

Nine months ago, Andrew moved from Chicago to San Francisco to do a startup. At his last job in Chicago, he worked on a highly successful project around skills management and skills development, and after the end of the project he decided there was clearly a great product waiting to be built. The product would help HR managers at technology companies more purposefully develop the skills of their employees.

The product he released was a web application that allowed HR managers to assess the technical skills of individual employees. His early excitement ran headlong into a brick wall: after all his planning and development, the app found a tiny audience, none of which stayed engaged, let alone converted to paid plans.

Discouraged, he considered moving back to Chicago- he was a star at his last job and there’d always be a position there waiting for him. However, after talking to a few friends who had been through startups of their own, he decided a more systematic approach to the venture might work, and that it was worth trying. In the worst case, at least he’d have some practical experience with new product innovation and that could be a great resume builder for his next role.

In this new take on the venture, he’d be treating the whole thing as a hypothesis to be tested instead of a plan to be implemented. Being ready to observe things as they are didn’t mean starting without an explicit idea to be tested. His current concept for the venture was as follows:

‘For [hiring managers] who [need to evaluate technical talent], [Enable Quiz] is a [skills measurement system] that [allows for quick and easy assessment of topical understanding in key engineering topics]. Unlike [formal certifications or ad hoc questions], our product [allows HR managers to create and apply systematic assessments of technical skills].’

How could he test the idea with a maximum of velocity and a minimum of investment? First off, he needed to see if he could find a problem/job that was important to the HR manager and their company. If and only if his interviews with customer showed him he’d found an important problem/job would he starting investigating possible solutions, like a web application. The idea of finding the right problem and then moving to finding the right solution was first described by pioneering design thinker Donald Norman.


Back to finding the right problem, Andrew’s hypotheses was that recruiting is an important problem/job and that screening for skill sets is an important component problem in that general area. However, he needs to test that hypothesis to see if it’s true or false. In retrospect, the mistake he made previously was leading his interview subjects to relatively predictable answers with simple yes/no questions (see: The Win You Design).

The Plan

A big shift in Andrew’s approach for moving the venture forward is that he’s now focused learning about a particular problem as opposed to selling a particular solution. The general problem/job area he’s focused on is hiring technical talent. He knows from his prior batch of interviews (and his own domain experience) that this is something that generally involves two actors: an HR manager and a hiring manager from the functional area.

Knowing who’s involved in the general area is important, but his view is that area is too broad even for early stage discovery, so he’s decided to focus (for now) on the problem of screening technical talent. This is something that the HR manager does with input from the functional manager and Andrew’s interested in finding out more about how that works and whether there are opportunities for him there.


Andrew’s next step is to get outside the building, as they say, and talk to some representative customers to evaluate whether he’s found a problem worth solving.

How might Andrew find subjects?
What screening, if any, is necessary to make sure he’s talking to a relevant subject?
How might he interview subjects so that they stay focused on his topic of interest but aren’t led to a particular answer?
Should he interview HR managers? Functional/hiring managers? Both? If both, how should he sequence his subjects? 

Exhibit A- The Customer Discovery Interview

The basic tenets of subject behavior are that:

  1. Subjects generally want to be nice and often just want to be done with the interview, so they’re very likely to tell you what you want to hear.
  2. Because of this, if you ask them whether they’d buy your product, they’ll almost always say ‘yes’.

However, if you ask subjects factual questions or to describe specific examples, they’ll generally tell you the truth. Your best bet is to move from relatively general questions to relatively more specific questions. This helps minimize the problem of leading the subject.

For notes how how to prepare customer discovery questions, see the section ‘Drafting Discovery Questions’ in the Personas Tutorial. For a template with interview questions, see: Template Interview Guide.

Exhibit B- Focusing Discovery with Personas and Problem Scenarios/JTBD

While the focus of this case is creating an interview guide, it may help you to begin with the end in mind. A great place to organize and focus your discovery is with personas and problem scenarios/jobs-to-be-done. For more on these in the context of customer discovery, check out the Persona & Problem Hypothesis section of the Customer Discovery Handbook.