Читать книгу Validating Product Ideas - Tomer Sharon - Страница 17

“What do people need?” is a very broad question that probably has an infinite number of answers. To make sure that your study results are effective and useful, first define a very clear scope for your inquiry. You probably have a domain in mind such as grocery shopping, photography, or enterprise sales. This means you will be focusing on this area during the study.

Try to get even more specific. What is it about grocery shopping that you want to learn? What would help you make a decision about your photography-related app? What aspect of enterprise sales are you interested in? Be as specific as possible when you decide how to best use research. For example, maybe your study scope involves challenges in remembering what to buy at the grocery store, or opportunities for saving time spent on uploading photos from a camera to a desktop, or ways in which salespeople prepare for a sales pitch to a potential customer. After you define the scope, it’s time to work on the question you will ask your study participants.

The specific question you ask experience sampling study participants must be carefully phrased and tested. As you phrase the question (or come up with a few alternatives), bounce it off other team members early on and run a pre-launch pilot before releasing it to the whole sample group. The most important thing is that the question has to speak to participants’ behavior that repeats itself. Remember, you are going to ask that question over and over, and you will expect different answers (both across the participant population and from the same individuals at different times) that provide insights into people’s needs, pain points, and delights.

Here are some good examples of experience sampling questions that target repeated behaviors:

• What was the reason you recently used a piece of paper to write something down?

• What was the reason you recently updated your website?

• What did you want to know recently?

• What was the reason for the last phone call you initiated?

• What were you frustrated about recently when you went grocery shopping?

The experience sampling question must also be specific; otherwise, you are destined to get a lot of irrelevant answers. For example, if you ask, “The last time you went to a shoe store, what did you do?”—that’s too general and vague. People will tell you they met a friend and talked about their recent spring break vacation or called their mother. If you are interested in uncovering needs related to buying shoes, be specific about it. For example, ask, “The last time you went to a shoe store, what frustrated you the most about buying shoes?”

The following are some questions to ask about behavior that is probably not so repetitive:

• What annoyed you the most the last time you moved to a new house or apartment?

• What was the primary reason for choosing your most recent vacation destination?

• Which laptop computer did you buy recently?

While the above questions are legitimate, good questions to ask, they are not recommended for experience sampling because the frequency of the behavior doesn’t make sense for this research format. If you are looking to uncover needs related to behaviors that are not repeated very frequently, consider interviewing (Chapter 2), observation (Chapter 3), or a diary study (Chapter 4).

Make sure that the question you ask helps you figure out what users need. There are several types of questions you should avoid in an experience sampling study because they will never help you uncover needs:

1 Questions about opinions: Asking for someone’s opinion about something several times a day or week is useless. Opinions don’t change five times a day or week, and there’s no point in asking for them that much. Examples might include:• What do you think about hiring a Web developer for updating your website?• Should links be blue or black?

2 Questions that speak to “average” behavior: A common trap people who phrase questions for experience sampling fall into is asking them in a way that vaguely refers to a time frame about which the question is asked. For example, when you ask, “What frustrates you most about boarding a train?” you are setting yourself up for failure. Let’s assume the person who tries to answer this question has boarded trains 300 times during her lifetime, 50 of them in the past year, one last week. This person will probably not remember all of the frustrations about boarding a train and will try to come up with an answer that “averages” the ones she does remember (maybe the last five). She will also try to satisfy you with the answer and give you a real interesting one with a nice anecdote. The answer will probably not represent many real frustrations very well. The best way to avoid this trap is to ask about the last time the behavior happened. This way, her memory is still fresh, and it is less likely the participant will pick and choose an answer she thinks will satisfy you. Here are some examples for questions that ask people to “average” their behavior:• What frustrates you the most when you board a plane?• How do you spend time while you wait in a long line?

3 Questions that are too general: While taking a broad approach to learning from people is usually a good thing to do, asking a general question in experience sampling is going to force you to deal with a lot of noise. If you are interested in one aspect of a topic, ask about that aspect rather than asking about the entire topic. For example, if you decided that the scope of your research was uncovering needs related to finding a parking spot, don’t ask, “What annoys you about driving in a city?” Instead, ask, “What was the most frustrating thing that happened to you the last time you were looking for a parking space?” Here are some more examples of questions that are too general:• What frustrated you recently?• How did you decide which smartphone to purchase?• What is email good for?

4 Yes/no questions: Experience sampling is a research method that integrates qualitative, rich data with quantitative, numerical data. Asking a yes/no question eliminates the qualitative aspect of the study to a point where you’ll have nothing actionable to do when you see the results. If you ask, “Did you update your website this morning?” and 78% of the answers are yes, then what are you going to do next? What did you learn about user needs? Which pain points did you uncover? Here are some more bad examples:• Did you buy milk today?• Was your bus late this morning?• Do you like your boss?

5 Quantitative questions: Similar to yes/no questions, quantitative ones are also not going to be very helpful. A number, an average, or a percentage tells you nothing about unmet needs, missing features, painful problems, or joyful delights. It’s just a number. Here are some examples:• How many emails did you receive in the past hour?• What time did you wake up this morning?• How many items did you purchase the last time you went grocery shopping?

As mentioned earlier, phrasing an experience sampling question is a critical factor in the success of your research. Before you launch your experience sampling study, refer to the following imaginary scenario and then read sample questions and explanations about why these questions are right or wrong in Table 1.1.

Scenario: Imagine that Stop & Shop, a grocery shopping retailer in the northeastern U.S., came to you with this challenge: they want to identify ways to improve their customers’ in-store grocery shopping experience with technology. In other words, they want to uncover user needs, or answer the infamous question “What do people need?”

To sum it up, after you phrase your question, make sure that it:

• Asks about repeated behavior.

• Does not ask about opinions.

• Does not ask to “average” a behavior.

• Is not too general but very specific.

• Is not a yes/no question.

• Is not a quantitative question in which the answer is a number.

TABLE 1.1 SAMPLE EXPERIENCE SAMPLING QUESTIONS AND EXPLANATIONS

Question	Good?	Why?	Better Question
1	How did you create your shopping list the last time you went to the grocery store?	Yes	Asks about repeated behavior and is open-ended enough for getting valuable responses.	-
2	What is the reason you use your mobile phone while grocery shopping?	No	Phrased as if it is asked once, not repeatedly. Also, it does not meet a most likely situation where a phone is used multiple times for different purposes during grocery shopping.	The last time you went grocery shopping, what was the reason you used your phone to help you shop?
3	How often did you check the time when you recently went grocery shopping?	No	Asks for a number. Numbers will not help in identifying needs. Also, people would be making a guess because they are not aware of the number of times they use their phone. It might seem to you that asking this type of question is helpful for validating a need (e.g., if it happened zero times, it’s not a real need; if it happened many times, it is a need), but you will need to make a big leap from a number to a need. Numbers will never tell you why something is happening.
4	Would you prefer to do your grocery shopping online to save time?	No	Asks for an opinion rather than a behavior. Also, this is a Yes/No question, which is useless for identifying needs.	What was the reason you recently grocery shopped online?
5	What did you do when you were last in the grocery store?	No	Too vague and general. People do many things while grocery shopping. Pick one thing and ask about it.	What was the biggest challenge for you the last time you went to the grocery store?
6	What part of the shopping experience is most difficult for you?	No	While asking about repeated behavior, this question is asking respondents to average their recent store visits.	What was most difficult for you the last time you went to the grocery store?
7	How do you decide what to buy?	No	Too general, asks respondents to average multiple experiences.	What was the most important consideration for deciding what to buy the last time you went to the grocery store?
8	What did you buy today?	No	Not a question for experience sampling. Can be answered by looking at store purchasing logs and cashier data.
9	How often do you come to Stop & Shop?	No	Asks for a number. Numbers will not help in identifying needs.
10	What would make finding items easier?	No	Asks for an opinion and predictions of the future, which are not helpful in identifying human needs.