Читать книгу Validating Product Ideas - Tomer Sharon - Страница 20

You need to make three decisions before you start the study:

• The number of study participants

• The length of the study

• The frequency you ask the question

First, think about how many valuable data points (single responses to the question) you want to have by the end of the study. You have two bad options and one good one:

1 Too much: Gathering 20,000 answers is going to be extremely time consuming to analyze. Even 2,000 answers are a lot to handle. As you’ll see in a later step, each answer will need to be read, understood, classified, and verified. It can take 1 to 5 minutes to complete this analysis per answer. This translates to 4–20 workdays of analysis for 2,000 answers. That’s too much.

2 Too little: 20, 50, or even 200 answers is not going to give you enough information to work with. You won’t feel very confident in your data, or be sure what to do next.

3 Just about right: 500–1,000 answers is a range of answers you can work with, be confident it’s comprehensive, and handle alone or with a team of people who support the analysis. This number of data points can take 1–4 days or a couple of hours to a day of teamwork to handle.

As soon as you have a target number of answers to be collected, calculate backward. Take into account that about a third of the answers you want will be lost (see Figure 1.2) due to your participants’ inability to answer (they don’t give you all the answers they committed to give), the fact that you have duplicates (they submit several identical answers in a matter of seconds), or the number of useless answers you received (they submit answers you don’t understand). For example, let’s assume that you want 1,000 valuable responses. Assuming one-third will be lost, you need to collect 1,500 responses. If you ask the question 5 times a day for 3 days and 100 people participate in the study, you will potentially get 1,500 responses (5 times a day x 3 days x 100 participants = 1,500 potential answers).

FIGURE 1.2 Only about ⅔ of the answers gathered in an experience sampling study are useful. The chart shows useful and not so useful answers to the question “What was the reason you recently used a piece of paper to write something down?”

The number of notifications you send each day is a trade-off between the times you think the behavior you ask about is happening and a number that would annoy or overwhelm your participants to the point where they would choose not to respond. For example, if you think people take notes on a piece of paper 10 times a day, ask them about it 3–5 times a day. Obviously, 10 times is too much. Also, your estimation might be wrong and if you ask 10 times a day about something that happens only 5 times a day, your participants will feel uncomfortable.

Another example is about behaviors that occur relatively rarely. For example, if you estimate your audience updates their websites twice a week, don’t ask about them every day. Ask the question once a week for a period of 10 weeks. If you do that with 50 people, you will get 500 reasons that people update their sites. That’s a good number of data points to learn from. Table 1.2 provides additional examples for frequency of asking an experience sampling question.

TABLE 1.2 EXAMPLES FOR FREQUENCY OF ASKING AN EXPERIENCE SAMPLING QUESTION

Behavior at Stake	Assumed Behavior Frequency	Question Frequency
Using paper to write something down	10 times a day	3–5 times a day
Updating a website	Twice a week	Once a week
Grocery shopping	2–3 times a week	Once a week
Searching on Google	5 times a day	1–2 times a day
Using a smartphone	150 times a day	5–8 times a day
Boarding a plane	1–2 times a year	Not a good candidate for experience sampling. Better to apply interviewing (Chapter 2), observation (Chapter 3), or diary study (Chapter 4).