Читать книгу Engaged - Amy Bucher - Страница 33

Randomized control trials (RCTs) are the gold standard of outcomes testing. Realistically, most companies won’t have the resources to do RCTs on their product, but understanding how they work can help with planning more realistic, scaled-down studies that target similar sorts of understanding. RCTs are in a sense the most scientific way to see the effects of an intervention on people’s behaviors. RCTs are a way to look at both within-subjects and between-subjects changes by looking at changes in people who use a product, as well as comparing them to people who do not. They include three elements, handily summarized in their name: randomization of research participants, a control condition, and a defined trial period.

Randomization means that the people who participate in the study are randomly assigned to either use the intervention or use something else. The randomization is important because, with a large enough sample size in the study, it theoretically eliminates the explanation that it’s something about the person that leads to the results. In the real world, people often self-select into experiences or products. They pick the things they enjoy and avoid the things they don’t. But then if something really works for them, you don’t know for sure that it’s because the product is good. It might be that this person liked a crappy program enough to stick with it long enough to get results. So in an RCT, people are randomly assigned to the product they use.

Control refers to eliminating other explanations for any outcomes. People in the control condition do not use the product, but are assigned to use something else that is reasonably similar. Research has shown that just becoming more aware of a behavior can change it. To rule out the explanation that an intervention works only because people are paying attention, the control condition will also do something that increases their attention. In a sleep-coaching program I worked on, the control group in the RCT completed sleep diaries and questionnaires, while the experimental group used our program. Because both groups were spending roughly the same amount of time each week thinking about and logging their sleep, it was easier to say that improvements were because of our program and not because of the heightened attention to their sleep habits.

Finally, trial just means that the study takes place over a contained period of time. Usually, the researchers who run an RCT will recruit people to participate for a defined length of time, chosen because it’s long enough for the outcomes of interest to be detected. At the start of the study, the researchers will take baseline measurements from all participants, regardless of whether they’re in the experimental or control condition. Over the course of the study, additional data might come from the product (or the control equivalent), as well as from additional surveys or observations from the researchers. Finally, at the conclusion, the researchers redo the measurements from the baseline in order to see what changes have taken place.

TIP WORK WITH THE PROS

Designing, running, and analyzing a real RCT is a specialized skill that you don’t find in most product organizations. If you want this kind of research, consider hiring a contract research organization (CRO) or partnering with researchers from a university.

RCTs are the closest thing in the behavior change world to prove that something works. They can be expensive and time-consuming, but when they’re successful, they can set a product apart from its competition.