Читать книгу Business Experiments with R - B. D. McCullough - Страница 31

The “Big Three” criteria for being able to make causal inference are as follows:

1 When changed, also changed. If changes and doesn't change, then we cannot assert that causes (sometimes this is useful information).

2  happened before . If happens after , then cannot cause . This issue arises sometimes in marketing research, where a commercial is shown one day and sales on that same day are measured. How can we know that today's sales weren't affected by something that happened yesterday?

3 Nothing else besides changed systematically. If variables and change at the same time that changes – not every time, but often enough – then we cannot rule out the possibility that and are causing the changes in . Observational data cannot rule out this possibility. The random treatment assignment of an experiment can rule this out.

Experimentation is the art of making sure these criteria are met so that valid causal statements can be made. Much more will be said about this in the next two chapters. The problem with observational data is that at least one of three is always missing, usually the third.

You should consider performing an experiment when you have lots of items on which to experiment (i.e. “experimental units”), you have the capability to take measurements on these units and the outcomes from the experiments can be measured easily, and you have control over the treatments. In manufacturing, for example, lots of items come off the assembly line, so there is an abundance of experimental units. Measurement typically is easy: Does the item work or how well does it work? Often it is very easy to apply treatments to some units and not to others.

In digital marketing, experimental units are available in very large quantities – think website visitors. However, measurement can sometimes be problematic – what constitutes a “successful” website visit? Is success an immediate purchase or a purchase three weeks later? How can you tell the same visitor returned three weeks later? Control of treatments can be difficult. We can put an ad on a webpage that a customer visited, but how do we know he actually saw the ad? In the case of television, we can run a commercial, but how do we know who actually saw it? How do we know which of the persons who bought our product this week have seen the ad? These are some of the problems we will deal with in later chapters.