Оглавление
B. D. McCullough. Business Experiments with R
Table of Contents
List of Tables
List of Illustrations
Guide
Pages
Business Experiments with R
Preface
Suggested Courses Using This Book
Acknowledgments
Bruce McCullough
About the Companion Website
1 Why Experiment?
1.1 Case: Life Expectancy and Newspapers
Software Details
Try it!
Try it!
Exercises
1.2 Case: Credit Card Defaults
Try it!
Software Details
1.2.1 Lurking Variables
1.2.2 Sample Selection Bias
Try it!
Exercises
1.3 Case: Salk Polio Vaccine Trials
Exercises
1.4 What Is a Business Experiment?
1.4.1 Four Steps of an Experiment
1.4.2 Big Three of Causality
1.4.3 Most Experiments Fail
Exercises
1.5 Improving Website Designs
Exercises
1.6 A Brief History of Experiments
1.7 Chapter Exercises
1.8 Learning More
Section 1.1 “Life Expectancy and Newspapers”
Section 1.2 “Case: Credit Card Defaults”
Section 1.3 “Case: Salk Polio Vaccine”
Section 1.4 “What Is a Business Experiment?”
Section 1.5 “Improving Website Design”
Section 1.6 “A Brief History of Experiments”
1.9 Statistics Refresher
2 Analyzing A/B Tests: Basics
2.1 Case: Improving Response to Sales Calls (Two‐Sample Test of Means)
2.1.1 Initial Analysis and Visualization
Software Details
2.1.2 Confidence Interval for Difference Between Means
Try it!
Software Details
Try it!
Software Details
Software Details
2.1.3 Reporting Results
2.1.4 Hypothesis Test for Comparing Means
2.1.5 Power and Sample Size for Tests of Difference of Means
Software Details
Try it!
Try it!
Software Details
2.1.6 Considering Costs
Software Details
Try it!
Exercises
2.2 Case: Email Response Test (Two‐Sample Test of Proportions)
Try it!
Software Details
2.2.1 Confidence Interval and Hypothesis Test for Comparing Two Proportions
2.2.2 Better Confidence Intervals for Comparing Two Proportions
Software Details
2.2.3 Power and Sample Size for Tests of Difference of Two Proportions
Software Details
Try it!
Exercises
2.3 Case: Comparing Landing Pages (Two‐Sample Test of Means, Again)
Try it!
Exercises
2.4 Case: Display Ad Clickthrough Rate
2.4.1 Beta‐Binomial Model
Software Details
2.4.2 Comparing Two Proportions Using the Beta‐Binomial Model
Try it!
Software Details
Exercises
2.5 Case: Hotel Ad Test
2.5.1 Tips on Presenting Experimental Findings
Exercises
2.6 Chapter Exercises
2.7 Learning More
Section 2.1 “Case: Improving Response to Sales Calls”
Section 2.2 “Case: Email Response Test”
Section 2.3 “Case: Comparing Landing Pages”
Section 2.3 “Case: Comparing Landing Pages”
Section 2.4 “Case: Display Ad Clickthrough Rate”
Section 2.5 “Case: Hotel Ad Test”
Notes
3 Designing A/B Tests with Large Samples
3.1 The Average Treatment Effect
Exercises
3.2 Internal and External Validity
3.2.1 Threats to Internal Validity
3.2.2 Threats to External Validity
Exercises
3.3 Designing Conclusive Experiments
Defining Treatments
Choosing a Test Setting
Choosing Response Measures
Selecting the Unit of Analysis
Selecting Subjects
Assigning Treatments to Units
Choosing Factor Levels
Making Sure You Will Have Enough Data
Avoiding Other Problems
Communicating the Design to Others
Exercises
3.4 The Lady Tasting Tea
Exercises
3.5 Testing a New Checkout Button
Exercises
3.6 Chapter Exercises
3.7 Learning More. Section 3.1 “The Average Treatment Effect”
Section 3.2 “Internal and External Validity”
Section 3.3 “Designing Conclusive Experiments”
Section 3.4 “The Lady Tasting Tea”
Section 3.5 “Testing a New Checkout Button”
4 Analyzing A/B Tests: Advanced Techniques
4.1 Case: Audio/Video Test Reprise (One‐Sided Tests)
4.1.1 One‐Sided Confidence Intervals
Try it!
Software Details%
4.1.2 One‐Sided Power
Try it!
Exercises
4.2 Case: Typing Test (Paired ‐Test)
Software Details%
Try it!
4.2.1 Matched Pairs
Software Details%
Software Details%
Software Details%
Exercises
4.3 A/B/n Tests
Software Details%
Software Details%
Software Details%
Exercises
4.4 Minimum Detectable Effect
Exercises
4.5 Subgroup Analysis
4.5.1 Deficiencies of Subgroup Analysis
4.5.2 Subgroup Analysis of Bank Data
Exercises
4.6 Simpson's Paradox
4.6.1 Sex Discrimination at UC Berkeley
4.6.2 Do You Want Kidney Stone Treatment A or Treatment B?
4.6.3 When the Subgroup Is Misleading
Exercises
4.7 Test and Roll
Exercises
4.8 Chapter Exercises
4.9 Learning More
Section 4.1 “Audio/Video Reprise”
Section 4.2 “Typing Test”
Section 4.3 “A/B/n Tests”
Section 4.4 “Minimum Detectable Effect”
Section 4.5 “Subgroup Analysis”
Section 4.6 “Simpson's Paradox”
Section 4.7 “Test & Roll”
4.10 Axppendix on One‐Sided CIs, Tests, and Sample Sizes
5 Designing Tests with Small Samples
5.1 Case: Call Center Scripts (ANOVA)
Try it!
5.1.1 Blocking
Software Details
Software Details
Exercises
5.2 Case: Facebook Geo‐Testing (Latin Square Design)
Try it!
5.2.1 More on Latin Square Designs
5.2.2 Latin Squares and Degrees of Freedom
Software Details
Exercises
5.3 Dealing with Covariate Imbalance
5.3.1 Matching
5.3.2 Rerandomization
5.3.3 Propensity Score
5.3.4 Optimal Matching
5.3.5 Sophisticated Matching: Selling Slushies
Exercises
5.4 Chapter Exercises
5.5 Learning More
Section 5.1 “Call Center Scripts”
Section 5.2 “Geo‐Testing”
Section 5.3 “Dealing with Covariate Imbalance”
6 Analyzing Designs via Regression
6.1 Experiments and Linear Regression
Try it!
Software Details
Try it!
Software Details
Exercises
6.2 Dummies, Effect Coding, and Orthogonality
Try it!
Exercises
6.3 Case: Loan Experiment Revisited (Interactions)
6.3.1 Interactions
Try it!
6.3.2 Loan Experiment
Software Details
Software Details
Software Details
Exercises
6.4 Case: Direct Mail (Three‐Way Interactions)
Try it!
Software Details
Try it!
Try it!
Software Details
Exercises
6.5 Pretreatment Covariates in Regression
Exercises
6.6 Chapter Exercises
6.7 Learning More. Section 6.1 “Experiments and Linear Regression”
Section 6.2 “Dummies, Effect Coding, and Orthogonality”
Section 6.3 “Interactions”
Section 6.4 “Three‐Way Interactions”
Section 6.5 “Pretreatment Covariates in Regression”
6.8 Appendix: The Covariance Matrix of the Regression Coefficients
Try it!
Software Details
7 Two‐Level Full Factorial Experiments
7.1 Case: The Postcard Example
Try it!
Try it!
Try it!
Software Details
Exercises
7.2 Case: Email Campaign
Try it!
Exercises
7.3 The Determinant of a Matrix
Software Details
Exercises
7.4 Aliasing
Software Details
Try it!
Try it!
Software Details
Exercises
7.5 Blocking (Again)
Exercises
7.6 Mee's Blunders
7.7 Chapter Exercises
7.8 Learning More
Section 7.1 “The Postcard Example”
Section 7.2 “Email Campaign”
Section 7.3 “The Determinant of a Matrix”
Section 7.4 “Aliasing”
Section 7.5 “Blocking (Again)”
7.9 Appendix on aliasMatrix and colorMap
8 Two‐Level Screening Designs
8.1 Preliminaries
Exercises
8.2 Case: Puncture Resistance (Small Screening Experiment)
Exercises
8.3 Case: College Giving (Big Screening Experiment)
Software Details
Try it!
Exercises
8.4 How to Set Up a Screening Experiment
Exercises
8.5 Creating a Screening Design
Software Details
Software Details
Software Details
Exercises
8.6 Chapter Exercises
8.7 Learning More
Section 8.1 Preliminaries
Section 8.2 “Small Screening”
Section 8.3 “The College Giving Experiment”
Section 8.4 “How to Set Up a Screening Experiment”
Section 8.5 “Creating a Screening Design”
9 Custom Design of Experiments
9.1 Case: Selling Used Cars at Auction I (Small Custom Screening)
9.1.1 Create the Design
Software Details
9.1.2 Evaluate the Design
Software Details
Software Details
Try it!
9.1.3 Use the Design
Exercises
9.2 Case: Selling Used Cars at Auction II (Custom Experiment)
Software Details
Exercises
9.3 Custom Experiment with Blocking
Software Details
Exercises
9.4 Custom Screening Experiments
Software Details
Try it!
Software Details
Exercises
9.5 More Than Two Levels
Software Details
Software Details
Software Details
Software Details
Software Details
Exercises
9.6 Chapter Exercises
9.7 Learning More
Section 9.1 “Case: Selling Used Cars at Auction I”
Section 9.2 “Selling Used Cars at Auction II”
Section 9.3 “Custom Experiment with Blocking”
Section 9.4 “Custom Large Screening Experiments”
Section 9.5 “More Than Two Levels”
10 Epilogue
10.1 The Sequential Nature of Experimentation
10.2 Approaches to Sequential Experimentation
References
Index. a
b
c
d
e
f
g
h
k
l
m
n
o
p
q
r
s
t
u
v
w
WILEY END USER LICENSE AGREEMENT
