Читать книгу The Success Equation - Michael J. Mauboussin - Страница 8

CHAPTER 1

SKILL, LUCK, AND THREE EASY LESSONS

LET ME START with a story that is likely to be familiar to you.

One of the greatest computer programmers of all time grew up near Seattle, Washington. He saw an upstart company, Intel, making computers on a chip and was among the first people to see the potential of these so-called microcomputers. He dedicated himself to writing software for the new device and, by one account, “He wrote the software that set off the personal computer revolution.”¹

In the mid 1970s, he founded a company to sell software for microcomputers. In the early history of the company, “the atmosphere was zany,” and “people came to work barefoot, in shorts,” and “anyone in a suit was a visitor.”² But the company was soon highly profitable, and by 1981 its operating system had a dominant share of the market for personal computers that used Intel microprocessors.

For all of its early triumphs, the company's watershed moment came when IBM visited in the summer of 1980 to discuss an operating system for its new PC. After some negotiation, the two companies struck a deal. In August 1981, retailers offered the company's software alongside the brand new IBM PC, and the company's fate was sealed. The rest is history, as they say.

In case this story's not familiar, here's the ending. This pioneer of computer technology entered a biker bar in Monterey, California, on July 8, 1994, wearing motorcycle leathers and Harley-Davidson patches. What happened next is unclear, but he suffered a traumatic blow to the head from either a fight or a fall. He left under his own power but died three days later from the injury, complicated by his chronic alcoholism. He was fifty-two years old. He is buried in Seattle and has an etching of a floppy disk on his tombstone. His name is Gary Kildall.³

You'd be excused for thinking that the first part of the story is about Bill Gates, the multibillionaire founder of Microsoft. And it is certainly tantalizing to ask whether Gary Kildall could have been Bill Gates, who at one point was the world's richest man. But the fact is that Bill Gates made astute decisions that positioned Microsoft to prevail over Kildall's company, Digital Research, at crucial moments in the development of the PC industry.

When IBM executives first approached Microsoft about supplying an operating system for the company's new PC, Gates actually referred them to Digital Research. There are conflicting accounts of what happened at the meeting, but it's fairly clear that Kildall didn't see the significance of the IBM deal in the way that Gates did.

IBM struck a deal with Gates for a lookalike of Kildall's product, CP/M-86, that Gates had acquired. Once it was tweaked for the IBM PC, Microsoft renamed it PC-DOS and shipped it. After some wrangling by Kildall, IBM did agree to ship CP/M-86 as an alternative operating system. IBM also set the prices for the products. No operating system was included with the IBM PC, and everyone who bought a PC had to purchase an operating system. PC-DOS cost $40. CP/M-86 cost $240. Guess which won.

But IBM wasn't the direct source of Microsoft's fortune. Gates did cut a deal with IBM. But he also kept the right to license PC-DOS to other companies. When the market for IBM PC clones took off, Microsoft rocketed away from the competition and ultimately enjoyed a huge competitive advantage.

When asked how much of his success he would attribute to luck, Gates allowed that it played “an immense role.” In particular, Microsoft was launched at an ideal time: “Our timing in setting up the first software company aimed at personal computers was essential to our success,” he noted. “The timing wasn't entirely luck, but without great luck it wouldn't have happened.”⁴

Defining Skill and Luck

The first step in untangling skill and luck is to define the terms. This is not a simple task and can quickly devolve into heated philosophical debates.⁵ We can avoid those, since pragmatic definitions are all that we need to think clearly about the past, present, and future results of our actions and to improve the way we make the decisions that lead to those actions.

But first things first. Before we can speak of skill and luck, we have to settle on the specific activity we're talking about. We can analyze what athletes, executives, or investors do. We just have to be clear on what elements of performance we are considering. Next, we want to agree on the measures of performance. For athletes, it is winning games. For executives, it is developing strategies that create value. The benefit of measurement is that it allows us to assign specific values to skill and luck.

Now we can turn to definitions.

Luck

Let's start with luck.

We can probably do a bit better than the average dictionary that defines luck as “events or circumstances that work for or against an individual.”⁶ This is a good starting point, but we can be a little more specific. Luck is a chance occurrence that affects a person or a group (e.g., a sports team or a company). Luck can be good or bad. Furthermore, if it is reasonable to assume that another outcome was possible, then a certain amount of luck is involved. In this sense, luck is out of one's control and unpredictable.⁷

For example, suppose that a teacher asks her students to learn one hundred facts. One student—call him Charlie—memorizes eighty of those facts, figuring that he'll always score 80 and earn a grade of B. Charlie likes this course, but his life doesn't depend on it, as long as he doesn't get a C. In this school, if you get a C, you fail the class. So a B is good enough for Charlie. Think of his strategy as employing true skill, because luck plays no role in how Charlie will perform on the test. He either knows the answer or he doesn't. And he can predict the consequences of his efforts.

However, instead of asking the class for all hundred facts, the devious teacher writes her test by randomly selecting twenty pieces of information out of the one hundred. Charlie's entire score is now dependent on which of those twenty facts match the ones he memorized. If you look at his predicament statistically, he has about a two-thirds chance of scoring somewhere between 75 and 85 percent. A grade of 85 would be okay, a high B. But a grade of 75 would not. Making matters worse, he has about a 30 percent chance of scoring either 90 or higher or 70 or lower. Suddenly, his perfect knowledge of those eighty facts can't shield him from luck.

His performance on the test is beginning to seem like a crap shoot. He'd be fine with scoring 90, of course, but he'll be doomed if he scores 70 or lower. In theory, Charlie could score zero if the teacher, just by chance, chose only the twenty facts that he failed to memorize. He could also score 100 if she chose none of those twenty. But the probabilities of those two extremes are vanishingly small. So Charlie's skill can easily be measured as 80 percent of perfect under one set of conditions. But under a second set of conditions, his score can vary wildly. Moreover, under the second set of conditions, measuring his skill in any meaningful way based solely on his score is much more difficult.

The second set of conditions introduces the element of luck into the process. And it satisfies the definition as I've stated it so far.

 The grade affects the student.

 It is either good or bad (he scores either above or below 80).

 It is reasonable to assume that another result was possible if only the teacher had selected different questions.

Standardized tests scores, including the SAT Reasoning Test used for admission to college in the United States, reflect the influence of luck in the same way. That's why the admissions officers who assess those tests recognize that the scores are an imprecise measure of true skill.⁸ Introducing a little bit of luck into a system can make the level of genuine skill very difficult to measure.

In this case, we assumed that Charlie's skill was fixed and not subject to variation. In fact, his declarative memory was very accurate. He knew eighty facts. If he was asked for them, he could regurgitate them reliably. We introduced the element of luck by varying the number of questions the teacher chose.

Skill was fairly well fixed in Charlie's example, but luck can also arise through the normal variation in other kinds of skill. Consider a basketball player who makes 70 percent of her free-throw shots over a long season. You wouldn't expect that player to make seven out of every ten shots she takes. Rather, some nights she might make 90 percent of her free throws and other nights only 50 percent. Even if she trains constantly at improving her free throws, she'll experience the variation that arises from the workings of the neuromuscular system, which relies on a completely different system of memory from the one that allows us to recall facts. An athlete can reduce that variation in performance through practice, but removing it altogether is virtually impossible.⁹

Randomness and luck are related, but there is a useful distinction between the two. You can think of randomness as operating at the level of a system and luck operating at the level of the individual. Say you ask a hundred people to call five consecutive tosses of a fair coin. The order of how heads and tails fall will be random, and we can estimate that a handful of people will call all five tosses correctly. But if you are one among the hundred and happen to get them all right, you are lucky.

My definition suggests that it is useful to develop an attitude of equanimity toward luck. The consequences of our efforts, both good and bad, reflect an element within our control—skill—and an element outside of our control—luck. In this sense, luck is a residual: it's what is left over after you've subtracted skill from an outcome. Realizing good or bad luck says nothing about you as a person. If you've benefited from good luck, be happy about it and prepare for the day when your luck runs out. And don't feel affronted when you suffer from bad luck. Provided that you have approached the activity in the correct fashion, you want to shrug off the poor results and go about your business in the same fashion in the future.

Most people have a general sense that luck evens out over time. That may be true in the grand scheme of things. But the observation doesn't hold for any individual, and the timing of luck can have a large cumulative effect. One well-documented example is the timing of graduation from college. Students who graduate at times of relative prosperity have an easier time getting jobs and enjoy higher pay than students who graduate during a recession or depression. Lisa Kahn, an economist at the Yale School of Management, studied this effect. For white male students at the time of graduation, the unemployment rate can be used to predict a loss of earnings. For each percentage point of unemployment, the graduate will earn 6 to 7 percent less. Fifteen years later, he'll still be below par.¹⁰ The difference in what people earn is strongly influenced by whether or not the economy is weak or strong when they graduate from college. In other words, it's a matter of luck.

Making Your Own Luck

Since luck is intimately intertwined in all of our lives, it comes as no surprise that there are plenty of aphorisms that address luck:

 “You make your own luck.”

 “Luck is what happens when preparation meets opportunity.”

 “I'm a great believer in luck, and I find the harder I work, the more I have of it.”11

Preparation and hard work are essential elements of skill. They often lead to good outcomes. But the aphorisms don't really address what's happening. If you prepare and work hard, you are successful not because your luck improves. Luck doesn't change at all. Only your skill improves. And you can work hard and prepare and build the best American diner on Route 66 just when the interstate highway bypasses your town and puts you out of a job.

There's another popular argument that says you can't get lucky unless you get in luck's way. For example, you can't win the lottery unless you play. On one level, of course, this is true. But it glosses over two important points. Luck can be good or bad. While winning the lottery does seem like good luck, it's hard to say that losing the lottery is bad luck. Losing the lottery is expected. Lotteries are designed to take in more money than they dole out, so they are a loser's game in the aggregate. The main issue is that putting yourself in a position to enjoy good luck also puts you in a position to lose.

The other point is that the very effort that leads to luck is a skill. Say that you need to complete ten interviews with prospective employers to receive one job offer. Individuals who seek only five interviews may not get an offer, but those who go through all ten interviews will have an offer in hand by the end of the process. Getting an offer isn't luck, it's a matter of effort. Patience, persistence, and resilience are all elements of skill.

The best-known advocate for the idea that you can create your own luck is Richard Wiseman, a professor at the University of Hertfordshire who holds Britain's Chair in the Public Understanding of Psychology. Wiseman's investigations are offbeat and fun. For example, he conducted a “scientific search” for the world's funniest joke. (The winner: Two hunters are out in the woods when one of them collapses. He doesn't seem to be breathing and his eyes are glazed. The other guy whips out his phone and calls the emergency services. He gasps, “My friend is dead! What can I do?” The operator says, “Calm down. I can help. First, let's make sure he's dead.” There is a silence, then a shot is heard. Back on the phone, the guy says, “OK, now what?”) He also argues that he has found “a scientifically proven way to understand, control, and increase your luck.”¹²

Wiseman collected a sample of hundreds of individuals and had them rate themselves on their beliefs about luck. He then sought to explain “the different ways in which lucky and unlucky people thought and behaved” and identified the “four principles of luck.” The principles include maximizing your chance opportunities, listening to your lucky hunches, expecting good fortune, and turning bad luck into good. Wiseman's research is unfailingly lively and provocative and he comes across as an energetic and intellectually curious man. Unfortunately, good science this is not.

In one experiment, Wiseman asked people playing the U.K. National Lottery to submit a form that included information on how many tickets they intended to buy and whether they considered themselves lucky. Of the seven hundred–plus respondents, 34 percent considered themselves lucky, 26 percent unlucky, and 40 percent were neutral. Thirty-six of the respondents (about 5 percent) won money that night, split evenly between the lucky and unlucky people. Individuals lost £2.50 on average, just as you would expect according to the number of tickets purchased. Wiseman points out that this experiment shows that lucky people aren't psychic (just in case you thought they were); he also rules out any relationship between intelligence and luck.¹³ Suffice it to say that there is no way to improve your luck, because anything you do to improve a result can reasonably be considered skill.

Skill

Now let's turn to skill. The dictionary defines skill as the “ability to use one's knowledge effectively and readily in execution or performance.”¹⁴ It's hard to discuss skill in a particular activity without recognizing the role of luck. Some activities allow little luck, such as running races and playing the violin or chess. In these cases, you acquire skill through deliberate practice of physical or cognitive tasks. Other activities incorporate a large dose of luck. Examples include poker and investing. In these cases, skill is best defined as a process of making decisions. So here's the distinction between activities in which luck plays a small role and activities in which luck plays a large role: when luck has little influence, a good process will always have a good outcome. When a measure of luck is involved, a good process will have a good outcome but only over time. When skill exerts the greater influence, cause and effect are intimately connected. When luck exerts the greater influence, cause and effect are only loosely linked in the short run.

There's a quick and easy way to test whether an activity involves skill: ask whether you can lose on purpose. In games of skill, it's clear that you can lose intentionally, but when playing roulette or the lottery you can't lose on purpose. Advocates for the legalization of online poker in the United States articulated this neat test. The law considers poker as gambling, a game of luck, and ignores the role of skill. But while luck certainly does influence who wins at poker, there should be no doubt that it is also a game of skill.¹⁵

Most people attain an acceptable level of skill in day-to-day activities after about fifty hours of training and practice. Examples include driving a car, learning to type, or playing a sport with basic proficiency. The process of acquiring a skill follows three stages:¹⁶

 In the cognitive stage, you try to understand the activity and you make a lot of errors. You might imagine a golfer learning to hold the club, thinking about how to position her body for a swing, and swinging poorly at first. The cognitive stage is generally the shortest.

 Next comes the associative stage. In this stage, your performance improves noticeably and you make fewer errors that are more easily corrected. A golfer would make regular contact with the ball but might not have full command of the direction it goes or the distance it travels.

 Finally, there is the autonomous stage, where the skill becomes habitual and fluid. Now the golfer can adjust her swing to accommodate the wind or the downward slope and break of a putt.

As your learning passes through these phases, there is a change in the neural pathways that the brain employs. If you become skilled in a physical or cognitive task, your body knows what to do better than your mind, and thinking too much about what you're doing can actually lead to degradation in performance. In these activities, intuition is powerful and valuable.¹⁷

Most of us hit a plateau in our skills and are perfectly content to stay there. Once at that plateau, additional experience does not lead to improved results (as my play in a recreational hockey league attests). What distinguishes elite performers, or experts, from the rest of us is that they advance beyond their natural plateaus through deliberate practice. Unlike routine and playful performance, deliberate practice pushes people to attempt what is beyond the limits of their performance. It involves hours of concentrated and dedicated repetition. Deliberate practice also requires timely and accurate feedback, usually from a coach or teacher, in order to detect and correct errors. Deliberate practice is laborious, time-consuming, and not much fun, which is why so few people become true experts or true champions.¹⁸

In activities where luck plays a larger role, skill boils down to a process of making decisions. Unlike a piano virtuoso, who will perform at a high level every night, an investor or a businessperson who makes a good decision may suffer unwelcome consequences in the short term because of bad luck. Skill shines through only if there are a sufficient number of decisions to weed out bad luck.

Jeffrey Ma was one of the leaders of a notorious team of blackjack players from the Massachusetts Institute of Technology. To make money, the team counted cards. Their system had two crucial components. First, team members fanned out and counted cards at a number of different tables in order to determine which tables were attractive. In this initial phase, the players stuck to small stakes. They were playing solely to determine if the cards that remained in the shoe had a relatively large number of high cards. The more high cards, the greater the chance that the player will win a hand. When a player found an attractive table, a teammate would join him and place large bets in order to win as much money as possible. As described in Ben Mezrich's best-selling book, Bringing Down the House, the team could express the attractiveness of the table and how large the bets should be with mathematical precision.¹⁹

Ma and his team were acutely aware of the influence that luck could have and therefore stayed focused on their decision-making process. Indeed, Ma recounts an instance when he lost $100,000 in just two rounds over the course of ten minutes, even though he played his cards just right: “The quality of the decision can be evaluated by the logic and information I used in arriving at my decision. Over time, if one makes good, quality decisions, one will generally receive better outcomes, but it takes a large sample set to prove this.”²⁰ In other words, he has to place a lot of bets in order to win, because this game involves a lot of skill but it also involves a lot of luck.

Developing skill is hard work whether or not luck is involved. But the feedback is very different, depending on the degree to which luck plays a role. With most physical tasks, there is a high correlation between skill and results. If you work diligently at increasing your speed at typing, the number of words you can type each minute will increase and the number of errors you make will decline. With tasks that depend on luck, making proper decisions using good skill can produce poor results over the short term. To use Ma's example, whether his team won or lost was not a reliable form of feedback in assessing skill unless and until they played enough games. The lack of quality feedback wreaks psychological havoc, too, creating false doubt in skillful people who are making good decisions and creating false confidence in those who are doing well simply because they're experiencing a streak of good luck.

In considering skill, it is also important to distinguish between experience and expertise. There is an unspoken assumption that someone doing something for a long time is an expert. In activities that depend largely on skill, though, expertise comes only through deliberate practice, and very few individuals are willing to commit the time and effort to go beyond a plateau of performance that's good enough. The fact is, most of us generally don't need performance that's better than good enough. An experienced auto mechanic, plumber, or architect, for instance, is often all you need. On the other hand, deliberate practice is essential to reach the pinnacle as a musician or an athlete.

The confusion between experience and expertise is particularly acute in fields that are complex and where luck plays a big role. One of the signatures of expertise is an ability to make accurate predictions: an expert's model effectively ties cause to effect. By this measure, experts who deal with complex systems fare poorly.

Philip Tetlock, a professor of psychology at the University of Pennsylvania, has done detailed research on experts in political and economic fields and found that their predictions were not much better than algorithms that crudely extrapolated past events.²¹ The record of people forecasting the behavior of a complex system, whether it's prices in the stock market, changes in population, or the evolution of a technology, is amazingly bad. Impressive titles and years of experience don't help, because the association between cause and effect is too murky. The conditions are changing constantly, and what happened before may not provide insight into what will happen next.

Professor Gregory Northcraft, a psychologist at the University of Illinois, sums it up: “There are a lot of areas where people who have experience think they're experts, but the difference is that experts have predictive models, and people who have experience have models that aren't necessarily predictive.”²² Distinguishing between experience and expertise is critical because we all want to understand the future and are inclined to turn to seasoned professionals with good credentials to tell us what is going to happen. The value of their predictions depends largely on the mix between skill and luck in whatever activity they're discussing.

The Luck-Skill Continuum and Three Lessons

To visualize the mix of skill and luck we can draw a continuum. On the far right are activities that rely purely on skill and are not influenced by luck. Physical activities such as running or swimming races would be on this side, as would cognitive activities such as chess or checkers. On the far left are activities that depend on luck and involve no skill. These include the game of roulette or the lottery. Most of the interesting stuff in life happens between these extremes. To provide a sense of where some popular activities belong on this continuum, I have ranked professional sports leagues on the average results of their last five seasons (see figure 1-1).²³

FIGURE 1-1

Sports on the luck-skill continuum (one season based on an average of the last five seasons)

Source: Analysis by author.

Where an activity lies on the continuum has important implications for making decisions. So our initial goal is to place activities properly on the continuum between skill and luck. Naturally, there are variables that make this an elusive task. For example, the skills of athletes shift as they age, and most companies lose their competitive advantages as new technologies emerge. But having some sense of where an activity falls on the continuum is of great value. Here are some ways that untangling skill and luck can be very useful in guiding our thinking and in evaluating events.

Take Sample Size into Account

To assess past events properly, consider the relationship between where the activity is on the luck-skill continuum and the size of the sample you are measuring. One common mistake is to read more into an outcome than is justified. Howard Wainer, a distinguished research scientist for the National Board of Medical Examiners and an adjunct professor of statistics at the University of Pennsylvania, makes this point by identifying what he calls, “the most dangerous equation.” Derived by Abraham de Moivre, a renowned French mathematician, the equation states that the variation of the mean (average) is inversely proportional to the size of the sample. This says that small samples display much larger variation (measured by standard deviation) than large samples in activities that involve a large dose of luck.²⁴ You can visualize the mean and standard deviation with the bell curve, the shape that traces the distribution. The largest number of observations is close to the top of the bell, near the mean, or average. From the top of the bell, the curve slopes down the sides symmetrically with an equal number of observations on each side. Standard deviation is a measure of how far the sides of the bell curve are from the average. A skinny bell curve has a small standard deviation, and a fat bell curve has a large standard deviation.

A small number of results tell you very little about what's going on when luck dominates, because the bell curve will look fatter for the small sample than it will for the overall population. Wainer deems this the most dangerous equation because ignorance of its lessons has misled people in a wide range of fields for a long time and has had serious consequences.

Wainer offers an example to illustrate the point: the rate at which people contract cancer of the kidney in the United States. He provides a map showing that the counties in the United States with the lowest rates tend to be rural, small, and in the Midwest, South, and West. He then shows a map of the counties with the highest rates. They tend to be rural, small, and in the Midwest, South, and West. This is simply de Moivre's equation at work: if you're closer to the luck side of the luck-skill continuum, small sample sizes will exhibit large variations and will lead to unreliable conclusions. Wainer then shows the rate at which people contract cancer of the kidney as a function of the population of any given county, and it is visually clear that small counties have the highest and lowest rates of incidence of cancer while large counties have rates that are closely clustered. A small population equals a small sample and therefore a wide variation.²⁵

Failing to understand de Moivre's equation has led to some significant blunders in making policy. One example is the effort to improve the education of children. Seeking reform, policy makers proceeded in a seemingly sensible way by asking what kinds of schools had children who scored well on tests. The next step was to restructure other schools to look like the ones producing the outstanding students. As you would guess by now, small schools are substantially overrepresented among the schools that scored the highest. This led to a movement toward reducing the size of schools. In fact, the private and public sectors spent billions of dollars to implement a policy aimed at reducing the size of schools.

A closer look at the data shows that small schools were not only overrepresented among the schools that scored the highest, they were also overrepresented among the schools that scored the lowest. Further, Wainer offers evidence that, toward the end of their secondary education, students at larger schools actually score better on average than those at small schools, because larger schools have the resources to offer a richer curriculum, with teachers who can specialize in a subject.²⁶

Here's the main point: if you have an activity where the results are nearly all skill, you don't need a large sample to draw reasonable conclusions. A world-class sprinter will beat an amateur every time, and it doesn't take a long time to figure that out. But as you move left on the continuum between skill and luck, you need an ever-larger sample to understand the contributions of skill (the causal factors) and luck.²⁷ In a game of poker, a lucky amateur may beat a pro in a few hands but the pro's edge would become clear as they played more hands. If finding skill is like finding gold, the skill side of the continuum is like walking into Fort Knox: the gold is right there for you to see. The luck side of the continuum is similar to the tedious work of panning for gold in the American River in California; you have to do a lot of sifting if you want to find the nuggets of gold.

Most business executives try to improve the performance of their companies. One way to do that is to observe successful companies and do what they do. So it comes as no surprise that there are a large number of books based on studies of success. Each work has a similar formula: find companies that have been successful, identify what they did to achieve that success, and share those attributes with other companies seeking similar success. The approach is intuitively appealing, which explains why the authors of these studies have sold millions of books.

Unfortunately, this approach comes with an inherent problem. Some of the companies were lucky, which means that there are no reliable lessons to learn from their successes. Michael Raynor and Mumtaz Ahmed at Deloitte Consulting teamed up with Andrew Henderson at the University of Texas to sort out how skill and luck contribute to the way that companies perform. First, the researchers studied over twenty thousand companies from 1965–2005 to understand the patterns of performance, including what you would expect to see as the result of luck. They concluded that there were more companies that sustained superior performance than luck alone could explain.

Next, they examined the 288 companies that were featured in thirteen popular books on high performance and tested them to see how many were truly great. Of the companies they were able to categorize, they found that fewer than 25 percent could confidently be called superior performers. Raynor, Ahmed, and Henderson write, “Our results show that it is easy to be fooled by randomness, and we suspect that a number of the firms that are identified as sustained superior performers based on 5-year or 10-year windows may be random walkers rather than the possessors of exceptional resources.”²⁸

The authors of those how-to studies found success and interpreted it to create lessons that they could peddle to a credulous audience. Yet only a small percentage of the companies they identified were truly excellent. Most were simply the beneficiaries of luck. At the end of the day, the advice for management is based on little more than patterns stitched together out of chance occurrences. You have to untangle skill and luck to know what lessons you can take from history. Where skill is the dominant force, history is a useful teacher. For example, by well-established methods, you can train yourself to play music, speak a language, or compete in athletic games such as tennis and golf. Where luck is the dominant force, however, history is a poor teacher.

At the heart of making this distinction lies the issue of feedback. On the skill side of the continuum, feedback is clear and accurate, because there is a close relationship between cause and effect. Feedback on the luck side is often misleading because cause and effect are poorly correlated in the short run. Good decisions can lead to failure, and bad decisions can lead to success. Further, many of the activities that involve lots of luck have changing characteristics. The stock market is a great example. What worked in the past may not work in the future.

An understanding of where an activity is on the luck-skill continuum also allows you to estimate the likely rate of reversion to the mean. Any activity that combines skill and luck will eventually revert to the mean. This means that you should expect a result that is above or below average to be followed by one that is closer to the average. Recall Charlie, the student who knew eighty out of one hundred facts but was tested on only twenty of them. If he scored a 90 on the first test because the teacher happened to select mostly questions he could answer, you would expect the score on the second test to be closer to 80, as his good luck would be unlikely to last.²⁹

The important point is that the expected rate of reversion to the mean is a function of the relative contributions of skill and luck to a particular event. If what happens is mostly the result of skill, then reversion toward the mean is scant and slow. If you're a highly skilled NBA player making free-throw shots, your shooting percentage will stand well above the average most of the time. Sometimes your performance will move back toward the average, but not by very much. If the outcome is mostly due to luck, reversion to the mean will be pronounced and quick. If you're playing roulette and win five times, you're better off leaving the table, because you can be sure you're going lose as the number of plays increases. These concepts are important and are often overlooked in business, sports, and investing, not to mention in the casino.

Take another example from sports. Tennis is largely a game of skill. Top professional men players hit in excess of six hundred shots during a best-of-five set match, providing plenty of opportunity for skill to shine through (large sample). As a consequence, the ranking of the best tennis players tends to persist from year to year. For instance, Roger Federer, one of the greatest players of all time, spent a total of 288 weeks—longer than five years—in the number-one spot. A look at the four top-rated players at the end of 2010 reveals that they were the same as at the end of 2009, with the only difference being that the top two players swapped spots. The same four players appeared in 2011. Reversion to the mean is muted because skill exerts the most powerful influence over who wins.

Baseball is another story. Even though its professional players are extremely skillful, baseball is a sport that involves a lot of luck. A pitcher can throw well but fail to get supporting runs from his teammates and thereby lose a game. A batter can put a ball into play and a slight difference in trajectory will determine whether it's a hit or an out. Over a long, 162-game season, the best teams in baseball rarely win more than 60 percent of their games, as reversion to the mean powerfully drives the outcomes back toward the average. In sharp contrast to tennis, baseball has a lot of randomness. Only the New York Yankees were one of the top four teams in 2009, 2010, and 2011 (based on wins), and they made it by a slim margin in 2010. Because there are nine defensive players on the field at any given time, and each player's performance fluctuates, one player's skill can easily be canceled out by another's mistake, driving the whole system back toward the average. So no matter how skillful the individual players, a system like this tends to look and behave much more like a game of chance than tennis does.

Naturally, for any particular individual or organization skill will change over time. The performance of a great athlete fades with age and a company's competitive advantage eventually gets whittled away. But from period to period, a sense of the ratio of skill to luck is of great value in anticipating the rate of reversion to the mean.

Interactions Vary, but the Lessons Remain

Some of the interactions featured in this book are focused on the individual, including cognitive tasks (music), physical tasks (gymnastics), or tasks in which an individual interacts with a system (the lottery). These activities tend to have a high degree of independence, which means that whatever happens next is not influenced by what happened in the past. In those cases, the skill of the players tends to dictate the results.

Still other activities have one person or entity competing against a few others. A company launching a new product amid a handful of rivals is one example. So is a team competing in a league, or even the performance of a player on a team. In these instances, what happened in the past does influence the future, a process known as path dependence.

Finally, there are cases in which one person competes with a crowd. Examples include betting on sports and investing, where an individual pits his or her skill against the collective skill of the crowd. History shows us that crowds can be wise or whimsical.

So far, I have depicted events as if they follow distributions that are known. For example, de Moivre's equation applies to events that follow a normal, or bell-shaped, distribution but doesn't apply in cases where some events are extreme outliers. The real world is messy, and there are myriad distributions that depart from the simple bell curve, as we will see. But if we approach these activities properly, the effort of untangling skill and luck will yield insights into how to assess past events and anticipate the future.

Limits of the Methods

Nassim Taleb offers a useful way to figure out where statistical tools are likely to work and where they fail. He introduces a 2×2 matrix, where the rows distinguish between activities that can have extreme variation and those that have a narrower range of possibilities.³⁰ The narrow distributions are the ones that de Moivre's equation handles superbly. The distribution of stature is a classic example, as the ratio between the tallest and shortest human on record is only 5:1. But extreme variation is a lot more difficult to deal with. For example, the distribution of wealth has extreme outcomes. The net worth of Bill Gates, in excess of $50 billion, is more than 500,000 times more than the median net worth of all Americans.

The columns of the matrix are the payoffs, and distinguish between the simple and the complex. Binary payoffs are simple: the team wins or loses; the coin comes up heads or tails. Again, modeling these payoffs mathematically is relatively straightforward. Complex payoffs would include the casualties from a war. You may be able to predict a war, but there's no reliable way to measure its effect. Figure 1-2 summarizes the matrix.

FIGURE 1-2

Taleb's four quadrants

Source: Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable (New York: Random House, 2010), 365.

Statistical methods tend to work well in quadrants one through three, and most of what we will be dealing with falls into one of those quadrants. Dealing with quadrant four is far more difficult, and there is a natural and frequently disastrous tendency to apply naively the methods of the first three quadrants to the last. While most of our discussion will dwell on areas where statistics can be helpful, we will also discuss ways to cope with activities in the fourth quadrant.