Part 1 (1/2)

NUMBERS RULE YOUR WORLD.

THE HIDDEN INFLUENCE OF PROBABILITY AND STATISTICS ON EVERYTHING YOU DO.

by KAISER FUNG.

Acknowledgments.

I would like to acknowledge the guidance and a.s.sistance of Grace Freedson, Michele Paige, Micah Burch, Kate Johnson, Steven Tuntono, Beth McFadden, Talbot Katz, and my editors, John Aherne and Joseph Berkowitz. My two sisters and brother made invaluable contributions as my most plain-spoken critics.

In addition, throughout this project, I was inspired by fans of my Junk Charts blog, .

Introduction.

This is not another book about ”d.a.m.ned lies and statistics.” That evergreen topic has inspired masterworks from Darrell Huff, John Allen Paulos, Ed Tufte, and Howard Wainer, among others. From the manipulative politician to the blundering a.n.a.lyst, from the amateur economist to the hard-selling advertiser, we have endless examples of what can go wrong when numbers are misused. Cherry-picking, oversimplifying, obfuscating-we have seen them all. This book takes a different direction, a positive position: I am interested in what happens when things go right, which is to say, what happens when numbers don't don't lie. lie.

The More We Know We Don't Know What will we learn from Bernie Madoff, the New Yorkbased fund managerswindler who impoverished an exclusive club of well-to-do patrons over three decades until he confessed in 2008? Or from the Enron executives whose make-believe accounting wiped out the retirement savings of thousands of employees? Perhaps we ought to know why the reams of financial data, printed statements, and official filings yielded few clues to the investors, auditors, and regulators who fell for the deception.

What will we learn from the Vioxx debacle in which the Food and Drug Administration conceded, five years after blessing its initial release, that the drug had caused ten thousand heart attacks? Perhaps we ought to know why widely available health and medical information and greater scale and sophistication of clinical trials did not spare Vioxx inventor Merck, doctors, or patients from overlooking the deadly side effects.

We ought also to ask why, despite having access to torrents of stock data and company reports, most of us have not made a killing in the stock market. Despite tallying up the nutritional information of every can and every packet of food, most of us have not achieved the hoped-for bodily downsizing. Despite heavy investment in information technology, flight delays and traffic jams continue to get worse. Despite detailed records of our shopping behavior, many companies have but the slightest clue when we call their service centers. Despite failing to arrest cancer in patients during large-scale clinical trials, beta-carotene and vitamin pills keep flying off the pharmacy shelves.

These examples reveal the unpleasant surprise that the modern obsession with measurement has made us none the wiser. We collect, store, process, and a.n.a.lyze more information than ever before-but to what end? Aristotle's wisdom has never been more relevant than it is today: the more we know, the more we know we don't know.

Stories of a Positive Nature We begin to overcome these failures by examining positive examples of how enterprising people are making sensible use of the new information to better our world. In the next five chapters, you will meet engineers who keep the traffic flowing on Minnesota highways, disease detectives who warn us about unsafe foods, actuaries who calculate how much Floridians must pay to insure homes against hurricanes, educators who strive to make standardized tests like the SAT fair, lab technicians who scrutinize blood samples from elite athletes, data miners who think they can detect our lies, lottery operators who face evidence of fraud, Walt Disney scientists who devise ever-clever ways to shorten queues, mathematicians whose ideas have set off the explosion of consumer credit, and researchers who offer the best tips for air travel.

These ten portraits feature some special men and women whose work is rarely celebrated openly. The reason for this neglect is that their achievement is not of invention, for which we shower awards and accolades, but of adaptation, of refinement, of salesmans.h.i.+p, and of perseverance. Their expertise is applied science.

The Statistical Way of Thinking For me, these ten stories ultimately merge into one: all of these exemplary scientists rely on the statistical way of thinking, as distinct from everyday thinking. I organize the stories into five pairs, each dealing with an essential statistical principle.

What is so unconventional about the statistical way of thinking?

First, statisticians do not care much for the popular concept of the statistical average; instead, they fixate on any deviation from the average. They worry about how large these variations are, how frequently they occur, and why they exist. In Chapter 1 Chapter 1, the experts studying waiting lines explain why we should worry more about the variability of waiting time than about its average. Highway engineers in Minnesota tell us why their favorite tactic to reduce congestion is a technology that forces commuters to wait more, while Disney engineers make the case that the most effective tool to reduce wait times does not actually reduce average wait times.

Second, variability does not need to be explained by reasonable causes, despite our natural desire for a rational explanation of everything; statisticians are frequently just as happy to pore over patterns of correlation. In Chapter 2 Chapter 2, we compare and contrast these two modes of statistical modeling by trailing disease detectives on the hunt for tainted spinach (causal models) and by prying open the black box that produces credit scores (correlational models). Surprisingly, these pract.i.tioners freely admit that their models are ”wrong” in the sense that they do not perfectly describe the world around us; we explore how they justify what they do.

Third, statisticians are constantly looking out for missed nuances: a statistical average for all groups may well hide vital differences that exist between these groups. Ignoring group differences when they are present frequently portends inequitable treatment. The typical way of defining groups, such as by race, gender, or income, is often found wanting. In Chapter 3 Chapter 3, we evaluate the mixed consequences that occur when the insurance industry adjusts prices to reflect the difference in the amount of exposure to hurricanes between coastal and inland properties, as well as what happens when designers of standardized tests attempt to eliminate the gap in performance between black and white students.

Fourth, decisions based on statistics can be calibrated to strike a balance between two types of errors. Predictably, decision makers have an incentive to focus exclusively on minimizing any mistake that could bring about public humiliation, but statisticians point out that because of this bias, their decisions will aggravate other errors, which are unnoticed but serious. We use this framework in Chapter 4 Chapter 4 to explain why automated data-mining technologies cannot identify terrorist plots without inflicting unacceptable collateral damage, and why the steroid-testing laboratories are ineffective at catching most of the cheating athletes. to explain why automated data-mining technologies cannot identify terrorist plots without inflicting unacceptable collateral damage, and why the steroid-testing laboratories are ineffective at catching most of the cheating athletes.

Finally, statisticians follow a specific protocol known as statistical testing when deciding whether the evidence fits the crime, so to speak. Unlike some of us, they don't believe in miracles. In other words, if the most unusual coincidence must be contrived to explain the inexplicable, they prefer leaving the crime unsolved. In Chapter 5 Chapter 5, we see how this powerful tool was used to uncover extensive fraud in a Canadian state lottery and to dispel myths behind the fear of flying.

These five principles are central to statistical thinking. After reading this book, you too can use them to make better decisions.

The Applied Scientist at Work These stories take a shape that reflects my own experience as a pract.i.tioner of business statistics. They bring out aspects of the applied scientist's work that differ substantively from that of the pure or theoretical scientist.

All the examples involve decisions that affect our lives in one way or another, whether through public policies, business strategies, or personal choices. Whereas the pure scientist is chiefly concerned with ”what's new,” applied work must deal with ”how high,” as in ”how high would profits go?” or ”how high would the polls go?” In addition to purely technical yardsticks, applied scientists have goals that are societal, as with the Minnesota highway engineers; or psychological, as with the Disney queue managers; or financial, as with hurricane insurers and loan officers.

The pursuit of pure science is rarely limited by time; as an extreme example, mathematician Andrew Wiles meticulously constructed his proof of Fermat's last theorem over seven years. Such luxury is not afforded the applied scientist, who must deliver a best effort within a finite time limit, typically in the order of weeks or months. External factors, even the life cycle of green produce or the pipeline of drug innovations, may dictate the constraint on time. What use would it be to discover the cause of an E. coli E. coli outbreak the day after the outbreak dies down? What is the point of developing a test for a designer steroid after dozens of athletes have already gained unfair advantage from using it? outbreak the day after the outbreak dies down? What is the point of developing a test for a designer steroid after dozens of athletes have already gained unfair advantage from using it?

Some of the most elegant achievements in pure science result from judiciously choosing a set of simplifying a.s.sumptions; the applied scientist adapts these results to the real world by noticing and then coping with inconvenient details. If you have read the writings of Na.s.sim Taleb, you will recognize the bell curve as one such simplification that demands refinement in certain situations. Another example, considered in Chapter 3 Chapter 3, is lumping together distinct groups of people when they should be treated differently.

Successful applied scientists develop a feel for the decision-making process: they know the key influencers, they grasp their individual ways of thinking, they comprehend their motivations, and they antic.i.p.ate sources of conflict. Crucially, they repackage their logic-laced messages to impress their ideas upon those who are more comfortable with intuition or emotion than with evidence. Because understanding the context is so valuable to the applied scientist's work, I have included a wealth of details in all of the stories.

To sum up, applied science has measures of success distinct from those used in theoretical science. For instance, Google recognized this distinction by rolling out its famous ”20 percent” time policy, which allows its engineers to split their week between pure science projects of their choosing and applied projects (with an 80 percent emphasis on the latter!).

More And there is something extra for those who want more. The Conclusion of this book serves a dual purpose of consolidation and extension. While summarizing the statistical way of thinking, I introduce the relevant technical language in case you should want to cross-reference a more conventional book. To ill.u.s.trate how universal these statistical principles are, I revisit each concept in a new light, harnessing a different story from the one originally selected. Finally, the Notes section contains further remarks as well as my main sources. A complete bibliography is available at the book's link on my website, .

Numbers already rule your world. And you must not be in the dark about this fact. See how some applied scientists use statistical thinking to make our lives better. You will be amazed how you can use numbers to make everyday decisions in your own life.

1.

Fast Pa.s.ses / Slow Merges The Discontent of Being Averaged Meter mystery If no one likes, why obey?

One car per green, please -HAIKU ABOUT THE M MINNEAPOLISST. P PAUL COMMUTE BY READER OF THE R ROADGUY BLOG.

Heimlich's Chew Chew Train Good film, big buildup, nice queue Twenty-second ride -HAIKU ABOUT D DISNEY BY A ANONYMOUS In early 2008, James Fallows, longtime correspondent at The Atlantic The Atlantic, published an eye-popping piece about America's runaway trade deficit with China. Fallows explained how the Chinese people were propping up Americans' standard of living. The highbrow journal has rarely created buzz on the Internet, but this article beat the odds, thanks to Netizens who sc.r.a.pped Fallows's original t.i.tle (”The $1.4 Trillion Question”) and renamed the article ”Average American Owes Average Chinese $4,000.” In three months, Internet readers rewarded the piece with more than 1,600 ”diggs,” or positive responses, which is the high-tech way of singing praise. Evidently, the new headline caught fire. Our brains cannot comfortably process astronomical numbers such as $1.4 trillion, but we can handle $4,000 per person with ease. Simply put, we like large numbers averaged averaged.

The statistical average is the greatest invention to have eluded popular acclaim. Everything has been averaged by someone, somewhere. We average people (”average Joe”) and animals (”the average bear”). Inanimate things are averaged: to wit, after the terrorist attacks of September 11, 2001, a security dispatch demonstrated how to ”weaponize the average water cooler.” Economic processes are averaged, as when a market observer in early 2008 proclaimed ”the new hope: an average recession,” presumably predicting a shallow one that would depart with haste. Even actions cannot escape: when Barack Obama's lawyer interjected on a Clinton conference call during the heated Democratic primary elections of 2008, the media labeled the occasion ”not your average conference call.”

Can rare items be averaged? You bet. Forbes Forbes magazine told us, ”The average billionaire [in 2007] is 62 years old.” Surely no one averages uncountable things, you think. Not so quick; the U.S. Census Bureau has devised a methodology for averaging time: on an ”average day” in 2006, U.S. residents slept 8.6 hours, worked 3.8 hours, and spent 5.1 hours doing leisure and sporting activities. It is a near impossibility to find something that has not been averaged. So pervasive is the idea that we a.s.sume it to be inborn and not learned, nor in need of inventing. magazine told us, ”The average billionaire [in 2007] is 62 years old.” Surely no one averages uncountable things, you think. Not so quick; the U.S. Census Bureau has devised a methodology for averaging time: on an ”average day” in 2006, U.S. residents slept 8.6 hours, worked 3.8 hours, and spent 5.1 hours doing leisure and sporting activities. It is a near impossibility to find something that has not been averaged. So pervasive is the idea that we a.s.sume it to be inborn and not learned, nor in need of inventing.

Now picture a world without averages. Imagine having the average child, the average bear, and the average such-and-so-forth punched out of our lexicon. We are dumbfounded to learn that such a world did exist once, before a Belgian statistician, Adolphe Quetelet, invented the ”average man” (l'homme moyen) in 1831. Who would have thought: such a commonplace idea is younger than the U.S. Const.i.tution!

Before Quetelet, no one had entertained the import of statistical thinking to the social sciences. Up until that time, statistics and probability fascinated only the astronomers who decoded celestial phenomena and the mathematicians who a.n.a.lyzed gambling games. Quetelet himself was first a distinguished astronomer, the founding director of the Brussels Observatory. It was in midlife that he set the ambitious agenda to appropriate scientific techniques to examine the social milieu. He placed the average man at the center of the subject he named ”social physics.” While the actual methods of a.n.a.lysis used by Quetelet would strike modern eyes as hardly impressive, historians have, at long last, recognized his impact on the instruments of social science research as nothing short of revolutionary. In particular, his inquiry into what made an able army conscript earned the admiration of Florence Nightingale (it is little known that the famous nurse was a superb statistician who became an honorary member of the American Statistical a.s.sociation in 1874). In this body of work also lay the origin of the body ma.s.s index (BMI), sometimes called the Quetelet index, still used by doctors today to diagnose overweight and underweight conditions.

Since the concept of the average man has been so firmly ingrained into our consciousness, we sometimes fail to appreciate how revolutionary Quetelet really was. The average man was literally an invention, for the average anything did not, and does not, physically exist. We can describe it, but we cannot place it. We know it but have never met it. Where does one find the ”average Joe”? Which ”average bear” can Yogi Bear outsmart? Which call is the ”average” conference call? Which day is the ”average” day?

Yet this monumental invention constantly tempts us to confuse the imaginary with the real. Thus, when Fallows calculated an average of $4,000 debt to China per American, he implicitly placed all Americans on equal footing, spreading $1.4 trillion evenly among the population, replacing 300 million individuals with 300 million clones of the imaginary average Joe. (Incidentally, the Netizens mistakenly fabricated only 300 million Chinese clones, rhetorically wiping out three-quarters of China's 1.3 billion people. The correct math should have found the average Chinese lending $1,000 to America.) Averaging stamps out diversity, reducing anything to its simplest terms. In so doing, we run the risk of oversimplifying, of forgetting the variations around the average.

Hitching one's attention to these variations rather than the average is a sure sign of maturity in statistical thinking. One can, in fact, define define statistics as the study of the nature of variability. How much do things change? How large are these variations? What causes them? Quetelet was one of the first to pursue such themes. His average man was not one individual but many; his goal, to contrast different types of average individuals. For him, computing averages was a means of measuring diversity; averaging was never intended to be the end itself. The BMI (Quetelet index), for good measure, serves to identify individuals who are statistics as the study of the nature of variability. How much do things change? How large are these variations? What causes them? Quetelet was one of the first to pursue such themes. His average man was not one individual but many; his goal, to contrast different types of average individuals. For him, computing averages was a means of measuring diversity; averaging was never intended to be the end itself. The BMI (Quetelet index), for good measure, serves to identify individuals who are not not average, and for that, one must first decide what the average is. average, and for that, one must first decide what the average is.

To this day, statisticians have followed Quetelet's lead, and in this chapter, we shall explore how some of them use statistical thinking to battle two great inconveniences in modern living: the hour-long commute to and from work and the hour-long wait to get on a theme park ride. A reasonable person, when trapped in traffic or stuck in a long queue, will suspect that whoever was in charge of planning must have fallen asleep on the job. To see why this reaction misplaces the blame, we need to know a little about the statistics of averages. Working with engineers and psychologists, statisticians are applying this knowledge to save us waiting time.

To label Dr. Edward Waller and Dr. Yvette Bendeck Disney World die-hards would be an understatement. On October 20, 2007, they toured every last open attraction in the Magic Kingdom in just under thirteen hours. That meant fifty rides, shows, parades, and live performances. Buzz Lightyear's s.p.a.ce Ranger Spin, Barnstormer at Goofy's Wiseacre Farm, Beauty and the Beast-Live on Stage, Splash Mountain, Mad Tea Party, Many Adventures of Winnie the Pooh, you name it-everything in the park! Nice work if you can manage it, no? Disney buffs know this to be a mission impossible; they feel lucky to visit four major rides on a busy day, not to mention the nonstop walking required within the hundred acres of park area. Waller and Bendeck had help from Len Testa, who devised the Ultimate Magic Kingdom Touring Plan. Testa's plan lays out precise directions for reaching every attraction in the shortest time possible. He warns unsuspecting novices that it ”sacrifices virtually all of your personal comfort.”

Len Testa is a thirty-something computer programmer from Greensboro, North Carolina. As the patron saint of disgruntled Disney theme parkgoers worldwide, he brought the gift of touring plans, which prescribe routes that guide patrons through a sequence of attractions in the shortest time possible. While the Ultimate Plan grabs attention, Testa creates touring plans for just about every need: for small kids, families, tweens, active seniors, grandparents with small children, and so on. He is mainly looking after rabid Disney fans, ones who are the most loyal-and easily the most demanding-customers. Sampling their typically breathless trip reports, posted on fan websites or relayed to journalists, one frequently comes across affectionate gripes like these: ”Going to Disneyland in the summer months is kind of like cruising to the Bahamas during hurricane season. You're just asking for it.””You haven't lived until you've stampeded to s.p.a.ce Mountain as the opening rope drops, alongside thousands of stroller-wielding soccer moms at a full run.””When those gates spring open at 8 A.M A.M., the weak and the semi-comatose will be left in the dust.””We felt we spent more time in lines than on rides-the fact is, we did! When a wait in the line is ninety minutes and the ride is only five minutes, you have to question your sanity!!””I've never really forgiven my brother for that one time he slowed us down with an untimely bathroom break at Disney's Epcot Center five years ago.”