What the Luck?

The newest book by the acclaimed author of Standard Deviations takes on luck, and all the mischief the idea of luck can cause in our lives.

In Israel, pilot trainees who were praised for doing well subsequently performed worse, while trainees who were yelled at for doing poorly performed better. It is an empirical fact that highly intelligent women tend to marry men who are less intelligent. Students who get the highest scores in third grade generally get lower scores in fourth grade.

And yet, it's wrong to conclude that screaming is not more effective in pilot training, women choose men whose intelligence does not intimidate them, or schools are failing third graders. In fact, there's one reason for each of these empirical facts: Statistics. Specifically, a statical concept called Regression to the Mean.

Regression to the mean seeks to explain, with statistics, the role of luck in our day to day lives. An insufficient appreciation of luck and chance can wreak all kinds of mischief in sports, education, medicine, business, politics, and more. It can lead us to see illness when we are not sick and to see cures when treatments are worthless. Perfectly natural random variation can lead us to attach meaning to the meaningless.