In the mid-1960s, Daniel Kahneman, a future Nobel Prize winner in 2002 for Economics, lectured to a group of Israeli air force flight instructors on the application of behavior modification techniques, specifically that “rewarding positive behavior works but punishing mistakes does not.”**

To his consternation, the flight instructors were emphatic that their experience was the opposite: “I’ve screamed at people for badly executed maneuvers, and by and large the next time they improve.”

Kahneman was puzzled. He believed in the research that said “rewarding positive behavior works but punishing mistakes does not,” but at the same time the flight instructors’ experience rang true. Then he had an epiphany: While the screaming may have preceded the improvement, the screaming punishment did not cause the improvement. It was a non sequitur.

The explanation lies in a phenomenon known as “regression toward the mean.” In any series of random events an extraordinary event is most likely to be followed, purely by chance, by a more ordinary event.

Obviously, all the students had ability or they would not have been selected. Also they were likely to be improving steadily due to that ability and as a result of the training they were undergoing. An event worthy of either screaming punishment or one worthy of being selected for individual praise can thus be defined as an extraordinary event, and by its nature as an extraordinary event it likely will be followed by a reversion to the mean—an ordinary event. In the case of a bad mess-up, the screaming will have seemed to work because the student reverts to his or her norm, fairly competent and improving. In the case of praise, the student’s extraordinary display of flying ability most often reverts to the norm, fairly competent and improving. Thus the praise seems not to have worked in the sense that more displays of extraordinary flying ability do not follow.

I love this fascinating example of an easy-to-make error in intuition, a common mistaken conclusion from observed data, because it leads me to question myself, to motivate myself to look deeper for explanations.

————
** This story is drawn from pages 7 and 8 of Leonard Mlodinow’s “Drunkard’s Walk: How Randomness Rules Our Lives” and contains direct quotes.

This is a classic from the NSC Blog archive, originally posted July 15, 2008.