David Brooks, in his Times column today, looks at the rise of “data-ism” — the rapidly spreading belief “that everything that can be measured should be measured; that data is a transparent and reliable lens that allows us to filter out emotionalism and ideology; that data will help us do remarkable things.” Brooks is wary of the worship of number-crunching. He worries, wisely, that as our stores of digital data swell we’ll “get carried away in our desire to reduce everything to the quantifiable.” But he grants that there are some obvious benefits to statistical analysis. For one thing, “it’s really good at exposing when our intuitive view of reality is wrong.”

As his prime example, he points to a perception shared by pretty much every sports fan and certainly every basketball fan: that sometimes players get in the zone and can do no wrong. They’re on fire. They’re on a tear. They’re HOT. Not true, says Brooks. The hot streak is a fiction, a figment born of a flaw in our mental makeup. Its existence was disproven, he says, in a famous paper from the 1980s:

Every person who plays basketball and nearly every person who watches it believes that players go through hot streaks, when they are in the groove, and cold streaks, when they are just not feeling it. But Thomas Gilovich, Amos Tversky and Robert Vallone found that a player who has made six consecutive foul shots has the same chance of making his seventh as if he had missed the previous six foul shots.

When a player has hit six shots in a row, we imagine that he has tapped into some elevated performance groove. In fact, it’s just random statistical noise, like having a coin flip come up tails repeatedly. Each individual shot’s success rate will still devolve back to the player’s career shooting percentage.

My own intuition howled in agony as I read this. I have, after all, watched Ray Allen go on clutch three-pointer sprees in the fourth quarter, effortlessly draining one bomb after another. Unconscious. And, sadly, I have seen the opposite: Ray Allen throwing bricks from the same spots in the same situations. But I’m no fool. I’m willing to accept the hard, spoil-sport facts. My intuition has to bow down to the stats.

Or does it? It turns out that this hot streak issue is not as clear-cut as Brooks makes it out to be. The data’s slippery.

The existence or nonexistence of the hot hand in basketball, and elsewhere, continues to be debated in statistical and economic circles, and there’s evidence to support both sides of the debate. Several studies have questioned the reliability of the Gilovich paper’s conclusions. This one, for instance, suggests that the sample size was too small, that the original researchers’ “statistical tests were of such low power that they could not have been expected to find a Hot Hand even if it were present.” And a series of recently published studies — this one, this one, this one — have found at least some evidence of a hot hand effect among basketball players. The most recent paper to call into question the Gilovich conclusions was published last year in The American Statistician. Written by Daniel Stone, a Bowdoin College economics professor, it presents evidence that “the widespread belief among players and fans in the hot hand is not necessarily a cognitive fallacy.”

After reading Brooks’s column, I sent an email to Professor Stone asking if he had any reaction to it and also asking whether the hot-hand question is in fact considered settled, as Brooks suggests. He soon wrote back. “I saw the Brooks article and cringed,” he said, “– as the answer to your question is no, it’s not settled. There is recent research showing there is a hot hand in basketball (Arkes 2010), and mine shows analysis may greatly underestimate the effect. Put those together and there could be major hot hand.” Stone did emphasize that fans often see a hot hand where none exists — “people are too quick to infer a player is hot based on limited data” — but that doesn’t mean that players don’t sometimes go on real streaks.

Stone also pointed me to a recent article he wrote with another researcher, Jeremy Arkes, that, in addition to showing how the hot hand remains a bone of contention, provides a quick explanation of why we should be cautious about accepting the received statistical wisdom. They conclude: “Our overall conclusion – based on the intuition, experience and judgment of millions of bball fans/players (that, of course, we only have a sense of), what’s been found and not found in the data (from bball and other sports), and our recent theoretical analysis—is that behavioral scientists have been too quick to conclude that there is no hot hand in bball, and in fact it’s likely that players do occasionally get hot, to varying degrees.”

After nearly 30 years of intensive analysis, the hot hand remains mysterious. Our flawed intuition may be seeing something—something real—that the data is missing. This ends up, I think, underscoring Brooks’s sense that we have to be wary about data-ism and its promises. A transparent lens can also be a warped lens.

AFTERTHOUGHT: By the way, isn’t it kind of asinine to look at the free throw line for evidence of a hot hand? Free throws are the hothouse flowers of basketball. You have to look at field goals.

Photo by Keith Allison.