First off, let's review the aforementioned bias. Miller and Sanjurjo, as have others, compared basketball shooters' hit rates when hot (in this case, following three straight made shots) to their hit rates following three-shot sequences other than three straight hits (when players are less hot or even cold). The authors' SSRN paper notes that distortion stems from the fact that "conditioning on a streak of three or more hits creates a selection bias in which these hits are removed from the sample, leaving a smaller fraction of hits, thus driving conditional performance on the subsequent shot below the base rate" (p. 9). Here's a concrete illustration. Using part of an example Miller shared in an e-mail, where H = hit and M = miss, the sequence [HHHMHHHM] would yield the not-so-hot result that the player was 0-for-2 on shots following three straight hits, even though the player's overall shooting (6-of-8) was very hot. Further, with a correction formula devised by Miller and Sanjuro, hot-hand effects now appear to be larger than previously thought (at least within this type of analysis).
This finding has sent statistically oriented bloggers to their keyboards with great urgency. Columbia University statistics professor Andrew Gelman headlined his July 9 posting "Hey -- guess what? There really is a hot hand!" The last I checked, Gelman's piece had received 105 comments! Then, on July 21, all-around sabermetrician Phil Birnbaum weighed in on his blog with a posting entitled "A 'hot hand' is found in the NBA three-point contest."
Though some of the commenters on these blogs have gone back and forth over the proper magnitude of the correction for the aforementioned bias and other methodological issues, I think it's easiest to take Miller and Sanjurjo's findings at face value. Table 1 of their paper is very informative, presenting results for 33 players, with and without bias-correction. The authors correctly note that, when averaging over players, those with negative results (shooting worse after a hot streak) can cancel out positive results. A simple look at the frequencies of different results therefore seems warranted, so I have summarized the results concisely from Miller and Sanjurjo's more-elaborate table. Even with the bias-correction (which enhances how streaky a player looks), here's how many players show different increases in shooting percentage conditional on making three straight shots:
Accuracy Gain
After 3 Straight
Hits ("Hotness")
|
No. of
Contestants
|
.34
|
1
|
.20 to .22
|
3
|
.14 to .18
|
4
|
.11 to .12
|
5
|
.05 to .07
|
5
|
.01 to .04
|
10
|
Negative
|
5
|
Miller and Sanjurjo's claim that some players exhibit quite appreciable streakiness is well-supported. What about the "typical" or "average" performance? The median for all players (which is unaffected by how far in a positive or negative direction the most extreme values sit) is a .05 or 5% improvement after making three straight shots (16 players above .05, 2 at .05, and 15 below it). This is indeed stronger evidence for basketball-shooting streakiness than we've seen before. For example, a Harvard study (Bocskocsky, Ezekowitz & Stein, 2014) found approximately a 2% hot-hand increase for NBA in-game shooting, using SportVU tracking-camera technology (here and here).
There are a couple of possible reasons why Miller and Sanjurjo's findings may overstate hot-hand effects. As Birnbaum notes, within the 25-shot sequence of the NBA three-point contest, there are five locations, from each of which the player attempts five straight shots. Thus, if a shooter hits his first shot from a given location, he can rely on the same motor/muscle memory in launching the next four shots.
Further, players invited to the NBA three-point-shooting contest are known to be great outside shooters, and players with high base rates of success appear more likely than those with lower base rates to go on hot streaks. Therefore, it would be interesting to see what would happen with a more representative cross-section of NBA players. (Both the motor/muscle-memory and base-rate issues are discussed in my book.)
Even if Miller and Sanjurjo's 5% median hot-hand effect is not inflated, it is still probably a smaller magnitude than most fans would associate with the term "hot hand," as the authors appear to acknowledge. The double-digit percentage-point increases some shooters exhibit after three straight hits, on the other hand, would seem to be closer to a lay characterization of a hot hand.
In addition, Miller, in comments on the Gelman blog, holds the Harvard study to a very high level of scrutiny, in my view. Arguably, too high. For example, Miller notes that it omitted some possible control variables, including "the quality and identity of the defender." However, the Harvard study did control for "Distance of Closest Defender, Angle of Closest Defender, Shooter-Defender Height Difference, and [whether the shooter was] Double Covered." Once all these facets of the defense are accounted for, I don't know how much incremental knowledge we gain from knowing the defensive-efficiency of the player guarding the shooter. I therefore take the Harvard study's 2% estimate of a hot-hand magnitude as having probative value.
In the end, I come down closer to Birnbaum's relatively skeptical view -- including his point that Miller and Sanjurjo's finding should be described as "a" hot hand, rather than "the" hot hand, because, like all studies, it is context-dependent -- than Gelman's more accepting position. Miller and Sanjurjo's hotness magnitudes for the hottest-shooting players are higher than I would have guessed. But the magnitudes for median shooters are only slightly higher than what I would have imagined.
