OT The "Hot Hand" is Real After All

Submitted by sadeto on April 15th, 2014 at 1:32 PM

This is way off-topic, especially on a day when we're probably saying goodbye to 40% of our starting basketball lineup, but it may be interesting to people like me who are both sports fans and spend way too much time trying to quantify human behavior. 

The 'hot hand" or 'hot streak' in basketball and baseball have been topics of debate forever, but for 30 years the academic consensus was that they were fallacy, that believers in such things did not understand the data. Well, new research from Stanford and Harvard - by business school faculty, interestingly - turns this assumption on its head: 

http://www.gsb.stanford.edu/news/headlines/jeffrey-zwiebel-why-hot-hand-may-be-real-after-all?utm_source=Stanford+Business+Re%3AThink&utm_campaign=7545c3ff4a-Stanford_Business_Re_Think_Issue_344_2_2014&utm_medium=email&utm_term=0_0b5214e34b-7545c3ff4a-73691613&ct=t%28Stanford_Business_Re_Think_Issue_344_2_2014%29

(The two studies cited in this article are a bit more difficult and require at least a basic understanding of statistics to follow).

In a nutshell, prior studies suffered from endogenous variable bias; e.g. a model designed to predict the probability of a player with a 'hot hand' making his next shot suffers from the relationship between a variable in the model, current or recent shooting percentage, and a variable not in the model, or lumped in the error: defensive reaction. This is much less of a problem in baseball. 

 

Comments

clown question

April 15th, 2014 at 3:01 PM ^

This comment bout sums it up "They’re studying a timescale where they’re selecting for stadium, and they’re totally mangling the treatment of park effects “We control for the stadium by calculating the success rate of all batters in that stadium over the course of the season.”

So if you have a team with good hitting and average pitching (and average schedule), that team’s home stadium will appear to be a hitter’s park by that methodology, regardless of what it actually is, and when a completely average team comes in, the home team’s pitchers will appear hot (because they’re pitching average in a supposedly hitter’s park), and will keep appearing hot as long as they’re at home, leading to a correlation when nothing is going on."

 

In otherwords, this study is COMPLETE AND UTTER BULLSHIT that will get forwarded everywhere on social media until it becomes the new norm. Even though it is wrong.

Gobgoblue

April 15th, 2014 at 1:57 PM ^

I think anyone who has played sports cannot deny it is a real thing.  If I didn't have the first hand experience I suppose I would doubt it.  

Great info.  Thanks for the post!

South Bend Wolverine

April 15th, 2014 at 2:42 PM ^

I've played sports all my life and think that the hot hand is nonsense.  So have a lot of people who adhere to more advanced stat approaches, etc.  Sure, I've had good games and bad games, but I don't attribute that to mysticism, I attribute it to practice, conditioning, tactics, adjustments, etc.  I find the "anyone who has played knows" line of reasoning presumptuous and unhelpful.

Gulogulo37

April 16th, 2014 at 12:43 AM ^

I haven't played much organized sports, but I don't really buy it. One of the things that's stuck with me the most from psychology has been how much people don't understand their own minds. People make up narratives that clearly aren't true and are convinced of it, everyone thinks they're better than the average person at everything, people ignore facts and reason (e.g., evolution), etc. You feel like you have a hot hand because up until that point...you have.

Knight

April 15th, 2014 at 2:04 PM ^

I just came across some "Hot Hand" studies while doing research for a paper. I think any athlete knows that there are definitely times when they are in the zone (or the opposite). Its the nerds that didn't play sports that deny it.

HipsterCat

April 15th, 2014 at 2:29 PM ^

As someone who is both a nerd and someone who has dabble in this "sports" you speak of I can say there have been pickup basketball games where I have hit multiple shots in a row and been as the kids put it "ON FIAH".

However the statistical impact of being "hot" is not really measurable and from a purely statistical point of view its just a small sample size creating an abnormal output. Its the same reason people dont believe in "clutch" being a thing.

When you are hot you may be more confident and you feel better but if recent success had a real bearing on your next attempt then once you are hot you would probably stay hot but eventually the gamebreaker meter runs out and you return to normal or "regress to the mean" as the nerds put it because your performance over time is a real indicator of your actual ability. So that hot streak leads to a cold streak punctuated with minor success and "Bronx cheers" from those around you when you finally do something semi decent.

Noleverine

April 15th, 2014 at 2:59 PM ^

There are certain models that support evidence of the "hot hand," in a sense. The concept of "flow" is related to this, where with a level of mastery and proper arousal levels, someone can reach "flow" or be in "the zone." As has been stated, empirical field studies are difficult. Anecdotal evidence is there but hard evidence is more difficult to obtain. "Clutch" is another topic where laboratory evidence shows some support, but in-game studies are almost impossible.

I'm on my phone or else I would post articles and go a little more in depth, but there is some evidence out there that these things exist. If people are interested in more info I can write a little more when I get out of class.

Gulogulo37

April 16th, 2014 at 12:35 AM ^

Has there actually been a lot of research on that with sports though? I usually heard about it more in terms of something like playing music, which makes more sense because it's just something in which you an let yourself go and play away continuously. It's not like someone's trying to actively stop you, unlike sports where you have more discrete events and you have to be more analytical about the situation.

MaizeAndBlueWahoo

April 15th, 2014 at 3:02 PM ^

The problem is that a purely statistical point of view is just wrong.  It just is.  There are plenty of studies that show that muscle memory and the subconscious mind are generally better at a task than the conscious mind.  If you want that illustrated, first type "example" and then try to think, with your hands over the keyboard, how many from the left is the "e" key, how many from the left is the "x" key, how many from the left is the "a" key, and so on, and in which rows also.  Are you good at typing because you can instantly say and think where every key is?

"Clutch" is a thing, perhaps one overapplied in a zeal to create a narrative, but it exists.  Some people are better than others at allowing their subconscious to do the job in front of them, like hit a baseball or sink a free throw.  Motivation is a thing too.  So is just simply a comfort level in what you're doing. 

Personal Cool Story Bro: as a very pedestrian high school swimmer, I got to my senior year without ever having qualified at the division meet, and when I got to my first swim in the prelims that year, I swam a time that I had never even come close to swimming.  I swam 50 yards on a dead sprint on three breaths of air.  Just try that sometime.  World class Olympic swimmers usually take a breath or two over 50m and they don't have to turn.  Statistical likelihood could not possibly have accounted for my swim.  Frame of mind matters and you can use it to gain a hot hand, perform in the clutch, or outdo your usual.  Anyone who claims otherwise hasn't been in a competitive enough situation.

MichiganTeacher

April 15th, 2014 at 2:18 PM ^

These studies are like academics arguing over whether some people are better at basketball than others or whether it's just statistically likely that some people make more shots than others.

 

 

swan flu

April 15th, 2014 at 2:27 PM ^

has the paper passed peer review? I am assuming it has not since the citation does not indicate this being published in a journal.

 

Not suggesting that the study is wrong, just that it is not appropriate to claim that it's valid until it passes peer review.

sadeto

April 15th, 2014 at 3:18 PM ^

A fair question, but I will point out that publication in a journal is not the only indicator of "peer review". It's the most common and very important in MOST academic fields, but not the only. That said, I don't know the answer. The Stanford paper is a business school working paper, a common model at major business schools, with varying degrees of "peer review" input and varying standards for publication. I just don't know Stanford's process.

The Harvard/MIT paper was submitted to a sponsored conference research paper competition, therefore subject to some sort of review but I couldn't tell you what. Obviously not at the level of a peer-reviwed journal. So take both with a large grain of salt, as interesting contributions to the debate that note an important flaw in prior studies but no doubt have their own flaws. 

As someone who publishes regularly in peer-reviewed journals, I can tell you that it is a process with inconsistent standards across journals for methodological rigor, bogus standards that get bent all of the time due to reviewer or editor bias, but it's the best thing we've got. Or so we keep telling ourselves. 

BiSB

April 15th, 2014 at 2:55 PM ^

Though there are some possible methodological problems, so you really can't say that this article replaces/subsumes all of the prior Hot Hand research.

clown question

April 15th, 2014 at 3:03 PM ^

appreciate the read, but:

1) stuff isn't "real", as your title states, just because someone somewhere said it was. I can probably find you a peer reviewed journal that says Elvis is alive on an island with Tupac. That doesn;t make it real.

 

2) The article has flaws which make most anyone in sabermetrics reject it.

clown question

April 15th, 2014 at 3:21 PM ^

I may have overreacted a bit, but check out the link posted earlier in the thread: http://tangotiger.com/index.php/site/comments/streaks-in-baseball#comme…

 

Not saying that there is no hot-hand effect, but that the study in question has methodological issues, and the general sabermetric community has rejected it because of those flaws.

sadeto

April 15th, 2014 at 3:34 PM ^

I accept your first point, I over-stated the impact of this research in my title, which makes me guilty of ... using a demonstrative title to attract attention to something I've written. I'll go flog myself later. 

On your second point, my impression of sabermetricians is that they are all over the place, not readily lumped together on an opinion as you do. The field seems to attract serious mathematicians and statisticians who love baseball, and baseball nerds who throw caution to the wind and wade into the muck of confounding variables that adhere to any phenomenon in the game. Show me a research paper without flaws, I'll gladly read it. 

JamieH

April 15th, 2014 at 3:07 PM ^

IMO People just misunderstand what a hot streak really is.

Every player has a maximum potential.  But they don't always perform at that maximum potential.  A shooter in basketball doesn't always shoot with a perfect stroke.  A hitter in baseball doesn't always swing purely.  All a hot-streak is is an athlete playinig at their max-level over a short period of time.  The player is just "locked-in" to thier perfect shot form or their perfect-swing.

For example, Nik Stauskas isn't a BETTER shooter when he is "hot", he's just the best shooter he can be.  When his form breaks just a bit, he is a bit "off".  All being "hot" means is that his form is good, he's focused, and he's shooting the ball as well as he possibly can.  If he manages to do that over an entire game, or over a series of games, he appears to be on a "hot" streak. 

So I guess what I'm saying is, you might be able to deny that players are "hot", but you absolutely can't deny that players slump or perform poorly over stretches.  Usually this coincides with a breakdown in the mechanics of their shot, or their swing, or something else they aren't doing right.  It could also be an injury that is dragging them down.   When they fix it, or when they heal up,  they appear to get "hot" again.

Assuming a player is a robot who shoots X% or hits a certain average and will do that everyt ime they shoot or swing is silly.  All players ebb and flow over the course of a season and are hotter or colder at different points.  You can look at the final number to gauge their overall season when you are done, but that number doesn't give you the exact reading on how they were playing on any given day.

JayMo4

April 15th, 2014 at 3:33 PM ^

This is one of the best responses to the "hot streak" arguments that I've read.  

 

I think the biggest problem is that people don't have an agreement on what "hot" means to begin with, and so we get into debates over whether "hot hands" is a thing and we're not even really in agreement as to what it is that we're arguing about.

If we're looking at things purely from a statistical standpoint, hitting a few shots in a row absolutely does not make you more likely to hit the next shot than you'd otherwise be - again, speaking from a strictly statistical standpoint.  If you force a bad shot because you think you're on fire, math is not going to bail you out and magically push that ball through the hoop.

But athletes do go through phases in which they are playing well - as you said, playing at their highest potential - maybe because they're confident, well-rested, focused, etc.  So stats aren't going to help me make that next shot.  But if I take a quality shot in terms of timing and form, I maximize my chances.  Apparent "hot" streaks are often just a series of "good" shots from guys that are playing to their capability, as opposed to forced shots, shots from a tired player, shots from someone that is stressed about having missed a few, and so on.

That is where the people that only want to look at numbers get it wrong - acting as if every shot happens in a vacuum, that it is like flipping a coin or rolling the dice.  If this were so, everyone that had ever picked up a basketball would shoot more or less the same percentage over the course of a lifetime.  That's clearly not how sports work, of course.

Nobody plays their best 100% of the time, and no one can play beyond their capability, regardless of romantic narratives we hear from announcers or sportswriters.  "Hot hands" is just a dude shooting the ball well, and all those other games are times when dude isn't playing quite as well, for whatever reason.  It isn't magic, but it's about more than just propability.

BiSB

April 15th, 2014 at 4:06 PM ^

is that no one has really ever found statistical evidence that those hot/cold streaks are predictive. In other words, if you've got a guy who's hit four shots in a row, statistics should show that his odds of making the next one are better than his overall average. But no one has ever really found that statistical link.

This paper purports to explain the previous studies that found no predictive relationship, and to detect such an effect by controlling for the alleged endogenous variable bias.

JamieH

April 15th, 2014 at 4:25 PM ^

There are too many variables.

 

So a guy hits 4 shots in a row.  A million things have just changed.

The defense now is keenly aware that this guy is killing them.  So they are going to pretty much double their efforts at stopping him. 

The player himself may be getting cocky, and may be willing to start launching up lower quality shots because he "feels good", thus lowering his percentage chance of hitting the shot.

The guy's teammates may start standing around waiting for Mr. Hot Hand to do all the scoring, thus making it even harder for him to be open for the next shots.

All of these things might contribute to that next shot NOT going in.  There are too many variables at play here. 

I'd be a lot more interested in a study that took guys in a gym shooting by themselves with no defense and measured whether they get "hot" or not, just shooting around.  You need to remove all variables other then the shooter to really find out what is going on. 

All I know is, some days an athlete feels good, and some days they don't.  And pretending that doesn't affect the athlete's performance is, IMO, rather ridiculous.   If every shot taken in basketball was the same, then why do athletes bother to have pre-game shootarounds to get ready?  It's because those first few shots don't usually feel nearly as good and you need to work the kinks out.  Same thing for guys coming right in off the bench--sometimes they have a hard time getting into the flow of the game.  Athletes aren't robots. 

BiSB

April 15th, 2014 at 5:05 PM ^

It's about whether there is any statistically significant difference in the likelihood of making the next shot.

The theory behind the article is partially predicated on the idea that players may start hoisting heat checks, or that the defense will start rotating more, etc. However, as has been pointed out, there are problems with the analysis that led to their conclusions.

JamieH

April 15th, 2014 at 6:53 PM ^

A guy is normally a 40% 3-point shooter.  He's 0-4 on the day.

Should he keep chucking, because statistically he is still a 40% 3-point shooter?  Or should he limit his shots because something is off and he's clearly not shooting well today?

I think a good shooter realizes something is not right with his 3-point shot and looks for other ways to contribute or get himself on track.  I think a bad shooter probably keeps shooting-----and probably isn't actually a 40% shooter from 3-point because he doesn't know to limit his shooting when he clearly isn't shooting well. 

notYOURmom

April 15th, 2014 at 3:15 PM ^

I mean is it driven entirely by defense? Is it possible that defensive reaction is driven by an (incorrect) belief in the hot hand? Could the hot hand in other words both be true (as a defensively driven empirically phenomenon) and not true (as a purely offensive matter)? So all the researchers are right?

Sorry not to read it in detail first but I hate reading academic journal articles from the phone

Fuzzy Dunlop

April 15th, 2014 at 3:18 PM ^

Good to know.  I am definitely a believer in advanced statistics informing our analysis of sports, yet every time I hear people claiming that statistics show there is no such thing as a "hot hand" or a "clutch performer" I have to stifle the urge to yell that they must have never played a sport in their lives, and scream at them to get off my lawn.

At the risk of coming off like a 60 year old print journalist, I do think that anyone who has ever played a sport knows that there are some days that you are "on" and some days that you are "off" -- the ball feels right in your hand and it seems like you can throw a watermelon through the basket, or everything feels discombobulated and the easiest lay up is no given.  It never seemed to make sense when people insist that, if a player is a 40% three-point shooter, the next shot has a 40% chance of going in, regardless of how the player is performing on a particular day.  Nice to have some actual analysis backing up my gut feeling.

repole

April 15th, 2014 at 3:21 PM ^

Regarding basketball, there was a paper on this at the Sloan conference this year that made a pretty easy to understand argument.

A player who is hot is:
-More likely to hit shots of the same difficulty.
-More likely to take more difficult shots.

The effect essentially balances out and hides the underlying "hotness." So a hot player may make an uncontested 3 at a 45% rate instead of his usual 40% rate, but he's more likely to settle for a worse shot precisesly because he (or his teammates, or his coach) senses that he's hot.

clown question

April 15th, 2014 at 3:33 PM ^

While this could be true, what about free-throws? I could be wrong here (and would love a link if I am!) but haven't people not been able to detect the hot hand effect in free throws?

 

There seem to be plenty of reasons why we wouldn't see a hot hand effect in basketball or baseball (such as agressiveness or defense), but shouldn't we be able to see it from the free throw line which is both standardized and involves only 1 player? To my knowledge, no effect greater than 1% has been found (again, please correct me if I'm wrong).

repole

April 15th, 2014 at 4:35 PM ^

Not a clue, I haven't read that paper yet, only saw a talk regarding it. A just total off the top of my head guess would be that free throws are such an isolated, out of rhythm event that they may not be prone to such swings. Wouldn't surprise me to hear someone say that they might over think at the line when they're "hot," which could offset that impact.

 

The thing with all of these types of studies is that there's so, so many dependent variables that it's really hard to control for things. This type of stuff always seems to start off with an implied assumption, and often there's no way to prove that assumption.  I'm a pretty big analytics guy, hell it's what I'd like to do professionally, but I think anyone who forgets about the human side of things is doing a great diservice to themselves.

 

EDIT: Here's a link to the actual paper by the way.

JamieH

April 15th, 2014 at 4:35 PM ^

You see guys who are bricking everything they shoot go to the line and hit nothing but net, and you see guys who are hitting everything go to the line and miss their free throws.  Free throws are a mental exercism that is somewhat independent of the rest fo the game, and hard to quantify.  They are also a somewhat different shot than a jump shot, so it isn't a one-to-one relationship IMO.

panthera leo fututio

April 15th, 2014 at 5:43 PM ^

"The myth of the myth of the myth of the hot hand."

FWIW, I think his 4th point is important and pretty incontrovertible:

"4. Whatever the latest results on particular sports, I can’t see anyone overturning the basic finding of Gilovich, Vallone, and Tversky that players and spectators alike will perceive the hot hand even when it does not exist and dramatically overestimate the magnitude and consistency of any hot-hand phenomenon that does exist."

MaizeAndBlueWahoo

April 15th, 2014 at 5:51 PM ^

I mean, we're talking about a subset of the human population that knows it makes absolutely no difference what color underwear they have on and behaves anyway as if it does.  Overrating the psychological aspects of sports is a given.  The only issue I have is when mathheads do a complete 180 and dismiss any performance as a mere phenomenon of statistics.

repole

April 15th, 2014 at 10:00 PM ^

Agreed that it's certainly overestimated by most. At the same time, Gelman notes in the comment section that for some guys the variance can be much higher than others (claiming that on average, the swing might be 2% points, but for some could be as high as 20%). It seems like a lot of people have taken "the average effect of any so called hot hand is small" to mean "the hot hand doesn't exist." It's essentially impossible to prove the latter, while there's plenty of data supporting the former.

I saw Gerald Green drop 25 points in a quarter against the Thunder earlier this season. It sure is hard to consider that as just some random variance.

Tater

April 15th, 2014 at 6:28 PM ^

Fuck the studies.  All of them.  Sorry to be such a simpleton, but when someone is hot, they are likely to hit their next shot.  

JamieH

April 15th, 2014 at 6:59 PM ^

I'd love someone to do a case study of Vinnie Johnson and try to claim that hot and cold streaks don't exist in basketball shooting.  In the playoffs, you could practically predict his final point total after watching his first two shots go up.  If he hit the first two shots he took, he was feeling good and was practically unstoppable.  If he missed, it didn't matter if he was wide-open, he couldn't buy a bucket to save his life.  He is about the streakiest player I can remember.

RJWolvie

April 15th, 2014 at 9:28 PM ^

As another huge sports fan who does statistical methodology for a living, I've always been interested in this sort of thing too. Guy named Robert Frank, mathematician-economist out of cornell did lots of studies on hot hand. Considered the microwave's career stats among many many other things. He was no more likely to make his next shot than his career average whether he'd hit the last one, two, three or four or whatever in a row. Sorry -- hot hand doesn't exist. Frank did much more on sports & stats. Contra the defense reacts hypothesis, he also tried with the cornell team (ok, ok...) at shoot around. Checking whether more likely to make the next after x in a row: nope. Also just asked them -- shoot and let me know when you feel you're hot. Nope: still not more likely to make the next after saying so than that player's average. And lots more cool studies like that. I'm just so jealous he got grant money to explore this stuff!!!

kb

April 15th, 2014 at 9:46 PM ^

about the original Gilovich article on the hot hand was that two or three out of the 30 or so players from the Sixers he used in his study actually did consistent produce the hot hand by showing streaks of makes that were outside of chance.....but somehow he fails to discuss it in his analysis.

RJWolvie

April 15th, 2014 at 10:04 PM ^

That's about how many you'd expect to show "significant" runs, at the .067-.100 level! (Sorry. Too geeky. But point is 2-3 of 30 is what you'd expect by chance at usual levels of "strength of evidence".)

kb

April 15th, 2014 at 7:49 PM ^

I have read all of the hot hand peer reviewed journal articles and did some research on it in grad school. The research basically says that the hot hand is not real and is a perception by players and spectators of streaks that occur normally within the distribution of makes and misses that occur by chance. The stats that are used to assess the hot hand are based on a runs test. The major issues with hot hand research are that 1) you cannot control for everything in our multivariate environment and 2) what constitutes being hot (e.g., how many makes or misses is being hot or cold). These limitations stack the deck in a way that makes it difficult to prove the hot hand exists and makes it easier to prove that it doesn't. As someone who played college basketball I tend to think that the hot hand does exist is some way, shape or form. Chance and ability interact and are not separate. The best example that comes to mind is coin flipping. If you flip a coin the chance of heads or tails is conceivably the same, but one can also train themselves through practice to flip a coin in a manner that generates longer streaks of heads or tails.