club_med

In the last installment, I investigated one case of what sports commentators refer to as “momentum” (where a team that makes a successful play will continue making to be successful): outcomes in overtime games. Looking through the CFBStats data from 2005-2011, I found that not only did teams that came from behind to force overtime fail to come out on top at an unusual rate, their outcomes were not affected by other factors such as being the home team or coming back from large deficits. However, I was not entirely exhaustive in my analysis, and two commenters, SpyinColumbus and cgnost, pointed out that it might be interesting to see what, if any role, rankings might play in determining outcomes in overtime.

As it turned out, integrating Sagarin rankings into the CFBStats data was fairly straightforward, and I created a table that matches the CFBStats ID codes (which are the same as used in the NCAA data that CFBStats is built from) with the names that Sagarin uses in his published data. So if you are working with these two data sets and want to put them together, here is the file to integrate these data sources, which is covered by an ODC PDDL (public domain).

With this in mind, the first order of business was to address the issue of what, if any, differences emerge in terms of Sagarin rankings in determining overtime outcomes compared to whether or not teams come from behind. In essence, do teams that come from behind beat their Sagarin predictions? If so, this might suggest that teams coming from behind are bringing some momentum into overtime.

Again, I am considering the set of 230 overtime games from 2005-2011 (dropping the 2005 Arkansas State-Florida Atlantic 0-0 EOR game). I will focus on Sagarin’s “PREDICTOR” model as he regards this as the most useful predictor of game outcomes, though I will also present some analysis using “RATING” and “ELO_CHESS.” PREDICTOR accounts for margin of victory, while ELO_CHESS only considers game outcomes (Sagarin describes it as more “politically correct”). RATING is a synthesis of the two. I also used the year-by-year home advantage values to adjust these ratings, including the 2011 addition of separate values for home advantage for each of the ranking systems. Neutral site games are not adjusted.

One important limitation of this analysis is that, because historical week-by-week Sagarin rankings are not available to my knowledge, all of this analysis is based on his year-end rankings. Because end-of-year rankings are determined by performance in-season, this brings up considerable endogeneity issues that cannot be easily dismissed. The best way to address this would be with the week-by-week rankings, and so if anyone knows of historical data, please let me know and I will see if this changes the results in any meaningful way.

To characterize the results, the first analysis I considered with regard to the ranking was general prediction of overtime outcomes. Sagarin’s rankings use scales with higher values indicating a higher ranked team, and, at least with PREDICTOR, the expected margin of victory. To predict outcomes based on Sagarin’s rankings, I subtracted the PREDICTOR, RATING and ELO_CHESS values of the losing team from the winning team. Thus, positive values indicate that the higher ranked team won (a “normal” outcome) and negative values indicate an “upset.” Based on this, we see the following results for overtime games:

Sagarin’s hit rate for overtime games is about 57% at best and 54% at worst, depending on which of his models is being used. It is worth noting that among non-overtime games, his hit rate is much better (between 78.4% and 80.2% in games between 2005-2011), but this is not surprising because overtime games represent a small sample of games between more closely matched teams (average difference between teams for the ranking systems in non-overtime games is between 10.0 and 10.2 while for overtime games it is between .3 and 1.1). How do Sagarin’s rankings look when considering the way in which overtime is forced?

To do this analysis, I modified my measures somewhat to make the results more interpretable. Since I was focused on teams coming from behind, I subtracted the PREDICTOR rating of the leading team from that of the team that came from behind. This difference therefore represents Sagarin’s predicted outcome for the team coming from behind – if it is less than zero, then the team coming from behind would be predicted to lose the game, while if it is greater than zero, they would be predicted to win.

The overall average for the from behind PREDICTOR difference score is -1.44, which is significantly different from zero (t(229) = -2.19, p < .05), indicating that, on average, teams coming from behind were predicted to lose. A logistic regression with the from-behind PREDICTOR difference score as the independent variable and the game’s outcome as the dependent variable revealed that these differences in PREDICTOR scores did not predict the games’ outcome (Exp(β) = 1.002, p > .85). To further clarify this relationship, I split the data into games where the team coming from behind was predicted to lose (that is, had a PREDICTOR score less than zero) and where these teams were predicted to win (PREDICTOR>0), and compared this to the games’ overall outcome:

(χ²(1) = .49, p >.48)

What this tells us is that rankings and game outcomes are independent of one another. More directly, while teams coming from behind to tie the game up are more likely to have been predicted to lose, these predictions did not affect how they performed in overtime.

In the context of momentum, this provides further evidence that coming from behind has no effect on game outcomes. Overall, Sagarin rankings are a barely weighted coin flip in overtime games, and how the teams became deadlocked in regulation does not affect this coin in any way.

Thanks, again, for reading, and to cgnost for prompting this analysis. In the next installment, I’ll continue the search for evidence of momentum in traditional defensive stops (those not ending in fumbles, interceptions or safeties), with a special focus on my favorite play in all of football: the goal line stand. Go blue.

For as long as I have had any statistical training, I have loathed the way that sports commentators use the term “momentum." I realize that I am probably not alone in this regard as member at MGoBlog, nor am I any where near the first to criticize the spurious use of “momentum” or related concepts (“Hot Hand” Fallacy, Gilovich, Vallone and Tversky (1985); Silver (2012); though see Berger and Pope (2009)), but I wanted to be a bit more concrete in my criticisms in the context of college football. Thus, I decided to work on a series of Diaries to try out various ideas about how momentum is discussed – the “Convential Wisdom of Momentum” (CWoM), and see what (if any) evidence there is to show for it.

Adapted from classical mechanics, momentum is commonly understood to mean that once an object or actor has been set in a particular direction, it will continue along this path until acted upon by another force. In the context of sports, momentum would suggest that teams that have had success tend to continue to be successful, and typically is discussed at the micro-level within individual games (i.e. play-to-play, drive-to-drive), as opposed to the macro level across games. Note that I do acknowledge “psychological momentum,” in that players may start to view the game differently depending upon recent events. Most athletes have had some experience of “flow” (Csikszentmihalyi 1988) wherein they get "in the zone," and their automated responses drive their performance. However, I think this plays a relatively small role at high-level athletic competition since most athletes are "flowing," having spent sufficient time training to develop it.

With this game-level focus in mind, the first example of "momentum" I wanted to consider was the idea that in overtime games, the team coming from behind has momentum, which became all the more germane after this weekend’s game. CWoM suggests that the team that has to tie the game up to force overtime would be more likely to win the game outright.

To investigate this hypothesis, I decided to look into the awesome CFB Stats data, which was released under an Open Data Commons attribution license. I only looked at complete years, so all of my analysis is based on games from the 2005-2011 football seasons. Out of 5,534 FBS and FCS games during this time period, 231 games went to overtime. I eliminated one game (the October 22, 2005 matchup between Arkansas State and Florida Atlantic) from the analysis because at the end of regulation, the game was tied 0-0 and thus neither team could be considered to be leading going into overtime. This left me with 230 games in the sample.

The basic prediction of CWoM in overtime is that the last team to score before the end of regulation will have a greater chance of winning in overtime. I’ll test this prediction, along with a couple of other ways of looking at the same underlying phenomenon. So, how do teams that come from behind fair in OT?

Not especially well. Out of 230 games, the team coming from behind won only 104/230 (45%) of the time. A Chi-square t test suggests that the differences observed here are random (χ²(1) = 2.10, p = .15), indicating that our data does not suggest that either come-from-behind or leading teams have an advantage in overtime. Breaking things out by conference does not suggest anything unusual, either, nor do the results of the statistical tests differ (χ²(11) = 6.11, p = .87). This result is also not changed by only considering the six biggest conferences (ACC, Big East, B1G, PAC-12, SEC and Big 12) where results could be less affected by the small sample sizes (χ²(5) = 2.46, p = .78).

What about home teams? Given the excitement of a come-from-behind score to tie the game and send it to overtime, it would seem very plausible that home teams might fare better in come-from-behind overtime scenarios. Note that for this analysis, neutral site games are treated as “away” games for both teams. Overall, home teams won 128/230 games (56%), and within come-from-behind games, 60% of the time the home team won. However, the difference observed is not significant (χ²(1) = 1.21, p = .27), suggesting that home teams do not perform unusually well in come-from-behind games.

So far, we have not found any evidence to support the CWoM hypothesis that come-from-behind teams perform better in overtime. There are two more factors I considered that might shed more light on the situation. The first was how close to the end of the game the final score occurred. If the final, tying score in regulation happened very close to the end of the game, this might make it more likely that the team coming from behind would continue their success into overtime. For this analysis, I only considered games wherein the tying score occurred in the fourth quarter, leaving 216 out of the 230 total games. To model this relationship, I used a binary logistic regression with the dependent variable as the outcome of the game for the coming-from-behind team, and the independent variable the number of seconds remaining in the game. The results of this regression indicate that the time at which the tying score occurs does not predict the outcome of the game (Exp(β) = 1.00, p > .65). Because logistic regressions do not lend themselves to obvious interpretation, I split the data into groups of more or less than two minutes of game time remaining. Across both groups, we see a similar pattern to what we observe in the overall data set – that the team that led most recently tends to win in overtime, but this difference appears a bit more pronounced when the tying score occurred in the last two minutes of regulation. This difference is not significant, though (χ²(1) = 1.27, p > .26)

The last component of the come-from-behind momentum hypotheses I wanted to investigate was how big the comeback was – that is, does coming back from a bigger deficit increase the odds of coming out on top? Again, I used a logistic regression with the dependent variable as the outcome of the game for the coming-from-behind team, and the independent variable the maximum point differential between the two teams through the game. The results of this regression indicate that the maximum point differential does not predict the outcome of the game (Exp(β) = 1.11, p > .71). Again, for clarity, I also provide a categorical analysis, with the maximum point differential divided into less than two TDs and more than two. In close games, we see largely the same pattern as before, with the team coming from behind winning only 42% of the time. However, in games with bigger differentials in points, 57% of the time, the team coming from behind comes out on top. This difference is marginally significant (χ²(1) = 2.42, p > .08), but I put more faith in the logistic result since it is a more robust test.

In conclusion, I do not see any evidence to support the CWoM hypothesis that the last team to score has “momentum” going into overtime. Even considering a variety of other factors that have some lay theoretical association with momentum (home team advantage, scoring late and big comebacks), nothing presents an even mildly compelling case for it. Our experience this weekend – tying the game following a dramatic Roundtree catch with less than 10 seconds on the clock – brought us to overtime with what one would call a great deal of momentum, which we capitalized on by winning decisively. However, the numbers just do not support this narrative. In reality, our odds of winning based on the way in which we got to overtime were not different from a coin toss.

I am hoping to continue this series by looking at various other situations that the CWoM views as “momentum swings,” such as 4^th down stops, kickoff returns, picks and safeties. If you have other ideas, would like to see more analysis within this data or have comments on this project, please let me know. Thanks for reading, and Go Blue.