Stopping momentum, part II

In the last installment, I investigated one case of what sports commentators refer to as “momentum” (where a team that makes a successful play will continue making to be successful): outcomes in overtime games. Looking through the CFBStats data from 2005-2011, I found that not only did teams that came from behind to force overtime fail to come out on top at an unusual rate, their outcomes were not affected by other factors such as being the home team or coming back from large deficits. However, I was not entirely exhaustive in my analysis, and two commenters, SpyinColumbus and cgnost, pointed out that it might be interesting to see what, if any role, rankings might play in determining outcomes in overtime.

As it turned out, integrating Sagarin rankings into the CFBStats data was fairly straightforward, and I created a table that matches the CFBStats ID codes (which are the same as used in the NCAA data that CFBStats is built from) with the names that Sagarin uses in his published data. So if you are working with these two data sets and want to put them together, here is the file to integrate these data sources, which is covered by an ODC PDDL (public domain).

With this in mind, the first order of business was to address the issue of what, if any, differences emerge in terms of Sagarin rankings in determining overtime outcomes compared to whether or not teams come from behind. In essence, do teams that come from behind beat their Sagarin predictions? If so, this might suggest that teams coming from behind are bringing some momentum into overtime.

Again, I am considering the set of 230 overtime games from 2005-2011 (dropping the 2005 Arkansas State-Florida Atlantic 0-0 EOR game). I will focus on Sagarin’s “PREDICTOR” model as he regards this as the most useful predictor of game outcomes, though I will also present some analysis using “RATING” and “ELO_CHESS.” PREDICTOR accounts for margin of victory, while ELO_CHESS only considers game outcomes (Sagarin describes it as more “politically correct”). RATING is a synthesis of the two. I also used the year-by-year home advantage values to adjust these ratings, including the 2011 addition of separate values for home advantage for each of the ranking systems. Neutral site games are not adjusted.

One important limitation of this analysis is that, because historical week-by-week Sagarin rankings are not available to my knowledge, all of this analysis is based on his year-end rankings. Because end-of-year rankings are determined by performance in-season, this brings up considerable endogeneity issues that cannot be easily dismissed. The best way to address this would be with the week-by-week rankings, and so if anyone knows of historical data, please let me know and I will see if this changes the results in any meaningful way.

To characterize the results, the first analysis I considered with regard to the ranking was general prediction of overtime outcomes. Sagarin’s rankings use scales with higher values indicating a higher ranked team, and, at least with PREDICTOR, the expected margin of victory. To predict outcomes based on Sagarin’s rankings, I subtracted the PREDICTOR, RATING and ELO_CHESS values of the losing team from the winning team. Thus, positive values indicate that the higher ranked team won (a “normal” outcome) and negative values indicate an “upset.” Based on this, we see the following results for overtime games:

Sagarin’s hit rate for overtime games is about 57% at best and 54% at worst, depending on which of his models is being used. It is worth noting that among non-overtime games, his hit rate is much better (between 78.4% and 80.2% in games between 2005-2011), but this is not surprising because overtime games represent a small sample of games between more closely matched teams (average difference between teams for the ranking systems in non-overtime games is between 10.0 and 10.2 while for overtime games it is between .3 and 1.1). How do Sagarin’s rankings look when considering the way in which overtime is forced?

To do this analysis, I modified my measures somewhat to make the results more interpretable. Since I was focused on teams coming from behind, I subtracted the PREDICTOR rating of the leading team from that of the team that came from behind. This difference therefore represents Sagarin’s predicted outcome for the team coming from behind – if it is less than zero, then the team coming from behind would be predicted to lose the game, while if it is greater than zero, they would be predicted to win.

The overall average for the from behind PREDICTOR difference score is -1.44, which is significantly different from zero (t(229) = -2.19, p < .05), indicating that, on average, teams coming from behind were predicted to lose. A logistic regression with the from-behind PREDICTOR difference score as the independent variable and the game’s outcome as the dependent variable revealed that these differences in PREDICTOR scores did not predict the games’ outcome (Exp(β) = 1.002, p > .85). To further clarify this relationship, I split the data into games where the team coming from behind was predicted to lose (that is, had a PREDICTOR score less than zero) and where these teams were predicted to win (PREDICTOR>0), and compared this to the games’ overall outcome:

(χ²(1) = .49, p >.48)

What this tells us is that rankings and game outcomes are independent of one another. More directly, while teams coming from behind to tie the game up are more likely to have been predicted to lose, these predictions did not affect how they performed in overtime.

In the context of momentum, this provides further evidence that coming from behind has no effect on game outcomes. Overall, Sagarin rankings are a barely weighted coin flip in overtime games, and how the teams became deadlocked in regulation does not affect this coin in any way.

Thanks, again, for reading, and to cgnost for prompting this analysis. In the next installment, I’ll continue the search for evidence of momentum in traditional defensive stops (those not ending in fumbles, interceptions or safeties), with a special focus on my favorite play in all of football: the goal line stand. Go blue.