[Ed.: as a basis for discussion. IME, the FO-based stats are the best available for reducing noise when you're evaluating how good of a team you've got.]
Hey guys, I don't know about you, but 99% of the conversations I've seen or heard about Rich Rodriguez's future at the University of Michigan hinge on how much each person thinks the team has improved. So obviously, the question is how much have we improved, exactly?
To start off, I'm going to make a few assumptions and attempt to defend them. First, very few people can simply watch the games, watch the highlights and determine if their own team has gotten better. Frankly, we don't know enough about the game on a micro level for our eyeball test to mean anything, not to mention the TV angles don't have large parts of the play, we don't know what play was called, etc.
Secondly, no mere mortal is actually capable of rating teams, especially the mediocre ones. There are around 50 games a week during the season, and while many of us wish we could be superfans, we simply are not capable of watching that many games in any meaningful sense. If you aren't watching the games, what are you basing your eyeball rankings off of?
Because of those two assumptions, the only place we can really look for improvement is found in statistics.
Statistics? @#$@, like math?
Don't they lie or something?
Well, yeah sometimes. There are many different ways to look at football statistically, and frankly, all of them have fairly severe flaws. Football simply has too many intangibles to model mathematically as well as baseball. However, that doesn't mean that all statistical analysis of football is useless, just that you have to be careful not to overstate your case and to look at the data in as many ways as possible. For this diary, we're going to look at three major ways of quantifying football games. The goal is to compare the results and see if we can get some sort of idea of what's going on.
OK so what are these different ways? Didn't Brian post about FEI or something?
The first, and most common, are methods that mostly rely on looking at who won against who and/or by how much. This is the type of method used by Sagarin, Massey and more. For the BCS formulations, Massey and Sagarin are not allowed to use margin of victory in their calculations. However, when Massey and Sagarin use margin of victory, their models are more accurate.
The second one we'll look at is basically drive analysis. This is FEI, and is best explained by Football Outsiders:
The Fremeau Efficiency Index (FEI) considers each of the nearly 20,000 possessions every season in major college football. All drives are filtered to eliminate first-half clock-kills and end-of-game garbage drives and scores. A scoring rate analysis of the remaining possessions then determines the baseline possession efficiency expectations against which each team is measured. A team is rewarded for playing well against good teams, win or lose, and is punished more severely for playing poorly against bad teams than it is rewarded for playing well against bad teams.
The last one we'll look at is an analysis that uses a play by play analysis. Again, Football Outsiders:
The S&P+ Ratings are a college football ratings system derived from the play-by-play data of all 800+ of a season's FBS college football games (and 140,000+ plays). There are three key components to the S&P+:
The situation: You are down 14 and probably only have 2 possessions left. Obviously, it will take two touchdowns to get back into the game. My question for you is, what combination of 2 point and 1 point conversions should you take to maximize your chance of winning the game?
Let's start off with a few assumptions. According to this rivals article, the average 2pt conversion rate in the NFL is 44%. I'll assume that it's about the same for CFB and that our team's conversion rate will be about the same in whatever specific situation we're in. We'll assume that we can estimate a PA kick as a sure thing. We'll also assume that we have a 50-50 chance of winning in OT.
So working with these assumptions, what is the optimal combination of 1pt/2pt tries?
Kicking 1pt tries only
This one is easy. Assuming we get 2 TDs to come back, taking 1pt each time will give us a 50-50 chance to win
In this situation, we get the first TD and take the 1pt. On the second TD, we 'man up' and go FTW BABY! Our chances of winning are equal to the chance of converting obviously, so 44%.
In this situation, we'll go for 2 after the first TD. If we convert, then we'll kick a 1pt try. If we do not convert, then we'll go for 2 again.
This is a slightly more complicated calculation, but here we go:
1.) 44% of the time we make the first 2pt conversion and go on to win the game.
2.) (.56)*(.56) = 31% of the time we miss both 2pt tries and lose despite making two TDs
3.) (.56)*(.44) = 25% of the time we miss the first but make the second 2pt. This ties the game and we go to overtime.
So what is our final equity? It is:
.44*1 + .31*0 + .25*(.5) = .57 or 57%
A quick explanation of this equation. We basically multiply the probability of an event by the outcome of the event. So 44% of the time we win (1), 31% of the time we lose (0) and 25% of the time we go to OT with a 50-50 shot (.5).
Now why isn't this done in the real world? Well part of it is that some of our assumptions aren't known. However, mostly it is coaches covering their ass. No one gets criticized for taking the safe route to force OT, only to lose. If you go for 2 twice and don't make it, you'll be torn apart in the press. Not to mention that football coaches don't focus much of their time on equity calculations.
The common belief of kicking 1pt to tie or going FTW! at the end with a 2pt conversion is clearly wrong, even if it is most commonly done.