[Ed.: as a basis for discussion. IME, the FO-based stats are the best available for reducing noise when you're evaluating how good of a team you've got.]

Hey guys, I don't know about you, but 99% of the conversations I've seen or heard about Rich Rodriguez's future at the University of Michigan hinge on how much each person thinks the team has improved. So obviously, the question is how much have we improved, exactly?

To start off, I'm going to make a few assumptions and attempt to defend them. First, very few people can simply watch the games, watch the highlights and determine if their own team has gotten better. Frankly, we don't know enough about the game on a micro level for our eyeball test to mean anything, not to mention the TV angles don't have large parts of the play, we don't know what play was called, etc.

Secondly, no mere mortal is actually capable of rating teams, especially the mediocre ones. There are around 50 games a week during the season, and while many of us wish we could be superfans, we simply are not capable of watching that many games in any meaningful sense. If you aren't watching the games, what are you basing your eyeball rankings off of?

Because of those two assumptions, the only place we can really look for improvement is found in statistics.

Statistics? @#$@, like math?

Yeah, sorry

Don't they lie or something?

Well, yeah sometimes. There are many different ways to look at football statistically, and frankly, all of them have fairly severe flaws. Football simply has too many intangibles to model mathematically as well as baseball. However, that doesn't mean that all statistical analysis of football is useless, just that you have to be careful not to overstate your case and to look at the data in as many ways as possible. For this diary, we're going to look at three major ways of quantifying football games. The goal is to compare the results and see if we can get some sort of idea of what's going on.

OK so what are these different ways? Didn't Brian post about FEI or something?

The first, and most common, are methods that mostly rely on looking at  who won against who and/or by how much. This is the type of method used by Sagarin, Massey and more. For the BCS formulations, Massey and Sagarin are not allowed to use margin of victory in their calculations. However, when Massey and Sagarin use margin of victory, their models are more accurate.

The second one we'll look at is basically drive analysis. This is FEI, and is best explained by Football Outsiders:

The Fremeau Efficiency Index (FEI) considers each of the nearly 20,000 possessions every season in major college football. All drives are filtered to eliminate first-half clock-kills and end-of-game garbage drives and scores. A scoring rate analysis of the remaining possessions then determines the baseline possession efficiency expectations against which each team is measured. A team is rewarded for playing well against good teams, win or lose, and is punished more severely for playing poorly against bad teams than it is rewarded for playing well against bad teams.

The last one we'll look at is an analysis that uses a play by play analysis. Again, Football Outsiders:

The S&P+ Ratings are a college football ratings system derived from the play-by-play data of all 800+ of a season's FBS college football games (and 140,000+ plays). There are three key components to the S&P+:

• Success Rate: A common Football Outsiders tool used to measure efficiency by determining whether every play of a given game was successful or not. The terms of success in college football: 50 percent of necessary yardage on first down, 70 percent on second down, and 100 percent on third and fourth down.
• EqPts Per Play (PPP): An explosiveness measure derived from determining the point value of every yard line (based on the expected number of points an offense could expect to score from that yard line) and, therefore, every play of a given game.
• Opponent adjustments: Success Rate and PPP combine to form S&P, an OPS-like measure for football. Then eachteam's S&P output for a given category (Rushing/Passing on either Standard Downs or Passing Downs) is compared to the expected output based upon their opponents and their opponents' opponents. This is a schedule-based adjustment designed to reward tougher schedules and punish weaker ones.

The S&P+ figures used in the tables below only look at the plays that took place while a game was deemed "close," or competitive. The criteria for being "close" are as follows: a game within 24 points in the first quarter, with 21 points in the second quarter, and within 16 points in the second half.

OMG Wall of Text! I'm Lost!

Think of it this way, we're looking at the game at three levels: final scores, drives, and plays.

OK that sounds more reasonable. Results?

Remember that a lower ranking is better. The average improvement from 2008 is about 48 places (from 89th to 41st) . Sagarin's BCS formula has the most improvement at 71. FEI (Drive analysis) is the smallest at 31 places.

So what the hell does that mean?

It means that Michigan improved a lot. In 2008, Michigan was ranked in between 68th and 105th. In 2010, Michigan was ranked in between 30th and 53rd. That is a huge leap.

The drive analysis and play by play metrics show the least amount of improvement for Michigan, however, those rankings had Michigan much higher in 2008 than the win/loss metrics.

Now, it's up to you exactly if it's enough to keep Rodriguez, but hopefully now you have a better idea of exactly how much that improvement was. 

Go blue!