Improvement, Quantified
[Ed.: as a basis for discussion. IME, the FO-based stats are the best available for reducing noise when you're evaluating how good of a team you've got.]
Hey guys, I don't know about you, but 99% of the conversations I've seen or heard about Rich Rodriguez's future at the University of Michigan hinge on how much each person thinks the team has improved. So obviously, the question is how much have we improved, exactly?
To start off, I'm going to make a few assumptions and attempt to defend them. First, very few people can simply watch the games, watch the highlights and determine if their own team has gotten better. Frankly, we don't know enough about the game on a micro level for our eyeball test to mean anything, not to mention the TV angles don't have large parts of the play, we don't know what play was called, etc.
Secondly, no mere mortal is actually capable of rating teams, especially the mediocre ones. There are around 50 games a week during the season, and while many of us wish we could be superfans, we simply are not capable of watching that many games in any meaningful sense. If you aren't watching the games, what are you basing your eyeball rankings off of?
Because of those two assumptions, the only place we can really look for improvement is found in statistics.
Statistics? @#$@, like math?
Yeah, sorry
Don't they lie or something?
Well, yeah sometimes. There are many different ways to look at football statistically, and frankly, all of them have fairly severe flaws. Football simply has too many intangibles to model mathematically as well as baseball. However, that doesn't mean that all statistical analysis of football is useless, just that you have to be careful not to overstate your case and to look at the data in as many ways as possible. For this diary, we're going to look at three major ways of quantifying football games. The goal is to compare the results and see if we can get some sort of idea of what's going on.
OK so what are these different ways? Didn't Brian post about FEI or something?
The first, and most common, are methods that mostly rely on looking at who won against who and/or by how much. This is the type of method used by Sagarin, Massey and more. For the BCS formulations, Massey and Sagarin are not allowed to use margin of victory in their calculations. However, when Massey and Sagarin use margin of victory, their models are more accurate.
The second one we'll look at is basically drive analysis. This is FEI, and is best explained by Football Outsiders:
The Fremeau Efficiency Index (FEI) considers each of the nearly 20,000 possessions every season in major college football. All drives are filtered to eliminate first-half clock-kills and end-of-game garbage drives and scores. A scoring rate analysis of the remaining possessions then determines the baseline possession efficiency expectations against which each team is measured. A team is rewarded for playing well against good teams, win or lose, and is punished more severely for playing poorly against bad teams than it is rewarded for playing well against bad teams.
The last one we'll look at is an analysis that uses a play by play analysis. Again, Football Outsiders:
The S&P+ Ratings are a college football ratings system derived from the play-by-play data of all 800+ of a season's FBS college football games (and 140,000+ plays). There are three key components to the S&P+:
- Success Rate: A common Football Outsiders tool used to measure efficiency by determining whether every play of a given game was successful or not. The terms of success in college football: 50 percent of necessary yardage on first down, 70 percent on second down, and 100 percent on third and fourth down.
- EqPts Per Play (PPP): An explosiveness measure derived from determining the point value of every yard line (based on the expected number of points an offense could expect to score from that yard line) and, therefore, every play of a given game.
- Opponent adjustments: Success Rate and PPP combine to form S&P, an OPS-like measure for football. Then eachteam's S&P output for a given category (Rushing/Passing on either Standard Downs or Passing Downs) is compared to the expected output based upon their opponents and their opponents' opponents. This is a schedule-based adjustment designed to reward tougher schedules and punish weaker ones.
The S&P+ figures used in the tables below only look at the plays that took place while a game was deemed "close," or competitive. The criteria for being "close" are as follows: a game within 24 points in the first quarter, with 21 points in the second quarter, and within 16 points in the second half.
OMG Wall of Text! I'm Lost!
Think of it this way, we're looking at the game at three levels: final scores, drives, and plays.
OK that sounds more reasonable. Results?
Remember that a lower ranking is better. The average improvement from 2008 is about 48 places (from 89th to 41st) . Sagarin's BCS formula has the most improvement at 71. FEI (Drive analysis) is the smallest at 31 places.
So what the hell does that mean?
It means that Michigan improved a lot. In 2008, Michigan was ranked in between 68th and 105th. In 2010, Michigan was ranked in between 30th and 53rd. That is a huge leap.
The drive analysis and play by play metrics show the least amount of improvement for Michigan, however, those rankings had Michigan much higher in 2008 than the win/loss metrics.
Now, it's up to you exactly if it's enough to keep Rodriguez, but hopefully now you have a better idea of exactly how much that improvement was.
Go blue!
November 30th, 2010 at 9:02 PM ^
No guarantee Mallett stays either, he could have just as easily transferred back to Arkansas anyway. It's what he wanted to do anyway, and ANY change in coach just made the decision easier. So, we still would have had Threet/Sheridan.
November 30th, 2010 at 9:47 PM ^
... dude jumped by bones for even hinting that Lloyd may not have been the world's greatest coach.
November 29th, 2010 at 5:27 PM ^
... was 7-5.
Carr lost to OSU three times in a row.
Carr won one bowl game in his last three years. (RR has a chance to tie that)
Stop comparing RR to Carr like Carr is some fucking God. He was not. He was not. He was not. He was not. He was not.
Enough.
November 29th, 2010 at 5:54 PM ^
are gravely mistaken. What is with RR defenders slamming Lloyd Carr? LEAVE HIM OUT OF IT. He was an excellent coach in his own right. Nothing to do with RR.
Lloyd Carr was not a deity, but he was a fantastic coach:
- Has a national championship-- may have been awhile ago, but that's a pretty amazing and rare accomplishment. Just ask any coach (including Rodriguez) in the Big Ten not named Tressel. Heck, you can ask him too.
- Never won fewer than 7 games (Rich Rod's best year so far)
- Was 6-0 against MSU in his final 6 years
- Was not winless against Penn State
- Was not winless against Ohio State
- Was not sub-.500 in the Big Ten for his coaching career
- Did not embarass the program, commit NCAA violations, et. al
- Won the first ten games of 2006, and was in the thick of the national championship hunt
- That bowl game win was against a reigning Heisman winner and the defending national champions (Florida), who would win it all the next year.
You may be annoyed by people reminiscing fondly of Lloyd Carr, but don't attack the man's legacy, especially in comparison to Rodriguez's. Petty and short-sighted.
November 30th, 2010 at 9:10 PM ^
Fantastic loss to Applicacian State while being ranked #2.
4 straight losses to OSU and 5 out of 6
Where are his recruits on D that should be Jr.'s and Sr.'s and playing now with experience?
He should have retired a couple of years earlier. We wouldn't be going through this.
Don't defend his leaving the bare cupboard in this program because he does have a bit of fault in this fiasco.
November 30th, 2010 at 9:24 PM ^
Mallett, Brown, Minor,Schilling, Molk, Hemingway, Koger, Matthews.. You cannot blame Carr for Mallett's departure. That is a solid nucleus of offensive talent.
Defensively, he left RR Terrence Taylor, Morgan Trent, Brandon Graham,Donovan Warren, Ryan Vanbergan, Obi Ezeh, Jonas Mouton, Mike Martin committed to Carr, Stevie Brown.Again, a solid nucleus of talent.
Most of those guys were 4 star players coming out of high school. How is that leaving the cupboard bare? Ezeh was solid his frosh season and then terrible since--all under RR.
This nonsense that Carr left it bare is spin pure and simple. Any person who has followed UM recruiting for more than two years know that Carr left RR a decent amount of talent. Granted, it did not fit RR's offensive system, but then it is up to RR to tailor his system to that talent.
November 30th, 2010 at 10:13 PM ^
Well said. The Carr left the cupboard bare is a myth fabricated by Rich Rodolytes and should be shouted down at every chance. Rich Rod emptied the cupboard is more like it.
November 30th, 2010 at 11:46 PM ^
"Rich Rod emptied the cupboard..."
This isn't merely ugly, it's fucking stupid. You're an ass.
Mallett? Was gone the moment Scott Loeffler left. Boren? Pulled out of Michigan by his enraged psycho-dad. Toney Clemons? Saw himself getting little playing time at Michigan. Who else do you want to talk about; Johnny Sears?
We did have Jason Forcier in 2006; did Carr drive him off?
November 30th, 2010 at 9:36 PM ^
... that Lloyd should be left out of it.
Lloyd is not the gold standard for college coaching, though he was a hell of a coach.
Don't fucking call me petty and shortsighted. Personal attacks can be left out.
November 29th, 2010 at 6:04 PM ^
what about the D?
November 29th, 2010 at 6:38 PM ^
It's not that hard to improve when you post one of the worst seasons in school history in year 1, and then follow it up with an even worse Big Ten the next year. For me, the question is this: does this staff seem likely to have us contending for national titles? I don't know how anyone can answer that affirmatively.
November 29th, 2010 at 6:58 PM ^
Any staff that can put together one of the 3 or 4 best offenses in the country with sophomores and juniors at essentially every position can compete for national titles.
November 30th, 2010 at 9:13 PM ^
I can affirmatively say that a staff that has assembled the three worst defenses in Michigan history is NOT capable of contending for national titles.
UM is basically Hawaii under June Jones.Look at all the national titles they have won.
November 30th, 2010 at 10:10 PM ^
Nah, Michigan fumbles more.
November 30th, 2010 at 9:15 PM ^
You are amazing. I've never seen you concede that a view other than your own could have merit.
December 1st, 2010 at 4:05 PM ^
Uh, I think there are a lot of people jmblue would agree with. His view also happens to be supported by fact.
December 1st, 2010 at 9:11 PM ^
So is just about everyones. They just choose to ignore those other facts that don't support their point of view. If you can't see that he continually cherry picks data that agrees with his POV, then you're as baised as he is. Seriously, who argues that close win/loss record is more compelling than overall efficiency data? jmblue, that's who.
December 1st, 2010 at 12:02 AM ^
December 1st, 2010 at 9:33 AM ^
Our previous assistants were known to us, not everyone else in the nation. DeBord was known for being a failed head coach that ran an offense that was capable of being a Maserati like it was a tractor. The guy was good at doing what he knew but the offense during his tenure, IMHE, was capable of much more, as evidenced by the bowl game in 2007 and games when we had to play catch-up.
November 30th, 2010 at 8:53 PM ^
but I am curious why does it seem so many posters have developed Brian's alter ego? Is it contagious? Should I be worried??
November 30th, 2010 at 9:23 PM ^
I hate facts too! RABBLE RABBLE FIRE RR!
November 30th, 2010 at 9:28 PM ^
It's contagious because:
1) It allows you to write pretty much exactly what you're thinking
2) It allows you to change topics quickly
3) It lets a casual reader know exactly what is going in the wall of text you just created
November 30th, 2010 at 9:07 PM ^
interested to see how much we regressed from the end of Carr - i do attribute a lot of the attrition to RR being not intelligent enough to use players he had to morph into his system -.. not saying mallet would have stayed, but wtf were you thinking running the threet/sheridan combo in the current system?
net-net - this improvement is significant if we accept mediocrity. when is the last time the program was ranked in the 30-58 range as we are now?
November 30th, 2010 at 9:14 PM ^
A Threet/Sheridan combo in any system would have been equivalent to a leaking anus. Have you ever seen them pass? Did you see our offensive line in 2008? McGuffie was getting tackled by our linemen's asses because they were getting shoved back so far against Miami of Ohio.
Edit: Also, do you seriously expect to jump more than 40-60 rankings in a year? Be as mad as you want that the first years sucked, but the improvement since then has been astronomical and the depth chart is only getting deeper.
November 30th, 2010 at 9:20 PM ^
I think RR has proven he's one hell of an offensive coordinator.
Don't get me wrong - i'm not a RR hater - i actually think we should give him another year. I don't buy the improvement aspect - there are 3 phases - our special teams (which i believe he coaches) and D seem to be getting worse every year.
(and isn't Threet doing well at ASU this year?)
November 30th, 2010 at 9:24 PM ^
As a RS Junior, he has 18 touchdowns to 16 interceptions and 19 rushing yards. He played as a RS Freshman for us. Also, this data takes more into account than just offense.
November 30th, 2010 at 11:30 PM ^
but for what it is worth, in 2006, Michigan was #2 at the end of the regular season; in 2007, Michigan was ranked #22 (then #71 in 2008, #59 in 2009 and #40 in 2010). I emphasize the phrase "for what it is worth." I don't agree with the conclusions you would draw from that.
I'll just note that West Viriginia was ranked #9 in 2006 and #3 in 2007.
November 30th, 2010 at 9:22 PM ^
Nice work putting that post together, it confirms the obvious improvement is happening and at rate which would bring unbridaled enthusiasm if the defense or kicking game would have kept pace. That did not happen which has some chorteling, some nashing of teeth, and some just pissed. I am ,just pissed,but did see enough improvement in the play of freshmen defenders to give me a good outlook for next year. I do want to see better play in our bowl game or will be back in the just pissed mode. The one thing I can't get a handle on is our inability to get a kicker who can put a kickoff into the endzone on every kick WTF dont we have enough to offer to have a choice.
November 30th, 2010 at 9:25 PM ^
Glad to see Michigan improving year to year.
To everyone continuing to bitch:
I am so sick of all you bandwagon fans infesting this board. You've made mgoblog depressing to visit. Simply put: get out. I support my school, my team, my coaches, my everything. Nothing you say matters in this decision. And you trash the team you're supposedly a fan of? You're throwing a tantrum because your team isn't winning for the first time ever. Yeah, it sucks. It hurts that the team is getting pounded like this. And I'll sit through every game and cheer our team on till the time runs out in the 4th.
If a coaching change is made, I'll support the next coaching staff with as much fervor as I do now. If the rest of you like Harbough so much, go cheer for Stanford.
November 30th, 2010 at 11:17 PM ^
This is not RichRodgoBlog (though it often seems like it). It's MgoBlog. M fans critical of the current staff belong here as much as those who support the current staff. We may disagree about the direction of the program, but we all support Michigan.
November 30th, 2010 at 11:51 PM ^
That's MGoBlog! Where else do you expect to find sexy pictures of Stanford's football coach? (Or is it a photo of team physician Dr. Gerry O'Connor, who just happens to be surrounded by Bo Schembechler and Jim Harbaugh?)
November 30th, 2010 at 9:26 PM ^
It would be more interesting to see how Michigan compares across a longer period of time. Was there quanatative decline at the end of Carr with 2008 being destruction and now clear rebuilding back to the average UM #s and beyond?
Be interesting to see what the typical range for the team has been the last ten years. Easy to request others do more work ...
November 30th, 2010 at 9:28 PM ^
Even if you accept that Michigan has, on the whole, improved from year-to-year (which I think is true, but not to this extent given that we still have massive problems with turning the ball over and our defense has regressed every year), then this cannot address whether or not Rodriguez has chronically underperformed every year.
November 30th, 2010 at 9:53 PM ^
November 30th, 2010 at 10:01 PM ^
We were 40th in the nation in least penalty yardage.
November 30th, 2010 at 10:14 PM ^
It's almost like you don't realize that this is accounted for. Weird.
November 30th, 2010 at 10:15 PM ^
that the "curve" (I know, only three points) for Sagarin's predictor is concave up while the Sagarin BCS is concave down?
November 30th, 2010 at 10:16 PM ^
is there are three very large problems on this team that have not improved during RR's tenure.
Defense, Special Teams and Turnovers.
The defense is young and seemingly lseverely lacking in Big Ten type talent and most-likely will be no better than the 80-90th defense in the country next season.
Special teams have been poor since RR got here and actually got much worse in 2010.
KR-77th; PR-T98th
in 2009 KR-25th; PR-62nd
The worst stat next to the defense is turnover margin. Possibilly the most embarrassing stat of all. 2010-110th (-9); 2009 115th (-12); 2008 105th (-10).
The combination of a poor, undersized defense; weak special teams(that's just the return game); and the turnover margin which shows no signs of getting better, I just can't see the improvement that many are pointing towards.
I don't care if you gain 700 yards per game between the 20s, if you can't score consistently against the better teams and at least slow down the better teams' offense improvement will be mighty slow.
The statisical eveidence is overwhelmingly against this staff. A lot of points and yards against the weaker teams and a lack of consistency and performance against the good teams, plus the inability to stop anyone with a decent offense does not look good for next year. Do we want to have to be back here again in 12 months?
November 30th, 2010 at 10:29 PM ^
You do realize the charts account for all that right?
November 30th, 2010 at 10:33 PM ^
the entire point of the original post. You're pulling out specific negative stats, cherry picked to make a case, while the FEI numbers use all aspects of the game combined.
You're basically starting with a conclusion then inventing arguments to support it.
November 30th, 2010 at 11:14 PM ^
"the turnover margin which shows no signs of getting better"
Seriously? I have no idea what your basis for this statement is. Look no further than who ahs been our starting quarterback (interceptions have accounted for 14-15 TOs/season)
'08 - quarterback was Threet/Sheridan
'09 - First-time starting quarterback (Tate)
'10 - First-time starting quarterback (Denard)
Given that next year will be the first time we will have a starting quarterback who is both experienced and talented, I see plenty of reason to think our TO margin will improve.
December 1st, 2010 at 8:33 AM ^
You can't be sure Denard will start next year. Tate could improve, Devin Gardner could step up, Denard could regress. No one thought Denard would start this year. Don't assume anything when it comes to Michigan in this day and age.
December 2nd, 2010 at 12:14 AM ^
This may come as a surprise but Michigan's use of a first year quarterback was a choice not some unfortunate fate due to injury.
December 1st, 2010 at 2:29 AM ^
tell Iowa and Wisconsin that their defensive talent is lacking too. Thus their shitty defenses......errrr, wait a second.
And you don't think a first year "project" quarterback at the helm and ZERO seniors (sans Schilling) has anything to do with the flaws in execution?
Are we happy with the current product? HELL NO! But Jesus man, give the guy a chance to put some upperclassmen out there. If this crap goes on again next year, I will be the first to show him the door. I don't think that's gonna happen though, honestly.
December 1st, 2010 at 8:06 AM ^
Come on dude. Those stats are useless. The difference between being ranked 77th and 15th in kick return average is 3 yards
November 30th, 2010 at 10:28 PM ^
go get 'em, kid.
'preciate the work.
December 1st, 2010 at 12:29 AM ^
I also appreciate your bolded alter ego. $#!% man, math
November 30th, 2010 at 10:33 PM ^
...but couldn't this quantitative analysis be summed up just as easily by saying
Sophomore Denard Robinson NCAA record-setter > Threetsherridammit. Ergo, Improvement between 2008-2010.
November 30th, 2010 at 10:49 PM ^
the defense, special teams and turnover number are atrocious. It's not an argument, these are the facts.
And if this team is improving so much why are they not competitive against ANY of the top five teams in the conference in 2010 when they were closer in 2008 and 2009?
No one says the offense is miles ahead of 2008, how could they not be, but the team as whole with the size makeup they have on both sides of the ball is simply not competitive against the top teams in the conference. Beating Indiana, Illinois and Purdue proves nothing.
The offense stalled down the stretch this season, look at the numbers in the final three games and look at the half time scores against the top five teams in the conference. We never really challenged any of those teams. And the fact that we cannot even attempt a field goal over 25 yards is simply ridiculous for a BCS school.
I'm not pulling a small snapshot of numbers to make a point. The numbers I provided are from all games and I specifically left out the defensive stats because we all know how bad they are. Aside from the offense, there are very few positives to take from this team right now. The kids play hard and never quit, that's huge, but they simply aren't executing, especially against better competition.
November 30th, 2010 at 10:59 PM ^
So you are saying we haven't improved as a whole, that we aren't on a positive tradjectary? Because the numbers say otherwise.
Comments