I like reading these diaries.
Week #7 Statistics and FEI Prediction for MSU
Prediction for MSU: The FEI Forecast for this Saturday is Michigan State 20 – Michigan 18 with a 56% Probable Win Expectation for Sparty? Basically a toss up but, like the Purdue game, FEI is wrong and Michigan wins the game 24 – 13. As you can see below, FEI has been really schizophrenic about M this year. M at #47 is the lowest ranked 4 win AQ team in FEI and MSU at #27 is the highest ranked 3 loss team in FEI.
Fremeau Efficiency Index: Week #7 is significant because all remnants of preseason projected data is removed from the formulas and all data represents 2012 games only. In addition both offense and defense efficiency are now opponent-adjusted and are referenced as OFEI and DFEI (up until this point OE and DE were just raw numbers).
That said, WTF!!!
After a 45-0 drubbing of an admittedly weak Illinois team, FEI blasted the overall rating to #47 (from #24 last week) and pummeled the offense efficiency to #63 (from #40 last week). Defense efficiency improved to #27 (from #33 last week).
The S&P Ratings (Also from Football Outsiders) is a play based analysis (rather than possession based) and M is ranked #9 overall, #7 in offense, and #22 in defense.
The FEI is a drive based analysis considering each of the nearly 20,000 drives each year in FBS college football. The data is filtered to eliminate garbage time (at the half or end of game) and is adjusted for opponent. A team is rewarded for playing well against good teams (win or lose) and is punished more severely for playing poorly against bad teams than it is rewarded for playing well against bad teams.
National Rankings: The rankings for Week #7 offense and defense are based on scoring (yardage statistics are inherently flawed). These are simply raw numbers without any adjustments for opponent, garbage time, or anything else. The data is from TeamRankings and includes only games between two FBS teams.
FEI Details: Here are the FEI numbers for Michigan and their opponent ( Football Outsiders FEI ).
Points Per Possession: The offense continued to rack up the points and the defense recorded their first shutout. Cumulative PPPo is 2.9 for the offense and 1.5 for the defense. M finished 2011 outscoring opponents by almost a 2:1 margin with PPPo for offense of 2.8 and defense of 1.4. The 2 charts show the raw data for offense and defense with the number of possessions adjusted for "kneel downs" at the half or end-of-game (maximum deduction = 2).
Using Scoring Offense and Scoring Defense National Rankings for the past 5 years (FBS AQ teams only), this table shows the percentage of teams that finish the season with a +WLM and a +5 WLM. For example, teams that finished in the Top 40 in both offense and defense had a 100% chance to be +WLM and an 82% chance to be +5 WLM (9-4 or better).
Had been poking around Football Outsiders yesterday, befuddled by our rankings drop, waiting for Enjoy Life's analysis. Thanks for the context.
"After a 45-0 drubbing of an admittedly weak Illinois team, FEI blasted the overall rating to #47 (from #24 last week) and pummeled the offense efficiency to #63 (from #40 last week)."
We go from 24 to 47, after winning 45-0? What would have happened if we won 35-10? We would be #95? It's useless (at the very least until the last 1-3 games of the season).
I also don't think that the FEI takes into account how a team changes over the course of the season. This just doesn't seem to be the same team that opened the season and lost to Alabama and Notre Dame the way they did.
I'd like to see some historical data on how accurate FEI predictions have turned out to be.
According to the FEI folks:
"Half of all teams finished within 10 ranking spots of their preseason FEI projection last year. 26 percent finished within five ranking spots. 20 percent finished within two ranking spots."
"the correlation of FEI projections to FEI final ratings at the end of the year is .785."
BUT, they also say,
"Projecting team ratings and game outcomes for 124 teams is relatively easy at the macro level. For the most part, the top teams consistently dominate college football and are easy to identify through the projection factors"
M was ranked #23 in the FEI preseason with 8 FBS wins projected.
Thanks! At first glance, a correlation coefficient of 0.785 looks very strong to me, but that's probably because I work with biological data (read: mass chaos) in which values of 0.4 can be very promising. In the context of college football, it's kind of like you mentioned - achieving a low correlation would probably be quite difficult considering the large-scale consistencies.
Additionally, looking at end-of-season rankings alone kind of smooths out a lot of stochasticity over the course of a season to generate an overall indicator of performance. I'm curious what FEI's rate of success is in predicting results of individual games (which is further complicated because the amount of predictive data increases as the season progresses).
I'd like to rearrange the more detailed breakdown you cited as well - let's start with the 20% that ended within 2 spots of their pre-season prediction. I imagine this chunk of data represents the perennial elites and FBS basement dwellers that are the easiest to guess. 26% ended within 5 spots, but this group would actually include the previous 20% we just talked about. That means that the difference - 6% - of teams ended 3-5 spots from their prediction. That's a pretty sharp drop-off, reflecting that they are a lot harder to dial in. The next group, which was called "half" of teams ending within 10 spots, could then be alternatively viewed as 24% of teams ending 6-10 spots from their prediction. That's not too bad, really.
Enjoy Life did post the FEI results (which admittedly seem flawed), but also took the time to post several other categories of statistics coupled with a few graphs and tables, all while putting some context into the numbers and whatnot. I'm not sure I would call these diaries pointless just because one category he tracks has some funky outputs.
So how did we drop that far after a 45-0 win and sparty is well above us after a loss at home? Ok, this FEI stuff seems legit.
In pure yards per play with no opponent adjusting, our offense is 13th and defense is 12. Might be reversed, but yeah...that's good
Our loss to Alabama or our win v AFA?
My guess is the reason for the drop is because preseason projections were finally completely removed from having any impact. So now, Illinois, AF, and Purdue wins are probably worth less than they were because no one figured they would be as bad as they have looked lately. The drop isn't because Michigan beat Illinois by only 45.
I can't figure out how people are confused by this if they read the OP.
That said, I find it kind of odd that we'd drop by that much. Did FEI really have Illinois, AF, and Purdue overrated that much more than they must've had ND underrated?