Expected Wins Model for 2011 Football Season
I briefly mentioned this before the start of the last season, but I have built a model for Michigan football wins, based on tracking data since the 1990 season. It is not that sophisticated, and I don't have any charts to put up on the front page, but I thought it was relevant because it projected us for 7.3 wins last season (clearly impossible, but I didn't round for precision's sake).
I won't get into all of the nuts and bolts, but the model's "expected wins" is based solely on the quality of play from all the major positions on a football team: QB, RB, WR/TE, OL, DL, LB, DB, and Special Teams. Each category receives a +1 (good), 0 (neutral), or -1 (bad). Below is a chart summarizing historical wins versus expected wins since 1997:
| Year | Wins | Expected Wins | Difference |
| 2011 | 8.2 | ||
| 2010 | 7 | 7 | (0.3) |
| 2009 | 5 | 6 | (0.7) |
| 2008 | 3 | 2 | 0.5 |
| 2007 | 9 | 10 | (0.8) |
| 2006 | 11 | 12 | (1.3) |
| 2005 | 7 | 10 | (2.8) |
| 2004 | 9 | 11 | (1.6) |
| 2003 | 10 | 10 | 0.2 |
| 2002 | 10 | 8 | 1.8 |
| 2001 | 8 | 8 | (0.2) |
| 2000 | 9 | 8 | 0.8 |
| 1999 | 10 | 11 | (0.6) |
| 1998 | 10 | 10 | 0.2 |
| 1997 | 12 | 11 | 0.5 |
As you can see, the model has been accurate in predicting the number of wins to within one win for 10 out of the last 14 seasons. The four years that stand out as abberations:
- 2002: Breakout years from Chris Perry and Braylon Edwards. Really solid year from Navarre. Beat expectations.
- 2004-2006: My model is a little tough on Lloyd and his coaching staff for these years. The model assumes that based on the quality of players, we should have had more wins during the winding down of Lloyd's tenure at coach. I could be being a bit harsh on Lloyd here, but I feel that many posters would agree - and the prior years are clearly not "anti-Lloyd" (he had a net positive influence in four of the previous six years).
Enough of the background. Where does the 8.2 come from in the 2011 column? Well, it's all based around what we'll inherit from the 2010 team (again, this is based on players only - the coaching shows up in the variance). I assume that "+1" will remain for QB/WR/OL, and the "-1" for DBs and Special Teams will go to net zeroes. Based on a total of +3, you get to 8.2 expected wins for next year.
Now, to take a look at the best and worst case scenarios.
- Best case: One of the RBs steps up and gets a stranglehold on the starting spot (+1). DL improves so much under a 4-3 scheme that it also becomes a +1. LB, DB, and ST play all remain a net zero. Under this scenario, the model projects +5, or 9.8 wins.
- Worst case: We don't sign a kicker in this class and ST remains -1. The young DBs remain a -1 as they are confused by a new scheme and are still relatively raw. No one is able to fill Mouton's role, and LB takes a dive to a -1. All offensive rankings stay the same. Under this scenario, the model projects breakeven, or 5.7 wins.
I know that range is pretty tight, but I believe in it. I do believe Best Case is more reasonable than Worst Case, because we've brough in Hoke-Mattison and to expect the defense to actually take a step back would be a hard pill to swallow. Also, the model lends itself to being more accurate in years where there isn't great attrition (such as this year).
I'm interested to get your guys' thoughts as I try to make some improvements and what categories I can include. Hopefully this is at least interesting for a Wednesday morning/mid-morning thread.
January 19th, 2011 at 11:46 AM ^
Right on! Expectations are high.
"This is Michigan, for God's sakes!"
January 19th, 2011 at 11:30 AM ^
Seems to me that if Hoke wants to get back to a more powerish running game Stephen Hopkins is the guy. Or perhaps Toussiant. The other guys are a bit too small, and those two are big with some speed. They should hire Mike Hart as a consultant on not fumbling.
January 19th, 2011 at 11:34 AM ^
Or a consultant on being Mike Hart. I'd sign up for that class.
January 19th, 2011 at 11:40 AM ^
Don't forget about Mike Cox. He's shown great speed, balance, and the ability to truck someone. I don't think we will have a hard time finding "the guy" this year.
January 19th, 2011 at 11:43 AM ^
I don't understand why people think a power running game requires a big bruising tailback. Hoke's best RB last year was a 5-10, 175 Freshman who flourished in his offense. V. Smith is our only RB that is shorter, and he actually weighs more than that. Fitz, Shaw, Smith, and backs like Dee Hart :( can all suceed in a pro-style offense. Not to mention, we still don't even know what type of scheme we will be running. Hoke has said all along he plans to adapt to his players, so we might see plenty of spread aspects in his offense.
Hopkins will do fine, however, the notion that smaller, speedy RBs can't be a part of this offense is simply not true.
January 19th, 2011 at 11:42 AM ^
If by power running game you mean Wisconsin with Clay and Ball then sure. But not all power running teams are equal. We dont have to have huge backs to have a power running team. Hopkins, Cox, Fitz, Shaw, etc could all be very good players in a power running scheme.
All power running really means is that you have a fullback and TE in the game most times and run downhill between the tackles. That could be done by a smaller back as well.
January 19th, 2011 at 11:51 AM ^
"hello everyone, and welcome to the michigan anti-fumbling clinic. my name is mike hart, and today im going to teach you how to not cough up the rock.
step number one: tightly hold onto the ball.
step number two: dont let go.
any questions? no?
well that concludes our clinic. thank you."
January 19th, 2011 at 11:54 AM ^
Why does everyone think Hoke likes big running backs? Hillman is a little darter.
January 19th, 2011 at 11:59 AM ^
Silly chunkums, facts have no place on the internets. When will you ever learn?
January 19th, 2011 at 12:25 PM ^
January 19th, 2011 at 12:38 PM ^
To each his own, but Shaw seems very much like a running back to me. Heck, that run against OSU was downright sexy. If only he could actually get on the field occasionally...
January 19th, 2011 at 11:33 AM ^
But it's not 9-3. Burn him!
Interesting that it's so accurate, though. I'd be interested to read all the inputs and how it all translates to your prediction. Nice work.
January 19th, 2011 at 11:33 AM ^
I'd be curious to see how your modeling relates to some of the more complex models out there, with the stipulation that more complex =/= better.
Just a question, did you begin your position rankings (+1,-1,0) before the season since 1997? or was it retroactive based on preseason hype? Just trying to understand your methods a little
Thanks
January 19th, 2011 at 11:42 AM ^
The current model has been the same since 2008. However, I've been collecting data points for it since I was a student (beginning in 2002).
So, admittedly, 2001 and prior are all based on my research of team stats, team rankings, and NFL draft picks (e.g., "Player X gets drafted in round 1, Player X is a LB, LBs are expected to take a dip next year").
Clearly It'd be great if I was collecting data for the past twenty years, but unfortunately I have not.
January 19th, 2011 at 11:51 AM ^
I was just wondering, my "predictions" preseason vs. postseason would be very different. that's why I was curious about the dataset. Thanks for the answer
January 19th, 2011 at 11:59 AM ^
My main question here is exactly when you collect the data for previous systems. If it's based on expectations/predictions going into the season, then there's no problem. However, if it's based on evaluating the players after the season, the model itself isn't predicting how the season would go in those years like you're trying to do now, it's trying to estimate how well the season should have gone, based on how the players actually did, which is much easier (though still impressive).
From your statement, I think your 2001 and before data is collected from actual performance rather than expectation so the success of those predictions is less impressive. Is the data from 2002 and forward collected before or after the season?
January 19th, 2011 at 12:08 PM ^
Clearly I shouldn't have spent so long typing up my previous response since there were 2 other posters saying the same thing.
As a researcher in Artificial Intelligence (which is basically what you're doing here, though a statistician would claim it as well--there's a lot of overlap), I want to add one other quibble with your results: since the current model was developed in 2008, you were presumably 'training' the model on the previous years--adjusting the model to come up with one that best explained those years. So the only real test of your model are the results it's gotten since it was developed (or on other data you added since it was developed). So the only true evaluations are either from 2008-2010 or just 2009 and 2010 (depending on when you developed it in 2008--whether all or even some of the results from that year were known at the time).
January 19th, 2011 at 1:16 PM ^
I get what you're saying and I was waiting for someone to say it.
You could absolutely assume that I "trained" this model, but I was very careful not to juke any of the stats when creating it.
You will just have to take my word for it that the collection of data since I was an M fan (2002-Current) is in line with the data I collected for previous years (1990-2001). I haven't juked anything and the creation of this idea was both out of boredom and a general interest to create something that was good at predicting M wins.
That's all I care about - being good at predicting wins. And clearly there are some stats snobs (which I expected) on this board that are ready to pee on my parade (or be the monkey who throws poop). To them I say - wait and see... wait and see... This model has already made me money betting on the Michigan O/U (got in at 6.5 this year). So here's to hoping there will be more to come in 2011. If not, then you guys can all call me an idiot.
January 19th, 2011 at 2:08 PM ^
Just out of curiosity, can you clarifiy one point: in order to grade the quality of each of the positions for each season, did you use the prior season's statistics, or the end-of-year statistics? For instance, to predict the 2001 season's wins, did you use subjective ratings of the players from the beginning (using 2000's data) or the end of the season (using 2001's data)? Not trying to be a stats snob; just trying to figure out what the model is actually explaining.
January 19th, 2011 at 11:49 AM ^
I like the data and impressed with the accuracy. I'm curious to know when you first started trying to make it predictive and how you ended up with the + and -. If you got them from judging performace after the fact, that makes sense as to how it seems to be so accurate. I don't have a problem with that, just interesting to know. I don't know how you'd be able to go back and get a good feel for hype for past seasons. Maybe returning starters / percentage of tackes or percentage of offense or something like that. Anyway, I know a lot of people don't necessarily want to know all of the methods, but a lot of us are probably legitimately interested.
January 19th, 2011 at 11:50 AM ^
Question to the OPer? How does the coaching change factor into your position ratings? Which groups would you have changed if RR had stayed?
January 19th, 2011 at 11:34 AM ^
does this include a bowl game?
January 19th, 2011 at 11:40 AM ^
My apologies, I forgot to mention this before - but no bowl game. Clearly there is an error in 2006 that should read (1.0) instead of (1.3).
January 19th, 2011 at 11:37 AM ^
I definitely think this is interesting, especially since it allows me to think that the 2011 season is right around the corner. Your model seems to be pretty accurate though.
January 19th, 2011 at 11:45 AM ^
I would have expected 8 wins going into 2011 had there been no coaching turmoil (maybe even 9?). However, with all the changes, the need to learn new playbooks and formations, new personalities and general viewpoints, I'm not sure if I feel comfortable expecting even 7 wins. For my own physical well-being, I'm going to go into September expecting 6 wins . . .
January 19th, 2011 at 11:40 AM ^
is about what I'm anticipating. The Notre Dame game will be a bellwether for the rest of the season.
January 19th, 2011 at 12:47 PM ^
Usually is. I remember in 2006, after crushing them and coming off the crappy 2005 campaign, thinking that we were in for something special. Turns out that turned out to be right.
January 19th, 2011 at 11:42 AM ^
I found this interesting, and I mean no disrespect, but since there is no objective data involved (e.g. returning starters, previous record, etc.) this seems like less of a "model" and more just a means of structuring your subjective guess. And I think it is a good means for doing that. But aren't you, with this model, essentially just carefully walking through your subjective belief in how good we'll be?
What you have is a model that runs not on data, but on a series of your subjective judgements. That is fine, and interesting. I would actually be curious to see how you rated our position groups through the years. It would be hard for me even to remember the cast of characters by year...
January 19th, 2011 at 11:47 AM ^
This comment is exactly spot on. There really is not modeling going on in the model.
January 19th, 2011 at 12:56 PM ^
Please see my comment below. What makes you a model (NNTM) expert? Are you still pissed because I said Mystery Science Theater 3000 sucks? (hint: it still does)
January 19th, 2011 at 12:04 PM ^
I was going to post some very similar comments, but this sums it up better than I could have.
Some other thoughts from a stats nerd:
Was a regression model (or something similar) used to model the subjective ratings vs the actual win/loss totals? If so, then the accuracy of the model (being off by only a game each season, on average) isn't that impressive; any good model should be fairly close used retrospectively unless there is a lot of unexplained variance. Perhaps that is an interesting conclusion in itself.
If regression was used, it would be interesting to compare the estimated coefficients for each of the positions. For instance: is a +1 for QB worth more than a +1 for WR? It might also be worth investigating whether moving from -1 to 0 is worth more than moving from 0 to +1.
Since you presumably assigned ratings retrospectively for the early seasons, I wonder how much your subjective ratings were affected by the win/loss total at the end of the season.
January 19th, 2011 at 12:15 PM ^
I was thinking the same thing. But that almost makes it more impressive that the predictions were so close year after year when they were only based on one person's subjective analysis.
January 19th, 2011 at 12:24 PM ^
But the "subjective analysis" can be made to fit the W-L result, calling into question the whole subjective part of it. This really isn't a model, it's just data fitting.
January 19th, 2011 at 1:03 PM ^
You are the monkey throwing feces at my parade. I hate you.
January 19th, 2011 at 12:55 PM ^
You've pretty much nailed it, but if I am allowed a retort...
- I work in corporate finance (M&A), and basically anything we put together in excel that has formulas, and is linked to other cells is a "model". What I have, while loosely based on qualitative observations, still uses formulas and calculations that a regular financial model would use. I am clearly not using a huge statistical database and running R^2's here, but I am still living within the financial model realm.
- The model's ultimate output (projected wins) is based on subjective data (e.g. "We had Henne in 2006, he played good, that means QBs get a +1"). However, the determination of whether someone was "good" is entirely based on stats. I didn't just think "Oh in 1996 we had Charles Woodson, so DBs are automatically a +1". I did the research on passing yards allowed, their rank in terms of passing yards allowed, etc.
Now for some of the posts below - I think it's pretty clear that my model isn't front-page worthy material (unless my 2011 prediction is similar to my 2010 prediction in terms of accuracy). I'm not running some version of SPSS here that would confuse someone who got an A in Stats 350. What I am running here is just a simple, "how did each position grade out in a season and how does that relate to expected win total".
Anyway, I appreciate the input. When I get a second I'll go over all the comments and start incorporating somethings into the model. My final prediction will be issued in August - but I don't expect it to change much (barring a mass exodus including Denard).
January 19th, 2011 at 11:44 AM ^
I have a hard time believing that you legitimately predicted the team would only win 2 games in 2008 though.
January 19th, 2011 at 11:44 AM ^
No matter the win total, if we can actually beat tsio, MSU, Iowa, and/or Nebraska, then things will be on the right track. Beating a team of substance will do wonders for my mental condition this year.
January 19th, 2011 at 2:50 PM ^
As well as your liver, or scratch that it might be worse. Who cares anyway.
January 19th, 2011 at 11:46 AM ^
It’s one thing to say, “Before the 1997 season, my model predicted 11 wins.”
It’s quite another thing to say, “My model, had it existed then, would have predicted 11 wins.”
Have you actually been making these predictions, every year, since 1997? Or has the model been developed retrospectively, based on experience?
January 19th, 2011 at 11:52 AM ^
Yep. I'll be interested to see what happens this season, and how close the prediction is without any tweaking.
At this point, it's essentially a best-fit curve based on the data.
(Unless the model was generated in complete isolation from the actual data, in which case the small sample size would make it less interesting anyway.)
To the OP: Have you tried using your model against other teams in the country? If you could show that it's highly effective in a large number of tests, that would hold a lot more weight.
January 19th, 2011 at 11:48 AM ^
I used this
and these
to simulate next season. Things were going great until Denard and half the O-line fell over for a full quarter during the Iowa game. If we can avoid that, I'm thinking B1G championship!
January 19th, 2011 at 11:59 AM ^
a lever/slider that adjusts for the vibration. If the coaches can find it, and tone it down a notch, I agree with your assessment. B1G championship!
January 19th, 2011 at 12:13 PM ^
If we had only used that strategy last season!
Trotting out two offenses instead of an offense and defense might have been an upgrade...
January 19th, 2011 at 12:39 PM ^
In my simulations, we recorded 0 field goals. Not too far off, I know, but still a regression.
Good news? We also allowed no field goals! Go GMAT!
January 19th, 2011 at 11:51 AM ^
Your analysis is interesting. 8 wins seems about right given the maturity of the team offset by scheme changes.
Do you take into account at all the coaching performance? Do you have similary +1, 0, -1 for the coaches? If so, what did you assume for last year and what are your assumptions for next year?
January 19th, 2011 at 11:52 AM ^
I like how you make it sound like their is some kind of science/math involved in your prediction. It's an educated guess just like everyone else has. I mean i personally think that we can easily can win nine games, but im not going to tell you its anything other than a huge Michigan fan with huge expectations.
January 19th, 2011 at 1:26 PM ^
. . .
January 19th, 2011 at 11:53 AM ^
| Opponent | Prediction |
|---|---|
| Western Michigan | W |
| Notre Dame | L |
| Eastern Michigan | W |
| San Diego State | W |
| Minnesota | W |
| @ Northwestern | W |
| @ Michigan State | L |
| Purdue | W |
| @ Iowa | W |
| @ Illinois | W |
| Nebraska | L |
| TSIO | W |
Michigan loses tough ones to ND, MSU, & NEB but finally gets the job done with TSIO at home to finish off the regular season at 9-3.