The Over and Underperforming, 2010 edition

Submitted by Ryano on

Last year I wrote a couple pieces on trying to measure team performance vs. expectations.  I came up with a system, and used 1999-2008 data to develop a benchmark for comparison.  After the end of the 2009 season, I meant to evaluate the year’s performance, but could not bring myself to revisit the season without weeping uncontrollably.  Now that 2009 is becoming a distant memory, I can finally face my fears.

To briefly summarize my logic: I am using each game’s spread as a means to measure team performance.  Not the team’s performance against the spread, rather whether they win or lose compared to what the spread predicts.  For each team, I am averaging the spreads for each game that had an upset.  For example, if this year Michigan pulls off upsets with spreads of 4 and 6, and loses one where they were favored by 8, in 12 games, the average is -.17.  Note that we are only including games with a line, i.e. no baby seals.  This is important because Michigan’s 2007 numbers do not include the HORROR, which would obviously make a huge difference. 

Using data from 1999-2008, I found that this average follows a pretty good linear trend when plotted vs. the predicted win % for each team.  Comparing variance from the trend line then tells one a little bit about how the teams are performing; teams above the line are beating expectations, below are not matching up:

So for 2009, I compared results versus this same trendline.  We will start with the Big 10:

# of Games:# of games played w/spread
W Upset: sum of spreads in upsets divided by # of games
Win %: actual win % in games w/spread
Pred Win%: percentage of games predicted to win
Plot: W Upset score calculated with linear trendline
Variance: variance from trendline
  
Team
# of Games
W Upset
Win %
Pred Win %
Plot
Variance
Michigan State
12
-1.50
41.7%
66.7%
-0.43
-1.07
Michigan
11
-0.91
36.4%
45.5%
0.10
-1.01
Illinois
11
-0.36
18.2%
36.4%
0.33
-0.69
Minnesota
12
-0.58
41.7%
50.0%
-0.01
-0.57
Indiana
11
0.64
27.3%
9.1%
1.02
-0.38
Purdue
12
0.00
41.7%
41.7%
0.20
-0.20
Penn State
12
-1.08
83.3%
100.0%
-1.27
0.19
Iowa
12
0.00
83.3%
58.3%
-0.22
0.22
Ohio State
13
-0.38
84.6%
76.9%
-0.69
0.31
Wisconsin
12
-0.33
75.0%
75.0%
-0.64
0.31
Northwestern
12
2.67
58.3%
33.3%
0.41
2.26
 
Once again, Michigan State is at the bottom of the pile, with Michigan basically right there with them.  It would appear that … based on this revelation … Michigan did not live up to expectations last year.

Purdue, who has historically the worst numbers in the Big 10, took a giant step into the middle of the pack … however it looks like they are already getting back to true form this year.  OSU is strong as always, and once again Northwestern is at the top by a long shot.  Every year, Northwestern seems to beat expectations, and nobody notices.  I blame it on all that purple; how can anyone take them seriously?

Here is how the rest of the league compares.  Starting with the bottom:

Team # of Games W Upset Win % Pred Win % Plot Variance
Vanderbilt 11 -1.73 9.1% 27.3% 0.56 -2.29
Tulsa 11 -2.64 36.4% 63.6% -0.36 -2.28
Oklahoma 12 -3.25 58.3% 91.7% -1.06 -2.19
North Carolina State 10 -1.40 30.0% 30.0% 0.49 -1.89
Southern California 13 -3.00 69.2% 100.0% -1.27 -1.73
Arkansas State 11 -1.73 27.3% 54.5% -0.13 -1.60
New Mexico 12 -0.50 8.3% 8.3% 1.04 -1.54
Eastern Michigan 12 -0.33 0.0% 8.3% 1.04 -1.37
Ball State 11 -0.55 18.2% 18.2% 0.79 -1.33
San Jose State 11 -0.36 9.1% 18.2% 0.79 -1.15

Vanderbilt just edges Tulsa to take the prize.  Vanderbilt was only expected to win 3 games, and managed to only get 1 (take that, Rice).  It was upset by Miss State and Army.  It’s interesting to see Oklahoma and USC so low in a section usually dominated by bottom feeders.

And here are the over-performers:

Team # of Games W Upset Win % Pred Win % Plot Variance
Idaho 13 2.08 61.5% 23.1% 0.66 1.41
Duke 10 2.20 40.0% 20.0% 0.74 1.46
Alabama 13 0.38 100.0% 92.3% -1.08 1.47
Iowa State 12 1.67 50.0% 41.7% 0.20 1.47
Washington  12 2.42 41.7% 16.7% 0.83 1.59
Boise State 13 0.54 100.0% 92.3% -1.08 1.62
Central Michigan 13 1.08 84.6% 76.9% -0.69 1.77
Brigham Young 13 1.23 84.6% 76.9% -0.69 1.92
Northwestern 12 2.67 58.3% 33.3% 0.41 2.26
Southern Methodist 12 3.25 58.3% 33.3% 0.41 2.84

Southern Methodist handedly takes the crown by winning 3 upsets with spreads over 10 points.  Boise State, who usually is at the top, still isn’t far behind.  Say what you want about them, they always manage to take care of business.  And finally, somehow Bama manages to beat the already lofty expectations for them.

How do these results look over time?  Here is a plot of Michigan, Michigan State, and Ohio State:

Again, recall that this does not include the HORROR.  If it did, one would see that Michigan has pretty much spent the past 5 years not getting it done.  State is very spastic; no one is ever sure when Sparty will Sparty itself.  And OSU has been pretty consistently outperforming since Tressel arrived on the scene.

So what does it all mean?  I don’t know, I was hoping you all could tell me.  It certainly doesn’t help Michigan to have these strong starts followed by a weak Big Ten performance.  That’s what’s hurting MSU in several of these years shown: great start leading up to a loss to Michigan, and implosion afterwards.  For the most part, these numbers confirm what I thought about some of these teams, but there are a few surprises.  Also, Southern Methodist and Northwestern would like your respect.  Please.

Comments

joeyb

September 28th, 2010 at 5:53 PM ^

One issue that I see with this is that spreads change throughout the year due to more information being available. Look Michigan this year. They were thought to be a 6-6 to 7-5 team before the season and now many people think 8-4 is a minimum. Because of that, the spreads have changed. We were an underdog in the ND game, but if we played them 4th in our OOC schedule, we would probably be the favorite because we would be 3-0 and they would be 1-2. If we keep winning, by the end of the season, we could be the favorites in the Wisc game.

The point is that end of season lines are going to be more accurate that pre-season lines and so the former should be weighted more than the latter.

Ryano

September 28th, 2010 at 6:07 PM ^

The lack of information works its way into the spread.  We were an underdog in the ND game, but not by much, specifically because of the lack of information.  Also, keep in mind that this is a dynamic evaluation of performance, not just versus pre-season expectations.

svf

September 28th, 2010 at 7:05 PM ^

If the line got more accurate over time you might expect to see the oscillations die out.  My first thought was that teams regress to the mean.  A peak (teams outperforming the spread) causes the next year's line to be too favorable.  Teams play a little closer to average (regressing to the mean) which puts them below expectations, hence the dip.  The cycle then repeats in reverse.

What's interesting is that it looks periodic with a 3-5 year period (length of a class, duh?).  It would be pretty cool to see this data over a longer time period, mabye w/some fourier analysis to extract the frequencies. 

oakapple

September 28th, 2010 at 9:26 PM ^

As I recall, throughout the Lloyd Carr era Michigan was generally reckoned a sucker bet: they tended not to beat the spread. I don't know if that tendency was true in the Bo or Mo eras, but it was perceived broadly to be true under Carr.

It could be that Carr routinely underperformed—many Michigan fans sure thought so. It could also be that Michigan attracted a lot of action because of its large fan base, and to compensate for that, Vegas had to give the Wolverines a larger spread than they actually deserved. The latter is widely known to be true of Notre Dame: they are practically always favored by more than they should be (or the underdog by less than they should be), simply because they are so popular.

In the first two years of the Rodriguez era, without question, Michigan has had trouble "beating the guys it should beat." Nobody, not even Michigan haters, predicted 3-9 followed by 6-6.

OSUstudentUofMfan

September 28th, 2010 at 10:09 PM ^

This seems like a pretty elaborate way of trying to understand whether a team wins when they are expected to. How is this an improvement from simply looking at a team's record vs. the spread? If they cover regularly it seems like they meet expectations and if they don't cover they are likely failing to meet expectations. I'd like to see your measure compared to data on spread covers. I bet they tell very similar stories.

Ryano

September 28th, 2010 at 10:52 PM ^

They are very different.  For example, Northwestern does very well in this analysis, but against the spread, they were very mediocre.  I came up with this because most of us don't really care about performance against the spread, we only care about W's and L's.  For example, Central Florida beat the spread regularly last season, but none of that translated into any upsets.