Michigan Under Harbaugh - Three Buckets Analysis

Submitted by bklein09 on November 2nd, 2021 at 4:57 PM

Like many others on the blog, I've spent the past few days pondering my Michigan fandom, and more specifically Harbaugh's tenure here. In trying to encapsulate his performance as Michigan's coach, I decided to take what I'm calling a "Three Buckets Approach".

In short, I used historical Vegas point spreads (Source: oddsshark.com) to put each Michigan game into one of three categories/buckets:

Clear Favorite / Should Win: = or > 7pt favorite

Toss Up: < 7pt favorite or underdog

Clear Underdog / Should Lose: = or > 7pt dog

Before we look at the data, I think it's important to point out some limitations to this method:

  • Point spreads are used for betting and aren't meant to assign win probabilities. They only provide a snapshot of how the teams are being evaluated in the days leading up to the game. Sometimes those snapshots are wildly off from how things end up at the end of the year. For those reasons, this analysis is somewhat limited.
  • Yes, the 7pt spread cutoff for the categories is completely arbitrary. However, I think it feels about right practically and spiritually. 
  • Full disclosure, I went into this analysis hoping to make myself feel better about the Harbaugh era. In general, I'm reluctant to see Michigan start over with a new coach, and I have concerns that things could get worse if/when we move on. However, the numbers may have had the opposite effect, as you'll see below.

The Numbers

* indicates Michigan losses

Clear Favorite / Should Win: = or > 7pt favorite

2015 8-1 (OrSt, UNLV, BYU, @MD, NW, MSU*, @Minn, Rut, @Indy)

2016 10-1 (Hawaii, UCF, Colo, PSU, Wisc, @Rut, Illini, @MSU, MD, @Iowa*, Indy)

2017 7-2 (Cincy, AF, @Purdue, MSU*, @Indy, Rut, Minn, @MD, SC*)

2018 10-0 (WMU, SMU, Neb, @NW, MD, Wisc, @MSU, PSU, @Rut, Indy)

2019 7-0 (MTSU, Army, Rut, @Illini, @MD, MSU, @Indy)

2020 1-1 (MSU*, @Rut)

2021 4-0 (WMU, NIU, Rut, NW)

Total: 47-5 (.904)

Takeaways:

  • 3 of 5 losses as clear favorites are to MSU.
  • A bit surprising Michigan was clear favorites vs Wisc (2016, 2018) and PSU (2016).

Toss Up: < 7pt favorite or underdog

2015 2-2 (@Utah*, @PSU, OSU*, Fla

2016 0-2 (@OSU*, FSU*)

2017 1-2 (Fla, @PSU*, @Wisc*)

2018 0-3 (@ND*, @OSU*, Fla*)

2019 2-1 (@Wisc*, Iowa, ND)

2020 1-2 (@Minn, @Indy*, @PSU*)

2021 3-1 (Wash, @Wisc, @Neb, @MSU*)

Total: 9-13 (.409)

Takeaways:

  • Home: 3-1 (not bad!)
  • Neutral: 2-2 (could be worse!)
  • Road: 4-10 (woof...)

Clear Underdog / Should Lose: = or > 7pt dog

2015 0-0

2016 0-0

2017 0-1 (OSU*)

2018 0-0

2019 0-3 (@PSU*, OSU*, Bama*)

2020 0-1 (Wisc*)

2021 0-0

Total: 0-5 (.000)

Takeaways:

  • Michigan is rarely a clear underdog, but when we are, it's a guaranteed loss.

Overall Takeaways and Future Directions:

  • Under Harbaugh, Michigan has been clear favorites in 65.8% of their games. Harbaugh wins 90+% of those, but the losses are brutal (MSUx3, Iowa in 2016). 
  • If the spread is less than 7 points in either direction, it's a likely loss. If that game is on the road, Michigan wins just 1 in 4 times.
  • When I have time, I'd love to do this same analysis for Hoke, RR, Carr, etc. to see how it compares. Anyone know a good source for very old CFB spreads?
  • Based on these data, I'm less concerned about taking a huge step back when we decide to change coaches. Sure, it's definitely possible, but there are certainly lots of options out there who could replicate the results shown above. 
  • Yes, all this is moot because we're going to (probably) win 9-10 games this year and keep the staff for 2022. And I'm fine with that. 

Comments

Vasav

November 2nd, 2021 at 6:25 PM ^

Thanks OP this is a neat analysis. one thing to note - the better we are, the more often we are going to be a favorite. One  thing I remember regularly with Hoke and Rod was finding ourselves in dogfights with the likes of UConn, UMass, and Toledo - yes, we won 2 of those, so I'm sure it will appear as winning as a favorite. But the only time I remember something similar with Harbaugh was Army in 2019. (Also - losing to MSU was almost guaranteed under those 2, and it was rarely close - 2009 I think was the only close one)

bklein09

November 2nd, 2021 at 8:26 PM ^

You make some really good points, especially in regards to point spreads being “self confirming” in a way.

It’s definitely not the perfect method to break it down, but my intent with this analysis was to remove/ignore the “feel” of wins/losses. Rather, I wanted to focus on more of an NFL “just win baby” perspective.

What winning percentage would be considered  successful by the fanbase when Michigan is favored? What about toss ups? And how often should Michigan win when they are a big dog?

I think Harbaugh’s percentage as a favorite is pretty solid. But the fact that 3 of those losses are to MSU makes it feel a lot worse.

Toss ups are another story. 40% isn’t going to cut it. It would probably have to be closer to 60%, and even then, people would still be grumpy if those extra wins weren’t against OSU.

The sample size as an underdog is too small to mean much IMO. 

charblue.

November 2nd, 2021 at 7:19 PM ^

Here's a real solid take: "Michigan is rarely an underdog, but when we are, it's a a guaranteed loss." 

Gee, confirmation of the obvious, rationalization of regression to the mean, acknowledgment that what we expect, frequently occurs when our perception is based on accurate understanding of clear win-loss opportunity.

The argument you are trying to make only emphasizes the fact that fans of every successful program expect to win more often than not, usually and virtually always when favored. This often happens whether we are paying attention to the details of probability or not especially in cases where fear of losing outweighs the expectation or unlikelihood of winning.

In 1969, there was scant chance that Michigan would beat Ohio State in a year when most football pundits considered the Buckeyes talented enough to play in the NFL. And yet, we won, cementing Bo Schembechler's career at Michigan. It was the greatest upset win in program history until another gateway win over the Buckeyes in 1997, the greatest win for me in my Michigan fandom. We subsequently won the Rose Bowl and a divided national championship.

The point is, it's glorious to win when you absolutely, positively shouldn't. It's fantastic when you pull the ultmate upset. But when you are favored most of the time, then achieve success on that level as a pedestrian response to expectation, it's only revealing when you don't win with attitude against that perception. Times change, things change, outcomes change, the world moves on from your past worldview. Evolve. Adapt. We are, deal with it. 

 

bklein09

November 2nd, 2021 at 8:32 PM ^

That point about being an underdog was a bit tongue in cheek. Captain obvious, I know. 

In reality, five games is too small a sample size  to say much about those games.

It would be interesting to know how often teams win as 7pt+ dogs across CFB. If I had to guess it’s maybe 10-15%?

blue in dc

November 2nd, 2021 at 8:08 PM ^

Thanks for doing this.   As others have pointed out, being a clear favorite in almost 2/3 of your games is not a given.   We like to think that anyone we would reasonably hire could get us to that point, but there is plenty of evidence both at Michigan and other schools that is not a given.

The MSU lost sucked, but I would argue that We are much closer to 201516 Harbaugh than we are to Harbaugh 2017 to 2020 Harbaugh.  2015 and 2016 Harbaugh was never a clear underdog.   Get to that point on a consistent basis and you solve the problem of never losing when you are a clear underdog.
 
Time will tell, but on the defensive side of the ball, I think Macdonald will improve.   I have much less confidence in Gattis, but I also think he is likely to move on after this season.  It is very clear that Harbaugh has taken away some of his responsibility and I can’t believe he is happy.

bklein09

November 2nd, 2021 at 8:43 PM ^

You make a good point that consistently being favored reflects well on what Harbaugh has accomplished here.

He’s steadied the ship and gotten us back to Michigan’s historical baseline performance.

Regarding this season, I tend to agree with you. We’re currently 3-1 in toss up games with maybe 3 left to play (@PSU, OSU, Bowl). If we can manage to win 2 of those, it would almost certainly be Michigan’s most successful season under Harbaugh. 

mbrummer

November 2nd, 2021 at 8:56 PM ^

Well FYI its been true for 40 years.  Spreads are always higher for Michigan.. Mainly because we have a huge fanbase who bets stupidly.

The goal of the spread is to have equal amounts on each team.  It's not entirely predictive, nor is it the goal

Its just like the Yankees.  

bklein09

November 2nd, 2021 at 9:06 PM ^

You’re correct, but to have equal amounts bet on each team you’d have to set a spread that seems at least somewhat predictive right? 

Even if you adjusted all Michigan’s spreads down by 3 points or so, it wouldn’t change these numbers much. It may make the toss up record look a bit better, but the underdog record even worse. Overall, the narrative (and reality) wouldn’t really change. 

Newton Gimmick

November 3rd, 2021 at 10:28 AM ^

Spreads are always higher for Michigan.. Mainly because we have a huge fanbase who bets stupidly.

I hear this a lot -- mostly by a few select pundits who hate Michigan -- but it never made sense to me for a few reasons:  

Do all teams with large fanbases have a poor record vs the spread?  OSU has a large fanbase as well but they have a great record ATS in recent years.

Do we know that Michigan fans bet on Michigan, or is that pure assumption?  If anything I'd prefer an emotional hedge against my preferred fan outcome.  

But most of all, wouldn't sharps keep hammering Michigan's opponents as a corrective to any (real or imagined) fan betting that moved the line?  It would seem that if the line was always skewed toward Michigan (and ND, etc) we could all retire by regularly betting against Michigan.  Why doesn't literally everyone do that?

Erik_in_Dayton

November 2nd, 2021 at 11:50 PM ^

I suppose I'd have to think more about this, but it seems to match what I believe most of us see in Harbaugh: he recruits reasonably well, he prepares players to play fundamentally sound football, his teams execute fairly well, but he is not going outwit opponents with a creative gameplan or motivate his guys to play beyond themselves. So he's often favored and generally able to beat teams that lack Michigan's recruiting and player development, but he never stuns anyone. If you've got better players than him, you win (as is true for most coaches).

This is admittedly an account of a coach that many programs would love to be able to give of their head man. I am not for firing Harbaugh. But I think that he's going to have to upset OSU at some point.

 

Collateral Whiz

November 3rd, 2021 at 4:56 AM ^

Nice post OP.  I did some similar research a few years back also using OddShark looking at all the Power 5 conference teams, and while it's a bit dated, I think it's interesting so I'll share some of the data here.  

My research went from the 2008-2017 seasons, or the last 30 games dating back from the end of the 2017 season (the database wouldn't show more than that at that time for this search).  At that time Michigan was one of only four teams to not win single game as more than a 7 point underdog, and they went 0-12 in that span.  The others were Tennessee (0-30), Alabama (0-1, yup only a major underdog one time in 10 years!), and LSU (0-5).  So, the inability to win as big underdogs has been a problem going back through the last 3 coaches, not just Harbaugh.  Some records of note as being quite good as big underdogs were Ohio State (2-4), Michigan State (4-13), Minnesota (8-22) and Northwestern (11-19!).  

Michigan was good as big favorites, but terrible in toss up games (toss up defined the same way the OP did them).  They were 19-36 in toss up games, and I found a website that gave the likelihood of wins based on pre game spreads, so then calculated that Michigan was expected to win 27.54 of those 55 games (I had to divide up games into more specific spreads to get that number).  That 8.54 games under expectations was the worst of all Power 5 teams with only Virginia at 8.35 games under expectation coming close.  No other team was at more than 6 games under expectations.  I think there is something to Michigan fans altering betting lines with over optimism (I'm a fairly optimistic fan, but I don't get how people are willing to put money behind that!), but at the same time Vegas is not going to alter the spreads a crazy amount lest they get burnt. These data tell me it's pretty clear that Michigan football struggles against the weight of expectations has been a problem, and continues to be a problem under Harbaugh (duh doi!)  

I wanted to share another set of data as well since you mentioned comparing Harbaugh's record with Carr's.  It's not quite the same as spreads, but I looked at Harbaugh's record vs teams in certain ranking ranges vs. Carr's, and I thought it was quite enlightening.  I used Sports reference's postseason Simple Rating System to get the rankings.  Oh, also I did this after the 2019 season so this doesn't include the last two years: 

Carr was great against top 25 teams, but really struggled against the middle of the pack teams.  That was the frustrating part of his tenure - we would look great in knocking off a top team, then play tight in a game against inferior competition.  Harbaugh is pretty much the opposite.  Terrible against top teams, but very rarely do his teams lose to inferior competition (last year was the exception, but that was a weird year).  He's steady Eddie.  The problem for Harbaugh is the schedule has been so skewed to playing Top 10 teams, with him having to face over 3 of these teams a year.  I can't imagine any other team in the country has faced that many top ten teams without a single conference championship game or being in the playoffs. 

Okay, that's enough.  What I thought was going to take me 10 minutes to write out just took me an hour. Hopefully someone enjoys this data dump.  

ca_prophet

November 3rd, 2021 at 6:27 AM ^

As you note, the issue with using point spreads is that the goal of the spread is to get equal amounts bet on either side.  This introduces distortion when actively-betting fanbases are involved.

One way to recast the effects of the distortion is to compare them to the average W-L record for each "bucket" e.g. if Harbaugh wins 2/3rds of his +7 games and Michigan historically wins 2/3rds, but overall teams win 3/5 of their +7 games, that corrects a little for the effect of our fanbase.

I did not run the number rigorously, but five minutes and an envelope back tends to confirm what I think most of us believe:  Harbaugh is performing within historical norms, has definitely pulled us back from our "wilderness wandering" years, and has underachieved in rivalry games for reasons both in and out of his control.

That last is the sticking point, because if Michigan doesn't understand the reasons, any decision they make is less likely to get anyone where we all want to go.

oakapple

November 3rd, 2021 at 8:05 AM ^

Points spreads are fine as a rough proxy. They don't reflect the true win probability, but they usually aren't way off either.

As I recall, someone did an analysis based on recruited talent. Take a composite of your last 4 classes and the opponent's last 4 classes. It provides a good rough idea of who is the more talented team.

The results are similar. Harbaugh slaughters the opponents when Michigan is much more talented, and gets slaughtered the other way (mostly Ohio State). That much would be true of almost any coach.

But when the talent is comparable or only a modest advantage for Michigan, Harbaugh's teams don't carry their weight. This suggests what the other studies are saying. Harbaugh usually wins when there is a huge talent disparity in his favor. But he seldom wins by outcoaching or outsmarting his opponent.

JBLPSYCHED

November 3rd, 2021 at 8:53 AM ^

Good point...which begets the question, which coaches do win more games than they 'should' by outcoaching or outsmarting their opponents? On Mandel and Feldman's podcast the other day they quoted Kirby Smart as saying something like, "No coach wins without talent," which I took to mean that recruiting is by far the most important factor, with development of recruited talent to be less important (but still important). I'm not going to do the research but I wonder what the results would reveal?

oakapple

November 3rd, 2021 at 9:44 AM ^

On average, recruiting is destiny. This has been deeply studied. The recruiting rankings are pretty accurate in aggregate, and the most talented team usually wins.

With that said, there absolutely are coaches who regularly out-perform their talent. Pat Fitzgerald at Northwestern is an example. This is not their best year, but over the long haul, he has won a lot more than he “should,” if you only considered his recruiting.

Even at the top levels, coaching matters. Compare Alabama and Texas. Both have access to incredible recruiting. The results are quite different.

WestQuad

November 3rd, 2021 at 10:06 AM ^

Seems like we should win at least 50% of the toss ups and win the same percentage of the underdog games as we lose of the favorite games.   

I think Harbaugh is a good coach, but whether it is refs or luck, he doesn't seem to get a lot of breaks.