As a follow up to my earlier thread here, I performed a deeper dive on offensive holding. In the initial thread, I found that in 2014-2017, Michigan's defense had only drawn 10 offensive holding calls, significantly less than all other B1G teams.
TL;DR: Defensive line performance is a poor predictor of offensive holding calls, but Michigan and Alabama both draw holding at a much lower clip than other NCAA teams.
About the Data
I downloaded the 2010 – 2017 Play by Play data from the ESPN API. This contained 1,234,271 plays, during which, 82,570 penalties were called. Additionally, I correlated this data with Football Outsiders’ Offensive Line and Defensive Line statistics, found here.
Each of the 1.2M plays has a “free text” field which describes the play, and if there is one, the penalty and whom it was called on. As an example, this means Michigan could be referred to as Michigan, UM, UofM, Mich, etc. I very carefully developed an algorithm to assign the penalty called, and who it was called on and drawn by, based on this field for all NCAA games involving an FBS team. While I’m sure it wasn’t 100% accurate, I do think I did a pretty good job and designed some good tests to check this process. Probably 99%+ accurate.
Are there any good predictors of Offensive Holding being called?
Honestly, none that I could find. I found evidence that holding calls are generally random…with a slight edge given to the underdog team or team currently losing. This means the argument “Michigan’s D-Line has been amongst the best in the country, we should be getting more holding calls than everyone else” probably lacks merit based on the data.
Comparing Football Outsiders Offensive and Defensive Line metrics (and score differential at the time of the holding call) to holdings drawn. All three lines are downward sloping – suggesting a slight edge is given to the underdog. However, the r-squared value is low; you can see visually it is generally quite random.
Michigan – we’re getting screwed, right?
I’m going to go with a yes. As it turns out, Michigan and Alabama are in a special kind of unlucky streak, stretching for many years. The below is a graph of all FBS teams 2012- present (B1G teams are represented by their logos) for all games played. Michigan is towards the top right – in the “we’re getting screwed” category. The only team that has fared worse is Alabama. A lot worse. Since 2012, Alabama has drawn a holding once every 260 plays. That is nearly 2.5 times the average.
If you zoom in on the B1G for in-conference games over the same timeframe, you see Michigan has drawn half as many holding calls as their peers.
Why limit the analysis to conference games?
Because if there is a bias within the conference, it wouldn’t perpetuate itself in nonconference games. Let’s take a look at holdings drawn in conference vs. out of conference games in the B1G, 2012-2017:
The red teams (Illinois, Michigan, and Nebraska) each have holdings drawn much more frequently in the nonconference schedule vs. B1G play. In reality, this chart likely has as much to do with scheduling choices than anything else. But you see Michigan has a very unfavorable variance, while Indiana and Purdue have a very favorable variance for in-conference play.
Alright. Let’s grab the torch and pitchforks. How long has this been going on?
Well, there is a reason why I keep showing 2012-present. Beyond this reflecting 5 full years (plus the partial season currently being played), it also reflects how long Michigan’s misfortunes on drawing offensive holding has been going on. The below table shows the rank within the B1G each year on plays per offensive holding. A “1” would represent the team that least frequently drew an offensive holding call in the current year. Michigan finishes in the top (err…bottom) 3 in 5 of the last 6 years. Before that time, Michigan fared much more reasonably.
What else did I find?
Not much else, but not for a lack of effort. I reviewed other attributes of when holding was called: Down and distance, quarter within the game, where the play was on the field, etc. – nothing jumped out. I even looked at other penalties to see if Michigan has had similar misfortune elsewhere…And couldn’t find much evidence behind it. So I’ll leave you with one final visual – the total number of penalties (holding or otherwise) called for each pairing of teams since 2010.
…And penalties called per game…as not every team has played an equal number of times:
Where do we go from here?
I don’t think the Big Ten is actively out to get Michigan. I probably should have led with that. There might be some subconscious bias, however. The way you combat that is by providing visibility behind trends in this data, and increased accountability for referees. Finding a way to better compensate officials - and not have this be their weekend job – is probably a good first step. But another step is to have this information be more readily available.
Umich1 is a data scientist in his professional job, and graduate of the University of Michigan. He spends a lot of time on Saturdays yelling at officials from row 67 in the big house.