Now that football season is over, occasionally a season ranking comes out based on team or offensive/defensive performance. Enter the self proclaimed college football professor. (I'm not here to dime the guy out, he's just what got me interested in working this up). Landof10 reported the other day that this guy's annual ranking of defensive coordinator performances came out, and that this past year Don Brown had the 24th best defensive performance in FBS, 6th best in the B1G. At this point you probably don't even care where he has Harbaugh. Wise of you. For any coaches out there trying to poach Don Brown, read no further!

My first reaction was "Look, if Don Brown's 2017 defensive performance isn't in your top 10, we can't even talk". However curiosity got the best of me, and in digging deeper, the statistics used don't measure what I would consider a Defensive Coordinator's performance at all.

Very briefly, he is including statistics that that are counted within other stats, such as number of sacks, and number of tackles for loss. To compare this to basketball, a slam dunk gives you two points and an emotional boost (maybe a sports center clip), but it's simply part of shot efficiency and total score the same way that sacks and TFL's are part of defensive yards, efficiency, and defensive 3/4th down conversion stops, its just how you get the job done, the result is what matters. Additionally he is detracting for recruit quality, which makes no sense. A coach's performance should most certainly include the quality of recruits they bring to the program, not detract from it.

A Simplified Defensive Theory

A defensive coordinator's main off field role is to recruit (and train) quality defensive players, study film, and develop game strategies. This should all directly relate to performance, so I don't believe it needs to be weighted against anything. Let's leave it all off the report card as it is included elsewhere in overall performance indicators.

A defensive coordinator's main on field role is to stop the offense from gaining any yardage, through either three and outs, or by turnovers gained, so as to provide maximum field position for the offense, with no points scored. 

Barring the ability to completely stop the offense from gaining yardage and first downs, the defensive coordinator's role is to stop the offense from scoring points. 

Applicable Scoring Indicators

While many relevant statistics are based on say yards per game, or score per game, these do not account for "offensive interference". A team with an offense that cannot advance the ball is likely going to have a defense that either has more time on the field or more defensive possessions. As such defensive performance needs to be based on its productivity per possession in cases where it makes sense to do so. Based on this theory, I examined the main applicable defensive statistics for all FBS teams.

Average Defensive Yards per Possession

Defensive 3rd and 4th Down Combined Conversion %

Defensive 1st Downs Allowed per Possession

Points Allowed per Possession vs Power5 Quality Teams

Defensive Turnovers Forced per Game

These five indicators to me measure the true performance of a defensive unit, and other important performance factors are already baked into at least one of these. While the first three indicators are somewhat related (actually there is a correlation between the first four, including points allowed), I felt it was important to examine all of them in totality as outliers could show say teams that gave up huge plays fairly frequently.

Points Allowed per Possession vs Power5 Quality Teams was used instead of the more common "scoring defense" (Defensive points per game/possession), as games against non-Power 5 conference quality schools do not provide a clear picture of defensive coordinator performance. For example, a late season game against Furman might allow for an excellent defensive performance without having to show any new looks, while an early season MAC opponent may strategically allow a DC to try and not show any new formations, or to play around with positioning, provided they maintain a comfortable lead. Also, some schools schedule slightly more non-Power5 quality games than others. A final note on this statistic: for all FBS teams measured, all games vs P5 teams and teams with similar/reasonable quality were considered.

It seemed most logical to rank these teams in order either by yards or 1st downs/possession, defensive stop %, or Points Allowed per Possession as the most important factors. I chose yards/possession based on my defensive theory above, and as Michigan came in first for three of these top four, with similar rankings for many others. Objective statistics below:

I ended up taking the top 19 teams by yards given up per possession because it covered all of the top 10 teams for the relevant statistics and looked fairly pretty. 

This is my attempt to do it right- accounting for offensive interference of true defensive performance

Moving past the issues that came up with the college football professor's version, a main concern with using simple statistics to quantify defensive (or offensive) performance is that the offensive and defensive performance is so intertwined. That is to say, number of defensive possessions per game, defensive field position, average number of offensive turnovers lost per game, number of offensive punts per game, and opponent quality (here measured as % of top 10 teams by final rankings played) will all reasonably factor into defensive points allowed, and are not logically true measures of defensive performance. Additionally I argue that loss in defensive production from the previous year is an additional factor in game performance to some extent that may vary by year to year. For this I did not measure starters lost, but total production as I think that's more accurate.

Offensive interference indicators:

%top 10 opponents in schedule

%lost defensive production from 2016

Average defensive field position

Average offensive turnovers/game

Offensive punts/game

Opponent possessions per game (past 10)

To stress, this is not a ranking of team performance or best teams, it's an attempt to quantify how well Defensive Coordinators did, or how well a FBS defense did in spite of the offense. Here I have named these indicators below as "offensive interference indicators", though some are external interference indicators (opponent strength) rather than purely offensive:

I chose to use Opponent possessions per game instead of average time played on defense for one main reason. Higher time on defense correlates directly to poor defensive performance: if you are allowing a team to slowly and consistently march down the field, that is not a offensive interference indicator, its an indicator of poor defensive performance. Conversely, higher defensive possessions per game can correlate either way and allow for exceptional defensive performance, with the assumption that the defense is still going at full speed for those shorter but more frequent possessions.

Once we accept (or don't) these six indicators as external or otherwise offensive interference, things start to become subjective. How much do each of these impact a Defensive unit's performance indicators? I was specifically interested in how they would impact Defensive points allowed per possession, as Michigan already appears to lead the way even excluding the offensive effect on yards, 1st downs, and 3+4d stop%. The first thing I did was to pull the points allowed per possession for the top dozen teams, and compared their offensive interference indicators to Michigan's. 

What I observed is that almost across the board, Michigan has stronger offensive interference indicators than these top 11 other teams by points allowed/possession. Of the six teams with better base points allowed than Michigan, only two had 1/6 indicator stronger than Michigan, the rest were 0/6. Of the five teams behind Michigan in base points allowed, only two had 2/6 stronger indicators, the other three were 0/6. My hypothesis was then that essentially Michigan was going improve more significantly by points allowed relative to its peers regardless of any reasonable weighting. I played around with various simulations (more on that below), using a subjective measurement that 2sd from the mean in either direction could result in 20% changes in defensive points given per possession, and ended up using this scoring mechanism below for two reasons 1) it felt the most logical, and 2) it weighted more heavily towards the indicators that were more consistent between teams but that have a direct impact on points per possession (field position and offensive turnovers/game):

Using Opponent Possessions per Game did not seem entirely appropriate without an adjustment, as the issue that translates from offensive interference is the "gas effect" (having to play that extra 1-2 possessions because the offense can't stay on the field (think back to the South Carolina game), which sort of loses its linearity past a certain number of possessions. As such I simply used possessions over 10 per game as a baseline. I weighted all six indicators to come to an average of one, and the multiplier is simply what is required for the weight vs the original type of number.

Some comments on these offensive interference indicators: Pnts/G is Offensive Punts/Game. The theory behind that being that if your offense is progressively punting a lot, your defense is going to become backed up to their own goal line by no fault of their own, similarly with offensive turnovers and average DFldPos. (Remember, that even with an average change of 4 yards field position, i.e. the difference between Michigan and Alabama, that's simply a mean that puts an offense within field goal range at the start of their possession on average once a game). Ignoring that offensive interference indicator would assume incorrectly that those 3 points given up per game are entirely the fault of the defensive unit. Results below:

In taking these offensive interference indicators and plugging them into defensive points allowed per possession, using this model Michigan comes out on top of points allowed per possession. As a side note various weights kept Michigan at or near the top 4 regardless of what sort of logic I used for the six indicator weighting, from 8% to 40% change in defensive points per possession, and did not vary significantly for Michigan's ranking within the top four in assigning different weights for each of the six indicators. For example Michigan still comes out on top if these six factors account for 15% swing points given up per possession, and is among the top four at 8-10%, making me fairly confident that Michigan's points given up per possession was within the top four or at the top of FBS performance, accounting for Offensive interference indicators. 

Confirmation Bias or just Confirmation

After publishing this originally I was curious and went back once more and ordered the original defensive point per possession between defensive yards per possession and the weighted defensive points per possession. I argue that defensive yards per possession shows a fairly accurate estimate of a defense's ability to stop an offensive movement towards points scored, and it isn't as influenced by offensive interference indicators such as field position or errant offensive turnovers in field goal range. As a result my hope was that weighted points per possession would more closely correlate to defensive yards per possession than unaltered points per possession. This appears to be the case.

That Defensive Turnover Rate

I'm going to try and dig into this deeper, as was suggested. I did not actively play with this rate or consider it relative to the offensive indicators, as it is simply a function of how the defense gets the job done; it is baked into yards/possession, down statistics, and offensive field position (not covered here). However, Michigan and Indiana were dead last at defensive turnovers gained per game in the top 19 defensive teams (though Indiana had the top defensive returning production in FBS at 96%. My guess is that Don Brown's strategy with so many younger players was to keep it simple, and play contain rather than attempt to make the big plays. Sort of how Don Brown kept things more simple for Rashan Gary in 2016. (below)

I'd say it worked, but I'd also hazard a guess that next year that turnover ratio is one of the things that significantly improves, which will likely result in better offensive productivity. Perhaps Gary will develop a defensive holding tell like throwing his arms up in the air to make it super obvious. Who knows. All in all this defense got me excited in review. Can't wait for next year, and go Blue!

(Thanks for reading, and feel free to provide comments, especially those of you more data/analysis oriented).