The GERG effect

Submitted by dnak438 on

[ed: bump.]

I hope this kind of analysis hasn't been done already; if so, my apologies. I was wondering, in light of Brian's analysis of the defense, if we could quantify the effect of having Mattison rather than GERG as our defensive coordinator. To that end, I have tried to quantify the effect of having Greg Robinson as defensive coordinator: the GERG effect. I looked up the scoring defense ranks of Michigan, Syracuse, and Texas for 2003-2010 (data from Rivals). They are as follows:

  2003 2004 2005 2006 2007 2008 2009 2010
Texas 33 14 5 26 45 18 12 49
Syracuse 61 71 70 54 104 102 81 17
Michigan 7 38 22 16 23 84 76 107
GERG   14 70 54 104 102 76 107

GERG was DC at Texas in 2004, HC at Syracuse in 2005-2008, and DC at Michigan in 2009-2010. We can therefore display the data graphically as follows (note that the Y axis is team defense scoring rank): We can also try to calculate a crude GERG effect by comparing the average rank of these defenses with and without GERG. This yields the following:

  non-GERG GERG difference
Texas 25.3 14 -11
Syracuse 57.5 82.5 25
Michigan 31.7 91.5 59.8

Note that positive is bad and negative is good. So GERG's Texas defense was 11 ranks better than the non-GERG average, whereas his Michigan defenses were 60 ranks worse than the non-GERG average.

To calculate the overall GERG effect, we simply multiple the differences in rank by the number of years at each school, divided by the total number of years (7), to arrive at our overall GERG effect of 29.77 [this figure has been updated]. That is, on average, GERG adversely affects the scoring rank of the defenses he is associated with by 30 positions.

If we take Brian's projection of Michigan's 2011 defense (82nd), and subtract 30 ranks to adjust for the GERG effect, we get to 52, a ranking that a number of commentators were predicting based on their "gut" feeling of player development and the new coaching staff's abilities (particularly Mattison).

Again, this is wild speculation, and incredibly simple -- hopefully it is not completely misguided. Other more advanced metrics should also be used. I am aware that there are far more variables at work that determine how good a defense is--and it is almost certain that GERG was not allowed to run his defense at Michigan. Also, it is likely that Mattison is an excellent DC, something that this analysis does not account for. I was still curious to see if anything could be done to account for the coaching change.

UPDATE: I made an arithmetical error which has been corrected. Also, I ran the same analysis with S&P+ play-by-play ratings from Football outsiders and got a GERG effect of 27.75 ranks (using only Syracuse and Michigan; S&P+ data are only available from 2005 and later).

[ed: We should be taking all of this, including my original post, with a grain of salt because of sample size issues.

That said, Michigan was an the extreme outlier because of its youth and trying to run two different schemes, one of which was something no one's ever tried before, and could expect to rebound further with Mattison--and more importantly, sanity--hanging around campus. The numbers offered here in the two posts (54 using S&P+ data and 82) seem like the ends of a range of reasonable expectations.

The moral of the story is the same one learned by the offenses of Notre Dame in 2008 and Michgian in 2009--you're going to be a lot better but still very far from good.]

Comments

74polSKA

July 13th, 2011 at 10:22 AM ^

I'm fairly new to mgoblog and in the last 2 days this GERG term has surfaced.  I searched and found a bunch of topics and I know it refers to Greg Robinson.  Without reading a bunch of old topics, is there a funny story behind this or just more mgoblog meme.  Someone help a newbie out.

JeepinBen

July 13th, 2011 at 10:28 AM ^

Some ways you might expand on an already good idea (if you REALLY want to try to quantify the GERG effect...):

Do the same comparison for other defensive metrics: Yards, passing defense, rush D, etc. 

Maybe just include conference games (i dunno how hard that data is to get) but with all the cupcakes on the schedule that can tend to skew data.

Again, thanks for what you've done already. If I had the time I'd do it myself. Very interesting post!

DixieWreck

July 13th, 2011 at 11:24 AM ^

piece of evidence that would need to be considered would be the upperclassman vs. underclassman component of which appears to be generally overlooked in my opinion.

Kilgore Trout

July 13th, 2011 at 12:11 PM ^

Very valid point.  The counter to that is of course the roster that he had in two years at UM. We've been over it before of course, but between poor retention, questionable recruiting, and being put into a system he wasn't familiar with, it's not really fair to look at the last two years at UM and say Robinson is the worst DC ever. 

An inverse way to look at it would be to take Shaefer's career and judge the "Michigan Effect" on his numbers (obvious sample size concerns apply).

dnak438

July 13th, 2011 at 8:44 PM ^

which is what you might expect in a fairly stable coaching regime at an established program -- every three-four years you get a great mix of talent and experience, while in down years you slip down to about 40th, presumably due to turnover, injuries, attrition, and so on. Michigan's curve looks like that, too, with peaks in 2003 and 2006... until the 2008 collapse.

Blue in Seattle

July 13th, 2011 at 12:04 PM ^

anecdotally the Syracuse improvement in 2010 is quite a jump upward.  So how did Scot Shafer achieve that?  Just colliding with a bunch of awesome recruits from GERG who were just too young in 2007-2009?

Statistics are all well and good when you can find enough data that is truly representative of the model you are trying to prove.  But statistics are a generalized representation of many many events.  Actually watching the singular events is what excites me,

"And that's why they play the game"

This team may not have the talent, and may be behind in their training on technique, but I believe it's got the passion, the bonding, and some very good teachers as coaches.  I'm excited for the coming season, and as Brian has pointed out, with a little luck swinging back our way, we could end up in the first Big Ten championship game at the end of the season.

 

oakapple

July 13th, 2011 at 12:11 PM ^

Statistics can be a useful corrective to anecdote and impression, but this sample size is really too small. Moreover, it ignores his successful tenure as a D.C. for the Denver Broncos.

As you note, he wasn’t allowed to run his defense at Michigan, a fact that seriously compromises one of the three data points. So what you’re left with is one good data point (Texas), one bad data point (Syracuse), and one that is difficult to assess (Michigan).

I’m certainly not making the case that Michigan’s defensive performance would have been great if GERG had been allowed to run it as he saw fit. But even if you put the failure on GERG’s shoulders entirely, three trials does not make for a valid sample.

dnak438

July 13th, 2011 at 3:40 PM ^

but I thought that might create more problems than it solved because it introduces issues of comparability. There are already big comparability issues as Gerg was HC at 'Cuse and not a DC. I'm not a coach but I assume that being a DC in the NFL is pretty different from being a DC in college. I could be wrong about that. But my general principle in running new analyses is to start simple and introduce more complexity later.

markusr2007

July 13th, 2011 at 12:23 PM ^

that GERG sucked universally as a defensive coordinator no matter where we coached.

And I thought I couldn't feel worse than I did before after watching Syracuse defense improve dramatically under Scott Shafer, the guy RR shit-canned in favor of....GOYG.

Thanks!

I wrote a piece earlier about GERG as a DL coach at UCLA, but I was way wrong.  The reason UCLA defenses kicked royal ass in the 1980s had nothing to do with GERG at all. It was all Bob Field and his recruiting.

Still, I'll miss that wonderful Centrum Silver coiffure gleaming on the sidelines more than anyone will ever know.  It was spectacular.

 

funkywolve

July 13th, 2011 at 12:29 PM ^

To me Gerg as head coach and not defensive coordinator at Syracuse is an interesting situation.  Obviously as head coach you are ultimately responsible for how your team performs.  However, how involved was Gerg with the defense at Syracuse?  Was he fairly hands off with his defensive staff or was he heavily involved in the coaching and game planning on the defensive of the ball?

 

news2me

July 13th, 2011 at 12:58 PM ^

Simple to see you have lightly explored a great line of thought: very well done dnak438. For me, yours is a seven year snapshot I find very compelling with no need for fine tuning. Therefore, knowing what I saw as a very sorry D-coaching performance under RR and GERG; my expectations are a top 50 D-ranking in 2011 or better: and I am guessing the glass slipper will fit Mattison's foot perfectly.

JohnnyV123

July 13th, 2011 at 2:03 PM ^

Real good work. Yeah there are criticisms that okay when GERG took over for Texas they were experienced with future NFL first rounders or that Michigan was playing inexperienced players when he took over here but it's nice to confirm his negative average effect on the defense in his career and not just Michigan.

The problem with looking at stats like this with such a small sample size is you could make the argument that he should only be judged on the defense as a defensive coordinator not as a head coach. Then you could go in two of the three years he was defensive coordinator the defenses improved from the year before, which far overrates GERG's ability.

I think it's also hopeful reaching to project our defense will be in the 50's by adding that to Brian's number just for the GERG effect but of course I would wish you are right.

If you want more hopeful stats maybe add what the Mattison effect would be if you can translate his college and NFL coaching experiencing to some number.

rlc

July 13th, 2011 at 2:31 PM ^

That huge drop in defensive rating between 2007 - 2008 pre -GREG should be considered. It seems like comparing the GREG ave against this non-GREG average, or even against an all-time average, would have to take that event into account.

It would be useful to see historically how fast a defense can recover after such a steep decline, and how other D coordinators have faired the two years following a similar event.