I think you will get your wish.
As Michigan enters year #3 with Denard starting the season at the helm and year #2 under OC Al Borges, I did a deep dive into teams that have made the leap into offensive greatness. A handy guide to going from a good offense to an elite offense in one year.
In making predictions you are always safer predicting things to regress towards the mean. I wanted to look at the teams that have gone from good but not great offensive teams to truly elite teams in one season. There were some surprisingly strong correlations within this group.
1. A quarterback with experience returning.
2. The same offensive coordinator as the previous year
3. Continuity in your receiving core
These three three aren’t sufficient conditions for making the leap, but they are necessary conditions.
From 2004 to 2011 there were 20 teams, including 2010 Michigan, that increased their offensive EV+ rating by at least 4 points per game and ended the year above 10. I wanted to take a look at these 11 teams to figure out what was necessary to make a jump like this, what might seem necessary but wasn’t and how many teams fit this same criteria but couldn’t make the leap.
It’s not rocket science to say that having a returner at quarterback is a good thing, but the exceptions to this rule prove it as much as anything. Of the 20 teams to make the leap, five didn’t technically have a returning starting quarterback. Two, Auburn 2010 and Stanford 2009, did it with future first overall draft picks. Wisconsin 2011 technically counts but Russell Wilson was a three year starter for NC St before arriving in Madison. Tim Tebow got quality field time on a national championship team despite Chris Leak’s position as starter. The final exception was Oregon 2010 when Darron Thomas stepped in to great success.
The returners themselves take all forms. There are runners like Josh Nesbitt from Georgia Tech in 2009, Pat White at West Virginia in 2006. There are pass-first athletes like Robert Griffin on Baylor 2011 and Vince Young on Texas 2005. There are future first round picks like Sam Bradford at Oklahoma in 2008 and Matt Leinart at USC in 2005.
Three times it was done with multiple quarterbacks seeing action. Michigan in 2010 saw returning starter Tate Forcier make regular appearances during the Denard Robinson injury hour every Saturday. Cincinnati didn’t miss a beat in 2009 when Zach Collaros stepped in for injured starter Tony Pike. Arizona State had the same story in 2005 when Rudy Carpenter replaced the injured Sam Keller.
How many years at the helm wasn’t a significant issue. There were as many teams who made the leap with second year starters as third and fourth starters combined. The only team to do it with a non-transfer four year starter was Northwestern in 2005 under Brett Basenez. Zac Robinson did it in his second year at Oklahoma St in 2007, Chase Daniel did it at Missouri in his third year of four running the offense at Missouri.
Not all the QB’s developed within the program, either. Cam Newton, Russell Wilson, Aaron Rodgers and Ryan Mallett all accomplished the task after transferring from a previous school or Junior College.
On average, about 35% of BCS programs turn over their offensive coordinators in a given year. For teams making the offensive leap it drops to 10%. Charlie Weis found his famed “Decided Schematic Advantage” in 2005 (only to lose it thereafter) with Brady Quinn. Arkansas and Ryan Mallett where the only others to accomplish the feat, but the presence of head coach and offensive guru Bobby Petrino indicates the continuity was likely high.
Teams making the leap returned 75% of their value from their receiving core, nearly fifty percent higher than the average team (53%). Only four teams returned less than 60% of their receiving value from the prior, and those teams all managed to make the leap due to ultra efficient passers, Tim Tebow, Russell Wilson, Aaron Rodgers and Vince Young.
Missing The Cut
Returning Running backs
My crusade to bury the position of the running back continues. Teams making the leap where all over the map on returning RBs. Texas and Northwestern both made the leap in 2005 with virtually no non-QB carries returning from the 2004 season. Baylor, Michigan, Florida and Cal all did with only marginally used carriers returning. On the flip side USC returned Reggie Bush and LenDale White in 2005. Georgia Tech, Oregon, West Virginia and Missouri all returned over 80% of the previous years carries.
It’s not that having a key ball carriers returning is a bad thing, it’s just not necessary to make the leap.
Just like the running back question, the recruiting rankings are mixed bag. Cincinnati, West Virginia, Cal, Northwestern and Baylor all made the leap without the benefit of a roster full of 4 and 5 star recruits. USC, Auburn, Florida, Texas and Oklahoma always enter a season stacked. Better recruits = better results is true but Better Results = Better Recruits is often incorrect.
Odds of Making It
Of teams that meet criteria 1-3 above (with 3 being defined at 60%) and are within 1 standard deviation of +10 EV+ (4.4 or higher) 25% make the leap to +10 or higher. 40% of this group improves but not to an elite level. That leaves 35% of teams to regress under this environment. The worst offender is Michigan State in 2006. Poised to become a potentially great offense under Drew Stanton, Sparty fell apart and dropped from a +7.2 in 2005 to a –2.6 in 2006.
Can Michigan be that team
With the loss of Junior Hemingway, Kevin Koger and Martavious Odoms Michigan falls below the 60% threshold (38%). All of the other criteria fit nicely for Michigan and when it has been done without the aid of returning receivers, its come on the back of a Heisman level quarterback performance. If year 2 of Al Borges Denard Fusion Cuisine comes together, that doesn’t seem out of the realm of possibility. I won’t be on record predicting a leap like year for the 2012 Michigan offense, but they are one of a handful of schools that I would even consider.
Evaluating coaches is a tricky thing. Ultimately it comes down to wins and losses but even comparing one situation to another in the unbalanced world of college football is a tricky proposition. Mike Shula has a higher career winning percentage as a head coach than Brady Hoke. However Hoke has spent all but the last year at non-BCS schools where Shula was at Alabama. School prestige, resources and recruiting all play major roles in team success along with coaching. Many of them often go hand in hand but I think I am finding some ways to parse out different pieces of the puzzle independently. This is my first of hopefully many off-season looks at coaches, and who at excels at what parts of coaching.
To evaluate how coaches develop and evaluate talent I needed a way to separate out better inputs (recruits) from the output (team success and draft placement). Team success is a viable way to look at it and at some point I would like to circle back to compare PAN and recruiting for a comparison, but for today’s exercise I am going to look at recruiting ranking to draft position.
The main challenge with this method is that draft placement is such a lagging indicator from recruiting. Since only some of the 2007 recruits and most from 2008 on have yet to be drafted, I am only looking at recruiting classes from 2002-2006.
I have now been able to add all four recruiting services to my database. Since we are only looking at classes up until 2006, that means just Scout and Rivals for all years except 2006 when ESPN came on board, as well. Recruits are given a number value based on national rank, position rank and stars. Each year has 25,000 points assigned across all players so the early years with fewer players have their individual ceilings a bit higher. Consensus 5 star players are typically 50-60 pts. Generic three stars are in the low teens and below. Anyone without a position rank or less than 3 stars is zero points.
Here is Michigan’s 2012 class for reference.
Evaluating Draft Picks
Because of the much higher value to higher draft picks, the draft pick evaluations are fitted using an exponential formula.
This works out to about 500 for the first pick and then each round is half of the same pick in the previous round (1st pick in second round about 250, 1st pick in the third about 125, etc.). This puts the total points for a 255 player draft at 24,600, almost identical to the total for a year’s worth of recruits.
Players are counted towards the coach that recruited them. This will only be somewhat an evaluation of player development since the coach gets “credit” for the player they recruited even if they leave the next year. I have also restricted the search to coaches with at least 1,000 total recruiting points over the five year period. This is about equal to two top 15 classes or five top 50 classes. This gives us 43 qualifying coaches to review.
First thing I did was look at each coach and how many recruiting points they accumulated versus how many draft points they had.
|Rank||Coach||Recruit Pts||Draft Pts||Ratio|
|41||John L Smith||1,187||273||0.23|
The first thing that jumped out at me was that there seemed to be a strong correlation between total recruit points and total draft points. This is going to be true to some extent, but it seemed that ability for the top schools to load up wasn’t properly accounted for. So I plotted the two versus each other and found a very strong correlation was present.
Since we are looking for more on talent evaluators and developers than MOAR 5 stars, I used the correlation between the two to adjust recruiting points to give a more fair comparison between the lower end and the top end. This allows for a more common evaluation tool between elite programs/recruiters and the rest.
|Rank||Coach||Adj Recruit Pts||Draft Pts||Adj Multiplier|
|35||John L Smith||474||273||0.57|
Now we have something to talk about.
One thing that jumped out at me was that NFL guys did seem to have a bit more success. Maybe their buddies were just doing them favors, but there are a lot more guys with NFL experience at the top than the bottom. Oh, except for the big guy coming in last at #43. Weis’s monster class of 2006 (934 team points, my #7 class of the last 11 years) yielded two 6th round draft picks. His first class which was much less regarded still only yielded a single fourth round draft pick. In the words of our fearless leader, #MissYouBigGuyXOXO.
Lloyd Carr comes in just below average on the adjusted scale. Barry Alvarez checks in at #1 among Big Ten coaches and #2 overall. Wisconsin’s lineman machine is real. The evil genius Nick Saban is #3 based on his last three classes at LSU. Ohio coaches new and old round out the top ten.
Of the nine elite recruiters (3,000 or more adjusted recruiting points) Pete Carroll and Jim Tressell come out on top, with Phillip Fulmer close behind. The bottom three are all southern coaches: Bobby Bowden, Larry Coker and Mark Richt. Bob Stoops, Mack Brown and Lloyd Carr make up the middle third.
Ted Roof takes home the prize for most recruiting prize without a single draft pick with 515 points and nothing to show for it. Top performers who missed the cutoff included Dan Hawkins, Bret Beliema’s first class, Ed Orgeron, Mike Stoops and Greg Schiano.
Many thanks to all who have helped populate the recruit database. We are 25% of the way done.
Still have lots of ideas for future posts including the final post on how to use game theory to maximize success based on the overvalued running back and success rates. If there is interest, I would like to do a retrospective on previous seasons through the eyes of advanced analytics and throw up some of the best WPA graphs of the season. Hopefully I can start with 2003 in the next month. I am open to any ideas you have out there, as well.
If you are on the twitters follow me at @the_mathlete. I am trying to post little snippets that aren’t quite column worthy there. Recently I have tweets about which state’s recruits stay in-state the most (Utah and Arkansas) and least (NY/NJ and Hawaii) and used my recruiting points ranking to list the top 4 Michigan high schools in producing 3* or better talent (Cass Tech, OLSM, Detroit Renaissance & FHH), correctly guessed by @Joshua_Block.
Thanks to all that helped build the coaching database. Now it's time to move on to recruits. I have uploaded all available recruiting sites databases back to 2002 in an effort to connect them to team rosters. Of 16,865 recruits, I have connected most of them to players in the databse. However there are about 2500 players still unconnected. Some of them were academic or legal casualties, some of them were transfers. Most of them are offensive lineman that never showed up on the play by play in the first place.
For those who consider themselves recruiting and or Google ninjas, I can use your help. I have listed the players, school they signed with and year they signed for all the missing entries. Whatever info you can help fill in would be a great help. There are more instructions in the spreadsheet and feel free and contact me with any questions you might have. My email is in the instructions. Thanks again for everyone's help.
Before signing day I took a look at how team recruiting rankings were predictive of future success. I found that good defenses almost always come with good recruits, but on offense great offense often comes without being fully stocked, although it doesn’t hurt.
This week I wanted to look more at the individual level by comparing recruiting rankings to draft success. For most positions college success is going to translate well into future draft status. Michigan might have the biggest exception to that rule in Denard Robinson (although some think he might be a top WR pick). For almost everywhere on the field but rushing quarterback, college success and production are highly correlated to NFL stock. It’s not perfect but it’s a great place to start.
The debate on do recruit rankings matter rages on. Dr. Saturday, may he blog in peace, annually refreshed his look to affirm their accuracy. Rarely do you find anything resembling an analytical take down but from even the best writers on college football can come the anecdotal dismissal. Hopefully those of us who prefer to use data have already won you over and this can be a nice look at some of the ups and downs within the overall success of recruiting rankings. If you’re there yet, hopefully you are after you read this.
The Data Sets
On the recruit side, the pool of players will be the recruiting classes of 2002-2006. All but 2-3 of those players have had their shot to be drafted between the 2005 and the 2011 drafts. I will only be looking at the players who were ranked for their position, as well. This means I have all 4 & 5 stars and the best of the 3 stars. I excluded fullbacks and specialists because the numbers are pretty low and they are mostly all 3 stars or less.
It’s All in How You Word It
There are two key arguments against recruiting rankings. The first is the one used by Bruce Feldman in his recent article on Stanford linked above. It’s the yeah but what about…argument. Ignore recruiting rankings because Stanford is good. Ignore recruiting rankings because JJ Watt is good. There of course exceptions. There are plenty of flameouts and come from nowhere success stories but this is a volume game and the exceptions don’t disprove the rule.
The second argument is the famed failure to divide. Here are two true statements:
If you are drafted, you are more likely to be a three star or less recruit than four or five star.
The more stars you have the more likely you are to be drafted.
The first statement is used by opponents of rankings but isn’t really a relevant statement. The second is the key point. If every single five star was drafted, there would still be six times more three stars and below drafted than five stars. Because four stars and above are so selective they can’t win the quantity game but they dominate the likelihood game. The NFL is full of unheralded recruits but for every five start there are literally hundreds of unheralded recruits playing college football. The pool just starts much bigger.
Tell Me Something I Don’t Know
So at this point we can all agree that recruiting rankings matter, right? If you’ve made it this far you’ve earned a chart.
Percent of Recruits Drafted
|Position*||5 star||4 star||3 star|
*Position based on recruited position, not drafted position
Across all positions, each additional star more than doubles your likelihood of being drafted. It’s not only true in the aggregate but at the position level, as well. There isn’t a single position where a 3 star recruit is more likely to be drafted than a four star. And this is a self-selected group of 3 stars and not the entire pool. In almost every case, a fifth star is another large bump from 4 stars. OLB, OT and WDE are virtually equivalent between 4 and 5 stars. Even a largely college specific position like Dual-Threat QB (RQB) and undefined positions like Athlete show the same trend.
The top positions for 5 star success are Athlete, DT, ILB and Safety at over 60% and the tight end position which was a perfect 4/4 in getting 5 stars drafted.
But getting drafted is only half the story, the other is draft position.
Average Pick For Drafted Players
|Position||5 star||4 star||3 star|
At the position level, the draft spot doesn’t hold up quite as well as the previous chart, but overall there is a strong trend favoring the higher starred players. On average, a drafted five star player will be picked in the middle of the third round, nearly a full round ahead of the average four star player and another 17 picks ahead of ranked three star players.
On twitter on Friday I teased a question about which position did five stars underperform four star counterparts. There is actually a position on each side of the ball. On defense it’s outside linebackers that don’t follow the trend and on offense it’s the tackles.
I think it’s interesting that Rivals has struggled to match top high school talent at position like tackle, outside linebacker and defensive end at the rate they have at other positions. Despite the weakness at these positions, similar positions like guard, inside linebacker and defensive tackle have had their rankings hold up quite well.
Don’t get too hung up on the magic of the fourth or fifth star. They are a nice aggregation but there isn’t going to be much difference between the last five start and the first four star. The bottom line is the higher ranked a recruit is the better they are likely to be, with plenty of exceptions. Positions like tackle, weakside d-end and outside linebacker the difference between a four star and a five is almost negligible. And there are no guarantees. Loading up on top talent gives you the highest likelihood of having team success and successful individuals, but when you get down to the specific player level it becomes a crapshoot. More 5 stars players never hear their names called than ones who do. For four stars it’s still a nearly 4:1 chance against getting drafted.
And now back to our regularly scheduled programming…
Previsouly: Parts 1a, 1b, 1c
I have done a terrible job of branding this series. The idea behind it is that football has changed and coaches haven’t. The game used to be about managing down and distance, putting yourself in a makeable third down, and hoping your defense can win with 17 points. Now offenses are more sophisticated at both running and passing. Third downs that used to be virtually out of reach are still tough but more possible and the upsides of going for bigger chunks of yardage on first and second down have begun to outweigh the risks of longer third downs. This changes how both offensive and defensive coaches need to think and how they allocate resources and personnel. Some pieces are now worth more and others less.
The traditional running game used to be the focal point of this philosophy. The traditional running game is the best football tool for limiting variance on a down by down basis. The quarterbacks job is to hand the ball off, throw a couple of beautiful play action deep balls a game, bail out a third down or two, then feed words like "focused" to the media.
As I spent the last several years combing through nearly ten years of play by play data, I kept coming back to the same question: Why do teams run the ball so much? I parsed the data time after time to try and find something I had missed and I couldn’t find it. Of the top individual PAN seasons among QBs and RBs since 2006, only 3 running backs (Boise St’s Ian Johnson in 2006 and Montee Ball and Trent Richardson this year) cracked the top 100. But PAN doesn’t take into account burning the clock at the end of a game. So I switched to WPA (Win Percent Added) which accounts for the clock. Under WPA rankings, Toby Gerhart in 2009 is the only running back to break into the top 200 seasons. 199 quarterback seasons and only 1 running back season.
Now this isn’t to say that a running game isn’t valuable. Of my ten highest rated offensive seasons noted below only Oklahoma, Hawaii and Houston didn’t feature prominent rushing attacks. In fact of the ten, I would categorize 5 as rushing spreads, 3-4 (Baylor is tough to categorize) as college passing spreads and Wisconsin as a traditional run-first offense.
The running game is alive and well but the traditional running back is harder to justify.
The Wisconsin Case
Montee Ball had an outstanding season and along with Trent Richardson clearly a top 2 back in the country. But was he the most valuable player on his own offense? Here are the traditional numbers for Ball and Russell Wilson
307 att, 1923 yards & 33 TDs rushing (NCAA record 39 overall TD)
225/309, 3175 yards & 33 TD & 4 INT (NCAA record 191.8 pass efficiency)
and the advanced metrics
+6.1 PAN and 0.10 WPA/Game
+11.4 PAN and 0.37 WPA/Game
The Wisconsin offense was a thing of beauty that could have been a national title contender if their –1 defense didn’t lead them to three losses while scoring at least 29 points in each of them.
So who was more responsible, Wilson or Ball? Wilson averaged more yards/play, had almost no turnovers and significantly higher advanced metrics. But let's dig down a bit and compare the two.
Nearly half of all Russell Wilson’s plays (rushes and passes) went for 7 yards or more. Ball had 28% of his plays go for the same distance. For negative plays, they are nearly even with sacks and all Ball without. The area were Montee Ball’s plays went was in the 0-3 yard range, i.e. the manage the down and distance range. This obviously wasn’t a bad season for Ball, it was a great season and he was still dominated by his quarterback in terms of output.
Now this take into consideration down and distance considerations so I put together a similar slide with EV.
Montee Ball had 15% of his plays go for at least a half standard deviation above average. Russell Wilson’s number was twice that at 30% with minimal negative offset.
Looking at a second way, here is there play EV value ranked.
As good as Montee Ball was last year, the offense should have even gone to Wilson, more.
RIP Running Back?
Obviously not as a position but as a premiere position I have a hard time justifying the running back’s historical position as at nearly the same level as the quarterback. Even at their best great running backs at similar value to decent quarterbacks. Two offseasons ago I did a study on returning starters and found that of all positions on the field, returning starts by running backs had the least effect of any position on future team success. Before signing day when I looked at the value of recruiting ranking to future team success, running back recruiting was one of the lowest correlations to future offensive success.
It’s not that running backs can’t be valuable. Montee Ball’s +6 PAN is outstanding. It’s more that a big upside for a running back is rare, hard to predict and is still less than you can get from a quarterback. Of the 29 QB’s and RB’s that were +3 or better last year only five were running backs, the rest were quarterbacks. Running back has become a low marginal production position.
Wrapping This Up Next Week
There is a good argument to be made that Wilson’s success is a byproduct of the attention paid to Ball. It obviously didn’t occur in a vacuum and I have no doubt that Wilson benefited from the attention paid Ball more than vice versa. In next week’s final part of this series we’ll look at how teams can adjust their strategies on both sides of the ball to maximize the new realities.
We now return you to your commitments in progress
I don’t think Success Rate is a misguided stat as much as I think it is a misguided strategy. I think the overall concept of S&P that Bill uses is very sound, I just think the emphasis should be more on the P than the S.
My biggest problem with the stat is that it is black and white. As comments on his article note, a metric that works on a sliding scale would a significant step in the right direction. On 1st and 10 losses and gains of 4 aren’t and shouldn’t all be treated the same. Just as gains of 5 and up are all valuable, just not equally as valuable. For my metric the sliding scale is factored into the expected points at any play. So there is some element of success rate built into PAN, but it is an integrated, sliding scale as opposed to a separate, black and white component.
There are only three things that matter for evaluating a team on a drive, where did you start, how many points did you score and what position did you give the ball back to your defense/special teams. Plays taken to achieve results and time elapsed off of the clock can be valuable in certain situations, but in general those three data points are the key. If we can effectively measure each play in how it contributes to those three key factors at once, why break it up into two pieces and why make it black and white?
Even though there are some differences and I got things off on a bit of the wrong foot, I think there is more in common than different with the two approaches. What I think is the ultimate issue, however, is coaches calling plays with success rate in mind. Advanced NFL Stats did a great article on this very subject (especially the Importance of Run Success Rate section). He found evidence at the NFL level that coaches are coaching to down by down success rate as opposed to drive success rate. Coaches appear to be attempting to win each battle and at times losing sight of the war.
The battle/war concept is what I think is the most interesting of this so you’ll have to wait until part 3 of this series where I’ll look at how strategy can adapt to score more points while risking a bit of short term success rate. Early next week I’ll post part 2, a look at how Wisconsin’s offense runs and how Russell Wilson was really the most dangerous part of the Badger offense.