Rank the Coaches: Talent Evaluation and Player Development

Submitted by The Mathlete on March 15th, 2012 at 8:45 AM

Evaluating coaches is a tricky thing. Ultimately it comes down to wins and losses but even comparing one situation to another in the unbalanced world of college football is a tricky proposition. Mike Shula has a higher career winning percentage as a head coach than Brady Hoke. However Hoke has spent all but the last year at non-BCS schools where Shula was at Alabama. School prestige, resources and recruiting all play major roles in team success along with coaching. Many of them often go hand in hand but I think I am finding some ways to parse out different pieces of the puzzle independently. This is my first of hopefully many off-season looks at coaches, and who at excels at what parts of coaching.

The Data

To evaluate how coaches develop and evaluate talent I needed a way to separate out better inputs (recruits) from the output (team success and draft placement). Team success is a viable way to look at it and at some point I would like to circle back to compare PAN and recruiting for a comparison, but for today’s exercise I am going to look at recruiting ranking to draft position.

Time Period

The main challenge with this method is that draft placement is such a lagging indicator from recruiting. Since only some of the 2007 recruits and most from 2008 on have yet to be drafted, I am only looking at recruiting classes from 2002-2006.

Evaluating Recruits

I have now been able to add all four recruiting services to my database. Since we are only looking at classes up until 2006, that means just Scout and Rivals for all years except 2006 when ESPN came on board, as well. Recruits are given a number value based on national rank, position rank and stars. Each year has 25,000 points assigned across all players so the early years with fewer players have their individual ceilings a bit higher. Consensus 5 star players are typically 50-60 pts. Generic three stars are in the low teens and below. Anyone without a position rank or less than 3 stars is zero points.

Here is Michigan’s 2012 class for reference.

Player Pts
Kyle Kalis 54
Joe Bolden 49
Ondre Pipkins 47
Erik Magnuson 43
Terry Richardson 39
Royce Jenkins-Stone 39
James Ross 37
Chris Wormley 30
Dennis Norfleet 29
Tom Strobel 28
Amara Darboh 27
Mario Ojemudia 21
Devin Funchess 19
AJ Williams 19
Jarrod Wilson 18
Matt Godin 16
Ben Braden 16
Kaleb Ringer 16
Blake Bars 16
Jeremy Clark 16
Jehu Chesson 16
Allen Gant 12
Willie Henry 12
Sione Houma 12
Drake Johnson 8

Evaluating Draft Picks

Because of the much higher value to higher draft picks, the draft pick evaluations are fitted using an exponential formula.

500*e^(-Pick#/50)

This works out to about 500 for the first pick and then each round is half of the same pick in the previous round (1st pick in second round about 250, 1st pick in the third about 125, etc.). This puts the total points for a 255 player draft at 24,600, almost identical to the total for a year’s worth of recruits.

Coaches

Players are counted towards the coach that recruited them. This will only be somewhat an evaluation of player development since the coach gets “credit” for the player they recruited even if they leave the next year. I have also restricted the search to coaches with at least 1,000 total recruiting points over the five year period. This is about equal to two top 15 classes or five top 50 classes. This gives us 43 qualifying coaches to review.

The Show

First thing I did was look at each coach and how many recruiting points they accumulated versus how many draft points they had.

Rank Coach Recruit Pts Draft Pts Ratio
1 Pete Carroll 3,605 8,069 2.24
2 Jim Tressel 2,920 5,642 1.93
3 Nick Saban 1,870 3,415 1.83
4 Phillip Fulmer 2,827 4,595 1.63
5 Tyrone Willingham 1,693 2,621 1.55
6 Bob Stoops 3,216 4,940 1.54
7 Al Groh 2,076 3,166 1.52
8 Mack Brown 3,225 4,537 1.41
9 Ron Zook 2,220 3,047 1.37
10 Joe Paterno 2,363 3,241 1.37
11 Urban Meyer 1,621 1,986 1.22
12 Bobby Bowden 3,244 3,956 1.22
13 Lloyd Carr 2,923 3,491 1.19
14 Tom O'Brien 1,415 1,580 1.12
15 Ralph Friedgen 1,925 1,914 0.99
16 Frank Beamer 1,618 1,539 0.95
17 Larry Coker 2,958 2,788 0.94
18 Houston Nutt 2,086 1,886 0.90
19 Kirk Ferentz 1,660 1,499 0.90
20 Walt Harris 1,392 1,217 0.87
21 Barry Alvarez 1,220 1,065 0.87
22 Chuck Amato 1,587 1,385 0.87
23 Tommy Bowden 1,787 1,549 0.87
24 Mike Shula 1,842 1,520 0.83
25 Jeff Tedford 1,748 1,411 0.81
26 Chan Gailey 1,043 840 0.81
27 Gary Pinkel 1,207 899 0.74
28 Mark Richt 3,305 2,379 0.72
29 Joe Tiller 1,407 956 0.68
30 Mike Bellotti 1,518 1,014 0.67
31 Lou Holtz 1,354 886 0.65
32 John Bunting 1,805 1,175 0.65
33 Bill Callahan 1,142 658 0.58
34 Tommy Tuberville 2,381 1,291 0.54
35 Mike Leach 1,500 681 0.45
36 Dirk Koetter 1,483 591 0.40
37 Gary Barnett 1,461 472 0.32
38 Dennis Franchione 2,034 656 0.32
39 Les Miles 2,170 686 0.32
40 Karl Dorrell 1,528 457 0.30
41 John L Smith 1,187 273 0.23
42 Rich Rodriguez 1,098 249 0.23
43 Charlie Weis 1,303 84 0.06

The first thing that jumped out at me was that there seemed to be a strong correlation between total recruit points and total draft points. This is going to be true to some extent, but it seemed that ability for the top schools to load up wasn’t properly accounted for. So I plotted the two versus each other and found a very strong correlation was present.

image

Since we are looking for more on talent evaluators and developers than MOAR 5 stars, I used the correlation between the two to adjust recruiting points to give a more fair comparison between the lower end and the top end. This allows for a more common evaluation tool between elite programs/recruiters and the rest.

Rank Coach Adj Recruit Pts Draft Pts Adj Multiplier
1 Chan Gailey 185 840 4.53
2 Barry Alvarez 539 1,065 1.98
3 Nick Saban 1,839 3,415 1.86
4 Tyrone Willingham 1,486 2,621 1.76
5 Gary Pinkel 515 899 1.75
6 Bill Callahan 383 658 1.72
7 Tom O'Brien 930 1,580 1.70
8 Pete Carroll 5,309 8,069 1.52
9 Urban Meyer 1,343 1,986 1.48
10 Jim Tressel 3,941 5,642 1.43
11 Al Groh 2,253 3,166 1.41
12 Walt Harris 883 1,217 1.38
13 Phillip Fulmer 3,754 4,595 1.22
14 Ron Zook 2,540 3,047 1.20
15 Frank Beamer 1,337 1,539 1.15
16 Joe Paterno 2,826 3,241 1.15
17 Lou Holtz 808 886 1.10
18 Bob Stoops 4,532 4,940 1.09
19 Chuck Amato 1,275 1,385 1.09
20 Kirk Ferentz 1,419 1,499 1.06
21 Joe Tiller 914 956 1.05
22 Mack Brown 4,550 4,537 1.00
23 Ralph Friedgen 1,950 1,914 0.98
24 Tommy Bowden 1,675 1,549 0.93
25 Mike Bellotti 1,137 1,014 0.89
26 Lloyd Carr 3,945 3,491 0.88
27 Jeff Tedford 1,596 1,411 0.88
28 Bobby Bowden 4,587 3,956 0.86
29 Mike Shula 1,784 1,520 0.85
30 Rich Rodriguez 296 249 0.84
31 Houston Nutt 2,273 1,886 0.83
32 Larry Coker 4,016 2,788 0.69
33 John Bunting 1,710 1,175 0.69
34 Mike Leach 1,100 681 0.62
35 John L Smith 474 273 0.57
36 Dirk Koetter 1,066 591 0.55
37 Mark Richt 4,711 2,379 0.51
38 Gary Barnett 1,023 472 0.46
39 Tommy Tuberville 2,861 1,291 0.45
40 Karl Dorrell 1,155 457 0.40
41 Dennis Franchione 2,169 656 0.30
42 Les Miles 2,440 686 0.28
43 Charlie Weis 705 84 0.12

Now we have something to talk about.

One thing that jumped out at me was that NFL guys did seem to have a bit more success. Maybe their buddies were just doing them favors, but there are a lot more guys with NFL experience at the top than the bottom. Oh, except for the big guy coming in last at #43. Weis’s monster class of 2006 (934 team points, my #7 class of the last 11 years) yielded two 6th round draft picks. His first class which was much less regarded still only yielded a single fourth round draft pick. In the words of our fearless leader, #MissYouBigGuyXOXO.

Lloyd Carr comes in just below average on the adjusted scale. Barry Alvarez checks in at #1 among Big Ten coaches and #2 overall. Wisconsin’s lineman machine is real. The evil genius Nick Saban is #3 based on his last three classes at LSU. Ohio coaches new and old round out the top ten.

Of the nine elite recruiters (3,000 or more adjusted recruiting points) Pete Carroll and Jim Tressell come out on top, with Phillip Fulmer close behind. The bottom three are all southern coaches: Bobby Bowden, Larry Coker and Mark Richt.  Bob Stoops, Mack Brown and Lloyd Carr make up the middle third.

Ted Roof takes home the prize for most recruiting prize without a single draft pick with 515 points and nothing to show for it. Top performers who missed the cutoff included Dan Hawkins, Bret Beliema’s first class, Ed Orgeron, Mike Stoops and Greg Schiano.

Other Notes

Many thanks to all who have helped populate the recruit database. We are 25% of the way done.

Still have lots of ideas for future posts including the final post on how to use game theory to maximize success based on the overvalued running back and success rates. If there is interest, I would like to do a retrospective on previous seasons through the eyes of advanced analytics and throw up some of the best WPA graphs of the season. Hopefully I can start with 2003 in the next month. I am open to any ideas you have out there, as well.

If you are on the twitters follow me at @the_mathlete. I am trying to post little snippets that aren’t quite column worthy there. Recently I have tweets about which state’s recruits stay in-state the most (Utah and Arkansas) and least (NY/NJ and Hawaii) and used my recruiting points ranking to list the top 4 Michigan high schools in producing 3* or better talent (Cass Tech, OLSM, Detroit Renaissance & FHH), correctly guessed by @Joshua_Block.

Comments

Blue in Seattle

March 15th, 2012 at 10:34 AM ^

Then you haven't found the "big program" factor and removed it. Assuming the present model is correct, then the present results state that Les Miles did a poor job while at OkSU of turning the talent. He had into NFL caliber players. Great work as always, and much appreciated. Just wondering if this answer means you don't think this is a true measure of a head coaches player development.

If there was enough data on more coaches like Saban, it might be interesting to compare there ratios at different schools. Like Saban's ratio at MSU versus LSU.

Good stuff as always, and this is definitely something I can't do, so thanks!

MGoDC

March 15th, 2012 at 11:06 AM ^

Great stuff as always Mathlete. Question for you, and again I'm sure you had your reasons but I'm just curious to follow your logic, have you considered ranking "draft points" on the same scale that NFL GM's use to make their trades? (http://www.sportznutz.com/nfl/draft/draft_point_value_chart.htm)

 

I just ask this because generally the #1 overall pick is worth significantly more than double the #1 pick of the second round. If you have your reasons for limiting the value of first rounders that's obviously interesting and I'd love to hear the rationale behind it. Impressive stuff though!

GrowBlue

March 15th, 2012 at 11:33 AM ^

Awesome stuff, as always. May I make a suggestion regarding a way to really dig into this more deeply? Perhaps you can model the relationship between recruiting ranking and PAN (ie, what is the typical PAN for a 4* RB, etc). Then take a look at how players perform relative to their expected PAN. Any difference between their Actual PAN and Expect PAN can perhaps be attributed to coaching (development) and/or superior talent evaluation.

dcblue92

March 15th, 2012 at 11:36 AM ^

Great analysis, thanks.  It would be interesting to tie this to wins and losses.  Guys like Phil Fulmer seemed to produce tons of NFL talent, but had very inconsistent results in wins and losses.  Two components of coaching skill are certainly player evaluation and development, but who are the best at producing results with limited talent?  Maybe this analysis has been done before.

hfhmilkman

March 15th, 2012 at 1:37 PM ^

It has been pointed out that many recruits get an uptick just because of the school they commit to.  For example ND gets excited about a recruit and he gets two more stars.

The other item that concerns me is that unless a recruit fails to develop physically, he will attract interest and be drafted regardless of how dismally he learned football skills.  If you run fast, stop on a dime, jump really high, and rep a lot of benches you will get drafted.  This may be because the former NFL'ers at power programs do well.  Even if they fail, the physical measurables will give that player a look.  Yet a 3 star who got that star because of physical liabilities might be a good college football player but just not good enough to go on.

What I think would be interesting is instead of using the NFL for a barameter for success something along the lines of how many starts did that player generate for the program and how did that player stack up against his peers.  A guy picked as the best DE and goes on to be a day one draft pick would be valued less then a 2 star who started 3 years, ranked in the top 3rd, but maybe just did not have the atheltic ability to go on.

Playing devils advocate on myself then how do you seperate someone who is just better at evaluting talent?  What if that coach knows that 3star is actually 5 star material and only he has figured it out?

 

 

 

ca_prophet

March 15th, 2012 at 4:10 PM ^

That is, given players with X,Y,Z recruiting points, you'd expect them to be drafted at M,N,O.  Where were they actually drafted?  If they were on average drafted higher, then you have the basis for thinking that they were coached-up for the NFL.

Of course, NFL success isn't necessarily the metric to use for a coaching staff - although it is definitely of interest to the players.  Perhaps a PAN approach is best - recruits with X recruiting points usually put up N PAN over their college careers, but this guy's recruits put up 20% more.

I'm not sure how to deal with transfers in all of this, either, but hopefully that's a small number and doesn't really matter.

 

Roachgoblue

March 15th, 2012 at 5:30 PM ^

2 turds ready to flush. I could win at WVU with one good qb recruit. Rich Rod was in charge of coaching and was weak. He was in control of the program and not the alumni. Booo hoo I didn't get support. He was recruiting to the Big House, yet still needed support. He chose Gerg, and his offense was not good against solid teams. What a little bitch.

Taps

March 16th, 2012 at 9:11 AM ^

Very interesting. But i think you've enormously overvalued top draft picks. A couple of your top coaches, Gailey and Pinkel in particular are being tremendously skewed by single players. Practically every draft pick has a good to outstanding college career. There are all americans and all conference players galore in the late rounds. A dozen 6th and 7th hrounders demonstrates way more developmental acumen than a single top pick.

MU9296

March 16th, 2012 at 11:36 AM ^

I think your argument is valid, but your use of Gary Pinkel as an example of that point is most certainly not.  Missouri has had a lot of players drafted recently, including five in the first round in just the last three years.  I think the above ranking of Pinkel is definitely indicative of his staff's ability to develop players.  See below for all of Pinkel's players who have been drafted:

 

 

Year Round Pick Player Name Team Position
2011 1 7 7 Aldon Smith 49ers DE
  1 10 10 Blaine Gabbert Jaguars QB
  7 31 234 Andrew Gachkar Chargers LB
2010 1 19 19 Sean Weatherspoon Falcons LB
2009 1 19 19 Jeremy Maclin Eagles WR
  1 32 32 Ziggy Hood Steelers DT
  2 23 55 William Moore Falcons DB
  3 34 98 Chase Coffman Bengals TE
  5 3 139 Colin Brown Chiefs T
  6 26 199 Stryker Sulak Raiders DE
2008 4 6 105 William Franklin Chiefs WR
  4 12 111 Martin Rucker Browns TE
2007 4 14 113 Brian Smith Jaguars DE
2006 4 6 103 Brad Smith Jets WR
  7 35 243 Tony Palmer Rams G
2005 3 25 89 Atiyyah Ellison Panthers DT
  5 6 142 Damien Nash Titans RB
  6 17 191 C.J. Mosley Vikings DT
2003 5 8 143 Justin Gage Bears WR
  6 41 214 Keith Wright Texans DT

 

http://www.drafthistory.com/index.php/colleges/m