so much for that
From 5 Stars to First Rounders, the Predictably Uncertain Path from Recruit to Pro
Previously: Predicting team success based on recruit rankings
Before signing day I took a look at how team recruiting rankings were predictive of future success. I found that good defenses almost always come with good recruits, but on offense great offense often comes without being fully stocked, although it doesn’t hurt.
This week I wanted to look more at the individual level by comparing recruiting rankings to draft success. For most positions college success is going to translate well into future draft status. Michigan might have the biggest exception to that rule in Denard Robinson (although some think he might be a top WR pick). For almost everywhere on the field but rushing quarterback, college success and production are highly correlated to NFL stock. It’s not perfect but it’s a great place to start.
The debate on do recruit rankings matter rages on. Dr. Saturday, may he blog in peace, annually refreshed his look to affirm their accuracy. Rarely do you find anything resembling an analytical take down but from even the best writers on college football can come the anecdotal dismissal. Hopefully those of us who prefer to use data have already won you over and this can be a nice look at some of the ups and downs within the overall success of recruiting rankings. If you’re there yet, hopefully you are after you read this.
The Data Sets
On the recruit side, the pool of players will be the recruiting classes of 2002-2006. All but 2-3 of those players have had their shot to be drafted between the 2005 and the 2011 drafts. I will only be looking at the players who were ranked for their position, as well. This means I have all 4 & 5 stars and the best of the 3 stars. I excluded fullbacks and specialists because the numbers are pretty low and they are mostly all 3 stars or less.
It’s All in How You Word It
There are two key arguments against recruiting rankings. The first is the one used by Bruce Feldman in his recent article on Stanford linked above. It’s the yeah but what about…argument. Ignore recruiting rankings because Stanford is good. Ignore recruiting rankings because JJ Watt is good. There of course exceptions. There are plenty of flameouts and come from nowhere success stories but this is a volume game and the exceptions don’t disprove the rule.
The second argument is the famed failure to divide. Here are two true statements:
If you are drafted, you are more likely to be a three star or less recruit than four or five star.
The more stars you have the more likely you are to be drafted.
The first statement is used by opponents of rankings but isn’t really a relevant statement. The second is the key point. If every single five star was drafted, there would still be six times more three stars and below drafted than five stars. Because four stars and above are so selective they can’t win the quantity game but they dominate the likelihood game. The NFL is full of unheralded recruits but for every five start there are literally hundreds of unheralded recruits playing college football. The pool just starts much bigger.
Tell Me Something I Don’t Know
So at this point we can all agree that recruiting rankings matter, right? If you’ve made it this far you’ve earned a chart.
Percent of Recruits Drafted
| Position* | 5 star | 4 star | 3 star |
| APB | 50% | 10% | 9% |
| ATH | 62% | 22% | 9% |
| C | NA | 11% | 6% |
| CB | 56% | 31% | 11% |
| DT | 64% | 21% | 12% |
| G | 50% | 15% | 10% |
| ILB | 64% | 18% | 8% |
| OLB | 22% | 23% | 13% |
| PQB | 44% | 21% | 9% |
| RB | 43% | 21% | 8% |
| RQB | 29% | 14% | 6% |
| S | 67% | 15% | 10% |
| SDE | 40% | 22% | 10% |
| T | 20% | 20% | 11% |
| TE | 100% | 30% | 13% |
| WDE | 30% | 30% | 11% |
| WR | 53% | 22% | 10% |
| All | 48% | 21% | 10% |
*Position based on recruited position, not drafted position
Across all positions, each additional star more than doubles your likelihood of being drafted. It’s not only true in the aggregate but at the position level, as well. There isn’t a single position where a 3 star recruit is more likely to be drafted than a four star. And this is a self-selected group of 3 stars and not the entire pool. In almost every case, a fifth star is another large bump from 4 stars. OLB, OT and WDE are virtually equivalent between 4 and 5 stars. Even a largely college specific position like Dual-Threat QB (RQB) and undefined positions like Athlete show the same trend.
The top positions for 5 star success are Athlete, DT, ILB and Safety at over 60% and the tight end position which was a perfect 4/4 in getting 5 stars drafted.
But getting drafted is only half the story, the other is draft position.
Average Pick For Drafted Players
| Position | 5 star | 4 star | 3 star |
| APB | 9 | 66 | 158 |
| ATH | 111 | 102 | 144 |
| C | 108 | 84 | |
| CB | 104 | 124 | 97 |
| DT | 120 | 102 | 119 |
| G | 82 | 124 | 137 |
| ILB | 33 | 124 | 118 |
| OLB | 125 | 118 | 118 |
| PQB | 39 | 98 | 88 |
| RB | 41 | 111 | 141 |
| RQB | 14 | 105 | 147 |
| S | 135 | 118 | 123 |
| SDE | 96 | 139 | 123 |
| T | 94 | 88 | 141 |
| TE | 63 | 114 | 124 |
| WDE | 53 | 70 | 124 |
| WR | 77 | 95 | 118 |
| All | 79 | 107 | 124 |
At the position level, the draft spot doesn’t hold up quite as well as the previous chart, but overall there is a strong trend favoring the higher starred players. On average, a drafted five star player will be picked in the middle of the third round, nearly a full round ahead of the average four star player and another 17 picks ahead of ranked three star players.
On twitter on Friday I teased a question about which position did five stars underperform four star counterparts. There is actually a position on each side of the ball. On defense it’s outside linebackers that don’t follow the trend and on offense it’s the tackles.
I think it’s interesting that Rivals has struggled to match top high school talent at position like tackle, outside linebacker and defensive end at the rate they have at other positions. Despite the weakness at these positions, similar positions like guard, inside linebacker and defensive tackle have had their rankings hold up quite well.
The Takeaway
Don’t get too hung up on the magic of the fourth or fifth star. They are a nice aggregation but there isn’t going to be much difference between the last five start and the first four star. The bottom line is the higher ranked a recruit is the better they are likely to be, with plenty of exceptions. Positions like tackle, weakside d-end and outside linebacker the difference between a four star and a five is almost negligible. And there are no guarantees. Loading up on top talent gives you the highest likelihood of having team success and successful individuals, but when you get down to the specific player level it becomes a crapshoot. More 5 stars players never hear their names called than ones who do. For four stars it’s still a nearly 4:1 chance against getting drafted.
March 6th, 2012 at 1:40 PM | 
Where have you been?
Where have you been?
March 6th, 2012 at 1:42 PM |
Consulting with
Consulting with
The Knowledge I presume ...
March 7th, 2012 at 1:44 PM |
So is BWC the 40%?
So is BWC the 40%?
Or is Hoke using his gold pooping magic to defy those odds as well? I know Mattison joked that Hoke chose to coach the NT spot because Martin played that position, but my emotional thinking likes to believe it was to get BWC on the correct track after three years of, let's say, questionable coaching.
The path from high school ranking to NFL draft position goes through a college program. The analysis I would like to see is on which programs are best at reducing high star flameout, and/or highest percentage of producing over-achievers. I would hypothesize that places like Alabama, where there is a disproportionate wealth of talent, also have a disproportionate amount of flameout or under performance from high school star to NFL draft position.
All of this can't be black and white, (although media writing articles unfortunately play upon the readers who want their answers diluted to black and white) but I am interested in knowing which "lever" has a larger effect; star rating or program development.
Overall a good analysis though, and very interesting.
"That's all there is...there isn't anymore"
March 6th, 2012 at 1:56 PM |
re: the relative success of guards vs. tackles
re: the relative success of guards vs. tackles
In terms of recruiting rankings predicting success--I suspect it has to do with the lack of guards being rated that highly. (Similar to how all five-star TEs have gotten drafted over the sample.) Because some positions are perpetually rated, when recruiting services do decide to rank an exemplary guard highly, it's because they have a high degree of certainty that said guard is freaking awesome.
March 6th, 2012 at 2:00 PM |
Thanks, Mathlete!
Thanks, Mathlete!
You provide significant added value to the college football discussion.
Formerly 'mgoblaniel.'
March 6th, 2012 at 2:01 PM |
None of the 5 star player
None of the 5 star player
None of the 5 star player counts are that high, but guards were 3/6 when it came to 5 stars getting drafted and tackles were 2/10. It's not just a more tackles are rated question. But there is definitely uncertainty tied to the small sample size. I didn't bring it out as an issue in the article because even with the small sample size the results were very consistent.
March 7th, 2012 at 1:56 PM |
Rivals vs. Scout
Rivals vs. Scout
I'm a Rivals guy also, so I appreciated you using those numbers, but the time honored debate is if one of the services is better than the other. It may be interesting to compare specifically the 3 categories that came out negligible - tackle, weakside d-end and outside linebacker and see if Scout is indeed better when it comes to evaluating the talent.
Bleeding Maize and Blue from the AZ
March 6th, 2012 at 2:20 PM |
Thanks, Mathlete. Very
Thanks, Mathlete. Very
Thanks, Mathlete. Very interesting. One of my favorite things to do is go way back to the old recruiting rankings and see where NFL stars were ranked as high schoolers. Granted, the systems probably weren't as accurate and well-organized back then, but still very interesting. A few of the best players at their positions in the NFL (from 2002 on):
Calvin Johnson: high 4 star, rivals 100 player.
Aaron Rodgers: generic 3 star
Adrian Peterson: #1 overall player in nation.
Derrelle Revis: generic 3 star
Mario Williams: low-end 4 star
These are just a few examples of how widely it varies. I don't know football compared to most people on this site, I just find it fun to go back and take a look. In my mind, the 2 players who are hands down the best at their positions in the league are Megatron and Revis, and neither of them were 5 stars, although Calvin was ranked pretty high.
I've decided to lay off the food for a while, and go on the booze.
March 6th, 2012 at 4:56 PM |
Those are all interesting
Those are all interesting
Those are all interesting examples- but the whole point (as I see it) of compiling data like this is to see general trends.
Many fans (and perhaps talent evaluators) get stuck up on examples that break from the norm to deny trends or to skew their perceptions of players- this data shows very clearly that the more starts you have the more likely you are to be drafted- with a couple unique counter-examples where for some positions 4* and 5* are not much different.
Yes 3* will outperform and get drafted above 4* and 5*- but there's a reason we get so excited about the massive number of 4* recruits we are pulling in- they do outperform 3* players as is demonstrated by a whole wealth of data just shared with us.
March 6th, 2012 at 2:47 PM |
I wonder if draft position doesn't correlate as well
I wonder if draft position doesn't correlate as well
because NFL teams don't draft players in the order of their goodness, but according to the perception of the team's needs and the relative value they put on each position.
The other thing I wonder about are the criteria for rankings. Players that do not project to be excellent college players may turn into excellent professional players.
March 6th, 2012 at 4:38 PM |
Rivals ratings
Rivals ratings
You're using Rivals ratings right? Have you tried matching up their numeric ratings to likelihood of being drafted? I'm curious if that would help give a more granular result?
I also think it's also interesting to compare this to All-American rankings. While five star players are just under five times more likely to get drafted than three star players per your analysis, Dr. Saturday found that five stars were more than eleven times more likely to make an All-American team than three stars. That's a pretty significant gap. I can speculate about a couple reasons for why, but the most obvious is a preference for placing a 5 star onto such a team over a 3 star (while the draft is more of an obvious meritocracy).
March 6th, 2012 at 6:10 PM |
60% of the time, 5 stars get
60% of the time, 5 stars get
60% of the time, 5 stars get drafted every time.
But did you know that Mike Hart was a one star? True story.
March 6th, 2012 at 7:53 PM |
Selection Bias
Selection Bias
Awesome analysis. But... because recruiting rankings are not static (ie. They change after a recruit commits to a school) there is ample opportunity for selection bias. The only analysis that would separate such bias would be an evaluation of recru
iting rankings' performance within each university's pool of players. Otherwise you are simply "proving" that players from big-time programs (with big time facilities, big time coaching, and big time media exposur e) are drafted at a higher rate than players from the lower level programs. We all know that a recruit at Michigan is more likely to go pro than a recruit at EMU. I would rather know if Michigan got better recruits than ohio.
Fanatic since 1978. Internet tough guy since 2012.
March 6th, 2012 at 10:46 PM |
Big time programs
Big time programs
I was thinking much the same thing but more on the recruit side of things and not the evaluater side. It seems (with zero data to back this up) that 5 star recruits almost exclusively end up at big time programs that are at least regular league championship contenders. This leads to more national exposure which leads to more NFL scouts being aware of the players which leads to a higher likelyhood of being drafted.
March 7th, 2012 at 1:14 PM |
National exposure
National exposure
In the modern NFL, scouts are aware of good players no matter where they play. If a player's college career is overlooked by the NFL, it's more likely to be because the player had injury problems or wasn't a schematic fit in college.
March 7th, 2012 at 1:50 PM |
Right
Right
I think the big-time college advantage is more self-reinforcing than anything else. Better raw material goes to school with better resources--and, boom, you get a multiplicative effect. The advantage in facilities, nutrition, and coaching is probably pretty big between big-time schools and, say, MAC School X.
March 6th, 2012 at 9:16 PM |
Mathlete, great work
Mathlete, great work
I wanted to point a few things out however. By your calculations a DT is over three times more likely to be drafted as a 5star than 4star yet you closed with saying that Rivals hasn't done a good of distinguishing between the two.
Also, what are we getting at with the draft position chart? Is it an "even if they do get drafted they won't be drafted as high as 5 stars" kind of thing? You seemed to draw a different conclusion with the DT's.
T
March 6th, 2012 at 11:19 PM |
Wow, what you want to be is a
Wow, what you want to be is a
Wow, what you want to be is a 5* TE. I'm pulling my kid out of school and getting to work on this right away.
M'Dog
March 7th, 2012 at 3:43 AM |
Great post, Mathlete
Great post, Mathlete
Also, JimLahey looks at how some of the best professional players were rated coming out of high school. I thought I'd look at some of the best college players, so the Heisman trophy winners of the last 7 years (looking at Rivals rankings, which I can only go back to Bush):
RGIII - 4 star, #4 Dual threat QB in the country, behind Pryor EJ Manuel, and MarQueis Gray (side note: imagine if RR had pursued him instead of Pryor, where would Michigan be)
Cam Newton - 5 star, #2 Dual threat QB, behind Tyrod Taylor
Mark Ingram - 4 star, #189 in the country
Sam Bradford - High 3 star, #12 Pro style QB, just behind Greg McElroy and ahead of Christian Ponder
Tim Tebow - 5 star, #22 in the country, #1 dual threat QB (interestingly enough Mattison was his recruiter at UF
Troy Smith - 4 star, #12 dual threat QB in the country
Reggie Bush - 5 star, #2 player in country
Looking at the breakdown, 3 5 stars, 3 4 stars, and 1 3 star in the group. Obviously, the scope of this is limited and skewed, because the best teams typically get the best talent around the higher rated recruits, but was interesting. A breakdown of All Americans and their ratings may be a bit more appropriate.
March 7th, 2012 at 12:47 PM |
mmmmmmmm...evidence.
mmmmmmmm...evidence.
mmmmmmmm...evidence.
March 7th, 2012 at 12:53 PM |
i bet the correlation gets
i bet the correlation gets
i bet the correlation gets better every year. scouts are better able to evaluate players than they used to be because tape is easier to send and because word spreads faster about good players b/c of social media and the like.
March 7th, 2012 at 12:53 PM |
LALALALALALALA 3 star mafia
LALALALALALALA 3 star mafia
LALALALALALALA 3 star mafia LALALALALALALALA. Oh wait we're getting 4 stars now? Nevermind.
Galileo was also chastised.
March 7th, 2012 at 1:05 PM |
Question for Mathlete
Question for Mathlete
I had actually attempted to a very similar analysis about 18mo ago, but I gave up because I had too many things on my plate. What I was attempting to do was see which recruiting service (ESPN, Rivals, Scout) was the best at predicting success. In a nutshell, I was going to do a series of regressions to see which service was the best predictor. For my own curiousity (and hopefully others'), do you think you could so something like that at some point?
March 7th, 2012 at 1:12 PM |
I am working on it. Only
I am working on it. Only
I am working on it. Only Scout and Rivals have data far enough back to do a good analysis with the NFL draft because of the lag time. Getting it all uploaded and managed is a bear. For any of you NOTY fans, you can understand how hard it is to link up all the creative spellings and name changes (like Maruce (Jones-)Drew or Patrick Johnson (Peterson). At some point I am hoping to get somewhere with it but don't count on it any time soon.
March 7th, 2012 at 1:19 PM |
This would be very
This would be very
This would be very interesting but the only problem is that there are a variety of confounding variables that really can't be accounted for. For one, coaching staffs and facilities differ and it's possible that the same 3* who was drafted in Setup A would not have been in Setup B. Also, where a recruit goes to school may lead to heightened exposure that could improve their draft stock.
I would be curious to see if the 3*s that are drafted in the first couple rounds come disproportionately from certain schools.
March 7th, 2012 at 7:39 PM |
double post*
double post*
*
March 7th, 2012 at 7:38 PM |
That's a very good question but it involves more than
That's a very good question but it involves more than
just certain advantages/biases that a recruit who goes to a big program may have over a recruit at a smaller program (frankly, I think the NFL generally does too good a job scouting to let talented guys slip through the cracks just because they are at a small school).
The bigger and more interesting question is whether the lower ranked players who go to big programs actually then to be better recruits than their similarly ranked counterparts. If Nick Saban is hot after a 3-star player, I suspect that kid may well be more talented than his ranking suggests, and perhaps Rivals just missed the boat. In other words, are all 3-stars created equal, or are some more likely to succeed than other?
Would be really interesting to compare, for example, whether a RIvals 5.7 3-star who is an otherwise highly ranked class is more likely to be drafted than a Rivals 5.7 3-star who is in a middling class, based on the theory that a lower ranked guy in a highly ranked class is much more likely to be a guy the coaches really wanted, rather than a guy the coaches merely had to settle for. On the other hand, a lower ranked guy in a highly ranked class might be LESS likely to succeed because the competition for playing time is greater. Either way, it would be interesting to know if there is any correlation between the overall ranking of a recruiting class and the likelihood of a lower ranked player in that class getting drafted.
March 7th, 2012 at 1:08 PM |
Mean vs median
Mean vs median
Mathlete, is this a case where median draft position might be a better indicator than average draft position? I can see an argument either way, but I'd think the median would be less sensitive to distortions from effects like picks being driven in part by NFL teams' positions of greatest need, which would probably alter the average draft position for the top few players more than the average position for the rest.
March 7th, 2012 at 1:19 PM |
I don't think Rivals does a
I don't think Rivals does a
I don't think Rivals does a very good job of projecting OL positions or DEs. They rate almost everyone as an offensive tackle (even though you need just as many guards on the team, and lots of the "tackles" end up at guard), and they often leave guys ranked as defensive ends who are clearly too big to be edge players. I think kids see the big money that OTs and DEs make, and they hate the thought of moving inside to lesser renowned positions.
March 7th, 2012 at 6:57 PM |
These statistics are great but are easily misused/misinterpreted
These statistics are great but are easily misused/misinterpreted
They suggest the odds for a RANDOMLY selected 5, 4 or 3 star player. However, it doesn't, for example, tell you that a particular 4 star player that your coaches just signed has a better likelihood of success than the 3 star player they just signed. It doesn't mean they necessarily make a mistake (even putting needs aside) when they go hard after a lower ranked guy and ignore a higher ranked guy. Because at that point you are no longer dealing with a random sampling from the pool--you are dealing with players that were specifically selected because the coaches saw something they wanted. You cannot forget that end of the day the rankings are just some guy's opinion, and opinions--even qualified opinions--vary. One person's three star maybe another person's five star.
On the other hand, while you expect some variation in opinions, you also expect a good deal of overlap. If your coaches are consistently pulling in ALL lower ranked players, it's POSSIBLE they are seeing something everyone else is missing, but it is far more likely they just aren't doing a very good job getting or identifying the best players available.
March 9th, 2012 at 12:26 PM |
Great writeup. Thanks.
Great writeup. Thanks.
Great writeup. Thanks. Accumulating a lot of 4-stars seems like a great strategy for success, exactly what Hoke & Co. are doing. A few 5-stars would be nice but probably not critical.
Where have you been?
And when we play as a team, when the old season is over, you and I know it’s going to be Michigan again. Michigan!