What’s a 5 Star Really Worth: Predicting Future Team Success Off of Recruiting Rankings

We all know it matters. Otherwise there wouldn’t be four major recruiting sites, countless team-specific recruiting blogs and grown men tweeting and facebooking 17 year old high school males, and breathlessly refreshing message boards for the next 14 days.

The question I want to answer is how much does it matter, and where do the numbers play out the most? How much of team success can be predicted based on recruiting profile of the present roster (not the JUCO-stuffed 38 member SEC class that the majority never shows)? Do recruiting services do a better job of predicting offense or defense? Which is more likely to win you conference and national championships, the 5 star running back or the 5 star linebacker?

Methodology

I have created a complimentary recruiting database that links into my PBP database. For a source I picked Rivals because I wanted to keep it relatively straightforward and they have a full 10-year history online. I only looked at the players who were ranked at their position. Each year that is about 1,000 players and virtually every signee from a major program. Anyone not ranked for their position was omitted. I only have comprehensive rosters for all teams for the last three years, so for that time period I did my best to link the two DBs together. I am sure there are a few that I am missing but I think I got all the Dee Harts linked up with Demetrius Harts and all the other weird things that happen to a recruit's name between recruitment and the official roster.

Each recruit is given an initial value. The value is roughly

[Percentile within position] * [# of stars] ^ 2

So a 5 star #1 at his position recruit is worth about 25 points and a 50th percentile 3 star would be worth 4.5 pts. The initial value is then adjusted based on how long the player has been in the program.

The recruits are then matched up with the final rosters. Players are only counted if they are still on the roster. So any players that have transferred, left school or gone to the NFL are excluded from the totals. The only major gap is transfers. For ones I knew of right away like Cam Newton or Ryan Mallet, they only count at their final school. Most other transfers will only show up at the original school for their time there and then disappear from the grid. Players are then given a “bonus” multiplier based on their experience. Players' initial values are doubled from their first year to their second year and tripled for every year after that.

That’s a lot fewer words than hours put in but in a nutshell, that’s the background for what I will show you below. The magnitude of the points isn’t relevant, all you need to know is the more points the better.

Answer Your Question Already

When you start talking to yourself within an article on mgoblog, there is only one appropriate response, CHART

Lot’s of variation within the numbers but definitely a strong correlation between recruiting points and team PAN [ed: points above normal, the Mathlete's SOS- and situation-adjusted stat]. For all the charts I put up the data will be BCS schools from 2009-2011. Recruits prior to 2009 will be included, but only the actual seasons of play from 2009 on.

There have been some really good seasons from teams with <1,000 pts like Oklahoma St this past season (896). There have also been some mediocre season from teams with 3,000+ points like Texas in 2010 (3,082 pts). But all in all more recruits is better, but we already knew that. So let’s dig a little deeper and see if recruiting rankings mean more for offense or defense and if any position groups are better indicators than others.

Who To Trust, Offense or Defense

Moving to specifics can become a bit more of a challenge. To ease that, I counted every recruit in the position they play, not the position that they are recruited for. They keep the same point total they would at the original position, it just counts in a different bucket. Whether its a WR moving to DB or an ATH finding a home, the points are set based on the initial group ranking, but they are allocated based on the roster position. On to the offense.

The correlation is still there, but it is much weaker for the offense as opposed to the team as a whole. In fact, most of the best offensive seasons were accomplished with relatively average recruiting talent. The ultimate loaded team, 2009 USC, only managed a 3.3 on offense with 10% pts more than any other team I have measured. Teams like the latest incarnations of Michigan and Oregon were able to achieve double digit offensive PAN without elite offensive recruiting classes.

Defensive recruiting is much more correlated with defensive success than offensive. The slope is nearly double and the R-Squared is much greater as well. There are still exceptions like 2009 Florida St who was almost –10 PAN despite over 1,000 defensive recruiting points. There is still success on the lower range but overall there are fewer failures at the top and less success at the bottom of defensive recruiting rankings.

Based on this data, system, player development and finding diamonds in the rough are more prevalent on offense than defense. On defense there is some variation but for the most part you are who you recruit. Unless you hire Greg Robinson and even your Never Forget roster still has 853 points to “earn” a –7 on the season.

The Best Position To Be In

Since the defense as a whole proved to be the most predictive, let’s look there first.

Being a good defense is all about your weakest link and based on that philosophy, you shouldn’t be surprised to see all positions play out relatively equal. None of the position groups is significantly better or worse than another at predicting defensive success.

Offense is where it really gets muddled. O-Line, tight ends and receivers all are moderate correlations between recruiting and offensive success and running backs (as I’ve stated elsewhere) are the most overrated position in football. Quarterback is far and away the highest correlation to offensive success of any position. Even with that QB, is still below all of the defensive positions when it comes to future success on that side of the ball.

Conference Variation

How recruiting matches up with success varies greatly by conference. Rather than throw up six more charts, I just put the R^2 values in a table:

Recruiting has virtually no correlation to success over the last three years in the Big East and the PAC 12 but for the other four conferences it's anywhere from a little (Big 12, land of Red River and everyone else) to a lot (the ACC and the SEC).

The Big Ten is in the middle; Ohio St has dominated at the top of both recruiting and success but Michigan’s underachievement and Wisconsin and Nebraska having strong seasons without top tier recruiting classes have thrown in enough variance to disrupt the correlation.

Your 5 Star Takeaway

Recruiting rankings have a huge correlation to future team success, especially on defense. Great teams can come from average talent, but more talent typically means more success. On defense it is virtually impossible to build an elite defense without elite recruits, and its equally true across all defensive positions. On offense dreams of 5 star skill position players are fun, but coaching, player development, system and luck play a much bigger role in future success than they do on defense. With top 20 and higher recruits at nearly every position on defense, Michigan is poised for a very strong future if they can keep the talent around.