“Star-system” losing value; Percentage of talent-pool recruited is shrinking

I don't think I've ever seen someone give real analytical legs to the old adage that recruiting stars aren't that big of a deal, but you pull it off exceedingly well. It's easy to view recruiting from the prizm that we're used to and not consider that the decreasing value of the star system is why programs like Iowa, Boise State and Wisconsin can consistently produce good to excellent squads while getting relatively few four-five star recruits. So in your opinion, would an ideal program focus more on depth and "fit" (i.e. whether a certain player has traits that make him more desirable to their specific program) than simply trying to stockpile highly recruited players (a la Ron Zook)?

Thanks again for this analysis, really great work, I feel alot better about our class after reading that. And if you live in NOLA (which I would assume you do) we should get a beer sometime as I live work in New Orleans as well. Cheers.

since your first 3 data points are extrapolated, I would focus on the 10 year span of 1997 to 2007, where you have actual data points. The ratio decreased from 0.24% to 0.23%, or 0.01%. Is that even significant?

Also, I would think that the increased popularity of non-traditional sports (skateboarding, BMX, X-Games stuff...) would take some good athletes away from football and there isn't really a foreign football payer (well, except Samoans) like in basketball to increase the pool.

But now I am going to be bummed all day because I failed to make 175 on the LSAT.

I think this is a very interesting analysis. I do wonder how strong of a conclusion we can really make from this data, but it is interesting anyway.

If your theory is true and that the high school talent pool is larger and possibly even shifted from a normal distribution, there's another circumstantial piece of evidence. We've heard a lot of people say that recruiting rankings are generally accurate because the 5-stars mostly succeed and populate NFL rosters.

Well, the recruiting services are very stingy with their 5th stars. They only give them out to a small number of kids. Your data suggests that maybe all high school players are getting better with coaching, camps, conditioning, etc. Additionally there are more of them. Yet the 5-stars represent an extremely small percentage. You estimated that the top 0.1% of high school players equates to about 900 kids. So the 5-star kids are actually like the top 0.01% of kids and the 3-stars are still in the top 500 or so.

When you think about it that way it isn't hard to realize why 5-stars are so successful and it also isn't hard to see why there are plenty of 3-star recruits who end up doing just fine.

I really like our class a lot, regardless of star ranking. I like the depth at positions of need and I like that most of the recruits have played their natural position in high school and will continue to do that in college. I think that is especially critical for LB and S - two things we're desperate for.

I would like to see more HS data, but this answers the Rivals/Scout (5 star talent equals NC contender) contention - which still has legs IMO. What you lay out here though does give more cred to 2 and 3 star players wrt recent years.

When did the star system start? I think only since the late 70's?? Injury and Junior day camping are huge factors in relative star power as well, besides supposed regional bias. Historically Paterno started the early recruiting game, I've always heard, but I would like to know when junior day camps / combine type player to player comparisons started to play a larger role.

There are clearly .05% of kids out there who excel in the Rivals / combine type system. I don't think there are sub 4.5 kids out there going unnoticed - at least not as a trend. With internet video and IT in general - there is more data just less analysis. There may indeed by more and better 2-3 star talent out there.

Ultimately a great team of good players beats a good team with excellent players. I'll still take the Shawn Crable and Gholston type talent, but there is more to it than Rivals alone. Thanks for this.

Very interesting account of the recruiting pool and having some numbers to back it up. The star system does have its flaws, and this analysis actually puts into perspective how close it is between a 2- and a 5-star prospect. Well done.

the fact that competition levels in high school football are very uneven. Evaluators have gotten somewhat better at adjusting for competition in recent years, but it is still a fact that while it's possible to identify the better players within a given competition level, comparing anyone but the very best players between levels is pretty much a crapshoot.

You're right to point out the increased role of college and other summer football camps/programs/combines -- they help to overcome such disparities.

As someone who got paid to cover recruiting (almost 20 years ago) for a little while, it's my opinion that the star-system largely sucks. First of all, it's a scale of 2-5. What the hell is that?

The five-star kids are easy to pick out. It takes absolutely no talent whatsoever to spot them. And it's pretty rare for those guys to fail at the college level ON THE FOOTBALL FIELD. The ones who disappear are usually victims of injury, get in trouble off the field, or fail academically. It's unusual for a five-star kid to just not be very good.

About half of your four-star kids are easy too. When you go to a Nike Combine, it won't take you long to pick them out. The front seven guys on defense are larger than everyone else and move like track guys. The skill position guys explode when they go all out.

Once you've gotten beyond those guys, it's just this big, foggy grey mass. You can tell who deserves a D1 scholarship and who doesn't. But the idea that any of the guys working for Rivals can tell you the difference between the #38 OLB in the country and the #39 OLB is simply absurd.

Everyone also knows that there's also an enormous amount of post-hoc revisionism that goes on. Over the last two years, for example, Tom O'Brien (who has always had a great reputation for spotting unknown talent - Mathias Kiawanuke, being a good example) has found, recruited and landed two linemen in South Carolina (Denzelle Goode and Thomas Teal) that no one knew about, evaluated, rated or even listed. After they committed, folks from Scout and Rivals went down and both came away openly wondering how they missed these guys.

Goode went to the Shrine Bowl last year and completely shut down Donte Moss (one of the top DEs in the nation). Thomas Teal went to the Shrine Bowl this year and everyone came away talking about how dominant he was.

Do you trust your coaching staff's ability to spot talent? If you do, you should worry more about getting the guys THEY want rather than where that guy is rated. For instance, if you missed on the first two DB targets and THEN took the unknown guy from Louisiana, that tells you something important. But if the two DB targets are still on the board and you take the kid from Louisiana early in the process...that tells you something completely different.

Most schools (I won't say "all" because I don't know) use recruiting services for their baseline information. They give you a database of several thousand names. From that database, a school will send post cards or literature to several hundred kids (maybe 1,000 at some school). It basically says, "We know about you."

A slightly smaller percentage will then get a letter that asks if the kids has ANY interest in the school whatsoever. If they reply that they do, they get moved to another list. If they don't reply and the kid is a good enough prospect, they might still get mail. If they DO reply and say "no," then usually, a school will move on. It's not the end of the road, but you can see it from here.

Then you start sending things like media guides and establishing contacts with high school coaches. Now we're down to maybe 300-500. From that, you build your recruiting board. I'm sure this is done on a computer spreadsheet now...but the boards I saw in the early 90s actually were big whiteboards. They were broken down by position and they had a wishlist by position. As the recruiting season moved along, guys would move up and down based on performance, response to recruiting, feedback, etc.

From that list of 300-500, not all will get offers. It all depends on how well recruiting goes. I think I saw somewhere earlier that Michigan has offered something like 180 kids this year. That sounds about average. The single greatest recruiting effort I ever heard of was the class at Notre Dame that had Jerome Bettis in it. They offered a grand total of 35 kids and got commitments from 24 of them. That's such an outlier, it's only worth mentioning as a statistical oddity.

When Rivals and Scout first popped up, coaches loved them. It gave them a chance to get feedback from the kids that was relatively neutral. Your big WR target just got back from a visit to Penn State. Did he like it? What did he like? Where does he rank you now?

The problem is that recruiting services call these kids so much that the players are used to it. They just give the same answers to everyone now. To let you know how far things have changed, I covered David Garrard (Jax Jaguars)'s first game as a varsity player. He threw for about 300 yards in a half at the JV game and they moved him up to varsity at S. Durham High School. A friend tipped me off so I went to see him. He blew me away (as a freshman, he was probably the best QB in the state. He struggled in the years after that when his Chron's Disease kicked in). I went down and talked to him afterwards and told him I was working with a recruiting magazine and he had no idea what I was talking about. He'd never heard of such a thing.

That's a very rudimentary outline of how it gets done and it's probably no surprise to most people. I'm just replying to Rasmus' question.

Coaches have networks that they listen to. One of the best things about the clinics that college coaches run is that it allows them to meet all of these guys that then act as their eyes and ears for players. The first thing most coaches do when they get hired is spend their first offseason going around to the high schools in the state and pressing palms and introducing themselves. You DONT want to make any enemies. Every coach has a guy in his corner that steers players towards him.

Those guys will also tip coaches off about players they might have, underclassmen to keep an eye on, and players at competing schools that they saw and liked.

It's entirely possible that the Michigan staff found Johnson that way.

I should also point out that the impact of this glance at the numbers is based on football talent adhering to a simple bell-curve.

Though you mention it (quoted), I think a normal distribution is a huge distortion. That is, unless you're assuming 100% of high school football players intend to play in college, on scholarship. You should revise this to have a double peak with a very large spike at talent level "0."

Following your analogy on the LSAT, you could assume that all college graduates are in the pool for being accepted to law school. For 2006: 48,171 LSAT takers with 1,485,242 total college grads -- about 3% of college graduates took the test. If your sensitivity is to the full range, it's obvious you wouldn't be able to differentiate between those at the top 3% (assume -- probably incorrectly -- that they're at or near the top). If you look at just the top 3% (pool of students getting 2-5 stars) there are absolutely going to be differences between the best and the worst.

I think where your piece really hits home is like Noahdb when you start looking at the 3 stars and such.

However, if you look at the top teams in college football they are the teams that clean up in recruiting - Florida, Texas, OU, OSU, USC, LSU and recently now Alabama. Starting with the 2003 BCS title game, these are the only teams to have participated in the BCS title game.

Ohio St has won 5 straight Big Ten titles. Starting with the 2002 season either OU or Texas has won the Big 12 every year except once (Kasnsas St - don't remember the year). This was the first year USC didn't win the Pac-10 title since 2002. Since 1999 I believe either LSU/Florida or Alabama have won the SEC title every year except once (Auburn). These teams aren't pulling in a bunch of 3 stars and coaching them up. They pull a few 5 stars, a good amount of 4 stars and fill out their classes with 3 stars.

Just because you have really good recruiting classes doesn't mean you're going to dominate on saturdays. Tennessee is good example of that. You need to have a decent coach too.

Iowa and Wisconsin are good examples of teams that aren't dominate in recruiting but usually put good products on the field. However, I believe the last time Iowa won a big ten title was 2004, and the last time Wisky won a big ten title was 1999.

Well researched and a very interesting take.

I'd be interested to see how more teams being added to D-I has affected the distribution of talent, if at all.

Most will agree that star ratings are more for bragging rights amongst fans. The parity in college football today is closer than it has ever been in the past.

With Sam Webb:

http://mvictors.com/?p=5913

Worth reading all the way through, but here's the relevant comments:

"The regional guys get together and come up with the rankings. ... in practice, I don’t think there’s a big difference between a three and a four star. It is all opinion and some of it politically influenced. Some networks are more SEC conference based. Some analysts are more tied to particular schools, so their recruits are rated more favorably. Notre Dame has benefitted from that. Michigan has benefitted from that in the past, not as much anymore [laughs]. There’s a lot in play when it comes to the rankings. I tell people to not take the rankings, or recruiting in general, so seriously. It’s a guide and it gives you a picture of what things look like and it gives you a glimpse at what kind of athlete a guy may be. But a recruiting analyst is never going to have as much information on kids as coaches, so what you have to do when it comes to recruiting classes is trust the scouting ability of your coaches and wait to see if they prove worthy of that trust. You’ll see that eventually on the field."

His comments are pretty much in line with my opinions, FWIW. I remember someone from Auburn saying once that there are three seasons in the state of Alabama: Football season, football recruiting season and the spring game.

"Bragging rights" is exactly right. People want to be able to talk smack at school or the office about how their team is going to rule because they got the #18 OLB in the nation...blah blah blah. And yes, you DO sound like a guy who just drank his first beer when you engage that guy.

Someone up above mentioned Tennessee as an example of how recruiting is just the first step. Coaching is huge, of course, but keeping the kids eligible and in school is even bigger.

Attrition is like termites. It gets in and it just erodes your program. When a kid quits two years in or gets thrown out of school, it just takes all of the time and effort you've spent on him and thrown it into the toilet. You can't get that time back and you can't replace that player. You just start from scratch with the next guy.

There is a certain amount of time that has to be spent just teaching a freshman how to be a member of the team. Here's how to keep your locker, here's how to stretch, this is where you need to line up and this is how you need to practice. When we say you need to be here at this time, this is what we mean.

When that guy quits on you, you have to go through all of that knuckle-headed stuff with someone new and that's time NOT being spent teaching that guy how to read the safety during the pre-snap cadence. Or how to shed a block at the second level or any of the other thousand things a good player has to do.

Back in 2001, I happen to see a list of Tom Lemming's top-100 players in the country. As someone who followed recruiting, I printed it out and stuck it in a file. Michigan had a bunch of players on there. Kelly Baraka, Pat Massey, Ernest Shazor...

(I found the list online...here it is, in case you were curious: http://espn.go.com/recruiting/s/2000lemmingtop100.html

I remember Massey being a solid player, Shazor being a colossal disappointment, and I have no memory of Baraka. I looked him up and apparently he got thrown out of school for smoking pot.

If you can't keep them in school, it really doesn't matter how good they were.

A few comments:

1) Regardless of how players are distributed, the most likely situation is that the gap between player 1 and player 50 is much higher than the gap between player 51 and player 100, which is higher than the gap between player 101 and player 150, etc. This is commonly seen in fantasy drafts, and applies here as well.

In the situation you've described, basically the bottom part of the distribution is getting cut off. But the distribution where the slope is the steepest is still there.

Now, there's still the issue of whether stars are properly applied to players. But assuming you had a perfect star system, the fact that there are more 'replacement level' recruits actually means stars/performance are even MORE important. Again, this is big in baseball, where 'value over replacement' or 'wins over replacement' have big sway among analysts. As the value of a replacement level player goes up, the worth of the super star actually increases, because it becomes so much harder to differentiate from the pack.

2) The best test of the star system is longitudinal -- how many 5* players pan out vs. 4*, etc. That analysis usually reveals a real effect -- the average 5* IS better than the average 4*. What confounds the issue is that exceptions abound -- there are 5* duds and 2* superstars.

3) Probably the best argument to be made is that the simple glut of players makes it difficult to get accurate ratings of players, so the star system has less meaning than it did before. But again, only a true longitudinal analysis will give the answer.

So, if there are more replacement level players available, then there's even more of a premium on getting great talent. But I do agree with your point: assuming that most teams are unable to evaluate and hoard that talent, then there is a definite premium on coaching, facilities, etc. Not to mention, dare we say, 'football smarts', which may be one way to distinguish two players who are athletically equal.

I wonder if the star rankings are more accurate for some positions than others. I'd think, for instance, that it's easier to project how good a running back will turn out to be, than an offensive lineman? The primary success factors for an RB - speed, vision, balance, agility - would seem to be more easily evaluated from tape than at least some of the success factors for an OL - mental agility, physical development, willingness to watch lots of film.

Even just watching YouTube of Dillon Baxter, you know the guy won't go the way of Dan O'Neill.

....because they effectively help discriminate among a larger group of candidates. Analogy: say (purely for the sake of argument) I went to FancyPants University back 30 years ago when they had 10,000 applicants. You could get admitted with good scores good grades and an ability to put a complete sentence together. Today, FancyPants gets 30,000 applicants. Now, you need to have been Touched By Greatness - some national award, or at least you should be made of dilithium.

In this case, the stars (national recognition) become increasingly important.

The statsgeek in me wants a complete roster of HS players controlling for personal and team records, size, academic ability, etc. AND the number of stars. The question is does getting the stars add to the quality of the team you land on over and above that other stuff, and has the size of that effect increased or decreased over time. (Don't mean to put you guys to sleep but that would really nail it.)

Let me play devil's advocate here and suggest that individual experts (coaches) don't necessarily know more (in the sense make better predictions) than some national system. Here's why: individual coaches predict player performance with some error. Any given player, some coaches will underestimate him and some overestimate him. If the star system takes into account the opinions of many (maybe it doesn't), it helps alleviate that problem.

It's like with finance - the expectation is that any one stock analyst has a larger error in valuing a stock than the market as a whole.

Of course, if the star system is just some self-appointed dude assigning stars (instead of some aggregation), that argument is toast.

that stars don't matter? I think that's the position of many posters since Michigan isn't getting the stars it used to get. By this logic, UM should get more lower rated players so they have an even bigger chip on their shoulder, maybe NR players.

And your basic conlcusion is that you can turn any one or two star into Charles Woodson or Lamarr Woodley, which I disagree with. And when you say coaching is a check, are you implying that RR is the best coach in the big-10?

Maybe I'm not reading this right, but .3% of 841,900 athletes is 2,526. While .23% of 1,108,286 athletes is 2,549. You are saying that schools are recruiting fewer athletes, but in actuality they are recruiting slightly more... Again there is the issue of extrapolating data and the addition of 14 teams, but it seems to me like the number of recruits has been relatively constant.

I don't think this really holds water. Yes, there's an incredibly long tail of high school football players who aren't good enough to play D-I college ball. That doesn't, in any way, diminish the gradation between the five-star guys and everyone else.

Let's take Terelle Pryor, who got more press than any recruit ever. He was the #1 five-star QB that year. The #1 four-star dual-threat QB was E.J. Manuel of FSU, who hasn't seen the field yet started a few games in injury relief this year. The #1 3-star dual-thread was Kody Spano of Nebraska, who may or may not ever actually play quarterback for the Huskers.

There could be a billion high school quarterbacks beneath that, who aren't fit to hold Spano's jock--but no amount of motivation or "rightness for the system" will make them perform like Pryor.

Peace
Ty

Manuel started several games this year for FSU when Christian Ponder went down.

Just sayin'...