Rankings: Are the "Experts" really experts?

Submitted by Wee-Bey Brice on July 17th, 2014 at 7:13 PM

I never really cared about rankings but from what I saw at the opening , all websites have the rankings so wrong

— Uncle Mike (@mikeweber25) July 14, 2014
 

Interesting tweet from Mike Weber expressing his thoughts on the rankings from his experience at The Opening. How wrong could they be? And how often is that the case?

Comments

gwkrlghl

July 17th, 2014 at 7:19 PM ^

It's generally regarded amongst people who see things other than "WOW TOO STARS GOT DRAFTED" that rankings are pretty reliable. There's variance in everything, but guys are generally rated somewhere for a good reason

RationalBuckeye

July 17th, 2014 at 7:23 PM ^

Whether the guys doing the rankings are personally ex-players and coaches or not, most have been doing this long enough to have a good idea of how high school kids fit into the system, and how the progressions occur at the next level. Not an exact science, of course, but probably quite a bit more accurate than a high school player who saw a microcosm of the nation's talent at a camp that primarily takes place in workout clothes.

evenyoubrutus

July 17th, 2014 at 7:37 PM ^

There are enough variables that no one can detect, measure or predict that there will always be variance and inaccuracies, but it is scientific enough that the rankings will average out to be quite accurate in large enough quantities and over long enough periods of time.

Unless you're talking about ESPN.

Farnn

July 17th, 2014 at 7:39 PM ^

I'd generally take a coaches opinion over that of the ranking services because one has been around football for 40 years and gets paid millions of dollars and the other has a journalism degree and makes 80k and has been doing this for maybe 10 years.  However, on the whole the ranking services are pretty accurate and a decent measure of the likelyhood of player success.

alum96

July 17th, 2014 at 10:23 PM ^

Wouldnt someone like a Tom Lemming on the other hand who owns a service make bookoo bucks?  Selling newsletters in the stock market - if you are one of the big ones - is massive cheddar so I can only imagine what Lemming does - he has books/pdfs... I bet he clears a very pretty penny.

But these guys on the ground who are regional and work for Scout or Rivals - who knows.

alum96

July 17th, 2014 at 10:24 PM ^

At its peak, Rivals.com employed close to 200 people, operated a network of 700 independent websites, filed for an initial public offering worth $100 million led by Goldman Sachs, and sponsored the Hula Bowl in Hawaii.[6] However, economic troubles and the collapse of the dot-com "bubble" soon led the Rivals Network, the parent company of Rivals.com, to cease operations in 2001, though it never sought bankruptcy protection.[6] Executives from AllianceSports purchased the Rivals.com assets and subsequently relaunched the website.[7] Heckman, who had been fired as chief executive officer, later started a competitor network initially named The Insiders, later renamed Scout.com[7] and sold to Fox Interactive Media in 2005 for a reported $60 million.[8]

Led by former AllianceSports executive Shannon Terry, Rivals.com became profitable. On June 20, 2007, Yahoo! agreed to acquire Rivals.com.[9] Terms of the deal were not disclosed, but several sources reported Yahoo! paid around $100 million.[10]

Wolfman

July 18th, 2014 at 12:24 AM ^

because they are in the mold of a Hand or Peppers and there simply aren't many of them around. Also considered is the age of the player so those doing the evaluating are aware whether they are evaluating a 17 year old kid who started school at 4 instead of one with a learning disability who received a waiver for same and was playing ball against competition over two years younger. 

Yes as to Lemming and others. There mags, for the most part have now gone the way of the internet, with many of them actually being employed by .com types instead of publishing their own mags.  You must also remember that just like selling magazines, these websites have to keep readers all over the country happy. As soon as a 4 or 5 star commits to a school, wait about ten minutes for another to take his place.  It does a site no good to not have their team in on the 4 and 5 stars so that is why they are passed along so freely.  Afterall, how much justification does it take to move a 5.7-Rivals 3 star to 5.8, thereby making him a four star.  1/10 of a point really is not that significant.  Even Mike Hart was not considered 4 * material, but the minute he committed to UM, they made him a 4* cb!!!!  Rojo, from Muskegon, a legit 5* as a cb suffered an injury his junior year and decided to primarily concentrate on his receiver duties in hiw senior year. Guess who all of a sudden went from the top cb in the country to the top receiver? Wasn't difficult was it.

One Rival rater went so far as to tell the truth one season.  He stated "The state of AL had so damn many good players this year, there were actually quite a few other kids who deserved a fifth star.  However, we can't simply give the coveted 5th star to the most deserving when we have so many readers from more populated areas of the country." 

I think the most important thing about ratings is to concentrate on those schools that get commitments from the kids at the earliest possible dates. These are the ones that more often than those named later that perform as expected. Also watch tape. One look at Ty will give a good indication of just how fast this kid is. Same is true of Harris.  The kids that simply run away from the defenders, and they do it continuously are the fastest kids in the nation. Those that manage to beat the defenders to the goal line, some of which are gaining on them, might not even be the fastest on the field. 

But as another poster correctly stated, way too many get caught up in measureables. There are coaches out there that have a real knack for predicting which kids will fit into their systems. That is why many coaches won't extend an offer until they get a kid on campus, do an in home visit and gain as much information on a kid, be it a 3* or 5* as they can, in an attempt to ensure they aren't wasting one of their most precious commodities.  The first time I saw Mike Hart run a football was against IA. He was on the ten and carried three of their linemen with him - including Roth - to the one yard line where he lost the ball on a fumble.  You knew right there why Carr extended the offer. Kid saw the flag and wasn't going to be denied, although in that case he was but the run itself was magnificent. And for the first time since Bo took the sidelines, I'm guessing, the kid that fumbled was the RB the next time the team took the field. 

Yes, they do know their football and they know which kid can "make all the throws" "tackle in space" and all the other cliches you see associated with this game.  At the same time they are selling their services. It's their life's work so the kid that gets bumped to a 4 when someone falls off the board was included in the disscussion before the first list was released. And remember, as stated above, there is simply not that much difference between 5.7 and 5.8, and often times, because it's a guessing game when they are so closely matched, the kid that gets the bump will turn out to be every bit as good as those listed from the start. 

JayMo4

July 17th, 2014 at 7:41 PM ^

It's always worth remembering at this time of year too that these guys haven't played their senior seasons or any of those post season all star games, either.  There's gonna be a lot of moving things around before everything is finalized.  In theory some of what Weber thinks they have wrong right now ought to be corrected in time.  In theory.

LSAClassOf2000

July 17th, 2014 at 8:27 PM ^

Multiple diaries just on this blog (and probably others) have shown that the correlation (down to R-squared values) between average or composite ranking and many statistics that might indicate success tends to be highly variable at best depending on the specific metric. That being said, however, if you try to link it to a broader view and not necessarily specific on-field statistics (although there are some that correspond well to average star rankings), the rankings do tend to accurately estimate overall success and when used as an input to team success, I know we've shown here that sustained higher average rankings in classes do correspond to sustained team success generally, although there are confounding factors like overall profile, size of class and whatnot. 

alum96

July 17th, 2014 at 10:32 PM ^

I think the general thought it more 5 stars will hit it big than 4 stars (as a %) and more 4 stars than 3 and more 3 stars than 2 stars.  But of a pot of 100 of any stars you will have big misses to the upside and downside.  You just increase your chances of missing out on busts the higher up the star food chain you go - and in theory increase your chance of finding players with higher ceilings.

We had a 2 star from the state of Michigan be the top pick overall in 2013.

My belief is a lot of 4 stars are just physically advanced human beings especially in their sophomore and junior years.  If you run faster, and hit harder, and weigh more than your peers you will look great at the HS level.  But most of these guys are not playing similar players at the HS level.  So it's a lot of extrapolation.  A lot of 3 stars could be the same as the 4 stars ,but just 18 months behind in physical development.

You watch some HS games and its dudes on the line who are 6'2 270 lbs going against guys who are going to be accounting nerds in college and are 5'10 215.  Or WRs who are 6'1 180 lbs going againt 5'9 150 lb corners.  How you judge that across thousands of games across the country and adjust for competition is very difficult.  A lot of "freak athletes" at age 16 might be just that - freak athletes who dominate at their age and are near their peak.  Meanwhile a "Caris LeVert" type in football would be a low 3 star whose body is way behind his peers and turn into a college star... etc.

Look how often the NFl gets it wrong after 4-5 more years of data on all these players when they are in their early 20s.  It's a lot of guesswork and prudent player evaluation goes a long way - ask Snyder at Kansas State and yeah Dantonio and a few others.

Roc Blue in the Lou

July 17th, 2014 at 11:22 PM ^

To the extent one's ranking is based on sexy, widespread offers...um, yeah.

Otherwise, so much of the rankings have to do with measurables, i.e. hight, weight, frame, wingspan, 40 time, vertical leap, ect., and these often correlate well to the next level.  You can't coach size or speed "up" very much.  But, S & C can add mass, muscle and strength.  Many of the experts talk about celiing, and guys who have the skills/technique, but lack in the measurables, are usaully deemed to have a lower ceiling.  That makes sense, because you can coach up a kid with great measurables, but lower skills, more so than the opposite.  I don't see this as all that different than the tension between NFL combine fanatics and old school scouting reports.  Personally, i think the rankings are more for us fans to stay highly involved on days other than gamedays...and that's just fine by me.  But, i like to think Hoke & co. types are watching for the next Jake Ryan or Mike Hart while their counting stars for studs like Peppers.

A Fan In Fargo

July 18th, 2014 at 3:00 AM ^

The only thing that matters to me is that they are wrong enough where it bugs you. Terry Richardson was ranked what? He start smoking diggity dank non stop or what? Peppers too, what a bunch of crap. This guy should've been #1 on all sites. Fournette on ESPN and Hand on Rivals. Ask any coach in the country who they would've took out of all three and I'm willing to let you bitch slap me in the face if the majority doesn't pick J-Rock a.k.a salt and peppers. That's all the haters not wanting to see Michigan get the number one pick on all sites. Think about it. Just remember I wrote those names too in case they stick. Knowing you guys, you won't let it happen. Your damn greed and arrogance. Anyway, there isn't a chance that Fournette or Hand has done or will do what Peppers is going to do on the biggest stage in the big house. He'll be a house hold name like Woodson by October 2015 if not sooner. Take it to the credit union.

steve sharik

July 18th, 2014 at 3:08 AM ^

...a lot of recruiting services do: base evaluation off one day, in the summer when motivation may be lacking by some, without pads, in a very non-football game simulation.

Best way to evaluate is to do it at actual games, in person, over the course of time.

ThadMattasagoblin

July 18th, 2014 at 6:16 AM ^

It ain't everything but I'd rather have a 5 star with offers from Alabama, USC, LSU, Ohio, PSU, Florida, Georgia, etc. than a 2 star with offers from Wake Forest, Bowling Green, Toledo, Western Kentucky, San Jose State, etc. I'm not saying that a 2 star can't become an all big ten or all american but those players are being recruited by MAC and Sun Belt schools for a reason.

Jimmyisgod

July 18th, 2014 at 12:10 PM ^

Not true, but I get your point, the services look at offers.

Kaleel Gaines, CB from Florida, offers from Bama, Auburn, Florida State, Ohio State, ND, Nebraska, TN, MSU, Penn State etc Kid is barely a 3 star recruit, 5.5 on Rivals, 84 on 247.  I think some kids they totally whiff on and then they're almost too pridefull to admit it and rank him accordingly.

In the case of Gaines, it sickens me to say it, but Dantonio and Saban are probably the best defensive back talent evaluators at any level in all of football, wherever this kid goes, he's going to be a stud most likely.  Unfortunately it will likely be for MSU and we'll all be wondering where the heck this kid came from in a few years when he's All-B1G.

What I find is extremely telling with the services is what they'll rank a previously unranked player who commits to one of the premier schools.  Saw a zero star, unevaluated player debut as a 4 star a couple days after committing to Georgia.  WTF?  You're telling me the services just happened to evaluate him the same week he committed to Georgia?  Not a chance, they looked at his height and weight, probably read a blurb about him, and maybe watched a 60 seconds of film on him and then said, "well, he looks the part, and he committed the Georgia, so, low 4 star!"  Now the services know almost nothing about this recruit, but he's suddenly a 4 star and no matter how he performs his senior season in high school, 4 star will be his baseline.  Kid could end up being terrible or he could end up being great, but there was zero reason other than where he committed for him to get 4 stars.

And I've seen it with guys who commit here too, I take it with a grain of salt. 

I also think position matters too, some positions are much harder to evaluate than others.  defensive back is tough, there are only a few good coaches in all of college at evaluating DBs.  O linemen is also tough because of the huge size disparity between a top level lineman and who he is playing, you also can't predict accuratley how a big kid will grow into his body.

I feel we are getting some great players and we have some coaches who obviously have a ton of experience evaluating talent.  But I don't get too excited about rankings.

jblaze

July 18th, 2014 at 9:27 AM ^

If journalists were actually good at predicting talent, they would be scouts, coaches, or something better than bottom feeders for scout/ rivals/ 247/ espn.

They have no clue and manufacture stories for page views (which apparently we love to give them).

This also explains the sudden bumps kids get when they are offered by big name programs and the ranking sites use offers to base stars on, so that at the end of the day, they look like they have a clue.

aplatypus

July 18th, 2014 at 10:05 AM ^

completing refuting what you're trying to say... be more adamant!

Every year this is questioned, and numerous times it's been shown that 5 star players pan out better than 4 than 3 than 2. It doesn't mean every one is better, no one above middle school thinks statistics work that way. 

The reason some players get bumps after an offer is that there aren't enough service scouts to track all 100,000+ potential prospects. There are 4 main services, with only a dozen or so regular scouts of their own going if that. There are over 100 universities with several coaches, scouts, and grad students to pour through film. Schools have incredibly more resources so when they offer, that's a sign to the recruiting services to check out guys they haven't yet, or look for more tape for someone they had limited scouting on. 

jblaze

July 18th, 2014 at 10:31 AM ^

4 and 5*'s are often awarded after a kid has been offered by a big time program. The scouts themselves have no clue. They rely on guys like Saban, Meyer... To offer kids and then re-rank guys based on that (although they never say this).

The "stats" that everybody and their mom use are the final rankings, which take rely heavily on offers (or coach choices). Nobody has looked at initial rankings and success.

TSimpson77

July 18th, 2014 at 9:51 AM ^

A lot depends on the player too, look at Big Will. He said he was his biggest problem. You can have all the talent in the world but if you don't have the work ethic you won't go far, talent can only take you so far.

Jimmyisgod

July 18th, 2014 at 11:43 AM ^

School is more important and I think that they give stars out based on school the majority of the time, that's the only way the rankings have much predictive value.  Certain schools are going to produce NFL players no matter what, a huge part of rankings is looking at those schools and ranking accordingly.

Plus, I think it's harder and harder to separate the top 500 players or more, sure, the top 50 are clearly a cut above most of the time, but with the internet you have more players with access to the best training methods, more high school coaches learning the best coaching methods, and access to the types of diets you need to have.  There are absolute elite level athletes all across the country who get overlooked.  The large number of great athletes makes it tougher and tougher to predict who will be great in college.  There are more elite athletes that are ranked as 2 stars than ever before, some coaches excel at finding them.

And high school film is almost pointless to watch for us fans and likely most people who work for the services.  Any mid to high level college recruit basically looks like a man among boys in high school.  That goes double for linemen, what can you really tell about a 4 star offensive lineman who is pancaking guys that he outweighs by 70 lbs all game?  I don't put much more faith in the services ability to evaluate high school film than I do myself, and that's almost none.  You really have to be a skilled football coach to be able to correctly evaluate talent based on film, and even the best get it wrong a lot.

Furthermore, the camps give a different look, it's elite talent against elite talent, but I find it amazing how different the guys from the services take will be from the coaches who are actually running the camps and putting these guys through the drills up close.  It's like the guys in the stands think they know more than the coaches on the field.

We're always going to get our share of highly starred recruits, it's making sure we get the right ones and then fitting them into our system and developing them that will make the difference.  But as for the recruiting rankings, it's more of a crapshoot than at an y other time in history.

vinsacco

July 18th, 2014 at 1:25 PM ^

They can't coach or evaluate.. Period.

My mother can identify the elite athlete. It's not rocket scientist material. These coaches have to forecast the future on the guy that fits their program design. They deal more with the norm per day the three star then the 4 and 5 they call it.

Those sites are money generators. Nothing wrong with it. Just know your reading it for entertainment purposes only.

Having coached and played collegiately and professionally there is so much that goes into evaluating and developing a player. Star ratings don't make players, coaches do.

Tater

July 18th, 2014 at 1:57 PM ^

In basketball, for exampkle, if anyone was a perfect evaluator of talent, he would be able to recruit players that other major schools aren't going after and turn them into first round NBA draft picks.  

Oh, wait...