Enough with the ESPN Rankings Bashing

Submitted by blueloosh on January 19th, 2012 at 3:30 PM

The most irritating refrain I hear constantly on this board is that ESPN does horrible recruiting rankings.  As I have maintained time and again, that just simply isn't the case.  There are objective ways you can evaluate the accuracy of the various sites' player evaluations (all-conference, NFL draft position, etc.).  Those analyses are useful but I think the most in-depth way I can look at rankings is to study, in retrospect, the evaluations of players with which I am very familiar.  Under that theory, below I am going to provide data on the evaluation of Michigan players from the classes of 2009 and 2008.  Why 2008 and 2009?  In most cases enough time has passed that you can make a judgment about how good a player turned out to be.  I think this data shows that ESPN does often zig when Rivals/Scout zag, but ESPN is hardly inferior to the other two.

I am going to present tables that show the rankings for the 2009 recruiting class broken down two ways.

First, star rating and position rating.  With regard to "star rating":  for Rivals, I am using the more detailed point system, and for ESPN the more detailed numerical rating.  For positional ranking, note that Rivals breaks out players into more categories so sometimes their ranking will look lower as a number.

2009   Rivals Scout ESPN   Rivals (pos) Scout (pos) ESPN (pos)
William Campbell DT 6.1 ***** 79   5 6 21
Justin Turner DB 6 ***** 80   3 3 21
Tate Forcier QB 5.9 **** 81   5 15 14
Jeremy Gallon ATH 5.9 *** 77   11 45 80
Craig Roh DE 5.9 **** 83   7 8 4
Je'Ron Stokes WR 5.9 **** 82   14 17 8
Vladimir Emilien DB 5.8 *** 80   14 42 21
Cameron Gordon WR 5.8 *** 78   36 83 62
Anthony LaLota DE 5.8 **** 80   11 15 13
Taylor Lewan OL 5.8 **** 80   16 20 12
Denard Robinson ATH 5.8 **** 81   14 16 7
Michael Schofield OL 5.8 **** 77   18 10 39
Fitzgerald Toussaint RB 5.8 *** 79   8 49 28
Quinton Washington OL 5.8 *** 82   8 19 6
Isaiah Bell LB 5.7 *** 81   26 46 11
Brandin Hawthorne LB 5.7 *** 78   35 33 39
Mike Jones DB 5.7 *** 77   25 49 54
Teric Jones RB 5.7 *** 78   37 44 48
Vincent Smith RB 5.7 *** 77   36 102 60
Thomas Gordon ATH 5.5 ** 77   X 115 77
Brendan Gibbons K 5.4 *** 77   8 11 16

Second, overall ranking (complication here is Rivals goes to 250, Scout to 300, and ESPN only to 150 -- so a guy that doesn't make ESPN's list could be #151...that is why I included the above data too):

2009   Rivals 250 Scout 300 ESPN 150
William Campbell DT 26 35 x
Justin Turner DB 35 26 x
Tate Forcier QB 164 137 144
Jeremy Gallon ATH 151 x x
Craig Roh DE 156 67 47
Je'Ron Stokes WR 104 169 67
Cameron Gordon WR 233 x x
Anthony LaLota DE 215 116 x
Taylor Lewan OL 194 274 148
Denard Robinson ATH 188 159 101
Michael Schofield OL 221 135 x
Fitzgerald Toussaint RB 239 x x
Quinton Washington OL 213 x 81
Isaiah Bell LB x x 94

FYI: for that set I only included the guys who made the "top whatever" list for one of the three sites.

Takeaways from 2009

Outliers, Good

  • ESPN - Will Campbell (he was a top 40 overall to the others, a good-not-great player to ESPN)
  • ESPN - Justin Turner (same deal)
  • Scout - Vlad Emilien (Rivals and ESPN thought he was a top 20-ish DB -- Scout was far more bearish)
  • Scout - Isaiah Bell (had him a sane 46 at his position, rather than 26 (rivals) or 11 (ESPN))

Could also make an argument for:  ESPN - Denard (101 overall vs. 159 and 188); Scout -J. Stokes (had him too high at 169 overall, but much better than Rivals (104) and ESPN (67); ESPN - Lewan (148 overall vs. 194 and 274); Scout - Q. Washington (had him 19 at his position, while the others overrated at 8 and 6).

Outliers, Bad

  • ESPN - Q. Washington (had him 81 overall - not a top 100 player)
  • ESPN - I. Bell (had him 94 overall - way off)
  • Scout - Fitz Toussaint (had him too low at #49 back -- jury is still out on whethe Rivals (#8) or ESPN (#28) had him pegged best, but as an optimist I would say it may be Rivals)
  • Rivals - Mike Jones (had him #25 at his position -- other sites much better ranking him 49 and 54)

Could also make an argument for:  ESPN - Stokes (had him #67 overall compared to #104 or #169 -- didn't include above only because position rankings are pretty close across the sites (14-17-8); Scout - LaLota (had him #116 overall which was less off-base than Rivals (#215) and ESPN (something above 150), but again positional rankings pretty close); Scout - Lewan (had him #274 when the other sites were much better at 194 (rivals) and 148 (ESPN).

There are many other conclusions you could draw from this data, and those conclusions will depend greatly on your opinion of a particular player.  For example, Rivals was much higher on Gallon.  But I'm not sure yet who was right or wrong -- depends what you think of Gallon.

But based on my own analysis ESPN and Scout did pretty well this year.  Rivals not so much.

On to 2008...

2008   Rivals Scout ESPN   Rivals (pos) Scout (pos) ESPN (pos)
Boubacar Cissoko DB 6 ***** 78   4 3 24
Dann O'Neill OL 6 **** 82   10 14 4
Darryl Stonum WR 6 **** 82   7 12 14
J.B. Fitzgerald LB 5.9 **** 80   18 10 14
Kevin Koger TE 5.9 **** 73   4 6 113
Michael Shaw RB 5.9 **** 78   7 29 59
Brandon Smith DB 5.9 **** 75   11 4 75
Ricky Barnum OL 5.8 *** 80   5 17 4
Kenny Demens LB 5.8 *** 78   23 23 35
Taylor Hill LB 5.8 *** 75   21 25 70
Mike Martin DT 5.8 **** 80   16 12 8
Sam McGuffie RB 5.8 **** 79   10 7 30
Elliott Mealer OL 5.8 **** 77   24 28 42
Brandon Moore TE 5.8 *** 81   8 43 6
Terrence Robinson RB 5.8 **** 80   9 16 18
Roy Roundtree WR 5.8 *** 76   44 89 104
Marcus Witherspoon LB 5.8 ****     20 14 x
Mike Cox RB 5.7 **** 77   x 35 70
Justin Feagin ATH 5.7 *** 40   41 93 x
Rocko Khoury OL 5.7 *** 70   41 65 99
Martavious Odoms WR 5.7 **** 78   71 49 56
Kurt Wermers OL 5.6 **** 78   37 11 20
J.T. Floyd ATH 5.5 *** 75   x 75 75
Patrick Omameh DE 5.1 *** 69   x 87 114

I have put emphasis on the obvious outliers above.  Sadly, the way a site looked smart this year was to doubt one of our recruits.

Takeaways from 2008

Outliers, Good

  • ESPN - Cissoko (others had him a top 5 CB...not so much)
  • ESPN - Mike Shaw (hurts to say, but I think they were right)
  • ESPN - Brandon Smith (elite to Rivals and Scout, ESPN had it right)
  • Scout - Ricky Barnum (I love Barnum but Scout had a more realistic take on him)
  • ESPN - Taylor Hill (see Smith, Cissoko)

(I am skipping McGuffie and Mealer because I think both ran into some horrible luck and may have turned out differently if that was not the case -- but based on an objective look at production ESPN had them right too)

  • Scout - Brandon Moore (Still hoping Scout is proven wrong, but....looks they were right and Rivals and ESPN were wrong to call him a positional top 10)
  • Rivals - Roy Roundtree (Finally Rivals gets one!  They had Roundtree much higher than the others (#44 WR vs. 89 or 104)
  • Rivals - Kurt Wermers (another case of Rivals getting it right; Scout's #11 positional ranking looks especially bad here)

Outliers, Bad

  • ESPN - Dann O'Neill (I am faulting ESPN here for pegging O'Neill as a super-duper star (#4 OT); the others thought he was good, but not that good)
  • ESPN - Kevin Koger (he may not have produced like a #4 or #6 TE, but I think Scout and Rivals were much closer to the mark than ESPN who had him as the #113 DE)
  • Rivals - Mike Shaw (talented guy, but not a #7 back)

ESPN has the biggest hits and misses here, but I think overall does the best in 2008.  Scout comes in second.  Rivals last again.

Overall Takeaways

The track record from these two classes does not support the notion that ESPN is out to lunch or does not know what they're talking about.  Personally?  I think they're wrong about Pipkins.  But I can't say that ESPN's track record shows I can discount their view.

By my count, here is the tally on major outlier picks that seem to have a definitive right/wrong result from 2008 and 2009:

ESPN: +6, -4 (net 2)

Scout: +4, -1 (net 3)

Rivals: +2, -2 (net 0)

Obviously my quantitative measures are subjective; I offer this as food for thought.  Please discuss and improve on what I did here.  But let's not dismiss sites out of hand.  As the above shows, there is no basis for that.  Certainly not with respect to ESPN.

Comments

JT4104

January 19th, 2012 at 3:44 PM ^

Mainly the 2008 class, whether is be injuries or whatever, it's hard to say what some of the guys did.

Who knows how Cissoko would have done had he you know actally been of age and such. Barnum has has injuries and was behind Schilling for 2 yrs.

I dunno, I would take a look after they leave school instead of some guys who haven't even left school yet.

RB's Mustache

January 20th, 2012 at 4:36 PM ^

The number of players is too low for an accurate statistical comparison of any meaning. The rankings assume a full career, and, more importantly, good coaching in the right system. With RR and his revolving door of crappy DC's who didn't teach the same system, nor positional play or proper technique, who is to say how good they'd be? ESPN rates our recruits lower (so people get upset), but pointing out that they have underperformed doesn't prove the writer's hypothesis because it fails to eliminate the LARGE numbers of environmental factors that stunted their development.

Magnus

January 19th, 2012 at 3:45 PM ^

FWIW, I have seen (and done) analyses that have shown Rivals to be superior to Scout at choosing 1st round draft picks.  The average 1st rounder has a higher star rating on Rivals than Scout.

ESPN (and 247) is too new to get a good handle on how they project down the line.

Obviously, being an NFL draft pick and being a successful college player are two different things...but it's one metric, at least.

blueloosh

January 20th, 2012 at 11:04 AM ^

I have actually looked at the NFL draft data and come to the same conclusion.  But that points to an independent question, which is:  what are the sites' ratings supposed to mean?  I have always assumed it means "how well this guy could contribute to a typical NCAA football team."  I assume that was at least the original point to the ratings.  Which college football team was getting the best future college football players.

Now the ultimate worth of a player is his ability at the NFL level.  But assigning recruiting rankings based on NFL potential seems like a different undertaking to me.  Guys like Eric Crouch or Desmond Howard are not potential All-Pro players, but had amazing college success.  I would not rank those guys too high out of high school if I knew I would later gauge my success by looking at how many of my Top 50 made it to the NFL as a regular starter.  The interesting question to me is:  what is the approach for the various sites?  I honestly have no idea and suspect it may not be the same.

mejunglechop

January 19th, 2012 at 4:11 PM ^

ESPN has some obvious problems with their rankings that probably are too small to be likely to show in a sample this small.

A few complaints:

  • ESPN has a stake in the grades of Under Armor bowl participants and those players frequently receive outlier grades.
  • Less transparency in the ranking process.
  • Less frequently updated grades/rankings.
  • Disproportionately many southern recruits and disproportionately few midwestern recruits with top grades (compared to other services... note this doesn't necessarily mean they're less accurate)

Good work putting this all together, but doing a persuasive analysis would require a metric shit ton more data.

bjk

January 19th, 2012 at 8:35 PM ^

is stronger than what the author at above link said:

However, the differences were minimal between the top two recruiting services. While many fans claim that one service is significantly better than another, actual results showed that the top two are similar, while the third is not far behind.

Edit: I guess it depends on what you mean by "signicantly."

bronxblue

January 19th, 2012 at 4:48 PM ^

Nice breakdown.  I still think ESPN is too new to have a good track record, but this list basically highlights the truth with all recruiting sites - they have their crutches that they fall back on, and with ESPN the issue seems to be with a strong focus on the UA game and the South.  I mean, the top 40-50 or so recruits seem to be consistent across the board, but after that you see these regional biases that drive people crazy. 

With ESPN, my other issue is that they focus WAY too much on pro prospects, at least in my opinion.  They love kids with speed and size, and sometimes they focus on that over technique because (they seem to think) those kids will mature and become draft picks.  Since they also run the TV coverage the draft, it helps for them as a branding entity to show a consistent narrative from HS to the pros, but that also feels like a conflict of interests, at least to me.

ninetysevennat…

January 19th, 2012 at 6:47 PM ^

Thank you amigo. I looked through the commits lists from the last 5 years the other day on espn (not pay-walled so no excuses) and came to an eyeballed conclusion: the order the players were listed in was the most consistent with how they've played, developed, been drafted, and/or done anything else awesome. It would be great to have Rivals be the standard for our own competitive pride in most cases, but as long as our classes are in the top 10 and the coaches are passing on highly ranked players who aren't highly ranked people, we are in great shape.

p.s. Our class is still listed ahead of Ohio on espn and it's the only site that has both Garnett and Kalis listed correctly as guards. 

ninetysevennat…

January 19th, 2012 at 6:56 PM ^

Having said all that, espn is way too conservative on Mr. Pipkins. Can't wait to see him and Richard Ash eating up o-lines and backs. Yeah, I said it, all you defensive tackle fretters out there! Who has ever let us down from the greater Pahokee area!? No one.