Can anyone help me get those two random bits of HTML out of this post?
I thought that myself when I read that article that talked about a Data Scientist(tm)
Note: This Diary comes with the standard disclaimer that recruiting ratings are subjective, and that this amusing exercise is not intended to answer the fourteen unanswerable questions.
Several mgocolleagues have objected to my disturbing avatar, the congenial visage of Gothmog, the Morgul lieutenant from the Lord of the Rings movies (not, for fellow nerds out there, to be confused with his namesake, Gothmog, the Lord of the Balrogs, from more ancient Tolkien history). I have a potential replacement for this avatar: Sylvester McMonkey McBean of Sneetches fame.
Why McBean? Well, Mr. McBean adds and subtracts stars with his Star On and Star Off machines, which is the exact skill set that we are looking for if we would like to answer the Pat White question once and for all.
Our new coach arrived in Ann Arbor with a reputation for finding diamonds in the rough, and for turning three stars into five stars with one part eye of newt, one part Mike Barwis, and two parts mad offensive genius. Was this by necessity or design? Rich Rodriguez appears to have an aptitude for spotting hidden talent and Mike Barwis is clearly able to turn 45-year-old couch potato fantasy leaguers into Ray Lewis, but are we making lemonade out of lemons? Has Rich simply been forced in the past to settle for a kettleful of lower-rated players and relied on outliers in a normal distribution to produce the occasional Pat White? <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
The debate has intensified since, after a three and nine season, Michigan recruiting has begun to resemble West Virginia recruiting a little too closely for comfort. The optimistic among us believe that Rich Rodriguez can create a juggernaut from three-star players that are a perfect fit for his philosophy and also happen to possess that killer attitude not present in such players as Gabe Watson or Alex Mitchell. Said killer attitude, combined with said killer, Mike Barwis, may be the foundation of Rich McMonkey McBean’s purported eye for diamonds.
How do we move this question from the realm of water cooler optimism to something more analytical? That is the purpose of this post. I am proposing a collaborative, ongoing post-recruitment rating system that will allow us to determine if, in the Rich Rodriguez era, perfect-fit three-stars are more desirable than random four-stars. Of course, we all acknowledge that perfect-fit four-stars (or perfect-fit five-stars, for that matter) are better than perfect-fit three-stars, but all these questions can be addressed with a system that continues to rate recruits after they begin playing football at the University of Michigan
Here’s where the mgocommunity comes in. As the work of one grotesquely deformed orc, this enterprise would hardly be taken seriously. However, if the career ratings that follow the recruitment ratings are the consensus of what can be objectively described as the most sophisticated and knowledgeable football blog community out there, then we might have something. I am not polishing the apple here; anyone who reads the Diaries on a regular basis knows that this is accurate.
The point here is that we want to expose where recruiting ratings were/are wrong. In the 2002 class above, for example, Rivals was slightly wrong about Gabe Watson and very wrong about David Harris. There was no malice or incompetence in their being wrong; they just were off in projecting those players. It is an inexact science. It is also important to note that this uses the recruiting ratings in a specific way: as a college career projection and not a snapshot of their status at the end of their high school career. Only in this way can the information be useful to us.
What now follows is a request for feedback regarding preliminary judgment calls I have made to launch this system. Which needs a name. Perhaps McBean Rating, but I am open to suggestions here.
The definitions are critical. These career ratings must mirror the intent behind the recruiting ratings for our conclusions to be useful. (Note: All recruiting ratings used are from Rivals.)
Question 1: Do you agree with these definitions for the two rating systems? If not, please suggest changes.
Question 2: What should be done about players that leave the team? Consider 2008. Dann O’Neill was a four star player coming in; does he count when he leaves after a year? What about a Justin Feagin, who played a little and was kicked off the team? Clearly, Taylor Hill or Marcus Witherspoon, who never put on a uniform, should not count…but is this so clear? Should there be no penalty for recruiting someone who leaves? Remember, the ranking of the class coming in is based upon Dann O’Neill and Taylor Hill being there.
This is a critical question in my opinion. If we start removing players who don’t play much or at all, then we’ve messed with that cryogenically preserved Team Ranking that is a touchstone in the Pat White debate. <?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Michigan has had six players from 2008 leave, including four 4-stars. Newsflash: that drops that class to 15th or 16th, all other classes staying the same. Which, of course, all other classes are not the same. Who the hell knows what Alabama’s ranking after the fact truly is? We obviously can’t re-rank the Rivals rankings; we must work with signing day class.
For purposes of making a start, I have judged that if the player sees the field for more than a cup of coffee, they get a rating. With this model, O’Neill, Wermers, Witherspoon and Hill do not get a rating from McBean, but Feagin and McGuffie do. The star calculation for a class over time, then, compares the final average from Rivals to only the McBean rated players. In the 2002 class above, all 21 players produce an average of 3.52 stars. The McBean average is 3.37 stars using a denominator of 19, not 21. Thoughts?
Question 3: Check my conclusions for lightly used players. This is where the mgocommunity will be critically helpful. I could barely remember some of these players and Googlestalking was marginally productive. For example, did Quinton McCoy and Tom Berishaj in the 2002 class ever play? The greatest inaccuracies will be for those players who played a little, like Feagin, but I don’t remember because it was years ago.
Question 4: If you disagree with my McBean rating for any player, please educate me. I am getting old and my memory is not what it used to be.
Here are the first three classes in which all the players have a final McBean rating (none are playing college ball any longer). It would seem, based on a preliminary review using the tentative rules that Lloyd’s classes underperform slightly. It will be very interesting over time to see how Rich’s classes do.
Here are the next two classes that have incomplete McBean ratings.
I will somehow compile the feedback and present a final McBean Analysis for each “closed” class. As each class becomes closed, I will present the final after feedback. Hopefully, Rich produces guys on the left.
Can anyone help me get those two random bits of HTML out of this post?
Ha! Got rid of one by stripping it in Notepad. But the other won't go. Aww, crap. It's lower, now. Help, someone.
gets 3 stars, heh?
Consider Sears officially up for 2-star status.
First, I think a five-star for Breaston is too high. I still don't see that you take NFL into it, and many have pointed out that the Super Galacticon Spread Rich Rodriguez-Only recruit is not rated as highly by the Rivals or Scout people because, well, they just don't understand it all, and they are total morons like everybody else in the world. So, I think it should be based more on what they accomplish at Michigan and not anything based on NFL other than possibly where they were drafted.
Second: I agree with you on players that leave. If they don't sniff the field, no rating either way. On players that do leave before their eligibility is used up, ya got to figure in some sort of penalty, though, because that is what you'd have to call a "miss"--but not much, maybe just a .10-.25 star.
Third: I love this concept and want to see how Rich Rod's recruits do over their careers, especially compared to Lloyd. I'm hoping the criticism of Lloyd and player development turns out to be correct when compared to Rodriguez.
1) I debated Breaston. And I agree about NFL play not counting, but draft status and NFL cache is clearly one measure of their career.
Breaston proposed for 4-star status.
He clearly, when not slowed by injuries, was an elite college player despite an inability to consistently catch the ball, which has mysteriously vanished at Arizona. But I wouldn't argue much either way.
Also, great post. I'm trying to collect my thoughts enough to respond appropriately later.
If he never played WR, and only returned punts and kickoffs - he would still be a 5* in my book. Let's go to the all time records in Michigan history.
Here are some of Steve's return records:
#1 all time most return yards in a game (223).
#1,2,3,4 in most return yards in a season. Seriously, this guy set four consecutive records in four years, with his last season (2006) actually being the lowest.
#1 all time return yards in a career.
#1 all time most TD's from returns in a career with 5.
We have a little on both sides of the Breaston argument. Can we get some tie-breakers out there?
Even with the last couple classes shown above, you can see how Lloyd's staff started slipping in recruiting toward the end. That 2003 class is awesome.
I'm not sure that there is a 1:1 correlation between the Rivals rankings and the McBean rankings. Example (5 star):
Elite, Dominant player, Complete package. Projects to the highest level.
Stud. Multi-year starter. All-something. NFL player.
I'll take "All-Something" to mean First team All conference in a BCS conference. That is 6 conferences x 22 players per all conference team = 132 -- plus kickers -- 5 star players. Does Rivals hand out 100 5 star ratings that often?
The NFL drafts (32 players/round x 7 rounds = 224 players) which leaves lots of room for the 4 stars to fit into the draft
I think that you are attaching some sort of superlative to the 5-star player that your definition doesn't hold (but may belong in it...) However, if you are attempting to compare Rivals Ratings to "McBean ratings", then you should (from a national perspective) have the same number of 5 stars...4 stars...3 stars...(which is admittedly hard to do). I would tighten up the definitions some more to give yourself a better guideline. 1st, 2nd team All-America and honorable mention definitely fit into this category. I think you have to ignore specialists (Sorry Zoltan--still a solid 4 star and probably Steve Breaston -- really only an AA KR, only a decent receiver. I'd ask about Chad Henne and Mike Hart too...but I could be convinced of 5 stars for them).
This is a very important comment and opened my eyes. The five-star definition has to be very much tightened up. Perhaps in light of your comment, the following should be downgraded to 4-stars:
Leaving up the following five stars over those five classes listed...
After seeing the first couple of images I expected your entire post to rhyme in the style of Seuss' writing. I was disappointed.
Lloyd Carr + Tom Brady = 6th round draft pick
Rich Rodriguez + Pat White = Nuff said
Michigan + RR + Barwis = the Victors, HAIL!!!
I'm not sure the NFL player stipulation for 4 and 5 stars is appropriate. We all have allowed that Rodriguez's players are great in college and being 5'3 might not make it to the NFL. Noel Devine isn't Adrian Peterson is what I'm saying, he also wouldn't be a 3 star. I mean Dan Mozes (sp?) on the weight training staff was an All American but barely got a phone call from NFL teams. The old Michigan would probably need the NFL Player distinction, and evaluating old classes could work for that, but moving forward with Rodriguez it's admitted his style is for college and success in the Pros is not that great a correlation for the skill and effectiveness of Michigan's current players.
Second comment of this nature and I agree that the NFL qualifier needs to be softened.
Excellent diary! Some possible star updates...
Chris Graham - should be 4 star, should not be rated same as Max Martin (3 star). 20 career starts, All b-10 Hon Men (2007)
Justin Schifano - should be 2 star
Brandon Logan - should be 2 star
Johnny Sears - should be 2 star
Andre Criswell - should be star
These 4 guys have had the same impact as D. Cone (2 star) on this team. Little or no impact, invisible.
Obi Ezeh - should be 4 star. Multi-yr starter, possible NFL player.
Greg Matthews - should be 3 star, weak or avg starter, similar career to Carlos Brown thus far.
Kates, Banks, Ferrara - should all be 2 stars, mostly invisible. Possible argument for Ferrera 3 star, but that was just because he was moved to OL for injury purposes. Probably will never see field again.
Cutting and pasting...Coments in CAPS.
Chris Graham - should be 4 star, should not be rated same as Max Martin (3 star). 20 career starts, All b-10 Hon Men (2007)...CANNOT...MAKE...CHRIS GRAHAM...FOUR-STAR...DOES...NOT...COMPUTE
Justin Schifano - should be 2 star
Brandon Logan - should be 2 star
Johnny Sears - should be 2 star
Andre Criswell - should be star
These 4 guys have had the same impact as D. Cone (2 star) on this team. Little or no impact, invisible. DULY NOTED. UP FOR TWO-STAR STATUS.
Obi Ezeh - should be 4 star. Multi-yr starter, possible NFL player. I THINK THE JURY IS OUT. FORTUNATELY FOR THIS SYSTEM, WE GET TO SEE HIM FOR ANOTHER YEAR.
Greg Matthews - should be 3 star, weak or avg starter, similar career to Carlos Brown thus far. AGAIN, WE GET TO SEE HIM FOR ANOTHER YEAR, BUT THIS IS REASONABLE.
Kates, Banks, Ferrara - should all be 2 stars, mostly invisible. Possible argument for Ferrera 3 star, but that was just because he was moved to OL for injury purposes. Probably will never see field again. NOTED.
They will never learn. No, you can't teach a Sneech...
This is a great idea.
I think you're being too generous in dealing with people who leave the program. We never hear the full story of why someone leaves, but with someone like Dann O'Neil, it seems pretty clear that a big part of it was that he wasn't going to get much playing time. As of right now, he looks like a recruiting dud. But if you give him an N/A, it shows up as a neutral in the "McBean" rating, not as a negative.
To fix this, I suggest you take into account each player's performance after they leave Michigan. If Ryan Mallett turns into an all-world player at Arkansas, give him a five-star McBean rating. If O'Neil is an average lineman at Western Michigan, he's a two-star. If someone suffers a career-ending injury, make them an N/A.
Of course, that assumes that what you care about is each recruit's actual ability to play college football (which to me is the best way to judge a coach's recruiting ability). If you care only about their contributions at Michigan, stick with what you have.
Good point, although daunting. I am wondering about how to do that. I doubt we hear of O'Neill...maybe he has a career as an average starter at WMU. What does that equal?
And Boren. He may turn out to be a five-star for the Bucks. Does that then influence how effective Michigan recruiting is?
I need more input.
Most of us know what happened to high-profile transfers like Jason Forcier after they left the team, so they're easy to keep track of. On the other hand, if you never hear from a player again, it probably means they did nothing and should be a 2-star.
If Boren becomes a five-star for OSU, I'd say make him a five-star on your list. It means that Lloyd Carr was able to bring in an excellent player, which is what you're interested in measuring. The ability to keep a player on the team is a different skill altogether.
Even like a Taylor Hill? Is he really a recruiting "trophy" that RR deserves if he goes on to five-star status? Seems to be a disconnect there.
That's a tough call. If a player doesn't qualify (like Weatherspoon, if I remember right), I don't think you get credit for recruiting them. I think I would exclude Hill since he left before his freshman season started. It was as if he was never on the team at all. On the other hand, you obviously have to count someone like McGuffie, who left halfway through his freshman year.
At some point, you'll have to draw a somewhat arbitrary line, and there's no way to make it perfect.
hmmm I really like the thought you put into the post, ultimately time will tell, but that is so darn hard to wait on huh? Rating of the recruits vs on-field results are much like predicting the weather too far ahead.
You also seem to imply later in your post that Gothmog was an orc not sure this is entirely true, more likely a dark Numenorean along the lines of the Mouth of Sauron.
At the risk of sending everyone screaming from this thread, I would say the jury is out on Gothmog. His race in not explicitly stated and the movie went with an orc, so I used orc.
This is an interesting topic and one of those wonderful topics to sit back and discus with buddies over a beer or three. But, my thought is that if it is to have any meaning, it has to use the same criteria as the major ranking services. Dan chose Rivals which is fine. After a few minutes of digging, I was able to find this (I was a little suprised that it was not on one of their main menus as in ESPN/Scout rankings).
My thought in evaluating players through this exercise is that the rankings project talent and success, that is, a future tense. Dan, on the other hand, I think is looking for past tense, did they live up to expectations. Perhaps one way to evaluate talent is in draft position for at least the top players, the rest may require a few adult beverages worth of conversation to sort out.
Rivals Star and Rating System
"Players are also ranked on their quality with a star ranking. A five-star prospect is considered to be one of the nation's top 25-30 players, four star is a top 250-300 or so player, three-stars is a top 750 level player, two stars means the player is a mid-major prospect and one star means the player is not ranked.
The ranking system ranks prospects on a numerical scale from 6.1-4.9.
6.1 Franchise Player; considered one of the elite prospects in the country, generally among the nation's top 25 players overall; deemed to have excellent pro potential; high-major prospect
6.0-5.8 All-American Candidate; high-major prospect; considered one of the nation's top 300 prospects; deemed to have pro potential and ability to make an impact on college team
5.7-5.5 All-Region Selection; considered among the region's top prospects and among the top 750 or so prospects in the country; high-to-mid-major prospect; deemed to have pro potential and ability to make an impact on college team
5.4-5.0 Division I prospect; considered a mid-major prospect; deemed to have limited pro potential but definite Division I prospect; may be more of a role player
4.9 Sleeper; no Rivals.com expert knew much, if anything, about this player; a prospect that only a college coach really knew about."
ESPN/Scout Star and Rating System
"Five-star player: 85 to 100
A potential immediate game-changer at the major college level who could push for all-conference recognition as a true freshman. Will enter college with elite skills needed to create mismatches against top-level competition. A future All-American candidate.
Four-star player: 79.5 to 84.99
A potential 3- to 4-year starter at the major college level with the superior skills needed to see early playing time and make a significant impact during his college career.
Three-star player: 75.0 to 79.49
A player with the skills to develop into a solid starter at the major college level. Potential high-ceiling prospect with the ability to make an impact during his career.
Two-star player: 68 to 74.99
A player who flashes developable major college talent and could be a late bloomer. Potential solid starter at the mid-major level.
One-star player: 55 to 67.9
Player does not show the physical tools or skill set at this point to project at the major college level, but could contribute at mid-major or FCS programs."
Thanks for that research! I couldn't find the Rivals definitions. I have already agreed with Wolfman that we needed to tighten up the five star standards to be more in line with Rivals, and I think I will review the others in light of those numerical guidelines.
I was heading out the door to an appointment when I quickly posted the above. As I was driving to and from said apointment, I had a couple of thoughts on how we could use a hybrid (how nouveau Michigan)of the two scouting service's evaluation criteria for a post factum evaluation of whether recruits fell short, lived up to, or exceeded recruiting service expectations. And as these evaluations happen, we may be able to get a sense of coaching diferences between the Old Regime of Camelot and the New Regime since the Barbarians have taken over in terms of how they are developing talent.
Post Factum Hybrid Star System
To be evaluated a player must remain in the system through their junior year. An earlier washout due to academic issues or injury would earn you a "washout" designation (a lowly single star). Transfers are simply removed from the books...they are like decommits. As per the New Regime, we only talk about players who play for Michigan. (This might make another interesting metric with which to evaluate a program: evaluating how each program's transfers fair once they are gone)
5 Star *****
This is a player who was drafted in the 1st round of the NFL entry draft and is thus regarded as one of the top 32 players in the nation (mirroring Rivals top 25-30 prospects) or baring that was an All-American and/or a 4 or five year starter (i.e. started as a freshman and started virtually every game they played at Michigan) and played at an "elite" level during their playing career.
4 Star ****
This is a player who was drafted 33rd-250thish (hard to peg an exact number due to compensatory picks) and thus compares to Rivals top 250 players or was a 3-4 year starter who made a "significant imapact" by their play.
3 Star ***
This is a player who was a 1-3 year starter (i.e. they started a minumum of 12 games) who made a "solid" contribution to the team.
2 Star **
Ordinarily a starter for a mid-major school. This is a player who ended up being a back up at Michigan. He started less than 12 games and chose to stay and get his degree here rather than transfer and perhaps start elsewhere.
1 Star *
A washout who ended up not playing because of injuries or academics or is kicked off for team rule violations, as long as they do not end up successfully playing football for another team, at which point they are simply deleted from the roster and if they succeed elsewhere we give ourselves a DOH!
Using the above critera, using the draft as the primary talent ranking tool (letting players earn a four or five star rating by other means is another more difficult discussion, although I would tend to be tighter and rank guys down rather than puch them up), it seems that Dan has been overly kind as he rated Michigan's outgoing talent. Acording to the draft, here are our 4 and 5 star players of the last few years:
5 Stars *****
Alan Branch (a 4/5 star player, drafted 33rd)
David Bass (a 4/5 star player, drafted 33rd)
4 Stars ****
Chad Henne (high 4 star)
Mike Hart (low 4 star)
Adrian Arington (low 4 star)
Lamar Woodley (high 4 star)
Tyler Ecker (low 4 star)
Tim Massequoi (low 4 star)
There may be a few undrafted players who may merit a 4th star and perhaps one or two 4 stars who should gain a 5th star, but for the most part, when I look at this list, it seems about right. The bulk of the rest of our players would then slot into the three or two star categories.
Excellent work, but I have a concern. Ultimately, we care about winning at Michigan, not the NFL career of the player (mentioned by others). If Steve Brown plays like he has always played this year, but gets drafted because he is a physical freak in the 3rd round, is he a four star? Do we then, in a water cooler debate, say to Sparty that Lloyd really knew how to pick safeties because of Steve Brown? Doesn't Michigan success ultimately have to be the yardstick?
Now almost always your system will correlate with a monster college career, so I think we'd simply have to introduce a check on the NFL metric for players who don't deserve to be that highly (or lowly - RR's tweener and tinies might really suffer in your system) rated.
he would have to continue to play to the expectations of that draft position. Since most draft picks do not become starters, some kind of Years In The League metric would probably be better. I've definitely seen value expectations by draft selection, not sure where though (Football Outsiders? Advanced NFL stats?). It would be nice to bring that all together.
The only problem with doing that is that it is exactly the exercise Dan is trying to work through in terms of our recruiting. How draft picks work out is a worthwhile question to answer (we might ask how many Michigan picks work out vis a vis other programs in terms of "productive years" or some such of our athletes vs. theirs) but is really beside the point. We are looking at the change between one moment in time in evaluating an athlete (at the time of recruitment) vs. another moment in time (when they leave the program). What we are trying to assess is whether our guys are failing to meet, meeting or exceeding expectations by the time they leave the program. It is a away of evaluating the success of our coaching staff beyond the metric of wins and losses.
could certainly be done incorporating NFL performance. If we're trying to evaluate the coaches, the biggest knock against them would be not making enough of a legitimately talented player.
Do we need to bring up Pat White again? He's to college football what Hitler is to history. He always comes up, given that a discussion goes long enough. But seriously, in a hypothetical college football draft, he would be a top 5-10 pick from his class. I don't care where he was picked in the NFL, all I know is what he did in college.
I propose that you break it down this way. With 20/20 hindsight, if you could recruit them onto your team for their senior season you would recruit them:
***** - obsessively, probably all-conference at least
**** - without reservations but not all-anything material
*** - with reservations, not thrilled about it
** - this player would probably never see the field
By senior season, you can finally correctly evaluate a player. Any player that leaves after their junior season is probably a 5 star, but you can extrapolate if you feel necessary.
This is the dilema of a star rating system. If all our recruits are "system" guys who help us win and we shred up college football now and forever, it might be possible that, in spite of winning, fewer of our players than might be expected are drafted into the NFL. I am sure that at the high school level there are countless players who made significant controbutions towards helping there teams win who never recieved scholarship offers to play at a major college program. There will always be players whose importance to the program and its success are such that they merit a high grade rating, in spite of not getting the external "guru rating" vis a vis the NFL draft.
But that said, the NFL draft provides an important external metric for evaluating Michigan's talent. It helps answer the question, "How does our talent stack up on a national level?" In other words, what does the NFL think of our talent as compared to say USC, Texas, OSU, Florida, OU and so forth? On the intake side, we all want Michigan to be recruiting the best athletes who will help us win. Recruiting system guys is important, but we also recognize that there is much to be made of the perception that this program is attracting the best talent in the nation, hence the importance of the class ranking each year. But irregardless of winning and losing, one of the questions in an incoming recruit's mind, and one of those metrics that measures what a program does with the talent they have beyond winning football games, is how many of the players go on to play in the NFL? In that regard, just as the recruiting services act as a filter on high school talent, determining which athletes have the best chance of success at the next level, so too the NFL draft acts in a similar manner.
There is no easy answer, then, on how much importance in terms of star rating to place on a player who helped our team out immensely, but whom the NFL sees as a "meh" talent. Mike Hart is perhaps the best example of this. Tom Brady is and excellent example of the opposite scenario.
To what end are we assigning these stars to players post factum? Is it to answer the question of the controbution they made to Michigan? Or is it to answer the question of how well Michigan is developing athletes during their stay here? My thought is that in the end we are trying to answer both questions in order to answer a third question, perhaps the most important question, "Are our athletes over or under performing in relation to their scouting service ranking?" Next to winning that is perhaps the most important metric of how our coaching staff are doing. In that regard, I would tend to tilt the weight of my criteria towards external validation vis a vis the draft before looking at percieved controbution to the program.
When one looks at it that way, it is not hard to come to the conclusion that the in the waning years of the Old Regime, our athletes under-performed expectations. We won enough games to remain respectable, but the draft numbers in terms of four and five star athletes is substantially smaller than the number of four and five star athletes who came into the program.
The question going forward is will that trend turn around? The past history of Coach Rodrigues is that that he has an eye for under rated talent and has a knack for coaching guys up. In the last presser video released through official propaganda (or it might have been his BTN interview) he was once again emphasizing how excited he gets teaching fundamentals to the young men on his team. My expectation is that the post factum guru rating for our team will rise and exceed initial expectations. In addition, there will be a lot of guys who do not get the guru ratings either on the incoming or outgoing side of things, who will make significant contributions to our team and who will also earn that extra star.
I like the direction you are going. My concern: while the comparison between Carr's program and Rodriguez's program might be somewhat meaningful, I don't think it really tells us all that much unless we're able to compare his body of work to his contemporaries. In other words, how did his "Class of 2009" turn out relative to other top recruiting classes of the same year? If his "McBean" rating is 0.5 lower than his Rivals overall score, is that dip normal across all of college football when converting from Rivals to McBean? Do top-level BCS teams fare better in this Rivals-McBean transform than mid-major programs? I wish such an in-depth analysis didn't require so much work.
While it doesn't show quite the same thing, an easier study would be to compare overall team recruiting scores with performance on the field. The arbitrary nature of both scheduling and voting makes the potential study somewhat less clear-cut than it otherwise might be, but performance on the field could be weighted with respect to opponents, and could be analyzed for comparisons within conferences.
That pretty much rounds up the issues. I think the problem becomes easier when we realize that we can use different metrics as tiebreakers, and that most players will be relatively easy to grade based upon either system.
I think the "underperformance" of Lloyd's recruits is basically what you'd expect to see under normal conditions. That is, regression. I presume that 3 stars is the mean for BCS recruits.
As for injuries and transfers, I'd prefer that be compared to some normal attrition expectation than simply thrown out. Staying healthy, as Will Carroll says, is its own skill and being able to identify and maintain health is a potential marginal (dis)advantage. The White Sox, for instance, have been cited in particular for their ability to keep pitchers healthy and performing where other MLB clubs do not. Given the Barwis' Gym Advantage™, that may be where Rodriguez's ability to "turn" 3-stars into better than average players lies. Healthier players mean relatively more lottery tickets.
My guess is that players that are smaller than positional norms are most prone to additional injury, which could have implications for the way Rodriguez runs his program. Barwis may simply be getting these smaller players to normal attrition expectation.
Obviously, this post and the idea are excellent. It would be nice if Scout/Rivals did this of their own accord. The alternative is like running PECOTA just once in a player's career. New data should lead to new assessments. It would be nice to have a continually updated idea of the talent available in every BCS program, that's certain. In the very least, we could certainly use the wisdom of crowds to have our players assessed, as you suggest. May I suggest google forms?
For the record, it's my guess that Lloyd's players had above average injury-related attrition. Probably above average transfer-related attrition as well, but that's to be expected for a top program. As a rule, a player will prefer to play over sitting and programs with less talent will always present that opportunity.
On the other hand, if you're transferring, that means there's a spot for you, which presupposes a departure elsewhere. If transferring is a zero-sum game, then high transfer levels might indicate something else about the program's quality.
Except transfers seem to happen downward. Besides Threet, I can't think of a Michigan transfer offhand, and that one sure evened out.
Dan, really enjoyed this diary as I've wanted to do something similar to test my hypothesis that the Northeast and Midwest is underscouted and whose players receive slightly lower guru rating than they deserve.
I read the diary this morning and reread it and I realized something. There's really two types of McBean Ratings. It really depends on what you want to measure.
1)The rating purely measures the contribution to the college footbal team over 4 years and nothing else. Thus tranfering would be a huge ding, perfomance on the field in college would be the main focus, and what they did in in the NFL would be irrelevant.
2)Purely a measure of what kind of player the prospect becomes. In this case the NFL is almost the only thing that matter and where no NFL data exists you would have to look at the player's most productive year(s) in college.
Comparing the guru ratings to McBean Rating 1 would tell you if the how well coaches developed talent, keep talent on campus, identified talent that fit their schemes, and something about their ability to judge talent in general.
Comparing the guru ratings to McBean Rating 2 would tell you purely how well the coaches identify talent when they are still prospects and also something about how well they develop talent.
I think most of us would care more about the first rating or scenario because it directly affects the performance of our team.
But if we compare these two McBean ratings together I think it would get really interesting. It would tell us how much we got out of our best players compared to their performance in the NFL. Most of the time a we would want our McBean 1 rating to be higher than the McBean 2 rating because it would show our players performed at a higher level in college than in the NFL and thus our couches got the most out of their talent as possible. The opposite would suck and mean that are good players were becoming great players in the NFL (think last few years of Carr era).
This was kind of wordy but I find this kind of study fascinating and depending on what we want to measure we can find out a lot about how our couching staff identifies, utilizes, and developes talent.
So what do you guys think? Is there a clear distinction between the two McBean ratings and would both ratings be useful?
Thanks for the extensive contributions so far. As this post has tumbled off the main page, I am going to process all the comments and post a McBean update this week.
Look forward to it...
Although the NFL draft is an important metric, I too agree with a number of you that it should not be the end all and be all in terms of handing out stars, we should use caution in too easily upgrading guys because we like them and their competition. The star rating system is not an internal only metric, but, as in recruiting, should measure how our athletes stack up against the best athletes in the nation from all the other programs. It is not unlike the question of how many stars would you assign the undrafted Texas Tech quarterback who leads the nation in passing yards? Is he as good as the guy who was drafted #1? There has to be other means of attaining a high star rating, but they should be used sparingly. Stars should be hard won through results. An undrafted All-American who started for 4 years should get that fifth star. That guy who started three years in our system and had success in it because he was a great system fit might end up being a high 3 / low 4 star simply because he would not have made such big controbutions elsewhere.
There are key words that the ESPN pole uses that I find helpful here:
5 star = "elite"
Should be reserved for the very best athletes in the nation only. It should be tough to earn that fifth star and when you are done there should only be about 25-30 of these guys nationwide.
4 star = "significant imact"
There might be a few undrafted system guys who earn that fourth star simply because of the huge impact they had on their team. Think a Mike Hart type of player who does not get drafted. Short of that type of controbution, they should remain a three star athlete.
3 stars = "solid"
Basically everyone else and most of the system guys. It is where they are now on the intake side and will likely be where they will be post factum. This is why winning football games should still be the most important metric of a coach's success or failure.
To be honest, I think some of the most important rating will take place down at the bottom of the scale. If you look at the classes above, what drags down a class (and gives Lloyd's classes a slight "underperform" grade) is guys ending up as two stars...being "invisible."
That is an area that has more impact on this analysis than whether Breaston gets a fifth. Since Michigan almost never recruits two star position players, is it fair that so many recruits end up as two stars because they can't get on the field?