Before that, I wanted to give a summary of the informal voting that took place in the last thread as it related to the 2002 class (note: it would be helpful if more readers would vote – the number of votes was small, although the justifications were scholarly). Here was the 2002 class career ratings proposed after the McBean Rating definitions were debated:
The following votes were then cast:
Five-Star: Two votes
Four-Star: Three votes
Three-Star: Three votes
Two-Star: Zero votes
Three-Star: Two votes
Two-Star: One vote
Single votes for Bihl as a four-star, Breaston as a four, Tabb as a three and Riley as a three.
All votes are not equal as arguments of varying persuasiveness were offered. In the debate, wolfman81 almost perfectly captured what still remains a point of contention: how to use the NFL draft in our assessment -
Notice that in all of these cases, I consider the college career first. If there is some doubt about what category the player should fit into, only then do I consider the NFL draft. The NFL draft is a tiebreaker, but only when there is a tie to be broken. In my mind, 1st team All-American = 5-star McBean. All-Conference = 4-star McBean. Starter = 3-star McBean. There can be exceptions to this rule (Tom Brady was not All-Conference, but I think he should be 4 star due to his leadership on the field and other intangibles. As he was drafted, the NFL assessment backs this opinion up. The fact that he will likely be a first ballot NFL hall of famer cannot bump him up to the 5-star discussion; at the same time, it does not harm my 4-star rating.) But, I think that this rule sets a minimum standard.
The new 2002 McBean Ratings are as follows with only one change – Harris drops to a four-star. Mark Bihl was pushed hard as a four-star, but his career falls short of making a significant impact, just as Harris clearly falls short of an elite career.
The conclusion one would then draw from this data is that the 2002 class dramatically underperformed. To give you an idea, this would represent a drop in the team rankings (if the rankings were based upon average stars) from 10th to 21st, using an average of the 2002 through 2009 data.
Now, the problems:
- How do we handle the math for players like Dann O’Neill, Quinton McCoy and Taylor Hill? Players who never saw the field and did not stay. My original method counts these players in the Rivals list, but does not count them at all in the McBean list. For example with the 2002 class above, the 21 players in the Rivals list are all counted and the denominator is 21. For the McBean review of their careers, McCoy and Berishaj didn’t have a career at Michigan, so they are not counted and the denominator to determine the average star rating is 19. If we eliminate the two players from the Rivals list, do we fail to take into account the reference point of our evaluation: the quality of the entire incoming class as judged by a recruiting service? Yet, by taking them out of the McBean career list, do we fail to punish the staff for recruiting players who could generally be considered a bust? However, if we rate them a zero or even a two, we ensure that pretty much every class will underperform badly (counting both as a zero, the McBean average would drop to 2.81!). I am both befuddled and in need of a paragraph break.
- Second, the ease with which we rate invisible players as two stars devastates our average rating. We rarely recruit two-star players, yet because every team logically has many anonymous backups (seventeen preliminary McBean two-stars over the first five classes I’ve looked at), we have exceeded a ratings Chandrasekhar Limit and pulling each class down into a black hole of underperformance. Case in point, I offer the 2003 class which, if this class doesn’t qualify as a recruiting success and overperformer, then none will:
EDIT, August 29th, 9:24am: Gentlemen, after reading below and thinking about the problems presented, I feel we made a mistake with the definitions. Any viable Michigan players - this includes backups who do not see the field - should be get three stars. We could use this yardstick: any player who makes the two-deep during his career gets a third star. Also, we should be more generous with four stars for good starters like Bihl and Kraus.
Here's my reason: we must remain connected to our baseline system, Rivals. It is our point of comparison. Without a close connection, we can make no claims about under and overperformance. We recruit many three-stars, and while individually each kid hopes to be the next Pat White, the 30,000 foot reality is that those three-stars, en masse, are destined to be the backups. They are not underperforming, they are performing as expected.
This pushes some - not many and not enough to unbalance the finite four-star numbers - up a notch, because Mark Bihl and Adam Kraus cannot coexist with a three-star backup.
So, I am thinking we need to predominantly loosen our three-star standards to allow us to recognize backups as meeting expectations.
The McBean Definitions discussion was lively.
Only minor changes resulted from the debate:
- I removed the reference to a “five year starter.” Amazing that so many different sets of eyes (except cali4uofm) could miss that, although it was referenced elsewhere.
- I clarified the McBean four-star rating a little bit. The 2002 class has two guys who I initially rated as four stars – Mark Bihl and Rueben Riley – that I have dropped to three stars given that they were undrafted.
Three other issues were discussed:
- brad voiced a concern that if all starters at Michigan get three stars, there is no differentiation between a weak starter and a solid three-star guy like Chris Graham, for example. This is a reasonable observation, because I had Mark Bihl and Rueben Riley rated as four-star starters but backed them down to three (as I considered them borderline four-stars) because they were undrafted. So are both Mark Bihl and Darnell Hood three-star players? That needs to be further debated in this thread.
- There was some debate about punters and kickers, but I think the exception to a punter or kicker as a lower rated player can be handled in the rare event a Space Emperor decides to play ball on our planet.
- The dominant concern was using the NFL to assign career star ratings. SanDiegoWolverine voices this concern:
I don't think the NFL is that relevant in the sense that how our players perform in the NFL shouldn't change our perception of their value/production when they were at Michigan. I'd rather have Rod recruit players that dominate while they are at Michigan and underperform at the NFL than vice versa.
The draft is a national comparison, a national measure of the athletes.
And summarizes, I think correctly:
I agree with BlueBulls that ONLY the draft can inform a McBean rating, and only as a tie-breaker. Steve Breaston is a classic example. Four-star or five? He seems to be on the cusp – look at all the Michigan records and his current value to Arizona – but in the end, he is a four and the draft – 5th round – gives us a valuable assessment tool of his potential (developed at Michigan) at the time his career in college ended. So the NFL draft stays as a tie-breaker, and the final McBean Definitions will be:
"Elite" = 5*
"Significant Impact" = 4*
"Solid" = 3*
I think that you will find that, barring a few exceptions, most of those that get that fifth star will be first-rounders or high second-rounders; the second group will get drafted; and the third group will make up the bulk of the rest of the starters.
Let's begin with the 2002 class. There are some interesting borderline cases in this class. Given the currently assigned career ratings, this class underperformed significantly.
Note: On Monday, I posted Sylvester McMonkey McBean Helps Settle the Pat White Question Once and for All, complete with annoying HTML that I couldn’t edit out of existence without starting over. Won’t make that mistake again. What follows is the digested feedback for improving the McBean Rating System, the purpose of which, to quote myself, is to develop “a collaborative, ongoing post-recruitment rating system that will allow us to determine if, in the Rich Rodriguez era, perfect-fit three-stars are more desirable than random four-stars.” Our goal is to settle the Pat White Question once and for all.
Thanks all for the quality feedback. In the original post, I proposed that players be rated over their career at Michigan using definitions that approximated the spirit of the Rivals star ratings to see if, on average, players exceeded or fell short of expectations. The effort would span the later Carr years and, of course, the Rodriguez era going forward. How players performed relative to expectations would give the mgocommunity interesting information on the ability of a coach’s system to evaluate and develop talent. For more detail, the original post.
We began with these definitions.
These were found to be too loose. SanDiegoWolverine dabbled with the idea of a two-faceted rating system, but my IQ will not permit that level of complexity, so I used the following comments to revise the definitions above.
I'm not sure that there is a 1:1 correlation between the Rivals rankings and the McBean rankings.UMFootballCrazy found some numerical guidelines in Rivals:
Example (5 star): Elite, Dominant player, Complete package. Projects to the highest level. vs. Stud. Multi-year starter. All-something. NFL player.
I'll take "All-Something" to mean First team All conference in a BCS conference. That is 6 conferences x 22 players per all conference team = 132 -- plus kickers -- 5 star players. Does Rivals hand out 100 5 star ratings that often?
Players are also ranked on their quality with a star ranking. A five-star prospect is considered to be one of the nation's top 25-30 players, four star is a top 250-300 or so player, three-stars is a top 750 level player, two stars means the player is a mid-major prospect and one star means the player is not ranked.It seems that unless there is some loyalty to the proportion of star ratings allocated, then the McBean Rating loses its credibility. For example, if we dole out twice as many McBean five stars, is that Michigan developing talent or is that rose-colored glasses? It seems the NFL draft can serve in an advisory capacity, part of a checks and balances system.
UMFootballCrazy develops this:
5 Star *****He has more good stuff, but I steal so much of it down below, that I won't repeat it here.
This is a player who was drafted in the 1st round of the NFL entry draft and is thus regarded as one of the top 32 players in the nation (mirroring Rivals top 25-30 prospects) or baring that was an All-American and/or a 4 or five year starter (i.e. started as a freshman and started virtually every game they played at Michigan) and played at an "elite" level during their playing career.
Nedved963 cautions, however:
[M]oving forward with Rodriguez it's admitted his style is for college, and success in the Pros is not that great a correlation for the skill and effectiveness of Michigan's current players.I think Colin boils it down:
How you determine those expectations could certainly be done incorporating NFL performance.
And I think that is key. We don't want the NFL draft or an NFL career to answer entirely the question as to whether a player fell short of or exceeded expectations at the University of Michigan (i.e., Tom Brady), but it can be a tie-breaker, if you will, a check against overrating our beloved players (Breaston a five-star?) and ensuring proportion to the Rivals system.
As a result, I offer for final review, the new McBean Rating System definitions. I use the Rivals definitions (d'oh!) for the Rivals definitions. Special thanks to UMFootballCrazy as I steal a great deal of his language, but shift the emphasis, using the NFL as sort of a tie-breaker for those players on the cusp in the McBean ratings. However, I move away from UMFootballCrazy's proposed definition for the lower star players because I think there is a potential hazard to mishandling the lower rated players*:
Thanks again. If we settle on these definitions, I will re-rate the 2002 through 2006 classes.
* Again, to quote myself: "To be honest, I think some of the most important rating will take place down at the bottom of the scale. If you look at the classes above, what drags down a class (and gives Lloyd's classes a slight "underperform" grade) is guys ending up as two stars...being "invisible."
That is an area that has more impact on this analysis than whether Breaston gets a fifth. Since Michigan almost never recruits two star position players, is it fair that so many recruits end up as two stars because they can't get on the field?"
Note: This Diary comes with the standard disclaimer that recruiting ratings are subjective, and that this amusing exercise is not intended to answer the fourteen unanswerable questions.
Several mgocolleagues have objected to my disturbing avatar, the congenial visage of Gothmog, the Morgul lieutenant from the Lord of the Rings movies (not, for fellow nerds out there, to be confused with his namesake, Gothmog, the Lord of the Balrogs, from more ancient Tolkien history). I have a potential replacement for this avatar: Sylvester McMonkey McBean of Sneetches fame.
Why McBean? Well, Mr. McBean adds and subtracts stars with his Star On and Star Off machines, which is the exact skill set that we are looking for if we would like to answer the Pat White question once and for all.
Our new coach arrived in Ann Arbor with a reputation for finding diamonds in the rough, and for turning three stars into five stars with one part eye of newt, one part Mike Barwis, and two parts mad offensive genius. Was this by necessity or design? Rich Rodriguez appears to have an aptitude for spotting hidden talent and Mike Barwis is clearly able to turn 45-year-old couch potato fantasy leaguers into Ray Lewis, but are we making lemonade out of lemons? Has Rich simply been forced in the past to settle for a kettleful of lower-rated players and relied on outliers in a normal distribution to produce the occasional Pat White? <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
The debate has intensified since, after a three and nine season, Michigan recruiting has begun to resemble West Virginia recruiting a little too closely for comfort. The optimistic among us believe that Rich Rodriguez can create a juggernaut from three-star players that are a perfect fit for his philosophy and also happen to possess that killer attitude not present in such players as Gabe Watson or Alex Mitchell. Said killer attitude, combined with said killer, Mike Barwis, may be the foundation of Rich McMonkey McBean’s purported eye for diamonds.
How do we move this question from the realm of water cooler optimism to something more analytical? That is the purpose of this post. I am proposing a collaborative, ongoing post-recruitment rating system that will allow us to determine if, in the Rich Rodriguez era, perfect-fit three-stars are more desirable than random four-stars. Of course, we all acknowledge that perfect-fit four-stars (or perfect-fit five-stars, for that matter) are better than perfect-fit three-stars, but all these questions can be addressed with a system that continues to rate recruits after they begin playing football at the University of Michigan
Here’s where the mgocommunity comes in. As the work of one grotesquely deformed orc, this enterprise would hardly be taken seriously. However, if the career ratings that follow the recruitment ratings are the consensus of what can be objectively described as the most sophisticated and knowledgeable football blog community out there, then we might have something. I am not polishing the apple here; anyone who reads the Diaries on a regular basis knows that this is accurate.
The point here is that we want to expose where recruiting ratings were/are wrong. In the 2002 class above, for example, Rivals was slightly wrong about Gabe Watson and very wrong about David Harris. There was no malice or incompetence in their being wrong; they just were off in projecting those players. It is an inexact science. It is also important to note that this uses the recruiting ratings in a specific way: as a college career projection and not a snapshot of their status at the end of their high school career. Only in this way can the information be useful to us.
What now follows is a request for feedback regarding preliminary judgment calls I have made to launch this system. Which needs a name. Perhaps McBean Rating, but I am open to suggestions here.
The definitions are critical. These career ratings must mirror the intent behind the recruiting ratings for our conclusions to be useful. (Note: All recruiting ratings used are from Rivals.)
Question 1: Do you agree with these definitions for the two rating systems? If not, please suggest changes.
Question 2: What should be done about players that leave the team? Consider 2008. Dann O’Neill was a four star player coming in; does he count when he leaves after a year? What about a Justin Feagin, who played a little and was kicked off the team? Clearly, Taylor Hill or Marcus Witherspoon, who never put on a uniform, should not count…but is this so clear? Should there be no penalty for recruiting someone who leaves? Remember, the ranking of the class coming in is based upon Dann O’Neill and Taylor Hill being there.
This is a critical question in my opinion. If we start removing players who don’t play much or at all, then we’ve messed with that cryogenically preserved Team Ranking that is a touchstone in the Pat White debate. <?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Michigan has had six players from 2008 leave, including four 4-stars. Newsflash: that drops that class to 15th or 16th, all other classes staying the same. Which, of course, all other classes are not the same. Who the hell knows what Alabama’s ranking after the fact truly is? We obviously can’t re-rank the Rivals rankings; we must work with signing day class.
For purposes of making a start, I have judged that if the player sees the field for more than a cup of coffee, they get a rating. With this model, O’Neill, Wermers, Witherspoon and Hill do not get a rating from McBean, but Feagin and McGuffie do. The star calculation for a class over time, then, compares the final average from Rivals to only the McBean rated players. In the 2002 class above, all 21 players produce an average of 3.52 stars. The McBean average is 3.37 stars using a denominator of 19, not 21. Thoughts?
Question 3: Check my conclusions for lightly used players. This is where the mgocommunity will be critically helpful. I could barely remember some of these players and Googlestalking was marginally productive. For example, did Quinton McCoy and Tom Berishaj in the 2002 class ever play? The greatest inaccuracies will be for those players who played a little, like Feagin, but I don’t remember because it was years ago.
Question 4: If you disagree with my McBean rating for any player, please educate me. I am getting old and my memory is not what it used to be.
Here are the first three classes in which all the players have a final McBean rating (none are playing college ball any longer). It would seem, based on a preliminary review using the tentative rules that Lloyd’s classes underperform slightly. It will be very interesting over time to see how Rich’s classes do.
Here are the next two classes that have incomplete McBean ratings.
I will somehow compile the feedback and present a final McBean Analysis for each “closed” class. As each class becomes closed, I will present the final after feedback. Hopefully, Rich produces guys on the left.
For you Rivals guys out there, is The Fort down for you as well? I haven't been able to get to it all morning.
I think we can all pretty much agree that Michigan’s top four rivals are OSU, MSU, ND and PSU. I was thinking about this the other day and trying to decide which ones are the most important to beat in 2009 and 2010? I thought it would be interesting to get everyone’s perspective and see how these four rank overall and for giving them a good beating this year and next. Rank the four teams in order of importance and, on a scale of 1-5, 5 being most important, that we beat them in 2009 and in 2010. Of course, Ohio State is the biggest, but the real question is who is number 2, 3 and 4 and why. Also, of these rivals, which ones are most important to defeat in 2009 and why? Which ones, if not all in 2010? <?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
Some of this may seem trivial, but if one assumes that Michigan will not beat each of its four biggest rivals this season, which ones are most important this year and next? The value of the question can be looked at from many different perspectives. Is a “progression” over the course of the season more important or simply beating the best teams? How would certain victories look to pollsters or bowl selection committees? Are bragging rights a major issue, or is it simply pride that makes one game more important than another.
With the debacles of last season still lingering, is revenge a huge factor or is simply ending a losing streak more important? Does the perception that one team is believed to be so much better this season make the game this year less important, but more important next year because we should be better next year? Does the fact that a game is at home make it more important? How much does recruiting affect your choices? There are many reasons why you might choose one over another.
With the longest off-season I can remember, (Nov.-Sept) the idea of which rivalry games are most important to me this season seemed intriguing. Boredom generates some crazy thoughts and this seemed like an interesting topic. Although I could be wrong I did a lot drinking while kayaking on the Kalamazoo River yesterday and my judgment could be very clouded. Lord knows, my head is.
1) OSU (5) Must end the streak. Respectability and maintaining this as truly a rivalry is huge
2) MSU (5) The first road game, setting a tone for the conference season and truly gauging the defense
3) ND (4) Need this to start the season right, build confidence and destroy Jimmah and the whale
4) PSU (3) Would like to get this, seems doubtful, but its at home and we have dominated them for so long
1) OSU (5) New Years Day bowl at stake, Restablish our place in the conference
2) ND (5) Will solidify that the program is back
3) MSU (5) Recruiting, bragging rights and keep little brother in place
4) PSU (4) Important for rankings and exposure at their place, and simply HATE happy valley
2011: Win ‘em all MNC dominate college football