Rivas as a four? I have to think about that. Here we get into that special grey area that is special teams.
How long did Richard hang around?
Before that, I wanted to give a summary of the informal voting that took place in the last thread as it related to the 2002 class (note: it would be helpful if more readers would vote – the number of votes was small, although the justifications were scholarly). Here was the 2002 class career ratings proposed after the McBean Rating definitions were debated:
The following votes were then cast:
Five-Star: Two votes
Four-Star: Three votes
Three-Star: Three votes
Two-Star: Zero votes
Three-Star: Two votes
Two-Star: One vote
Single votes for Bihl as a four-star, Breaston as a four, Tabb as a three and Riley as a three.
All votes are not equal as arguments of varying persuasiveness were offered. In the debate, wolfman81 almost perfectly captured what still remains a point of contention: how to use the NFL draft in our assessment -
Notice that in all of these cases, I consider the college career first. If there is some doubt about what category the player should fit into, only then do I consider the NFL draft. The NFL draft is a tiebreaker, but only when there is a tie to be broken. In my mind, 1st team All-American = 5-star McBean. All-Conference = 4-star McBean. Starter = 3-star McBean. There can be exceptions to this rule (Tom Brady was not All-Conference, but I think he should be 4 star due to his leadership on the field and other intangibles. As he was drafted, the NFL assessment backs this opinion up. The fact that he will likely be a first ballot NFL hall of famer cannot bump him up to the 5-star discussion; at the same time, it does not harm my 4-star rating.) But, I think that this rule sets a minimum standard.
The new 2002 McBean Ratings are as follows with only one change – Harris drops to a four-star. Mark Bihl was pushed hard as a four-star, but his career falls short of making a significant impact, just as Harris clearly falls short of an elite career.
The conclusion one would then draw from this data is that the 2002 class dramatically underperformed. To give you an idea, this would represent a drop in the team rankings (if the rankings were based upon average stars) from 10th to 21st, using an average of the 2002 through 2009 data.
Now, the problems:
Rivas as a four? I have to think about that. Here we get into that special grey area that is special teams.
How long did Richard hang around?
The two star problem is a genuinely thorny one. I might phrase the question this way:
How many of our bench warmers are in fact "top 700" athletes when compared nationally?
As I think about it now, this is perhaps a more difficult question than evaluating the 5,4 & 3 star athletes, as we have tools like the draft and recorded stats and awards to help us out. But these are players who did not accumulate a tonne of stats for us and were largely invisible. Should we penalize a mid to low three star athlete [and the program and the coaching staff] because he was outperformed by other three star or more star athletes? Is there a reliable way to sort out the question of whether a guy was still solid, but just not good enough to see the field at Michigan, someone who might have performed better at another school with a different depth chart, or was he genuinely overrated, or did the staff do a lousy job of developing an otherwise solid athete?
Again, this should be a question of national rankings. The hope is that we are loaded with talent and that there are some very "solid" athletes who have warmed the benches at Michigan. Or is a bench warmer a bench warmer and for whatever reason, either on the recruiting end or the development end, the staff dropped the ball and deserves to take the hit? Would this downgrade affect every program, or would other programs have enough guys coached up to offset the down guys? It seems to me that every athlete's precious [Monty Python skits aside] and that it is the coaching staff's job to ensure a recruit with a "solid" potential actually does contribute up to their potential and should take the hit if they don't. Its cold, yes, but 2003 just shows, in my opinion, how the coaching staff had checked out and was just relying on "superior talent" to win games.
I think the solution might lie in what wolfman suggested: some formula that values five stars more than it values a two star. In other words, a five star and a three star would not have an average of four stars, but over four stars.
wolfman, explain your formula in more detail.
However, you run the danger of not judging a staff for player development across all players equally...the new formula pays off for getting or developing rare stud talent and not letting Barwis turn all the three stars into men of steel.
How many Rivals three star players are there?
So here's what I'm seeing as a problem:
Under the current metric, if I'm understanding it correctly, we're better off having 3* players leave without seeing the field than having them stay on as career backups.
Our recruiting average star rating will always be between 3* and 4*. If everyone on the team ended up with the same McBean rating as Rivals rating, every 4* or 5* departure is a penalty, though not a severe one. That seems right. But for every 3* that leaves, the average McBean ratings actually increase (i.e., player defections are getting recorded as player development). Moreover, some of those 3* left because they would have been invisible (read: 2*) had they stayed. So we get a McBean ranking benefit when they leave and would suffer a McBean penalty had they stayed. That seems wrong to me.
Not sure how to fix it, just thought I'd point it out.
To restate, every player that leaves below the average ends up raising the average, and those players below the average are more likely to not get playing time and leave. Correct? I see that, but it seems to me that history shows more highly rated players not getting the playing time their egos expected, so you get an O'Neill and a Threet and a Mallett.
I think this one works itself out.
included in his 3* McBean rating?
A good question. As of right now, no. Many have argued for it, however. If we are trying to evaluate a staff's recruiting and player development skills, do we want to give Barwis over at WVU credit for our development?
But if we are just including Mundy's UM stint - I think 3* is way too generous.
i think you're being tough on Kraus. He came in as (and rated as) a TE. He then got moved to OL and basically transformed his body. He also was one of our better OL for a solid 2-3 years. I'd give him the 4 stars.
Here's where our draft metric comes in...if we have a borderline player after reviewing his college career, then the draft determines which way to move. Is Kraus borderline? If so, he's a three as he went undrafted. If Kraus is not borderline, then we can ignore his undrafted status.
We need a vote.
Legit starter for a few years, two-time all big ten - I think that is enough to warrant 4 stars.
It's the Two First-Team All Big Ten selections that really seals this one.
Compare Prescott Burgess (a McBean ****):
Kraus: 4 letters, 35 games started, 2 time All Big Ten, signed then cut as UFA.
Burgess: 4 letters, 20 starts, 2 time Big Ten Honorable Mention, 6th Round pick.
I don't think that draft slotting, especially when it is the difference between a 6th round pick and a UFA, should mean that much more than college performance (as measured by starts and all-conference voting).
Either Kraus is a 4 star or Burgess is a 3.
I changed my mind back to 3* a high 3* but still a 3*...he almost gets into the top 250-300 in the nation, but not quite. It is the draft that is the tie breaker for me in this case.
I am persuaded that Kraus is a 4.
You should expect there to be a drop-off from the Rivals ranking to the McBean ranking, simply because so many highly regarded players (at every school) don't pan out. It's not necessarily a flaw in your rankings or an indication that the Carr regime couldn't develop talent.
It's hard to find data that matches up perfectly with your rankings, but Brian recently linked to a study that comes pretty close. It found that only 33% of five-star recruits in BCS conferences, and 20% of four-stars, went on to make an all-conference team at some point in their careers, which I believe would qualify them as four-star McBean players. In other words, most four- and five-star recruits would end up as no better than three stars in your system. Any team like Michigan that recruits large numbers of four and five stars is almost guaranteed to wind up with a lower McBean ranking than its initial Rivals ranking.
And that is the point of the McBean system: to be used as a tool in judging how good coaches are at finding that talent that will not bust and also a measure of how well they help all talent get better. The Pat White Question.
My concern is, as you point out, that there is a tendency for the average player to do less well in college. Then our system won't allow us to compare ratings.
An apt comparison from sales, Dan. Even when you screen out prospects, and focus in on those potential high value clients, you are, for whatever reason, not able to convert every prospect into a sale. The advance work that the coaches do in recruiting, the recruiting services, all these things help limit the number of prospects in the pipeline that do not work out. [I am just being struck by how apt this comparison is and how overlapping the lingo is...just never really sank in before] In that regard, as with all sales, there has to be an acceptable conversion ratio, an acceptable number of guys in every program that do not work out. To draw a base line we might have to sample a number of top programs like Florida, Texas, USC, Oklahoma, and even OSU and run a very "tight" study on them using only the draft and starters as a metric and see how many of their recruits in a given year do not see the field and thus potentially drop into the two start category.
This will give us an acceptable drop off rate from recruiting rankings and give us an idea if our drop off is more than other schools.
This is good, but it would change our use of the McBean rating from a numerical comparison to the incoming class to a numerical comparison to the target "drop off" value. We would be unable to claim, as we did, that the 2002 class underperformed until we know what the acceptable drop off would be. Interesting.
Dan, it seems that the more that we as a group bore into this subject, that it is becoming obvious that in every program we can expect some recruits to not work out. What is the acceptable level of washouts in the program? When you look at the numbers, there is not a lot of wiggle room. When you hear about programs run the way everyone wishes their school’s program ran, they seem to recruit talent, develop talent and are able to keep talented guys around.
There are what, 80 or 85 scholarships, if memory serves me correctly. That means that in an athlete’s 3-5 years time with program with variances depending upon redshirts or early exodus to the pros will look something like this:
Year 1 – on the three or four deep, with little or no playing time.
Year 2 – on the three or two deep, with some playing time.
Year 3 – on the two deep, with more playing time and perhaps a few starts.
Year 4 – Starter.
When you hear Coach get all misty eyed about the future that is coming when he has the talent here, this is the kind of picture I hear him painting. This is kind of what you see at places like USC where they are stacked three or four deep with talented guys. And when you look at the numbers of scholarships, you realize that with 22 starters plus kickers, that you cannot divide that same 80-85 scholarships evenly by four and have an even four deep at every position. In that sense you are always using up athletes faster than you can replace them, assuming an equal distribution. That means you have to develop talent at a faster rate than a four year equal distribution.
I guess what I am trying to say is that the potential is there that every athlete can be used given an ideal set of circumstances.
It is understandable that not every prospect will work out and to this end, some benchmark should be set, and that means doing some sample looks at various other top flight programs. At the same time, it is not an unreasonable expectation that it is at least in theory possible for every athlete to start and be a “solid” contributor at least in their final collegiate year.
In this regard, I would still hold out that a scholarship athlete that does not start for whatever reason should be downgraded to a 2** as their lack of production can be attributed to either a bad recruiting job [staff's fault] or a poor development job [could be part athlete, but also a good part staff's fault as well]. Given the number of scholarships and the need/opportinuty for every athlete to work out, there is no reason why every recruit cannot at least in theory have at least one start in four years.
I hear you, but if the result is every class falling well short of the Rivals rating, then we don't have much. To solve that, you propose creating some sort of benchmark, which will be time-consuming and have questionable validity (who do we choose, what's the n, what years, etc.).
However, I am leaning towards allowing the proliferation of two-stars because it has a certain logic and avoids ridiculous scenarios like Will Paul and Ruben Riley sharing three stars.
I think KISS should hold here. Keep it simple, go with what we have and let an internal Michigan benchmark emerge. When I finish, we'll be able to see that, maybe, the 2002 class underperformed because it falls short by the greatest amount.
For example, just to give an idea of where we stand for the 2002 year. Here is a quicky look at the 2002 recruiting class for USC for their 10th ranked class[sorry no charts, but it should be fairly straight forward...first number is rivals star rating, the second is their McBean number][our class was ranked 13th]:
Darnell Bing -- 4* -- 4* (4th round/101)
Van Brown -- 3* -- Transfer
Dominique Byrd -- 4* -- 4* (3rd round/93)
Hershel Dennis -- 4* -- (starter, career cut short due to injury)
Chris Doyle -- 4* -- Quit
Brandon Hancock -- 4* -- 3* (Starter)
Winston Justice -- 4* -- 5* (2nd round/39)
Oscar Lua -- 4* -- 4* (7th round/211)
Tom Malone -- 3* -- 4* (All-American Punter)
Fred Matua -- 4* -- 4* (7th round/ 217)
Chris McFoy -- 3* -- 3* (Starter)
Jason Mitchell -- 3* -- 3* (Backup who played consistently)
Ronald Nunn -- 4* -- 3* (Starter)
LaJuan Ramsey -- 2* -- 4* (6th round/204)
Mike Ross -- 3* -- Transfered
Dallas Sartz -- 4* -- 4* (5th round/143)
Danny Urquhart -- 4* -- Career ending injury
Kyle williams -- 4* -- 4* (undrafted free agent, still with Seahawks)
Mike Williams -- 4* -- 5* (1st round/10)
Manuel Wright -- 4* -- Academic drop out -- Starter and All-P10 honourable mention
Justin Wyatt -- 3* -- 3* Undrafted NFL free agent
Ryan Kileen -- 0* -- 3* Undrafted NFL free agent kicker
Rivals Rating 77/22 = 3.5
McBean Rating [discounting drop outs, career ending injuries and transfers] 63/17 = 3.7
Conclusion: Pete Caroll and his staff do a great job of getting the best out of their players and a great job of coaching them up. It is not hard to see why he is 88-15 at USC and has the best winning record of any coach in D-1 with more than five years experience. With a class ranked very close to Michigan's, Pete Caroll actually saw his McBean rating go up! It is hard not to come to the conclusion that in the waning years of the Old Regime, that if we did well it was because guys stepped up and not because they were coached up. Too many guys ended up on the bench and did not perform. Too much talent was squandered. It is not hard to see from just this snapshot why Pete Caroll's record is the best in football for his coaching tenure and why guys line up to play for him. My guess is that you will find a similar pattern in any year you take at USC. USC represents the ideal of how athletes should progress through the system as I described above. Us, not so much. And you will find a similar pattern for Michigan in any year of the Old Regime. It is sad and disheartening to think of all the wasted talent.
For our discussion...I am more firmly set than ever that if a guy does not see meaningful playing time at any time during their career, they get two stars and if the ratings suffer so be it.
Scary that this was before Carroll started rattling off #1 classes.
Good research although you are one reviewer and knowledge of different teams' marginal players to make an effective McBean rating average is beyond us. Look at the trouble we're having with just the first two classes. Who's going to break down Will Paul for me? How about LSU's version of Will Paul?
I hear you, though. Reviewing all of this, I am going to proceed as we defined the ratings before and see where it leads us.
I concur. The more we go down this rabbit hole the scarier it gets.
Seems to me that whether a class overperforms or underperforms is a lot more dependent on the original Rivals ranking than on the McBean ranking. I think the Rivals ranking would be a lot more suspect than McBean, which is after the fact. Who's to say Rivals is anywhere close to right, especially when some of their high rankings don't even show up in college.
A valid observation and one of the motivations of McBean - to see who finds and produces the real the real stars.
Please see EDIT note at the bottom of the original post.
1. Every class is destined to "underperform" given your current rating system. If the average M recruiting class has, say one five star, 10 four stars, eight three stars and two two stars, you will have a roster with almost 50 players that Rivals rated as four or five stars. There is no way every one of these guys on any team in the country can be a starter and All-Conference, which is the limit for a McBean four star ranking. So, saying the classes underperform is a misnomer in my opinion.
I think the thing to look for would be the average or expected return on four or five star recruits after your re-ranking. That number would be the one to make comparison to, and the comparison would be of the most value if we also incorporate other teams that recruit at a high level consistently.
2. The McBean system is missing a golden opportunity to incorporate the quality of play over all four years of player eligibility. For example, a one year starter gets the same raking as a three year starter if the three year starter did not earn All-Conference status. In my opinion, a three year starter was by definition more valuable to his team than a one year starter, and should be ranked higher as a result.
Also, was a three year All-American like Charles Woodson of greater or equal value to a one year All-American like Earnest Shazor?
3. We have only four categories to drop guys into in this system. Rivals already has at least 13 categories, from 4.9 to 6.1. This ranking should do something similar, again to help differentiate between the masses of guys that will end up in the three star category.
4. Based on point No. 2 you might do well to create a composite ranking that does not exactly match the 1-5 star Rivals system, but calculates a cumulative ranking incorporating all four years of eligibility and media honors.
For example, if a guy does nothing much for two years (2* + 2*, or something like 5.0 + 5.0) , starts his third year (3*, or 5.6-ish) and earns all conference honors his senior year (4*, or 5.9-6.0), you can add that all up and get a four year ranking of:
2 + 2 + 3 + 4 = 11, or
5.0 + 5.0 + 5.6 + 5.9 = 21.5
This kind of composite rank will allow to much more easily differentiate between overachievers, underachievers and your average M player.
brad, I think your points are valid, but you suggest a level of complexity that would be difficult and time-consuming and would probably not deliver a great deal more value. Point #3 I considered, but that would eliminate two of the four completed classes that will serve as our benchmark (no RR for 02 and 03). And if we've had hot debates over who is a three or a four, I can only imagine the debates over who is a 5.7 vs. a 5.8.
I think the first pass, anyway, needs to be simple to see if there is value. If everyone says: "Wow, cool. Wish we could look deeper, though," then that might be the next step.
However, you will have a really hard time getting a valuable result if you don't go there. About every class you rank under this method will get you around 3.1 or 3.2. First, going season by season, and second using a little more discreetized range of values will help you a lot, possibly to the point of getting proper results. Just my opinion, obviously.
And a good one. Just a hard one. Going to finish out with the current definitions and see what we get. We can then perhaps go further...
I think once we are done with our classes 2002 and on...that we do some analysis of other programs, recognizing the tricky nature of evaluating the lower/marginal players to give us an idea of where we stand. A look at Pete Caroll's 2002 recruiting class ar USC disputes the inevitability of the drop from 3.5 to 3.1. Their McBean rating actually went up. What this exercise is exposing is how much talent has been squandered at Michigan in the early 2000's and makes you wonder what could have been had we developed out talent the way Caroll does.
If a guy eventually starts, it means his talent/skill was developed to a degree. If he earns All-Conference, All-America and/or gets drafted, he was developed to a higher degree. The concept of attrition is irrelevant since you are talking about player development of guys that stayed on the team. The difference between single- and multi-year starter is irrelevant since the point is that he was made good enough to start at some point, and therefore developed to that *** level.
In that context, I take my previous criticism back as the current ranking system will be able to give you what you are looking for, and the difference in the USC numbers now makes sense to me.
I think I was subconciously looking for a grade of a recruiting class w.r.t. overall team quality, or the net positive or negative impact a class has on its team. For example, the McBean ranking method might show that the guys who dodged fate and were able to stick out their playing days from the 2005 class performed just slightly below their Rivals ranks, if taken as a whole. The highly rated guys that stayed, Manningham and TT, were both drafted so ****, Grady and Zoltan will cancel each other out, a few *** who started at some point, Savoy who may not start, then Moosman and Harrison who were 5.8 four stars that will probably rank at McBean ***. Thus, the class was slightly underdeveloped upon leaving UM, a point that could and probably should be criticized in its own right.
In my own private ratings, the 2005 class was an almost unmitigated disaster that set the program back at least a year (2008) and has a visibly negative impact on two others (2007 and 2009). A lot of that has to do with attrition, and only a little with underdevelopment of talent. But this appears to be a different point altogether.
busted coverage! Don't look here.
OK, we're going to proceed with the agreed-upon definitions, which will produce low McBean ratings, and see what we get.
Now, let's settle some player ratings in the 2003 class above. All looks good except some disagreement on Rivas, Richard (N/A vs. **), and Mundy (overrated?). Kraus will be a 4.
PLEASE VOTE ON THESE THREE PLAYERS.
Rivas -- 3*
Richards -- N/A
Mundy -- 4*
Kraus -- 3*
aka Backup Level as per Rivals definitions, which is pretty evidently for Michigan 3 stars. All backups almost must be 3 stars. So really there shouldn't be anything worse than excepting attrition issues. So anyone filling in at a position that lacks depth or can't consistently remain a starter should be 3 star. Consistent starter is 4 and an excellent starter is 5. I don't see any reason to really nitpick over one or the other. As long as the input errors are random, it's not going to matter much.
I don't do weekends...so it was sad to miss out on most of this debate.
I like the new/revised standard definitions that have evolved:
5-Star - Elite player. Will almost certainly be a 1st team All-American. Non-All American five stars will be rare. To help decide a borderline case, a 5 star player will almost certainly be drafted in the first round of the NFL draft. (Kickers: All-American punter/kicker.)
4-star - Excellent player. All-Conference performer and/or starter. Multi-year contributer. This player will usually be on the two-deep for several seasons, ending his career at the top of the two deep. These players are the backbone of the program. Often, these players will be drafted or sign as undrafted free agents--and remain for several seasons--in the NFL. (Kickers: Multi-year starter.)
3-Star - Functional player. Infrequent starter and spends about 2 seasons on the two-deep. Special teams contributor. This player will, almost certainly, not play in the NFL.
(Kickers: Unreliable kicker. Has some deficiency...only does kickoffs or can't go beyond 30 yards...Only plays because there is no better healthy option. Transfers: Almost certainly a ceiling--maybe not for a few notable exceptions--as they don't have the talent/skill to compete here for playing time.)
2-Star - Invisible. No appreciable playing time. Rarely sees the two deep. (Transfers: Never had a shot here and they left once they figured that out.)
N/A - Does not exist in this system.
I don't know if we have addressed this, but for players who get career ending injuries, they should have one of two things happen. 1 - Rate them on their career here if they get injured late. Bump them up in the case of a tiebreak. 2 - Remove them from the calculation for the McBean rating.
Lastly, you asked me about my formula. It's really just the Root-Mean-Square. So add up for each player (star rating)^2. Then divide by total number of players. (This is the mean of the squares.) Now take a square root so that the numbers are comparable.
Example: Compare these 2 person classes (2 4 stars, vs. 1 5-star and 1 3 star)
2 **** -> Avg = 4.0, RMS = 4.0
1 ***** + 1 *** -> Avg = 4.0, RMS = 4.123
I'll ask the question this way. Would you prefer a class that is half 3-stars and half 4-stars (remember, I'm talking about McBeans here--so 12 All-Conference players and 12 servicable backups) or a class that is half 2-stars and half 5-stars (so we have 12 All-Americans in a single class and 12 guys who never play.) I know what my answer is (especially if we consistently recruit and develop that kind of talent). Think about it, if we do this every year, we have 12 All-Americans every year...And each All-American is backed up by next year's (or the following year's) All-American...This will never happen, because everyone wants to play now, but can you say, MNC every year?
Wolfman, sorry I missed this. I will repost some of it in the next summary.