is there such a thing as an etsy genuis? if so, this is it.
doing it with pan
Close followers of this site are well aware that for all his faults, Rich Rodriguez was not handed a classic, loaded Michigan roster when he arrived. If you followed his recruiting at all you know he didn’t do the Wolverines any favors with his recruiting during his three years in Ann Arbor. Based on 11 years of recruiting and roster history, I have compiled a look at how Michigan’s talent and experience have ebbed and flowed since 2006 and how many elite caliber classes like the currently assembled 2013 class it will take for Michigan’s talent base to catch up with the nation’s best recruiting programs.
For those interested in how the roster values were calculated, I’ve added a section to the end to explain the methodology.
The Lloyd Peak
How long before he retired he knew when the end was is hard to know, but Lloyd Carr certainly had the roster set up for a great 2006 and 2007. In 2006 Michigan entered the season with the highest rated roster in the Big Ten and trailed only LSU and Miami (YTM) for top marks nationally. Both sides of the ball were well represented, ranked third nationally on both offense and defense. The stacked lineup led Michigan to the brink of their first BCS National Championship game before losing to top ten talent teams in the final two games of the season.
After a disappointing finish to 2006, the roster was even more setup for a big send off for Lloyd Carr in 2007. The roster trailed only USC in terms of overall talent and the offense had a sizeable lead as #1 nationally. Unfortunately, the losses on the defensive side dropped the defense out of the top 10 and were exposed in the first two games, ending Michigan’s national title hopes before the calendar turned to October.
The RichRod Hangover
The 2008 version of Michigan was not a team lacking talent (except for quarterback), but it was a ways from the talent level in Ann Arbor the previous two seasons. The defense rose a bit to sneak back into the top 10, where the loss of the Henne/Hart/Long dropped the offense from #2 in the nation to #16. By the 2010 season things hadn’t gotten better but they hadn’t gotten worse, either. The offense and the defense consistently ranked in the top 15 nationally but the results, especially the GERG effect on the defense never matched up with the talent-level on campus. The roster left for Rodriguez didn’t do him any favors but it was also far above the record and defensive performances that came from it.
The High of Hoke…
…will have to wait. As great as the Hoke recruiting era has begun, it will likely be a few seasons before the dividends begin to payout. The next two seasons will be the two where the roster effects of the Three and Out era will come due. This season the talent level will drop down into the 20s overall and likely stay there for the 2013 season as well before a big jump two years from now when the majority of the Hoke classes move from observers to contributors.
Michigan Vs The Rivals
Michigan dominated the end of the Lloyd era versus all the big three rivals but by 2008 Ohio and Notre Dame both moved into a better roster position. Despite the head to head losses, Michigan maintained an overall roster advantage over Michigan State. That nearly closed over the next two seasons, before the Hoke recruiting pushed Michigan out front once again. By 2015 Michigan should move close to par with Ohio and Notre Dame for the first time since 2008.
Michigan Vs The Best
Despite a lull coming up over the next two seasons, Michigan is poised to move into the upper echelon of national programs in terms of overall program talent. If Michigan can maintain top 5 level classes, by 2015 Michigan will move into the elite group of rosters in the country. Michigan’s projections begin to look a lot like that opening day opponent, with a five year delay. Nick Saban had similar rosters his first two seasons that Michigan is facing in the coming years, before taking off into elite status. Hopefully Michigan doesn’t need to go through the roster manipulation to get there, but with the first two Hoke classes, Michigan is making the first step to achieve a similar roster look as the team that has won 2 of the last 3 national championships.
What it All Means
Even the most die-hard numbers guy like myself knows that at the end of the day these are just numbers on a computer screen and the game is played on the field. The numbers are far from everything. Coaching, luck, player development and other factors are all major contributors to team success. With that said, the composition of the roster does mean a lot. As far back as I have reliable data for roster composition (2005) only Auburn has won a National Championship and not been ranked in the Top 10 for roster talent, and it took one of the greatest individual single-season performances ever for that to happen. Conference championships can be won with middle of the road talent for the conference. Last year Wisconsin won the Big Ten with the 7th highest rated roster in the conference. But over the long haul talent will win out.
Michigan will have more to overcome in the talent department than at any other time in the internet era. 2012 and 2013 will be lean years by Wolverine standards. Thankfully, we have the kind of coaches that have a high likelihood of mitigating that drop through their other talents of player development, in game coaching and for the future with recruiting.
Unlike Lindy’s I’ll do my best to give you a rundown of how I arrived at the valuations for each roster. Each player when recruited is ranked by each service. If they fall out of the Top X for any site, I do my best to approximate a ranking based on stars, position rank and grade where available. That rank is then translated into points. The formula I use is -4.5*ln([Rank]+11)+36. The specific formula was generated to more heavily weight the top players and produce an output where a consensus #1 rated prospect is worth a total of 99 points (summed across the four sites) and bottom of the barrel anonymous 2 stars are worth only a couple points.
Using this system, Michigan’s highest rated recruit was Prescott Burgess in 2003 at 90 points, Kyle Kalis was tops for the incoming class at 69 points and Shane Morris is currently at 79 points. Players are then weighted based on years in the system. Freshman only add 25% of their points to the roster total. 2nd year players (red-shirt freshmen or true sophomores) add 75% of their recruiting value to the overall total and all players in at least their third year in college football get 160% of their recruiting value. These numbers were derived based on actual usage of running backs (split of carries), quarterbacks (split of attempts), receivers (split of catches) and defenders (split of tackles) and then normalized so that the total roster number would approximately equal the sum of the unweighted recruiting points.
The total only counts players still on the roster at the beginning of the season. On the recruiting side a thousand points is usually at least a top ten or better class (The 2013 class is sitting at 1034 points). From the charts above you can see that the best rosters reflect 4-5 great classes and are 4-5,000 points.
Matthew Stafford (Rivals #6 overall, 2006) and Mitch Mustain (Rivals #10 overall, 2006)
With the ESPN150 hot off of the presses yesterday, all four major sites now have an updated Top 150/247/250/300 list available for the 2013 class. I wanted to dive in and look at how each service has performed over the years. Would any site stand out as doing a better job of predicting success? Do allegations of regional bias play out for any specific services?
Typically considered the gold standard (unless another site has the player you like rated higher), Rivals online archives are available back to 2002. They have produced a ranked Top 250 since 2008 (Terrelle Pryor was the original #1 recruit) and an unranked Top 250 list in 2006-2007. Since 2002 they have classified between 25 and 33 players as five stars and typically have about 300 four-stars per class.
Like Rivals, online archives go back to 2002. They're a bit more generous on the five-stars, extending the honor to exactly 50 high school seniors per class since 2008. The larger group of five-stars is offset by smaller group of four-stars that round out the rest of the Scout 300 ranking that has been available since 2005.
The Worldwide Leader joined the recruiting party in 2006. No one is stingier with the fifth star than ESPN, offering up between 11 and 18 each season since they went crazy with 42 in their first year. The ESPN four-star threshold is also a bit tougher. Last year the number peaked at 238 but prior to that the total was closer to 200 per class.
The newest service is 247 Sports, which did a barebones review of the 2010 class before jumping in head first for the last two completed classes. Their best-of list ranks the top 247 players (just like their name, get it?) and is in line with Rivals in terms of number of five- and four-star rated players. Their later entry into the group has allowed them to provide what is, in my opinion, the best website in terms of navigation and ease of use. For the most part they are excluded from these evaluations since the first class they fully rated were only freshman last season.
This is where it gets tricky. Do you evaluate on hits or misses or both? Based on available, accessible information I decided that hits would be easier to quantify and really what you want to know about a service. Who does a better job of predicting future stars? By stars I defined them as players who earned all-conference or AP All-American status for a BCS conference school. First team all-conference honors were weighted double and AP All-Americans were weighted triple. If a player earned awards for multiple years, their value was weighted for each season depending on their level.
Each recruit was given an overall ranking for each service in each season. If there was a formal ranking, I used my method to complete the rankings behind them. I used star values and position rankings to approximate a ranking. All five-stars were ranked first, then four-stars and finally three-stars. Each player was ranked in position order and the positions were allocated based on total quantity in each group. This way the #4 fullback wasn’t rated the same way as the #4 wide receiver. Kickers and punters were excluded.
The square root of the rank was then used to further accentuate the differences at the top end of the rankings. Ratings were capped at 1000 and any unranked player was given that value. ESPN was evaluated solely based on the 2006 and later classes.
Who Rates the Best (at Rating)
Overall, Rivals gave the highest average rating to a future star. The weighted average ranking of a player to earn post-season honors from Rivals was #268. Scout wasn’t far behind at 281 and ESPN lagged further back at 329. Here is how each service did by recruiting class (lower is better):
Rivals dominated from 2002-2006 before Scout picked up a couple seasons in 2007 and 2008. With plenty of eligibility left for the 2009 class it’s still anyone’s game. Rivals has jumped out to an early lead for the 2010 class and the 2011 class is 60% comprised of Sammy Watkins and generally pointless at this point in its lifecycle. ESPN has failed to come close for any completed classes, although the 2009 class to date has been neck-and-neck between all three services with probably two-thirds of the results still outstanding.
Offensive Ratings-Weighted Average National Rank of Post-Season Honorees
ESPN finally picks up a win in the tightly contested quarterback evaluations. As you can see by the lower numbers, picking future all-conference quarterbacks has proved to be one of the easier tasks among rating services. ESPN’s average rank of 135 puts them ahead of both Rivals and Scout.
Scout does the best at wide receiver with Rivals a bit back. ESPN is not very close to the leaders at any offensive positions other than quarterback. Their results for both offensive linemen and running backs are particularly lacking.
Defensive Ratings-Weighted Average National Rank of Post-Season Honorees
It’s a clean sweep for Rivals on the defensive side of the ball. Scout is never far from, but still consistently behind, Rivals. ESPN is a distant third across all position groups, at least 20% higher than Rivals in every category and nearly 25% overall.
Conference Ratings-Weighted Average National Rank of Post-Season Honorees
The criticism of ESPN having an SEC bias and west coast neglect certainly shows up in the evaluations. SEC is the conference ESPN clearly wins and the ACC is a narrow win. All the other conferences are just carnage for ESPN while Rivals again takes a majority of wins. Scout is virtually tied for the Big Ten, Big XII and Pac-12 while lapping the field for the always crucial Big East rankings. It is difficult to tell whether the ESPN success is due to better rankings on players ultimately landing at ACC and SEC schools or if they are just giving a flat lift to those conferences. The fact that 1 in 3 players on the 2013 Top 150 are from Florida or Georgia probably indicates that at least some of the success comes from allocating preferential spots to players from the SEC footprint.
Here is how each service allocated their ranking slots to geographies. Higher rankings are weighted heavier and each player’s home state was allocated to one of the five major conferences (sorry Big East) based on their geography.
Players from ACC territory were the most consistently allocated across all four services. ESPN and 247 allocate fewer prime slots to the Big Ten versus Rivals and Scout. ESPN is a major outlier out west as the Pac-12 footprint garners much lower rankings there than at any of the other three. The SEC evaluations pick up about 40% from 247 and ESPN versus 35% and below from Rivals and Scout. This gap has narrowed some in more recent years, but there is still a strong bias from 247 and ESPN towards players from the SEC footprint.
In terms of ability to predict future success, Rivals stands out as the clear winner among all of the services. Scout is not significantly behind and has closed the gap in recent seasons. Rivals predictions proved more accurate at five of the seven position groups and overall for both sides of the ball. ESPN is a distant third in almost every sub-category with the exception being quarterback where they lead the most closely contested position group.
The services appear to be mirrored in their regional biases. ESPN and 247 slant to the Southeast and Mid-Atlantic regions with a clear sub valuation of the West Coast. Rivals and Scout don’t have dramatic swings to any one region but do give less value to the Southeast with the extra spread across the rest of the country.
While Michigan’s 11 [players in the...] Top 150 showing is absolutely a good thing, ESPN has proven to be the least reliable of the three established services. Historically, 30.5% of the weighted post-season honorees originally appeared in the ESPN 150 while Scout and Rivals have each had at least 34% in their Top 150’s. The differences aren’t massive and all sites have had their misses, but overall there is clear evidence that Rivals is the most consistently correct and that Scout is a strong #2. Although the individual players fluctuate, the overall ratings for Michigan’s class to date are essentially identical between Rivals, Scout and ESPN with 247 being lower on it than any other service.
Bonus: Protecting Conference Turf
Not looking at recruiting services but conferences now, I wanted to see which conferences did the best job of keeping the best players from their region in-house. Each state was split between conferences based on number of schools in a given state. States without BCS conference teams were excluded. This isn’t perfect because there is no way Cincinnati and the Big East have the same share of Ohio as the Big Ten and its school in Ohio. But it makes each school among the Big Six theoretically even and provides a good starting point.
|Conference||Total FP Pts||Split FP Pts||Signed FP Pts||Signed/ Total||Signed/ Split|
The Pts are an estimate of the total value of the recruits within a footprint. The total includes points for all players to any conference with a school in that region. The split is an allocation based on number of BCS conference schools in that state in each conference. Not surprisingly, the conferences with the smallest geographic competition, the Big XII and the Pac-12 signed the highest percentages of their available recruits. After splitting up the states, the SEC actually signs more than their allocation of the footprint. The Big Ten is close behind but work from a much smaller pool. If the SEC is able to make make gains in Texas (they currently have a 12% “share”) with the addition of A&M, the talent gap between the SEC and the rest of the conferences could widen.