Stars Do Matter: The Over-achievers and Under-achievers

Submitted by The Mathlete on

Matt Hinton has annually done the yeoman’s work showing year after year showing how recruiting rankings matter. Brian linked to this year’s edition already and it’s worth taking a look at. Building off of that idea, I wanted to look at which teams and coaches were the biggest over and under achievers in the business.

Methodology

Every recruit gets a numerical rating from 0 (anonymous MAC recruit) to 99 (consensus #1 rated recruit) based on all available recruiting services.

Each player on the roster is given an adjustment factor based on how old they are. 75% reduction for first year players up to a 60% bump for upperclassmen.

This generates a total point value for each roster based on attrition, age/experience and recruiting rankings. The top teams since 2003 are dominated by Pete Carroll era USC. His 2005-09 teams take up 5 of the top 7 spots along with 2006 LSU and 2006 Miami.

Each game since 2003 is then compared against each team’s roster rating and the final score. The resulting best fit is 0.007 * (Home Team’s Roster Rating – Road Team’s Roster Rating)+4 –> Final margin. The R squared is .17.

There is obviously a lot of variance that goes into every game but there is no doubt that there is a strong correlation between recruiting success and team success. Today we’ll take a look at the teams and coaches that have maximized and wasted their talents the most.

Stars Don’t Matter (In a Good Way)

Using the above calculated fit, I looked at each true home game since 2003 and compared roster predicted results with actual game scores.

Over the last 11 seasons, the biggest overachieving teams have been:

Team Points vs Recruiting
Boise St +20.6
Oregon +12.4
TCU +12.2
Northern Illinois +11.9
Wisconsin +10.6

Not a lot of surprises on the list, five teams that have certainly had success beyond their recruiting profiles. Boise dominates the list with their amazing run of late and little to no major recruiting wins.

Northern Illinois made the list with four different coaches while TCU did it all under Gary Patterson.

Here is how the Big Ten stacks up (Big Ten games only):

Team Points vs Recruiting
Wisconsin 11.3
Iowa 4.2
Michigan St 4.1
Ohio St 2.8
Penn St 1.1
Northwestern 1.0
Minnesota -1.3
Purdue -1.3
Nebraska -2.7
Michigan -5.2
Indiana -7.3
Illinois -8.8

Wisconsin has been the clear leader and right behind are Iowa and the Big Ten’s biggest cheerleader for Star’s Don’t Matter, Michigan St. Despite being the Big Ten’s strongest recruiter, OSU has still managed to generate above average results from their talent. Michigan sits at the low end of the spectrum, down over 5 points per game versus talent and ahead of only Indiana and Illinois.

Other conference results from conference games only:

SEC

Best: Texas A&M (+13), Missouri (+7), Alabama! (+7)

Worst: Tennessee (-10), Ole Miss (-5)

Big 12

Best: Missouri (+9), Oklahoma St (+7)

Worst: Kansas (-7), Texas A&M (-6)

Pac 12

Best: Oregon (+13), Oregon St (+9)

Worst: Colorado (-19), UCLA (-6)

ACC

Best: Virginia Tech (+11), Georgia Tech (+7)

Worst: Miami (-9), Duke (-5)

Big East

Best: West Virginia (+10), Virginia Tech(+9)

Worst: Syracuse (-7), Pitt (-3)

Ranking the Coaches

For some programs the coach and the team are interchangeable but here are how the 2014 Big Ten coaches have done as head coaches at all of their D1 stops since 2003.

Coach Team Points vs Recruiting
James Franklin Penn St +12.0
Gary Andersen Wisconsin +6.7
Urban Meyer Ohio St +6.7
Kirk Ferentz Iowa +5.0
Jerry Kill Minnesota +4.7
Mark Dantonio Michigan St +4.3
Randy Edsall Maryland +2.7
Pat Fitzgerald Northwestern +1.4
Brady Hoke Michigan +1.4
Bo Pelini Nebraska +0.9
Tim Beckman Illinois -3.9
Darrell Hazell Purdue -4.0
Kevin Wilson Indiana -5.2
Kyle Flood Rutgers -6.1

The East and the West may be highly imbalanced in terms of recruiting profile but they are pretty balanced in what the coaches have done with the talent. James Franklin has the second shortest head coaching tenure on the list, so there are some sample size issues, but James Franklin has so far proven himself to be a significant over achiever with his talent. Coach Hoke sits in in the middle of the back at slightly above average. Hoke’s numbers have progressed at each stop in his career, going from –0.4 at Ball St, +1.0 at San Diego St and +4.9 so far at Michigan.

Lloyd Carr’s five eligible years would have put him towards the bottom of the list at –2.9. Carr presided over some stacked teams and Michigan’s style often meant closer games than the talent would dictate. Of all the disastrous metric for RichRod, this may be the worst. After going +14.5 at West Virginia, his three years at Michigan inverted and were –13.9 versus what the roster would project. In his two years Arizona he has moved toward the middle at +4.3.

Here are your top rated coaches of the last 11 seasons with at least 4 season.

Coach Points vs Recruiting Primary School
Chip Kelly +21 Oregon
Chris Petersen +20 Boise St
Kevin Sumlin +15 Texas A&M/Houston
Bobby Petrino +13 Louisville/Arkansas
Glen Mason +12 Minnesota
Gary Patterson +12 TCU
Jimbo Fisher +11 Florida St
Paul Johnson +10 Georgia Tech/Navy
Frank Beamer +10 Virginia Tech
Bret Bielema +9 Wisconsin

Looking Ahead To 2014

I am hoping to have a post up later this offseason about the “secret sauce” for BCS champions, but one thing they all share is a place in the top 10-12 spots of roster success. As noted above, there is lots of variance in the middle, but if you want to play for a championship, you have to have an elite roster. Projecting 2014 rosters is a bit tricky on a large scale, but here are my early projections for roster strength for this coming season.

Projected Rank Team Conference Coach Rating
1 Texas Big 12 +7.7
2 Alabama SEC +8.3
3 Georgia SEC +0.1
4 Florida SEC -6.2
5 Ohio St Big Ten +6.7
6 USC Pac 12 +0.2
7 Auburn SEC +6.6
8 Florida St ACC +11.2
9 LSU SEC +3.3
10 Oklahoma Big 12 +4.4
11 Clemson ACC +2.4
12 Michigan Big Ten +1.4

Each of the five auto-qualifying conferences have at least one team on the list and the Pac 12 is the only one with a single entry. Unsurprisingly, the SEC leads the list with five entries.

If you are looking for early title favorites, take the top coaches ratings for the 12 on this list and that leaves Texas, Bama, Ohio St, Auburn and Florida St as the first teams I would look at.

Comments

YoOoBoMoLloRoHo

February 14th, 2014 at 1:01 PM ^

The headline could read: Stars and Coaching do matter. The last chart and your top 5 favorites clearly reflect the importance of both. And your chart highlights why Brown, Kiffin & Chizik are gone while Richt & Muschamp are on the hot seat. Under-achievers. So Hoke has clearly gathered the stars for elite status. This year seems critical to his coaching rank as UM has the stars for potential 11-1 - that's a high bar to avoid Under-achiever status.

ca_prophet

February 14th, 2014 at 8:30 PM ^

... That #12 rating hardly argues for 11-1, especially with our strong opponents all on the road. Those teams are usually in the 9-3 range, which is an optimistic prediction for a team with a new offense and missing its starting tight end. A better title is that Stars Do Matter Over Time - note the Mathlete is explicitly adjusting the star content by eligility. People may be tired of hearing about youth, but our roster reflects it none the less.

YoOoBoMoLloRoHo

February 14th, 2014 at 10:02 PM ^

for all programs ranked below UM for 2014, it can be inferred by the final chart that only OSU has a stronger roster (stars and experience) than UM on the schedule for 2014. My statement was UM "has the stars for a potential 11-1" and that was "a high bar". Playing on the road and Hoke's middling coaching score create under-achiever status, which means we win fewer games. Perhaps your 9-3 guess is right.

DrMGoBlue

February 18th, 2014 at 9:31 PM ^

I may be biased, but I think senior leadership is very important--especially at a place like Michigan. And to see all that attrition from players mostly 3-4 stars really hurt us in developing the younger players. With fewer seniors there are fewer leaders for the younger guys to learn from. That has to have some kind of an effect.

Wolfman

February 15th, 2014 at 5:47 AM ^

I won't even go through the boring details of how so many coaches at premier programs have paid their due in the same manner. I am looking solely at Brady.  His coaching bio points to programs that were, of course, non-BCS level, but nonetheless in piss-poor condition when he took over. After his third or fourth year, he had built programs to basically the same level as the one he inherited at Michigan in his inagaural season. If you look at his first year at MI, it's easy to see his defense contained the type of leadership and experience that he had to work three seasons to attain at his previous stops. He inherited Martin, Kovacs, VanBergen, and some lbers that, through Mattison's teaching, were able to actually read keys and act accordingly. I think we can all close our eyes and recall Ryan VanBergen not only chasing down an opposing RB headed for the end zone, but also stopping him and stripping him of the ball, thereby allowing Denard and Co. one more opportunity to put points on the board for UM. This type of play was typical of 2011 and why we went from one of the worst to one of the best defenses in the nation.  Leadership, which can only be developed through experience and belief in the system, were on display every Saturday. I cannot recall the last time we had a turnover ratio in our favor to that degree. We all suffered while watching and probably all bitched while watching these young(key word) players suffer through wins over teams, that as far as star watch, should not have been on the field with us, and then worse, losses to teams like PSU that were as equally ill-equipped as those teams we somehow beat. But, we also watched Mattison, and you have to love him for this, toss almost his entire roster into every game to get them the invaluable PT that their talent, lessons learned while playing against far more experienced players, no matter their star ratings, is going to pay off handsomely as suddenly, they will be the team similar to the 2011 unit, only far more talented as an entire 42-44 man unit. No one can discount Kovacs value to the team, but because of Hoke's recruiting, we won't see too many more walk-ons be co-captains and major contributors. And if they are, although it will be rare, it will be a bonus.      And just as the subject-matter of the initiator of this thread stated, those recruiting classes full of blue chippers, many of which are now upper-classmen will execute in the manner their numerous offers of top D1 BCS coaches throughout the land suggested they would during their recruiting cycle.                              ^And the offense is no different.  Hell, as difficult as it was for Greg to watch his kids go through that learning curve, remember the record setting offense from 2011, although headed by one of our most exciting offensive players ever, along with another 1,000 yard rusher-two in the same backfield is always a good sign- and soon to be the single season receiver in M history, Jeremy Gallon, that offense also contained an AA center, a LT that would become a two time AA and arguably the best in the nation at his particular position, but buffered by experience throughout, and actual honest-to-goodness experienced depth. We were solid, and these were kids recruited for a completely different brand of football.    The production, or lack thereof, of last season's oh-so-young OL was probably something we should have expected, not unlike the disaster that was 2008 when we only fielded one returning starter among the entire eleven. And this brings me back to the original point. At both of Brady's last two HC stints his teams went from cellar-dwellers to the top of the league based almost exclusively on his ability to outrecruit every other coach in his respective conference. But that took three to four years to accomplish. His first season here was basically a bonus in terms of talent inherited, but at the same time we all knew how thin we were going to be where the game is decided for the next couple of seasons, that being in the trenches, and even accepting that truth, I don't believe I am out of line by saying we should have easily beaten OSU twice out of the gate when considering Al's inexplicable second half play calling in 2012 when we continuously picked up an easy 8 on first down and his refusal to keep pushing it until stopped and instead displaying an almost Lloyd Carr stubbornness, continuously bashing ahead on the ground on 2nd and 2 and again on 3rd and 2 to keep coming up with the same results. Yes, we all are aware of Einstein's famous quote.  To hold them to six for an entire half and still lose is something you can't do at Michigan, thus the "Adios Coach Borges. Then again, maybe there were a couple, i.e., the ND end results that could have just as easily turned out in the opposite manner.                 ^But as the initiator of this thread suggested, stars matter, and other than OSU, Brady has been able to separate himself from the rest of the league. Inasmuch as this is his fourth year at the helm, this is his statement year. Aside from gathering the necessary talent, something he has been very successful in, it's now time for him to make a personal statement, one that says, "I'm as good as anyone in this league."  There are some very good to great coaches in this league. And as much as so many wanted Harbaugh, and to a far lesser degree, Miles, Brady has met the challenge at every level he's coached. He has the staff, he has the players and he's had the time. And I believe I did mention his wins and losses in his very first season here with far more many three stars than four and fives. I think this season will be a fun one for all of us.

Pinto1987

February 19th, 2014 at 4:06 PM ^

...I think a few facts regarding Hoke's "coaching bio" need clarification:

Ball State Football Results

2000 Bill Lynch 5-6

2001 Bill Lynch 5-6

2002 Bill Lynch 6-6, then Lynch was fired.

2003 Brady Hoke 4-8

2004 Brady Hoke 2-9

2005 Brady Hoke 4-7

2006 Brady Hoke 5-7

2007 Brady Hoke 7-6

2008 Brady Hoke 12-2

Then Hoke bolted his alma-mater for the greener pastures of SDSU, where he stayed for only two years - going 4-8 (2009) and 9-4 (2010).

Hoke’s record for EACH of his first four years at Ball State was WORSE than the three years prior to his arrival.  It was only in Hoke’s fifth year that he exceeded Bill Lynch’s results for the three years before Hoke arrived – and then only by the narrowest of margins.

It looks an awful lot like Hoke cratered an average program at Ball State, taking five years to improve upon the results of the year immediately before he arrived - and then only by one game.  He then parlayed a senior QB coached by Stan Parrish (who just happened to be the guy who coached/developed a guy named Tom Brady) to a 12-2 season and a $600k contract at SDSU.  After Hoke left, Parrish took over and imploded to 2-10 (2009) and 4-8 (2010) records (so much for recruiting - and the talent of the guys assembled by Hoke).

That’s not one or two bad years on the way to recovery under Hoke – that’s FOUR bad years, one year back to average, and then a SINGLE very good year.  Was Ball State’s 2008 season a trend or an aberration?  We’ll never know (but 2009 and 2010 argue for aberration).

I have no idea what Hoke inherited when he arrived at Ball State, but I think that Lynch’s last three years with 5-6, 5-6, and 6-6 records indicate that Hoke inherited an average, mid-level MAC team.  What followed for the next four years – under Hoke – was demonstrably worse than Lynch’s final three years.  To the extent that Hoke was the architect of Ball State's resounding, single-year, success, it's clear that the recovery was from a hole that Hoke dug for himself.

LJ

February 19th, 2014 at 9:15 PM ^

I don't think Hoke's track record prior to coming to M was all that impressive, but this analysis strikes me as pretty unfair.  Hoke takes the blame for having poor records during his initial transistion years at Ball State because he "cratered the program," but then he ALSO takes the fall for Parrish's bad years after Hoke left because he didn't recruit well?

Hoke goes 12-2 in his sixth year at Ball State, but gets no credit for that year because Stan Parrish coached the QB.  Nevermind that Parrish was there for the 3 prior years as well--if Parrish was responsible for all those wins, why weren't they good the other 3 years?  And why did Parrish do so badly after Hoke left?

Pinto1987

February 20th, 2014 at 10:18 AM ^

...and I probably didn't do a good job of explaining myself.

I was responding to the ideas that (1) Hoke had "built programs" that were in "piss-poor condition" when he took over, and (2) he did it on the basis of his recruiting prowess.

Hoke should get credit for one year of success at Ball State, but one year of success that was preceded by five years of nothing and followed by two years of nothing (a) does not constitute a "built program", and (b) does not look like he recruited to even an average MAC program level.  

Hoke should also get credit for one year of success at SDSU but, again, one year of success does not constitute a "built program".  Hoke and his team didn't recruit and build SDSU to a higher level - they reorganized the mess they found when they got there, and apparently did a good job of it for one year.  Hoke didn't have enough time at SDSU to reasonably gauge his recruiting ability (he started in Dec 2008, and finished the 2009 recruiting season there; had a full 2010 recruiting season; then left at the end of the 2011 recruiting season).  Did his ~2 recruiting seasons work?  Well, SDSU has bounced around with 9-4 and 8-5 seasons in the MWC, so maybe it did.  Is that any evidence that he can do better in Ann Arbor?  Of course not.

Wolfman

February 16th, 2014 at 5:46 AM ^

this is not true. Simply ask yourself if you, an 5 star qb, would rather be protected by a 2 star center and two 3 star guards or would na 4 star for each position suit you better? I think your answer will make my point. All great teams in any sport start with defense in the middle. What's so cooln with today's "mind in the moment" OCs is their ability to gain advantage because they are aware of the status quo and what they must bring, often three stars, or less even, to combat it. Pure, nunadulterated speed will mitigate a defense rostered by your typical AA. Proof thein lies w/in the W.V. offenses unbder RR and the same to be said of ORE. They won'tn beat you every game butn 10 out of 11 is about as good as anyone gets nowdays.  Hell, even one of the BCS opponents usually have one loss, but like the Bear said, "Ifn you're going to lose one, do it early."

gustave ferbert

February 19th, 2014 at 2:24 PM ^

I always wondered what happened to him?  He seemed to be doing well at Minnesota then he wasn't there anymore.   I never understood why, and I never really researched it.  But if anyone knows the story, please fill me in!

gustave ferbert

February 19th, 2014 at 6:37 PM ^

I remember the 2005 horror and watching that game in the first quarter and seeing how incredibly sound their O-line technique was.  One kid from Minnesota put his head on Rondell Biggs' hip and literally steered him with his elbows out of a hole for a huge gain. . .It was some of the most beautiful fundamental football you ever saw. . . 

 

bronxblue

February 19th, 2014 at 3:51 PM ^

Great stuff, as usual.

I guess my only quibble is that we've come to realize that star rankings are based on a specific view of what is a "good" player, and doesn't usually take into account the system he will be playing in except, I guess, the pro/spread QB split.  So with teams like Wiscy and Boise, you probably see some of this disparity because the ranking sites are comparing apples to apples when those teams are definitely looking for oranges.  Obviously coaching has an impact, but Wiscy has a very specific style of play that works for them, and they recruit to it regardless of what the star rankings say.

funkywolve

February 19th, 2014 at 3:54 PM ^

are almost a perfect example of why stars matter.  When they were in the mid major conferences Patterson was one of the hottest names in the coaching circle and TCU was almost an annual BCS buster.

Since they moved to the Big 12, they've been nothing special.  Their two years in the Big 12 have seen seasons of:  7-6 (4-5 in conference) and 4-8 (2-7).

Pinto1987

February 19th, 2014 at 4:11 PM ^

I suspect that you're right re: Franklin and sample size is important (I don't think that any number of stars could overcome a Threet / Sheridan QB combo, for example), but this is fantastic data nonetheless.  Thanks for the effort.

BlueKoj

February 20th, 2014 at 9:15 AM ^

Just for clarity, Hoke's +1.4 (and all ratings) were based only on home games, correct? If that's true, is that surprisingly low given the 19-2 home record (often against poor competition), and how scared should we be based on @ND, @MSU and @OH in 2014?