Analysis of 2009 Draft; Thoughts on the Correlation of Stars

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;}

After reading through, and enjoying, many comments on yesterday’s diary post I was driven to give more contemplation to the subject of “stars.”  The first premise derived from yesterday’s data is that a very, very small sliver of high school talent is of the caliber to even be recruited (currently, 0.23% of the talent-pool receives a scholarship to a Div 1-FBS school.)  Moreover, the top 0.1% of the talent pool would now include somewhere between 780 and 1002 high school seniors each year, depending on whether you are of the ilk that 70% or 90% of the top high school football players are seniors (or anywhere in between.)  Either way, this large number of players from within a very tiny sliver of a talent-percentile casts some serious doubt (in my mind) on the ability of scouts to truly differentiate among anyone outside of a football “prodigy.”  See yesterday’s post for a more comprehensive analysis of the “diminishing sliver.”

Among the comments were some great links posted by “Oregon_Alum,” credited to “mejunglechop and others” who had “brought this to the fore.”  The links (below; a couple of which had died before I had the chance to click on them) detail the success of “stars” in correlating to measurable outcomes.  I found the first article, from athlonsports.com, to be very convincing and well-stated.

http://www.athlonsports.com/college-football/16635/recruiting-the-nfl-draft
http://www.sundaymorningqb.com/2008/3/17/71811/4082
http://rivals.yahoo.com/ncaa/football/blog/dr_saturday/post/Hug-your-fri...
http://www.athlonsports.com/college-football/13422/nfl-stars-how-recruit...
http://www.sundaymorningqb.com/2008/1/21/1614/43228

The athlonsports.com article uses an analysis of the 2008 and 2009 NFL drafts to point out that the “stars” a player was previously given by talent scouts directly correlated with the likelihood of being drafted.  According to their data analysis of the 2008 draft, a player given that 5^th star coming out of high school has a 40-48% chance of being drafted in the top 3 rounds; 4-star 9-11%, 3-star 3.6% and 2-star less than 1%.  And, giving more credibility to the college scouts’ “eye,” I found that the NFL scouts had a pretty good eye as well… 7 draftees from 2008 were subsequently selected for the Pro-bowl (I know I don’t need to say it, but that includes our very own Jake Long) with draft-order actually seeming to represent the spread of talent: 4 future Pro-bowlers in the 1^st round, 2 in the second, and 1… well he went undrafted (apparently, even the NFL scouts miss one now and again.)  As a side note, these 7 elite players came from (in draft-order): UofM, Boise St., Tennessee St., East Carolina, Cal, Rutgers and Fresno St… I found that interesting.  Also: 4-star, 2-star, 4-star, 2-star, 5-star, 3-star, and 3-star… maybe the “chip on the shoulder” has a bit of a lasting effect.   I digress…

I cannot argue against “stars” correlating well with an eventual NFL selection, the numbers seem clear.  But, given that I am still leery of scouts’ ability to differentiate between non-prodigal high school players (3-17 players each year might be football prodigies, OK maybe 22… ) from within the top 0.23%, I am left questioning where such a correlation might stem from.  My initial theory is coaching and facilities (aka “player development”) as well as media exposure.

At this point my thought goes something like this…

I am currently studying tuberculosis in Moldova.  It is estimated that 1/3 of patients with an active case of TB will spontaneously resolve without treatment.  In contrast, somewhere between 60-65% of patients who undergo pharmaceutical treatment will be healed (note: that is a very low success rate for treatment, not applicable to those seen in the US… hence why I am here studying the system.)  But, let’s say that the infrastructure is overwhelmed and some people’s diagnoses are missed.  Those people will not be admitted to the facilities which (in this case) would have doubled their odds of survival.

So, in the football context:  a certain percentage of the high-school players (and I am not yet talking about the “prodigies,”) who continue on to play NCAA football, will someday develop into the mold of a potential professional player (likened to the cases of TB which spontaneously resolve.)  This is irrespective of their star-ranking, as well as exposure to coaching and facilities (likened to TB-treatment.)  In other words, there is a specific number of recruits from across the star-spectrum who are “destined” for a slot in future NFL draft; based on future physical and mental development, as well as intelligence, self-motivation and character.  This explains the numerous 2-stars we see drafted from the FBS subdivision each year, as well as the 22 players drafted from outside of the FBS subdivision in 2009.  Obviously, the baseline of “destined-success” is nowhere near 33% of all new college players.  My posit is that a longitudinal study would reveal a baseline of inherent draftability among players, measured by the success of Div 1-FCS players in achieving such outcome.  Granted, there are some great coaches and nice facilities in Div 1-FCS; but, the facilities’ levels could certainly be considered “basic” in comparison with those of Michigan, LSU, USC etc., while successful coaches are continually “poached” through the ranks to end up at… Michigan, LSU, USC etc.  Remember, FCS division players still come from the top 0.5% of high school football talent, and apparently this talent pool is still of high enough quality (in comparison with that of the FBS subdivision) to have accounted for 8.6% of the 2009 NFL draft.

  Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;}

List of schools from outside of FBS represented in 2009 NFL Draft:

Abilene Christian**			Sam Houston St.
Cal Poly SLO			St. Paul’s College
Furman			Stephen F. Austin
Liberty			Stillman
McNeese St.			Tennessee St.
Monmout			Weber St.
Nebraska-Omaha			West Texas A&M
Nichols St.			W. Illinois
Norfolk St.			Western Ontario
North Dakota St.			William and Mary
Richmond
**Indicates 2 draftees

Perhaps this means that upwards of 20% of the NFL draft is filled with players that develop regardless of coaching and facilities.  8.6% of draftees came from approximately 11,500 FCS athletes, correlating to an expectation of an additional 10.5% to have come from the approximately 14,000 participants in the FBS subdivision.  How is the other 80% filled?

Well, my theory is that the other 80% is first filled by the correctly identified prodigies, with the remainder arriving in a disproportionate manner based on attending schools with the best facilities, the best coaches, and the most media exposure.  So, back to the star-system.  What is it truly correlated with?  Well, first, the talent scouts seem to correctly identify most of the prodigies; the 5-star ranking is sort of a highly sensitive measure with a poor selectivity (identifying 90% of the prodigies with a rate of false-positives hovering around 50%.)  This is why we see that 5-star ratings correlate so well with future success (in this case measured by draft-status.)  Secondly, the star-system is correlated to the student-athlete’s college destination.  As “Noahdb” pointed out (among yesterday’s comments,) most schools are using recruiting services as the preliminary filter.  I wouldn’t doubt that the 4 and 5 star kids, at positions of need, are given the first-looks by the big time schools (enter first source of data bias…)   It is also pretty obvious that the recruiting services freely modify recruits’ star-levels after seeing where the recruit is likely to commit (enter second source of data bias…)  These biases add up to meaning that the major, automatically qualifying BCS schools have classes filled with a disproportionate number of 4 and 5 star athletes (regardless of the actual talent of the athletes.)

If the star system was actually a predictor of success, then we should see that 4 and 5 star athletes are drafted from “big-time” schools at the same rate.  But, when analyzing the 2009 draft by universities the draftees are selected from, we see 4 university categories emerge:  Over performers, Average performers, Underperformers, and the dreaded Underperforming Outliers.

The Over-performers:  split into two categories, those who successfully recruited at least one player from the Rivals’ 2005 Top 100 and those who did not.

School		‘09 Draftees	% of Total            Draftees	2005         Top 100	Rate          Proportion
Alabama		4	1.5625	2	1.28
Clemson		4	1.5625	1	0.64
Georgia		6	2.3438	4	1.706667
Iowa		4	1.5625	5	3.2
LSU		6	2.3438	3	1.28
Maryland		5	1.9531	1	0.512
Ohio St		7	2.7344	3	1.097143
Oklahoma		5	1.9531	8	4.096
Ole Miss		4	1.5625	2	1.28
Oregon		5	1.9531	1	0.512
Penn St		5	1.9531	2	1.024
S. Carolina		7	2.7344	3	1.097143
Texas		4	1.5625	3	1.92
USC		11	4.2969	9	2.094545
Virginia		4	1.5625	1	0.64
Wisconsin		4	1.5625	1	0.64

 

Of note: a randomized distribution of draftees would mean an average of 2.15 draftees taken from each FBS school (assuming none come from FCS, Div II, Canada, etc.)  Therefore, I have considered all teams which supplied 4 or more draftees to be over-performers.  The number of draftees from each is accompanied by the percentage of total draftees represented by that team.  For example, USC supplied 11 draftees, which was 4.3% of the total taken across all 7 rounds of the ’09 draft.  The numbers in the next column indicate the number of Rivals Top 100 recruits signed by each school in 2005; 2005 is the class with the biggest impact on the 2009 draft (although I concede the early attrition of juniors would be expected to affect the stability of this figure.)  The final column is a simple comparison of the % of Top 100 talent acquired in 2005 vs. the % of draftees produced in 2009.  If the top 100 recruits are truly more likely to be drafted, then the teams which acquire the highest % of them should produce a comparably disproportionate number of draftees.  A score above 1 indicates a team which is taking a high level of talent, but not matching that rate with NFL talent.  A score below 1 means the team is producing NFL talent at a greater rate than it is taking in top 100 talent; “0” would be the best possible score, meaning a team is producing NFL talent without the aid of any Top 100 recruits.

Speaking of ratio scores of “0,” the next set of draft Over-performers did so without taking a single top 100 recruit in 2005.

School		‘09 Draftees	% Total            Draftees	2005         Top 100	Rate           Proportion
Cinci		6	0.023438	0	0
Georgia Tech		4	0.015625	0	0
Mizzou		6	0.023438	0	0
N. Carolina		5	0.019531	0	0
Oregon State		8	0.03125	0	0
Pitt		4	0.015625	0	0
Rutgers		5	0.019531	0	0
TCU		5	0.019531	0	0
Texas Tech		4	0.015625	0	0
U Conn		4	0.015625	0	0
Utah		4	0.015625	0	0
Wake		4	0.015625	0	0

 

Analyses of these two groups: 

There were 28 teams in total.  26 are from the automatically qualifying BCS conferences.  The other two are… TCU and Utah.

The first group of 16 teams produced a total of 85 draftees.  That is a rate of 5.3 draftees per team, for a total of 33.2% of all 2009 draftees.  In order to accomplish this feat, these 16 teams (13.4% of Div 1-FBS teams) signed 49% of the top 100 recruits in 2005, according to Rivals.

The second group of 12 teams produced a total of 59 draftees.  That is a rate of 4.9 draftees per team, for a total of 23% of all 2009 draftees.   These 12 teams (10.1% of Div 1-FBS) signed zero top 100 recruits in 2005, according to Rivals.

In other words, the first 16 teams signed an average of 3.1 recruits, each, from the top 100, and produced only 0.4 more draftees, each.  Remember there’s basically a 50% chance of a 5-star being a “prodigy”… the first 16 teams signed 14 of these athletes.  This means that 7 of their draft slots were due to the odds of a 5-star being a prodigy… therefore, their expected non-prodigal rate is 4.9 draftees per team (the same as the other 12 teams in the Over-performers category.)

In order of gross output:  the top performers are USC, Oregon St., Ohio St. tied with S. Carolina, followed by a four-way tie for 5^th between Cinci, Georgia, LSU and Mizzou.  Ordered in terms of their “Rate Proportions:”  Oregon St., Cinci tied with Mizzou, Ohio St. tied with S. Carolina, LSU, Georgia and USC.  Based on this data, if I were looking for a coach I would look at Oregon St. (USC tried and failed,) Cinci (Notre Dame tried and succeeded) and Mizzou.

The Average-performers:  since we would expect 2.15 draftees per team after a random distribution, I considered those teams sending 2-3 draftees to the NFL as average.  I excluded teams which finished with 2-3 draftees but had at least 3 top 100 recruits in 2005 (they fall into the Underperforming Outliers.)

School		‘09 Draftees		2005        Top 100
Abilene Christian		2		0
Arizona		2		0
Arizona St		2		0
Auburn		3		1
Ball St		2		0
Boston College		2		0
BYU		2		0
Cal		2		2
Florida		3		1
Fresno State		2		0
Hawaii		3		0
Illinois		3		1
Louisville		2		0
New Mexico		2		0
NC State		2		2
Purdue		2		1
Rice		2		0
San Jose St		3		0
Southern Miss		2		0
Syracuse		2		0
W Michigan		2		0
West Virginia		3		1

 

Analysis:  There were 22 schools which finished in the “Average” category, 12 are auto-qualifying BCS schools.  There were also 9 Rivals Top 100 athletes among these twelve schools, with 3 five-stars.

The Dreaded Under-performing Outliers:  The schools in this category have the distinction of having excelled in recruiting while performing average or below in producing 2009 draft prospects.

School		‘09 Draftees	% Total             Draftees	2005         Top 100	Rate           Proportion
Florida St		1	0.003906	9	23.04
Miami		1	0.003906	5	12.8
Michigan		2	0.007813	7	8.96
Nebraska		3	0.011719	4	3.413333
Tennessee		1	0.003906	7	17.92
Texas A&M		2	0.007813	3	3.84
Va. Tech		1	0.003906	2	5.12

 

Analysis:  Seven schools, all from among the auto-qualifying BCS conferences.  I named this category as being outliers not because their proportional representation is sufficiently small to be a complete anomaly (5.9% of total Div. 1-FBS schools is a pretty large segment of the total.)  It is because each had its own set of circumstances leading up to the 2009 draft.  Michigan and Tennessee finished with uncharacteristically bad records (for different reasons,) same goes for Texas A&M, and who knows what happened with Florida St…. you would think that 9 recruits from the top 100 who helped piece together a 9-4 season would have been a sure-sell for a few more draft slots than 1.  Maybe Bowden really had lost his knack for coaching.

At any rate, these seven schools accounted for a whopping 37% of the Top 100 Rivals recruits in 2005; 10 of which were five-stars!  Four years later they combined for a horrendous showing at the draft:  11 draftees (4.3% of the total.)  Assuming the “50% of five-stars turn out to be prodigies” theory, 5 of the 11 drafts slots were a given, irrespective of coaches and facilities.  That means these seven programs could take credit for developing only 6 NFL caliber players.  Finally, according to the “rate proportions” Florida St. was the most colossal failure, followed by Tennessee and Miami.  I’ll stop there.

Conclusions:  After viewing three categories, we are left with the knowledge that in the 2009 draft, the 66 auto-qualifying BCS schools (67 with Notre Dame) split into 26 Over-performers, 12 Average, 22 Underperformers, and 7 Underperforming Outliers.  Among the Over-performers, there was no difference in draftee output (after 5-star talent was accounted for) between the group of 16 teams with a large representation (49%) of 2005’s top 100 recruits and the group of 12 teams with none of 2005’s top 100 recruits among them.  The impact of recruiting-stars took another hit when teams accounting for 37% of 2005’s top 100 talent combined for a miserable 4.3% of draftees in 2009.  Finally, 8.6% of the draftees in 2009 came from the Div 1-FCS subdivision.  This gives an initial estimate of 19.1% of draft-slots being accounted for by a baseline of players who will develop into NFL talent irrespective of differences in coaching, facilities and media exposure (and stars allotted to them.)

This has been an analysis of a single draft.  The data from the 2009 draft supports that while “stars” may correlate with draft success, it is likely a correlation due to “stars” predicting the collegiate destination of athletes as opposed to describing a differentiable talent level.  The two sources of bias aforementioned would suggest this possibility, as well as the fact that after accounting for 5-star athletes, the presence of “top 100” talent did not impact the NFL-production of universities with comparable levels of coaching and facilities.  In fact, a large concentration (37%) of “top 100” talent within 7 traditionally successful schools’ 2005 recruiting classes had no positive impact on the 2009 draft results.   I acknowledge the value of 5-star athletes; 50% of them are probably the much searched for football “prodigies” while the rest are just mis-rated and subject to the need for successful development.  I am not convinced that there is a discernible and/or long-term difference between 2-, 3- or 4- star athletes; rather, the appropriate coaching and facilities can turn any of these athletes into future NFL studs.

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;}