Splitting a White Rainbow

In a previous diary I used passer rating as a well known and objective grade for the relative value of a quarterback’s stat line in order to determine if there were any trends in player development, and if so, how strong those trends were. However, in the diary I noted that passer rating is not without its issues and pointed those interested in finding out toward other people’s work and went on with it.

Most Declarations of Grievance attack the adequacy of the the formula used saying that the scale is unintuitive, some of the components are not orthogonal (total yards, completion %), some components are irrelevant (touchdowns), and other components are omitted (rushing stats, and sacks). These are valid arguments but the alternatives presented are unfamiliar, come with their own set of complexities, and are often difficult for fans to calculate on their own.

In this diary I don’t want to generate a new formula, that has been done. Rather, I want to accept the current formula for what it is and develop new benchmarks for what it shows us in modern context. The two problems I have with it are that it’s clearly outdated and that it obliterates information.

Problem 1: It’s Old and Busted

The current NCAA passing efficiency formula (shown below) was developed in 1979 and was generated using passing data since the beginning of the modern two platoon era which began in 1965. At the time, the rating was calibrated to yield a rating of 100 for the average passer. If a QB had average values for all 5 components (attempts, completions, total yards, touchdowns, interceptions) his passer rating would have been 100.

Here’s the rub, major rule changes have been implemented in favor of the passing game since two-platoon football started, and so the majority of the data set used to calibrate the formula was skewed toward weak passing numbers by today’s standards. The major rules changes are:

• 1976: Offensive blocking changed to permit half extension of arms to assist pass blocking.
• 1980: Retreat blocking added with full arm extension to assist pass blocking, and illegal use of hands reduced to 5 yd. penalty.
• 1985: Retreat block deleted and open hands and extended arms permitted anywhere on field.

And these aren’t even all of them. Behold, further evidence of Anthony Carter’s ridiculousness: he thrived in an era where the rules were stacked against the pass. Before these rules were implemented, offensive linemen could not really be aggressive in pass blocking.  They were forced to be either turnstiles (before 1976) or turnstiles with their elbows sticking out. Before ‘85, linemen could not have their palms facing the opponent. Back in the day illegal use of hands and holding penalties were 15 yards assessed from the spot of the foul. Cloud of dust football so popular back then for a reason. For a modern taste of what this might have looked like check out Michigan v. Notre Dame 2007. The mismatch between Michigan’s D-Line and Notre Dame’s O-Line in that game was obscene. Despite that Jimmy Clausen’s freshmen year performance at Notre Dame, on that terrible offense, was slightly above average by 1979 standards. 

Due to the rules changes, passing stats have inflated but the formula has not adapted along with them. That is not to say that it has no value, just that our understanding of that value is outdated.

Problem 2: It’s A White Rainbow

Imagine if a rainbow were a brilliant white arc in the sky; still interesting, but less so than what we usually see. If the water droplets in the air can not produce a prismatic effect, they just diffract the light and we can’t see the individual colors. BTW, white rainbows are real.

Getting back to football, the passer rating formula looks at, yards per attempt, completion %, touchdown rate, and interception rate, then assigns weights to those values and blends them together to provide football fans a single number to use to compare QBs against themselves and each other. All in all that is a useful tool, but the blending process obliterates some very interesting information. Passer rating is a great coarse filter but it’s inadequate for picking up subtle differences. Not all 130’s are created equal.

Re-Calibration

In order to address the first problem, it is necessary to decompose the formula into it’s base components to see what the new definition of average is for each category. For college players, I think it is also useful to split the data by recruiting ranking (Rivals.com Star Rating) and Experience (Years as Starter) to really understand how well a kid is performing relative to history.

For this project I’ve taken only players who played on teams in BCS conferences and who were rated as a Rivals.com 3-star recruits or higher. The data plotted is the average for all players within a given category (ex: all 3 start players in their 1st year as Starter is a group, and so on).

One thing I should note up front is that there are fewer and fewer players in each category as the number of years as starter increases; only about 10% of QB recruits in each group start for four years. For the 3-star and 4-star groups this isn’t a huge problem because they survive the attrition fairly well and still have 7 or 8 players to use for averaging purposes; not great by any means but workable. The 5 star group ends up with 2 players in my data set that have started 4 years (Chad Henne and Trent Edwards). A sample size of 2 is not workable and has therefore been omitted.

For completion percentage we see that the average QBs gradually improves his accuracy and approaches 61% in the long term. The higher a player is rated coming out of high school, the sooner he is likely achieve steady state.

With Yards per attempt we see a more subtle upward trend and also more separation between rating groups. I think this separation makes some sense. For one, Rivals explicitly accounts for the players physical assets; it stands to reason that 5-star players are more likely to develop NFL-level arm strength and will therefore be able to push the ball up field without sacrificing accuracy significantly. Another potential factor is that a high level QB recruit is likely to attract high level WR recruits that help improve YPA significantly. I think the long term standard that should be applied for this category is 7.6 yards per attempt.

Touchdown rate is a factor that many people argue against including in the passer rating formula. The argument goes that a TD is as much a result of the WR’s ability as it is the QB’s. The Roundtree hawk down at Illinois is an example: Edwards, Manningham, Breaston, Odoms, and a bunch of other guys would have taken that ball to the house. I think this chart shows this effect pretty dramatically. The 5-star recruits tend to go to high level programs and are surrounded by high level offensive lines, running games, and receiving corps, thus making it easier for them to throw touchdowns. Oh yeah, and they’re more likely to have the skill to exploit their advantages. Long term target: 6.0%.

Interception rate is the only negative factor in the formula, so a lower number is better (duh). Again we see 3-star recruits significantly lagging the other two groups. I suspect that not only is there the experience issue, but 3-star recruits are likely to need more time to develop proper mechanics. By year 3, all groups are about as good as they’re going to get. Long term target: 2.7%.

The New Hotness

Cherry picking the long term values for these parameters allows us to assemble a passer rating that is a true indication of good passing efficiency in college, not just objectively point but also subjectively; that value is 139.2. This is a stout target to hit and the player needs help from his team mates to get there, but it is achievable for all BCS level recruits by their 3rd year as starter. In 2009, 33 QBs put up this level of performance or better with another 10 or so within reasonable striking distance.