Behind the Numbers (9/16) Part II - A Possible Alternative to Efficiency Rating?

Submitted by CollegeFootball13 on September 16th, 2009 at 2:28 PM
This is a follow up to this post. If you haven't yet, read that first.

So quite a bit of discussion has opened up in the original post about the Efficiency Rating only taking into account the passing efficiency, and in today's College Football world, quarterbacks are much, much more than that.

In this post I'll take a closer look at the current efficiency rating and how it turned out last year in terms of ranking the quarterbacks, as well as taking a stab at my own Quarterback Efficiency Rating, which will hopefully take into account the broader, tangible aspects of a quarterback's game.

Last Season's Results

Top Ten in Passing Efficiency - 2008
Player, School
1Sam Bradford, OklahomaSO 180.8410.02
2David Johnson, TulsaSR 178.698.9
3Colt McCoy, TexasJR 173.758.07
4Tim Tebow, FloridaJR 172.378.23
5Zac Robinson, Oklahoma St.JR 166.847.43
6Mark Sanchez, Southern CaliforniaJR 164.648.29
7Chase Clement, RiceSR 163.928.31
8Graham Harrell, Texas TechSR 160.047.97
9Case Keenum, HoustonSO 159.917.96
10Chase Daniel, MissouriSR 159.447.11
*Rank is only out of these ten players
(For comparison's sake, Pat White's QER was a 6.3)

Most of the names on the list are pretty obvious ones. The who's who of College Football Quarterbacks last year. Bradford, McCoy, Tebow, Robinson, Sanchez, Harrell, and Daniel all had phenomenal seasons and were in the spotlight of College Football because of it.

Johnson, Clement, and (to an extent) Keenum, however, weren't mentioned too much. They don't play at high-profile programs, and don't play against Grade-A competition, but you can't make an argument that they had great seasons. is David Johnson a better quarterback than McCoy, Tebow, and Sanchez? Almost certainly not. He is a good passing quarterback, however, and his rating shows that.

Key Players Not in Passing Efficiency Top 10
26Pat White, West Virginia 142.35
39Michael Desormeaux, La.-Lafayette 135.01
75Julian Edelman, Kent St. 118.83

Maybe a bit of a stretch to call White, Desormeaux and Edelman "Key Players", but there's a method to my madness. Edelman, Desormeaux, and White had the 11th, 32nd, and 52nd highest rushing yards per game, respectively.

Obviously this didn't suddenly make them some of the best quarterbacks in the nation, as none of the three were invited to the Heisman Ceremony, but it's just an example of the aspects of the game that Passing Efficiency doesn't take into account.

Quarterback Efficiency Rating?

So if Passing Efficiency isn't a great way to evaluate the overall quality of a quarterback, what other ways are there?

Well.. there aren't too many.

So I took a stab at my own Quarterback Efficiency Rating. It has its flaws but it's a more comprehensive, all-encompassing look at what a quarterback does and evaluates them based on a multitude of other statistics, beyond just passing.

Quarterback Efficiency Rating (QER)

      (Completions) + (Passing Yards x 0.5) + (Passing Touchdowns x 50) +
   (Interceptions x -25) + (Rushing Yards x 0.5) + (Rushing Touchdowns x 50)
                            (Rushing Attempts) + (Passing Attempts)

In this formula, not only is the best possible rating just over 100 (no one would ever realistically reach over 100, or even close to 100) for an easier analysis of the rating, but a pocket passer:

26/32, 280yds, 3 TDs (QB Rating: 9.875)

Has a comparable rating to a dual threat or even a running quarterback:

14/21, 150yds, 1 TD, 10att, 96yds, 2 TDs (QB Rating: 9.26)

There's no arguing that, in this example, the pocket passer had a better game, but at least with the QER they were about on the same level, whereas the Passing Efficiency Rating would have given the pocket passer a 185.7 rating, and the dual threat quarterback a 142.4.

Again, not perfect. But neither is the Passing Efficiency Rating. It might not make it into NCAA Recordkeeping, but it might help us in the bloggosphere rate quarterbacks on more than just their passing ability.

That's all for this installment of Behind the Numbers, please feel free to let me know if you have any constructive advice for the QER. Thanks for reading!



September 16th, 2009 at 2:34 PM ^

...but you might be able to make your case more strongly by adding QER to the chart at top, at least, since it's probably difficult to generate QER for any significant number of D-I QBs from last year (and thereby be able to show the top QERs and where the generally-accepted "good" QBs ended up).


September 16th, 2009 at 2:40 PM ^

Because there's *so* much that is left out - the specific value of obtaining first downs (versus just yards/touchdowns), or considering sacks taken (as a lot of recent work has shown the importance of the QB in avoiding sacks).

Looks like a decent start, however, given the data limitations. Are you able to re-run last year's ratings with your new formula? (I'm guessing that would be difficult, as rushing data is not always available in the same tables which have passing efficiency ratings in them).

B Ready

September 16th, 2009 at 3:27 PM ^

If there have been any studies done on the importance of touchdowns, first downs, yards per completions, etc. and the potential penalties that come with interceptions and negative yardage through sacks then that would make any formula that much better. Otherwise, throwing random numbers in there to multiply things by in any formula is a cause for it to be picked apart.

If anyone is familiar with sabermetric thinking in baseball, then what I would propose is a formula similar to the one used for wOBA. Find out how much a touchdown contributes to a win, or first downs, or yards per completion and how much interceptions contribute to a loss, etc. and then calculate a formula off that. It is certainly possible, but it would take a lot of research.

Enjoy Life

September 16th, 2009 at 6:24 PM ^

So, wouldn't it be a good idea to apply your formula to the 10 guys in your original chart and show us the results?

Please, pretty please, with a touch of honey on top?


September 16th, 2009 at 6:24 PM ^

You'd want to account for strength of schedule, if possible, by considering the quality of the defenses (perhaps weighted to pass defenses?) that each QB faced. Once a season is over, I'd probably recommend FEI (Fremeau Efficiency Index) as a measure of quality of team defense.

Enjoy Life

September 16th, 2009 at 6:57 PM ^

Wow, I'm shocked and the TT lovers will be totally pissed! They have contended that, "If only you could include TT's running stats, he would obviously be the highest rated QB."

Oops, your data shows he goes DOWN in the rankings.

Ahhhh, that is sweet.


September 16th, 2009 at 7:12 PM ^

I think this formula passes the smell test, except for the touchdowns.

Let me give a quick summary of what I know about football sabermetrics:
The "neutral point" on a football field in Major College and NFL football is your own 25 yard line (give or take 5 yards or so). In other words, if a team has the ball at its own 25, first and ten, it is a 50/50 proposition of which team is going to score next. So we can say that having it on your own 25 is worth exactly 0 points.

If you lose 25 yards from there, it is worth 2 points to the other team. 25 yards/2 points = 12.5 yards/point. If you gain 75 yards from there, it is worth 6 points for your team. 75 yards/6 points = 12.5 yards/point.

Obviously, it's not perfect, but it gives us a starting point. Now...I'm not going to show my math, but this means that a turnover is worth 4 points, or 50 yards.

Try it some time:

[(total yards gained) - (total yards allowed) + 50 * (takeaways - giveaways)] / 12.5 will get you pretty close to the final scoring margin.

So the formula in the OP, which awards 0.5 units for each yard gained, and subtracts 25 units for each INT, has this ratio spot on. Adding an extra unit for a completion sounds about right; this makes two 5-yard completions the same as one 12-yard completion plus one incompletion. I buy that.

I think the OP has greatly overvalued touchdowns, though. The difference between a 49-yard pass that ends up on the 1 yard line (24.5 units) and a 50-yard touchdown pass (75 units) is too much. Maybe 10 x touchdowns is better than 50 x touchdowns, making a touchdown worth the equivalent of 20 extra yards, instead of 100.

Other than that, it seems to be just right.

B Ready

September 16th, 2009 at 11:16 PM ^

So, essentially, the trouble is finding out what values to give to touchdowns. I think sacks should also be discounted for b/c new info is coming out that suggests that QB's have a lot more ability to avoid sacks than we originally thought/is commonly believed. I think setting up a team for a FG try should also be included (not sure how that would be possible to account for, though).

Other things... we would need to account for the strength of the defense the QB played and the drops his WR's had.

Enjoy Life

September 16th, 2009 at 7:42 PM ^

Here is a comparison of how M's top 15 QBs (based on college rating) end up when you use the NFL rating.

Name College Rating Rank NFL Rating Rank
Grbac 148.14 1 99.29 1
Harbaugh 145.64 2 92.60 5
Wangler 145.17 3 91.37 6
Collins 145.03 4 95.60 3
Taylor 141.60 5 94.72 4
Henson 140.82 6 96.18 2
Brady 136.43 7 90.64 8
Henne 133.90 8 90.81 7
Griese 131.37 9 86.81 9
Leach 130.03 10 76.76 12
Franklin 129.37 11 81.24 11
Brown 127.89 12 73.94 1 5
Dreisbach 126.00 13 79.71 14
Navarre 125.77 14 85.09 10
Smith 124.51 15 76.03 13

So, the weighting factors (which are somewhat or greatly arbitrary) have a significant impact on the evaluation. I have no idea how anyone could "prove" their rating factors are "correct".


September 16th, 2009 at 8:24 PM ^

I think Rushing Attempts need to show up in the numerator. A 10 yd pass is worth 6 points, but a 10 yd run is only worth 5.

Having RuAtt in the numerator is kosher here because the denominator is basically total plays.

I'll pull up my QB injury database and add this version of QER (plus Alton's comments) to it to see how the 2008 rankings change. I'll also add a fumble penalty of the same amount as interceptions.


September 16th, 2009 at 9:54 PM ^

Nothing really changed in the top 10 but Pat White took a monster hit sans spread 'n' shred, not a huge surprise.

Also, I think its safe to say that Nate Davis and David Johnson's numbers would not hold up in the SEC or Big12.

Also also, McCoy is a undoubtedly a better passer than Tebow but (last year) he also had better rushing stats overall than Tebow too. Again, Big 12 defenses are not SEC defenses but the stats say McCoy had a better year than Tebow did in 2008.

2008 Rank Player Team QER_mod
1 Sam Bradford Okla 5.91
2 Colt McCoy Tex 5.15
3 Nate Davis BaSt 5.11
3 David Johnson Tuls 5.11
5 Chase Clement Rice 5.08
6 Tim Tebow Fla 5.06
7 Case Keenum Hou 5.01
8 Graham Harrell TexT 4.97
9 Zac Robinson OKSt 4.88
10 Mark Sanchez USC 4.84
30 Pat White WVU 4.07
- Pat White - 2005 WVU 4.53
- Pat White - 2006 WVU 5.39
- Pat White - 2007 WVU 4.96
- Dennis Dixon - 2007 Ore 5.09
- Tim Tebow - 2007 Fla 5.24

*No fumble penalty b/c I don't have that data.


September 17th, 2009 at 8:59 AM ^

I did find a stat out there that in the NFL, having the ball first-and-goal at the opponent's one yard line is worth 5.5 points. Obviously, a touchdown is worth almost exactly 7.0 points.

So it looks like a touchdown being worth about 20 yards is very close to the right answer. I'm not going to say that a touchdown should be worth 18.75 yards, because that is pretending that we are working with much more precision than we actually have. Anything from 10 to 35 yards would be plausible to me.

I think it would be great if we could include sacks given up, fumbles lost and even first downs in a passer stat, but I think we need to limit ourselves to what is found on a standard stat sheet, because I am sure that Collegefootball13 wasn't planning on going into the play-by-play records of the games to calculate the new efficiency stat.