Behind the Numbers (9/16) Part II - A Possible Alternative to Efficiency Rating?
So quite a bit of discussion has opened up in the original post about the Efficiency Rating only taking into account the passing efficiency, and in today's College Football world, quarterbacks are much, much more than that.
In this post I'll take a closer look at the current efficiency rating and how it turned out last year in terms of ranking the quarterbacks, as well as taking a stab at my own Quarterback Efficiency Rating, which will hopefully take into account the broader, tangible aspects of a quarterback's game.
Last Season's Results
Top Ten in Passing Efficiency - 2008Rank | Player, School | YR | PER | QER | Rank* |
1 | Sam Bradford, Oklahoma | SO | 180.84 | 10.02 | 1 |
2 | David Johnson, Tulsa | SR | 178.69 | 8.9 | 2 |
3 | Colt McCoy, Texas | JR | 173.75 | 8.07 | 6 |
4 | Tim Tebow, Florida | JR | 172.37 | 8.23 | 5 |
5 | Zac Robinson, Oklahoma St. | JR | 166.84 | 7.43 | 9 |
6 | Mark Sanchez, Southern California | JR | 164.64 | 8.29 | 4 |
7 | Chase Clement, Rice | SR | 163.92 | 8.31 | 3 |
8 | Graham Harrell, Texas Tech | SR | 160.04 | 7.97 | 7 |
9 | Case Keenum, Houston | SO | 159.91 | 7.96 | 8 |
10 | Chase Daniel, Missouri | SR | 159.44 | 7.11 | 10 |
(For comparison's sake, Pat White's QER was a 6.3)
Most of the names on the list are pretty obvious ones. The who's who of College Football Quarterbacks last year. Bradford, McCoy, Tebow, Robinson, Sanchez, Harrell, and Daniel all had phenomenal seasons and were in the spotlight of College Football because of it.
Johnson, Clement, and (to an extent) Keenum, however, weren't mentioned too much. They don't play at high-profile programs, and don't play against Grade-A competition, but you can't make an argument that they had great seasons. is David Johnson a better quarterback than McCoy, Tebow, and Sanchez? Almost certainly not. He is a good passing quarterback, however, and his rating shows that.
Key Players Not in Passing Efficiency Top 10
26 | Pat White, West Virginia | 142.35 |
39 | Michael Desormeaux, La.-Lafayette | 135.01 |
75 | Julian Edelman, Kent St. | 118.83 |
Maybe a bit of a stretch to call White, Desormeaux and Edelman "Key Players", but there's a method to my madness. Edelman, Desormeaux, and White had the 11th, 32nd, and 52nd highest rushing yards per game, respectively.
Obviously this didn't suddenly make them some of the best quarterbacks in the nation, as none of the three were invited to the Heisman Ceremony, but it's just an example of the aspects of the game that Passing Efficiency doesn't take into account.
Quarterback Efficiency Rating?
So if Passing Efficiency isn't a great way to evaluate the overall quality of a quarterback, what other ways are there?Well.. there aren't too many.
So I took a stab at my own Quarterback Efficiency Rating. It has its flaws but it's a more comprehensive, all-encompassing look at what a quarterback does and evaluates them based on a multitude of other statistics, beyond just passing.
Quarterback Efficiency Rating (QER)
(Completions) + (Passing Yards x 0.5) + (Passing Touchdowns x 50) +
(Interceptions x -25) + (Rushing Yards x 0.5) + (Rushing Touchdowns x 50)
__________________________________________________________________
(Rushing Attempts) + (Passing Attempts)
In this formula, not only is the best possible rating just over 100 (no one would ever realistically reach over 100, or even close to 100) for an easier analysis of the rating, but a pocket passer:
26/32, 280yds, 3 TDs (QB Rating: 9.875)
Has a comparable rating to a dual threat or even a running quarterback:
14/21, 150yds, 1 TD, 10att, 96yds, 2 TDs (QB Rating: 9.26)
There's no arguing that, in this example, the pocket passer had a better game, but at least with the QER they were about on the same level, whereas the Passing Efficiency Rating would have given the pocket passer a 185.7 rating, and the dual threat quarterback a 142.4.
Again, not perfect. But neither is the Passing Efficiency Rating. It might not make it into NCAA Recordkeeping, but it might help us in the bloggosphere rate quarterbacks on more than just their passing ability.
________________
That's all for this installment of Behind the Numbers, please feel free to let me know if you have any constructive advice for the QER. Thanks for reading!
September 16th, 2009 at 2:34 PM ^
...but you might be able to make your case more strongly by adding QER to the chart at top, at least, since it's probably difficult to generate QER for any significant number of D-I QBs from last year (and thereby be able to show the top QERs and where the generally-accepted "good" QBs ended up).
September 16th, 2009 at 2:40 PM ^
Because there's *so* much that is left out - the specific value of obtaining first downs (versus just yards/touchdowns), or considering sacks taken (as a lot of recent work has shown the importance of the QB in avoiding sacks).
Looks like a decent start, however, given the data limitations. Are you able to re-run last year's ratings with your new formula? (I'm guessing that would be difficult, as rushing data is not always available in the same tables which have passing efficiency ratings in them).
September 16th, 2009 at 3:14 PM ^
The first chart has been updated with the players QERs.
Thanks for the input!
September 16th, 2009 at 3:27 PM ^
If there have been any studies done on the importance of touchdowns, first downs, yards per completions, etc. and the potential penalties that come with interceptions and negative yardage through sacks then that would make any formula that much better. Otherwise, throwing random numbers in there to multiply things by in any formula is a cause for it to be picked apart.
If anyone is familiar with sabermetric thinking in baseball, then what I would propose is a formula similar to the one used for wOBA. Find out how much a touchdown contributes to a win, or first downs, or yards per completion and how much interceptions contribute to a loss, etc. and then calculate a formula off that. It is certainly possible, but it would take a lot of research.
September 16th, 2009 at 6:35 PM ^
Just for jollies, last year I used the NFL formula for passer rating (which is considerably different) to compare Michigan's Top 15 QBs.
It dramatically changed the rankings. I'll work up the table and post it in a while.
September 16th, 2009 at 6:24 PM ^
So, wouldn't it be a good idea to apply your formula to the 10 guys in your original chart and show us the results?
Please, pretty please, with a touch of honey on top?
September 16th, 2009 at 6:24 PM ^
You'd want to account for strength of schedule, if possible, by considering the quality of the defenses (perhaps weighted to pass defenses?) that each QB faced. Once a season is over, I'd probably recommend FEI (Fremeau Efficiency Index) as a measure of quality of team defense.
September 16th, 2009 at 6:57 PM ^
Wow, I'm shocked and the TT lovers will be totally pissed! They have contended that, "If only you could include TT's running stats, he would obviously be the highest rated QB."
Oops, your data shows he goes DOWN in the rankings.
Ahhhh, that is sweet.
September 16th, 2009 at 8:10 PM ^
More likely, it shows that passing stats are valued more highly by CF13's initial formula than rushing stats are. That's why the need to better understand relative value between passing stats and rushing stats are so important to getting this right.
September 16th, 2009 at 7:12 PM ^
I think this formula passes the smell test, except for the touchdowns.
Let me give a quick summary of what I know about football sabermetrics:
The "neutral point" on a football field in Major College and NFL football is your own 25 yard line (give or take 5 yards or so). In other words, if a team has the ball at its own 25, first and ten, it is a 50/50 proposition of which team is going to score next. So we can say that having it on your own 25 is worth exactly 0 points.
If you lose 25 yards from there, it is worth 2 points to the other team. 25 yards/2 points = 12.5 yards/point. If you gain 75 yards from there, it is worth 6 points for your team. 75 yards/6 points = 12.5 yards/point.
Obviously, it's not perfect, but it gives us a starting point. Now...I'm not going to show my math, but this means that a turnover is worth 4 points, or 50 yards.
Try it some time:
[(total yards gained) - (total yards allowed) + 50 * (takeaways - giveaways)] / 12.5 will get you pretty close to the final scoring margin.
So the formula in the OP, which awards 0.5 units for each yard gained, and subtracts 25 units for each INT, has this ratio spot on. Adding an extra unit for a completion sounds about right; this makes two 5-yard completions the same as one 12-yard completion plus one incompletion. I buy that.
I think the OP has greatly overvalued touchdowns, though. The difference between a 49-yard pass that ends up on the 1 yard line (24.5 units) and a 50-yard touchdown pass (75 units) is too much. Maybe 10 x touchdowns is better than 50 x touchdowns, making a touchdown worth the equivalent of 20 extra yards, instead of 100.
Other than that, it seems to be just right.
September 16th, 2009 at 8:06 PM ^
What can be said about the relative worth of rushing yardage/TDs? I guess it could be reasonable to conclude that a yard is a yard, but I'd be surprised if that were the case.
September 16th, 2009 at 11:16 PM ^
So, essentially, the trouble is finding out what values to give to touchdowns. I think sacks should also be discounted for b/c new info is coming out that suggests that QB's have a lot more ability to avoid sacks than we originally thought/is commonly believed. I think setting up a team for a FG try should also be included (not sure how that would be possible to account for, though).
Other things... we would need to account for the strength of the defense the QB played and the drops his WR's had.
September 16th, 2009 at 7:42 PM ^
Here is a comparison of how M's top 15 QBs (based on college rating) end up when you use the NFL rating.
Name | College Rating | Rank | NFL Rating | Rank |
Grbac | 148.14 | 1 | 99.29 | 1 |
Harbaugh | 145.64 | 2 | 92.60 | 5 |
Wangler | 145.17 | 3 | 91.37 | 6 |
Collins | 145.03 | 4 | 95.60 | 3 |
Taylor | 141.60 | 5 | 94.72 | 4 |
Henson | 140.82 | 6 | 96.18 | 2 |
Brady | 136.43 | 7 | 90.64 | 8 |
Henne | 133.90 | 8 | 90.81 | 7 |
Griese | 131.37 | 9 | 86.81 | 9 |
Leach | 130.03 | 10 | 76.76 | 12 |
Franklin | 129.37 | 11 | 81.24 | 11 |
Brown | 127.89 | 12 | 73.94 | 1 5 |
Dreisbach | 126.00 | 13 | 79.71 | 14 |
Navarre | 125.77 | 14 | 85.09 | 10 |
Smith | 124.51 | 15 | 76.03 | 13 |
So, the weighting factors (which are somewhat or greatly arbitrary) have a significant impact on the evaluation. I have no idea how anyone could "prove" their rating factors are "correct".
September 16th, 2009 at 8:24 PM ^
I think Rushing Attempts need to show up in the numerator. A 10 yd pass is worth 6 points, but a 10 yd run is only worth 5.
Having RuAtt in the numerator is kosher here because the denominator is basically total plays.
I'll pull up my QB injury database and add this version of QER (plus Alton's comments) to it to see how the 2008 rankings change. I'll also add a fumble penalty of the same amount as interceptions.
September 16th, 2009 at 9:54 PM ^
Nothing really changed in the top 10 but Pat White took a monster hit sans spread 'n' shred, not a huge surprise.
Also, I think its safe to say that Nate Davis and David Johnson's numbers would not hold up in the SEC or Big12.
Also also, McCoy is a undoubtedly a better passer than Tebow but (last year) he also had better rushing stats overall than Tebow too. Again, Big 12 defenses are not SEC defenses but the stats say McCoy had a better year than Tebow did in 2008.
2008 Rank | Player | Team | QER_mod |
1 | Sam Bradford | Okla | 5.91 |
2 | Colt McCoy | Tex | 5.15 |
3 | Nate Davis | BaSt | 5.11 |
3 | David Johnson | Tuls | 5.11 |
5 | Chase Clement | Rice | 5.08 |
6 | Tim Tebow | Fla | 5.06 |
7 | Case Keenum | Hou | 5.01 |
8 | Graham Harrell | TexT | 4.97 |
9 | Zac Robinson | OKSt | 4.88 |
10 | Mark Sanchez | USC | 4.84 |
30 | Pat White | WVU | 4.07 | - |
- | Pat White - 2005 | WVU | 4.53 |
- | Pat White - 2006 | WVU | 5.39 |
- | Pat White - 2007 | WVU | 4.96 | - |
- | Dennis Dixon - 2007 | Ore | 5.09 |
- | Tim Tebow - 2007 | Fla | 5.24 |
*No fumble penalty b/c I don't have that data.
September 16th, 2009 at 10:35 PM ^
Edit.. nevermind. I think you're right, but it's too late to re-calculate everything.
September 17th, 2009 at 8:59 AM ^
I did find a stat out there that in the NFL, having the ball first-and-goal at the opponent's one yard line is worth 5.5 points. Obviously, a touchdown is worth almost exactly 7.0 points.
So it looks like a touchdown being worth about 20 yards is very close to the right answer. I'm not going to say that a touchdown should be worth 18.75 yards, because that is pretending that we are working with much more precision than we actually have. Anything from 10 to 35 yards would be plausible to me.
I think it would be great if we could include sacks given up, fumbles lost and even first downs in a passer stat, but I think we need to limit ourselves to what is found on a standard stat sheet, because I am sure that Collegefootball13 wasn't planning on going into the play-by-play records of the games to calculate the new efficiency stat.
Comments