A Short Success Rate Follow Up

Submitted by The Mathlete on February 17th, 2012 at 11:56 AM

Background: Original article, Bill Connelly’s follow-up

I don’t think Success Rate is a misguided stat as much as I think it is a misguided strategy. I think the overall concept of S&P that Bill uses is very sound, I just think the emphasis should be more on the P than the S.

My biggest problem with the stat is that it is black and white. As comments on his article note, a metric that works on a sliding scale would a significant step in the right direction. On 1st and 10 losses and gains of 4 aren’t and shouldn’t all be treated the same. Just as gains of 5 and up are all valuable, just not equally as valuable. For my metric the sliding scale is factored into the expected points at any play. So there is some element of success rate built into PAN, but it is an integrated, sliding scale as opposed to a separate, black and white component.

There are only three things that matter for evaluating a team on a drive, where did you start, how many points did you score and what position did you give the ball back to your defense/special teams. Plays taken to achieve results and time elapsed off of the clock can be valuable in certain situations, but in general those three data points are the key. If we can effectively measure each play in how it contributes to those three key factors at once, why break it up into two pieces and why make it black and white?

Even though there are some differences and I got things off on a bit of the wrong foot, I think there is more in common than different with the two approaches. What I think is the ultimate issue, however, is coaches calling plays with success rate in mind. Advanced NFL Stats did a great article on this very subject (especially the Importance of Run Success Rate section). He found evidence at the NFL level that coaches are coaching to down by down success rate as opposed to drive success rate. Coaches appear to be attempting to win each battle and at times losing sight of the war.

The battle/war concept is what I think is the most interesting of this so you’ll have to wait until part 3 of this series where I’ll look at how strategy can adapt to score more points while risking a bit of short term success rate. Early next week I’ll post part 2, a look at how Wisconsin’s offense runs and how Russell Wilson was really the most dangerous part of the Badger offense.



February 17th, 2012 at 1:16 PM ^

Man, I always have a nerdgasm as soon I see a Mathlete post on the sidebar!

I can't see how this particular topic has become a big debate. It's obvious that a more granual description is superior to a binary one. If someone wants to simplify to the binary lens, that's fine, but I prefer more information.

Anyway, a bit offtopic to this particular discussion, but a question to the Mathlete in general: It seems that most of your efforts to this point have been geard toward explaining and/or describing past events. I know you post your weekly matchup predictions during the season, but I don't recall seeing any more detail on the effectiveness of PAN and your other stats on predicting things. I know you've done some one-off studies, such as the write-ups on recruiting rankings and returning starters, but I'm talking of a more broad approach. Is this something you intend to focus on? I ask out of curiousity, since in general, I like to use data to tell the future story, and I'd be willing to help out. My day job is spent building quantitative models to predict stock returns, so I'm guessing I can provide something useful.