At Least Nick Sheridan Has People Skills

Submitted by colin on February 13th, 2009 at 9:15 AM

One of the most lamentable aspects of being a college football fan as far as I'm concerned has long been the lack of quality stat keeping, as well as analysis. Matt Hinton (currently Dr. Saturday) and Chris at Smart Football are great, and if CFB Stats didn't exist, this post wouldn't exist, but it ain't no Fangraphs and those fellas ain't quite Tom Tango, who literally wrote The Book on baseball. Not that it's a fair comparison.

I bring Tango up because his stat wOBA inspired this post. wOBA (weighted On Base Average) is basically on base percentage gone plaid. Instead of dividing times on base (1B+2B+3B+HR+BB+HBP+ROE) by plate appearances, you decide how valuable in terms of runs each of those individual events are and then proceed (hence weighted). OBP is transformed into runs per plate appearance. Multiply times total PAs and you have the runs that batter was responsible for in that season. And scoring (or preventing) runs are the bottom line in baseball. In sum: bases get you runs get you wins. In football, it looks like this:

Yards - Turnovers = Points

This isn't exactly groundbreaking. It's a fundamental assumption behind Dr. Saturday's Life on the Margins, iirc, and I'm pretty sure this is what I'm going to find in Pete Palmer's Hidden Game of Football if and when it eventually ships to a2. And it's sorta-kinda what David Romer did, though not nearly exhaustive. The theory is good. The actual arithmetic is kind of annoying and is summarized in the following paragraph. Feel free to skip to the part where we find out just how crippling the impact of Nick Sheridan was and how much worse it could have been.

The key to being able to do this yourself is to figure out yards and turnovers in terms of points. I ripped the drive logs of every Big Ten conference game in 2008 from Yahoo. That'll give you yards/point, which came out to about 15. Then I plotted, in buckets of 10 yards, the percent of drives that resulted in a TD or FG based on the drive starting field position, except the last 30 yards which I averaged at the opponent's 15 due to relatively few samples.* This gives you average expected points based on field position. That plus average field position equals the average value of a possession, which is what you lose in a turnover. Not only that, but you give expected points to your opposition. According to my math, an INT was worth about -4 points. Thus points per throw is (Yds/15 + INTs*4)/attempts.

Feel free to comment

I Am Not An Expert. If my math is off, then suggest different constants/methods. They pass the sniff test to me; I ran assorted regressions on excel to test assumptions and it looked right. I'd be glad to share the drive chart database. Onward...

The Part Where We Find Out Just How Crippling The Impact Of Nick Sheridan Was

go here

It's sorted by pts/attempt, the relevant measure. Average was .33. Mr. Sheridan was dead last with those over 50 attempts with .15 points per attempt. An all around average team wins 4 games. The results indicate that an all around average team that replaced its average quarterback with Nick Sheridan would win 2 (converting to wins over average is easy enough). But it would also have tremendous team chemistry and at least one valedictorian. Wins aren't everything.

Also, check out Terrelle Pryor's numbers. Remember, this is just per throw. Rushing and sack yards are not included, nor is it defense adjusted. Having rewatched the Texas and Michigan games in HD (being able to see the d-backs helps), I was impressed. Tressel used the threat of Wells inside and Pryor's skills when bootlegged on the edge to great effect. The playbook seemed cut down, but his athleticism made it work. The sack numbers (scroll right in the g-doc) and somewhat inconsistent mechanics are the most glaring issues, but they were exaggerated by a bad pass blocking unit in front of him. In conclusion: barring injury, Pryor is going to be a terror. Surprise! Rivals #1 overall prospect in 2008 is projected to dominate. At least he'll probably be gone after his junior year.

*It's a shortcut and it probably understates how valuable possessions that start inside the 15 are. I actually think inside the 15 the function is probably no longer linear. I'm also sorry that this is isn't the most thorough or transparent presentation. It's a start though.

Comments

1M1Ucla

February 13th, 2009 at 12:51 PM ^

He was likely referring to his quarterback play -- Nick just seemed to fold in the games (save Minnesota). Without question, he was put in some really untenable spots to get his experience. Just count up the number of times he went onto the field with the ball inside his own 15 yard line. But I really think the game was just too fast for him. Watch his fakes with the ball, his footwork -- he rushed everything, couldn't get comfortable. I'm sure I'm along in this, but I think if Nick were (and Steve, for that matter) had taken a more typical route to the field, not starting until Jr or Sr year, they would not have been less effective than the kid at OSU last year or any of the NU, IU, Minn or MSU quarterbacks. Among those guys, Nick and Steve would have been in a relatively comparable cohort. A comparison I'd like to see would be those guys vs all other freshman qb starters in the nation, or vs second-stringers with no starts across the nation.

Glen Masons Hot Wife

February 13th, 2009 at 10:47 AM ^

"Pryor is going to be a terror. Surprise! Rivals #1 overall prospect in 2008 is projected to dominate. At least he'll probably be gone after his junior year."

What, about his floating-duck throws, makes you think he'll leave early to be an NFL quarterback?

stubob

February 13th, 2009 at 10:59 AM ^

I had briefly examined Yards Per Point earlier this season to determine offensive efficiency. My conclusion was that it may be useful for overall offensive efficiency, but week-to-week it's too variable of a predictor. I should re-run the numbers for the top 25 now that the season is over and see if the best teams have the highest YPP.

My reasoning being that a good defense and a "lucky" team would have 1. , on average, better field position and have less distance to go to score, and 2. manage to convert more chances into points of some kind. I hadn't thought about using it as a metric of individual offensive efficiency, but I like it.

It seems biased toward high-scoring teams, but I guess that IS a mark of offensive efficiency. If you've got Missouri putting up 60 points per game on 500+ yards of offense, it's hard to argue that that's not the best offense in the country. Now, to be fair, it should factor in the average defensive yards per game or something, so blowing out West Texas Teacher's College doesn't skew the numbers.

I guess it's a simplified calcuation of Passing Efficiency, but just as useful.

colin

February 13th, 2009 at 11:06 AM ^

you're not going to get anywhere near a useful sample. i've run both offense and defense numbers and gotten a Pythag type number and it did a pretty good job predicting 2008 results, actually.

this was really a preliminary post. supposedly, SBN is going to get a college football sabr site up and running. that would be a big help, since i'm already drowning myself in spreadsheets.

stubob

February 13th, 2009 at 12:34 PM ^

I recognized that as a problem fairly quickly: year-to-year, the stats wouldn't transfer very well (too much variation in team), and you need enough data to make a reasonable prediction. With only 12-17 games, there's too much variability.

I think having 162 games makes baseball just much more useful for statistical analysis. Your sample size is 10x what it is for football.

I got flamed for this before, but I also think that the simplicity of the outcomes of baseball also helps statistical analysis. Football, basically anything can happen on any play. Offense scores, defense scores, anything in between. Baseball you've got: ball, strike, out, walk, hit, run.

But good post, I'd be curious to see the top QBs using that and compare it to the Passing Efficiency stats (not that PE is the gold standard).

colin

February 13th, 2009 at 1:13 PM ^

i have to say the amount of variance and flux that everyone is talking about is overstated. it looks like a very stable population to me. you have to remember that the baseline here is plays, of which any one offense gets at least 60 of per game. it's not that big a deal.

stubob

February 13th, 2009 at 6:02 PM ^

I think you're right, points per drive is probably a more useful metric. Looking at it from a game perspective, there's not enough data, but per drive (or per play), you have a lot bigger statistical set to draw from.

I may have to play with that and see how it looks. I'm still not sure how to compensate for weak competition, so the Florida / /Oklahoma game properly represents the level of competition.

colin

February 14th, 2009 at 7:03 PM ^

it seemed easier to control. if you have an easier means to get the per play/per possession data, i would definitely go conference by conference and then make weights by looking at OOC games. i did the drive log rips by hand, c&ping my ass off, so i'm not making any commitments as far as that goes.

iawolve

February 13th, 2009 at 11:01 AM ^

Granted, his resulting numbers look pretty good. I still think you need to consider how few passes he through. Part of that was a reduced playbook, part was that he can create with his legs. My hope is that he is not able to develop more as a passer. I was more surprised at how efficient Clark was which I guess should not be surprising to me considering they won the conference.

PeterKlima

February 13th, 2009 at 11:16 AM ^

Sorry, there is a reason football is not so "stats laden" as baseball and that is because football stats are EXTREMELY misleading.

A QB's performance depends, in LARGE part, on the team around him catching the ball, blocking, threatening to run. This is even more true when you ascribe the "teams points" to his statistics.

This is not a batter against a pitcher. While some QBs are better than others, looking at broad stats gives you more of an indication about how well the team performed. (although there is some reflection of the QB's performance).

I know all football stats have this problem, but this analysis seems to stretch it even more.

Personally, I prefer a UFR based system that ascribes some fault or credit (somewhat objectively) on one or two of the eleven players on the field.

Regardless, I am sure Nick Sheridan is a good guy, but he didn't do too well by any assessment.

1M1Ucla

February 13th, 2009 at 12:35 PM ^

misleading is fallacious, though your rationale is not. It's not that football stats can't exist, it's that they haven't been developed to a point that appropriately teases out individual performance from team, or even, system performance.

This calls out for a multi-variate-type of analysis that would permit the scoring positions' stats to be weighted with all the other stats on the field. Colin's analysis actually takes a net-effect correlation which is relatively effective for comparison's sake, but has a fairly large error bar.

One way to look at this is, as Colin suggests, in a VORP-type way -- where does each guy sit with regard to a mean -- and estimate their individual impact in replacement. The range of the statistic is wide enough (0.16 - 0.47, +/- some 50% around a mid-point of 0.32 or so) that it discriminates the extremes really well -- Clark = Sheridan times 3 -- that's a bunch.

Using a UFR to mediate the numbers makes a lot of sense, as it goes the direction of a multi-variate analysis, and would tend to make it more predictive -- take this guy out of that offense and stick him in another and see how he does. Less time on his back or running for his life, more Tacopants or less, and you'll see the difference. How much? Would sitting in the comparative comfort of the Penn State personnel make up the factor of 3 difference for Nick, or the factor of 2 difference for Steve? Not likely, but could it make up the factor of 1.4 for Steve vs Pryor? Hmmmm, I'm intrigued by that proposition because tOSU's O-line wasn't too special, but Terrelle had Beanie and the other guy, several competent + receivers, and his own escapability (which basically offsets the OL suck factor). Count me as a Pryor skeptic because I hate the way he throws -- I'm not sure his mechanics are fixable. Without Beanie, that offense didn't accomplish much.

So, I think there's a lot of room for improvement in football stats, but it's a doable project, despite your characterization that football is inherently different.

My biggest surprises?

That Steve's number wasn't negative and that Clark produces a point with every couple of throws.

Ernis

February 15th, 2009 at 5:43 PM ^

Agree on this being useful for comparison and not much else. Using this to compare Threet to Sheridan is pretty useful because the majority of confounding factors (that is to say, the entire fucking team) are held more-or-less constant between the two (obviously some error [for example, different opponents, variation within teammates on the field, etc.], but as long as you compare within-team QB's this stat isn't terribly misleading). It falls apart when you compare across teams.

I'm curious about your proposed method of multivariate regression analysis. I agree it could be done, but you have to consider the law of diminishing returns. At what point of time, effort, and sophistication would you be creating a meaningful statistical analysis? My thinking is that the value of those results would not be worth the time put into the method, compared to the understanding one can achieve simply by watching the game carefully.

It's like using calculus to plot out a ball's trajectory to catch it, instead of letting instinct tell you where to put your hand. At some point, your calculation is a barrier to successfully catching the ball.

Cosmic Blue

February 13th, 2009 at 11:27 AM ^

As an engineer, i eat this stuff up. i wish i had more patience to do this kind of thing on my own. maybe ill be more inspired to do something like this next football season. there are lots of things you can build into this formula to give it more accuracy

Michigan Arrogance

February 13th, 2009 at 11:27 AM ^

i like it, but i think peter brings up a good point about the nature of the game. it's really hard to separate an individuals worth from the team in this sport. that's why i just try to think about separating the offense's quality from the defense's and STs. points and yards per possession for an offense and defense are my thing.

Little Bro

February 13th, 2009 at 11:47 AM ^

I believe that's part of the theory behind adjusted statistics as well as just evaluating the why of a statistic. In adjusting the statistics to account for different defenses and offenses you can start to adjust where teams actually rank. Stats have to start somewhere before you can begin to make them better.

Starko

February 13th, 2009 at 11:50 AM ^

This does seem like as significant statistic, but of the efficiency of the offense, and not necessarily of just the quarterback.

What may be more telling on an individual level is where two quarterbacks on one team have different results, as in the case of Indiana.

Cosmic Blue

February 13th, 2009 at 11:52 AM ^

yes, this does not take into account all considerations. football does have a lot of other variables involved and it is not as statistics driven as baseball. of course sheridan wont get exactly 0.15 pts/throw. does that mean we shouldnt attempt to come up with metrics to show relative production? no, this can be a useful tool for evaluating player ability and should not be disposed of just because its not perfect

Cosmic Blue

February 13th, 2009 at 12:09 PM ^

and all you had to go on was this information (pretend you were in a coma all last year and noone told you about anything) and nothing else. would you bet that sheridan or clark (again, pretend both are playing) would have a better results/success/failure?

you go ahead and flip a coin cuz you dont want to use this as a predicter - ill take clark (and your money)

gsimmons85

February 13th, 2009 at 12:27 PM ^

that clark was loosing his recievers, his entire ol, and his coach...

and that michigan had the top recruiting class in the nation, and returned all 11 starters..

i would think that would change the bet a bit dont you?

unfortunatley sports do not live in vacums where status quos remain, and nothing changes. Im not degrading the value of such statistics, im just meerly pointing out that it doesnt NECCESARILY corilate to future events.

Cosmic Blue

February 13th, 2009 at 12:37 PM ^

there are definitely scenarios where these kind of things fall apart - you described several. clark could get injured or decide to try throwing with his other arm for some reason. theres tons of ways things could change. im merely saying its not a worthless indicator and more often than not it would apply

gsimmons85

February 13th, 2009 at 12:46 PM ^

and obviously i could use the ole' "they dont play the game on paper analogy"...

there are patterns, but very often sport breaks patterns, thats why people watch it...

otherwise we would give a 300 hiter 3 out of ten hits, before his next ten at bats.

but what if he is aobut to go on a tear? or about to go into a slump?

most stats to most relavent coches today are great guides, but dangerous rules..

again not discrediting anything any of you guys have done that is really great stuff, or trying to discredit any other opinion

gsimmons85

February 13th, 2009 at 12:51 PM ^

can not account for the things on a game to game basis that makes all the difference in the world. What the person had for breakfest, the air temperture, the argument he had with his wife, the comfort level against a pitcher, the length of his cleats, the 3rd light in the second tower being out, the crick in his neck... etc etc etc.

good for guiding expectations? sure, accurate predictor of the future, obviously not.

dex

February 13th, 2009 at 1:03 PM ^

right - if you are trying to use the numbers to predict exactly what ARod is going to do in a specific at-bat, your odds of success are low.

But, as I said down below, PECOTA has been a very reliable predictor of team success for upcoming seasons. It's obviously not right every time for every team, but it gets within a close range a significant portion of the time.

addendum:

I look at it the same way I do recruiting rankings. Trying to pin it down and use the rankings to say "this guy will fail" or "this guy will be a 1st round pick" isn't very accurate. But on the macro level, they tend to be much more accurate. Will Campbell might be a specific 5* that doesn't pan out, but most of the 5* guys from this year will end up being decent. Looking at individual events (like one at bat) or players (like Pat White) is always going to be impossible to predict.

1M1Ucla

February 13th, 2009 at 1:10 PM ^

predicting game-to-game individual performance. A guy who completes 60% of his throws doesn't get there by hitting 30% one game and 90% another. All game-planning is based on tendencies and a DC prepping for Chad Henne was going to prep for knowing between 55 and 65% of his throws were going to be completed -- the game plan goes to trying to keep that at the low end and not let too many of that 60% get to the end zone.

The toughest parts to predict are the low-incidence game-changing plays: Beanie breaking off tackle for 60 yards and 6, #2 pulling an impossible INT on the sideline, #21 laying out and catching a 6-ball in ND's endzone -- every coach sets up to try to get that one or two plays for himself, and prevent the other guy from getting his.

I'd say one of the reasons football stats haven't advanced much is that they have been adequate predictors of future performance -- there just hasn't been enough incentive to develop much that is more sophisticated.

gsimmons85

February 13th, 2009 at 1:27 PM ^

this makes even more of a mystery to me why you guys get so emotionally involved in games. Sounds to me like you already know what a person is going to do, what he isnt going to do, and how the game is going to end up before it even happens.

yet after the games i spend half my afternoon trying to talk some of you down from the cliff, and keeping you positive about things.

But then again maybe thats why some of you blame coaches for everything, you expect the stats to be true every time, and if its not, its the coaches fault. But when teams are expected to beat us, why woudl you get upset? i dont get it.

truth of the matter is that most time we expect stats to not play out. if beenie is averaging 100 yards per game, we are trying to keep him under 100... if chad henne completes 60% of his passes, AS A DC. I AM ABSOLUTELY TRYING TO KEEP HIM FROM THAT, because that is obviosly the strength of the offense. We try to take away strengths, we use stats to tell us what a team has been good at, then devise a plan to make them suck at it. IF we do it right we win, if we dont they win.... Teams win when stats dont play out, teams loose when they dont play out. Stats help me prepare for what i need to stop, and what i can expect to see, but often go out the window when the games begin becasue of adjustments, and other unforseen circumstances. Im not playing The state champions average for the year, when we meet them in the playoffs, im playing their stats for that game, if i shut down their running game, i have to be ready for what they will do next, reagardless of what their stats say. If their QB is off that night, i have to play for that tendecny, not the tendency over the course of the entire year. at least in my 25 years of playing at every level, and coaching a 4-a highschool to a top ten defensive season in the state, thats what i have seen.

but obviously i dont know everything otherwise i would be more famous than i am. And i very well could be wrong about everything...

dex

February 13th, 2009 at 1:36 PM ^

I don't know if that was directed at me, but I explicitly said that using the numbers to predict specific events like "will player x complete this particular pass" or "will he strike out this at bat" isn't going to be super accurate outside of general stuff like "he's struck out 30% of the time against this pitcher so there's like a 1 in 3 chance he will" or something like that.

If I could use any statistic to predict outcomes of individual games with super accurate consistency I wouldn't be posting here, I'd be sleeping with high priced escorts on top of a pile of money.

chitownblue (not verified)

February 13th, 2009 at 2:12 PM ^

Well, "sample size" is a huge issue. Over the course of 162 games, Albert Pujols will hit something like .330/.430/.580 and hit 40-odd homers. Fine. But that really isn't predictive for an individual at-bat, game, or even month of the season. So I think stats can predict an aggregate performance, if you can find the right ones for a given sport. But stats, like Chad Henne's 2006 performance, contained booms and busts that really can't be predicted.

chitownblue (not verified)

February 13th, 2009 at 12:42 PM ^

For those "this works in baseball, not football" people, this is a valid point. When the batter is in the box, there really isn't any "team" context to seperate his performance from. You need to account for the ballparks they play in, but that's relatively simple.

colin

February 13th, 2009 at 1:03 PM ^

if we write off the idea that this can't work in football, we'll never get to the point where we have people devoted to getting stats right. i mean have you seen pitch f/x? mlb releases that data free of charge to its fans. hit f/x is supposedly on its way and in general both mlb.com and milb.com are great resources. the ncaa doesn't have near that kind of commitment to its fans on this point, largely because it's not demanded of them. they only recently started recording sacks!

the long term goal, in my opinion, should be to improve the dialogue about college football.

1M1Ucla

February 13th, 2009 at 3:17 PM ^

doesn't mean it's unknowable.

To assert that it is amounts to the fallacy of relevance called argumentum ad ignorantium.

The coverage of football has been woeful in its lack of detail and inquiry. There is a lot of inside knowledge that doesn't get anywhere out into public at large, knowledge that people find exceptionally interesting. The sort of inside detail that "everyone" knows about baseball adds to its appeal -- don't swing at a first pitch breaking ball, the call between short and second with a man on first as to who covers the bag, signals from infielder to outfielder indicating pitch to influence positioning, tips by the pitcher on what pitch is coming or whether he is throwing over -- all that adds to the fraternity of baseball fans.

And a lot of that has been missing for the football fan. The average fan's lack of knowledge of coverages, blitzes, blocking scheme, reads and the like makes football fans a fraternity of emotion -- are they fired up enough? They don't hit hard enough, they should have run on second and 4 from the 20 because Bilenzeiwicz looks like he has the "hot hand". Blah, blah, blah. It's a fool's dialog most of the time -- that's actually why Brian's offering is such a success. It's an intelligent alternative to Sports Illustrated and the daily newspaper.