TO THE HOT TAKE CANNON
[Ed: MCalibur, apparenly an economist found himself collateral damage on today's shotgun blast at "X is stupid" sports economists. Maybe I should have come up with a label like "freakonomists" so as to not implicate people who are just interested in the numbers without the look at me pub. Anyway, here's an excellent diary on what your goals should be on second and third down. Implications for a second and medium are interesting.]
A while back The Mathlete sent out a Thundercat signal for some help shucking data for his database; at least that’s how I remember it. Any un-lame kid of the 80’s knows that when you see the Thundercat Crest you put on your spiked suspenders, pick up your laser shooting panther paw nun chucks, jump into the tank you built singlehandedly, and you roll; that’s all there is to it. I had no choice.
Anyway, we voltroned* our abilities together and came up with something pretty sweet. I have put together my own database, with Mathlete’s help, and can now do some of the same tricks he can. I’ve focused onto BCS-BCS matchups extending the thought of excluding mismatches; Michigan v. Eastern Michigan is still a significant mismatch.
*Oops, wrong cartoon but, then again, you simply cannot over-reference 80’s cartoons/shows. I pity the fool that disagrees. I feel bad for youngins that don’t know the glory of 80’s children’s programming. Also, am I the only one who thinks that Voltron and Zoltan might be related?
When I’m not eliciting unreasonable responses from otherwise reasonable people, I’m usually crunching numbers of some kind as if they were a motley band of mutants and aliens led by a grody and ancient mummy demon priest. Very often the numbers have something to do with football in general and, most often, Michigan football specifically. This time I wondered “how do we know if a play was successful or not?” This question has been asked and answered by some smart people before, but being the curious little twit that I am, I wanted to gauge it on my own.
One way to go about it is Mathlete Style: Expected Points, a good but abstract method. One potential problem with focusing on EP is that doing so can drive you to scoring points where as the real goal is to win. It’s a subtle but important distinction. Depending on the situation, maximizing EP might not be the same as maximizing the probability that you will win. Maybe you would rather not score if doing so means giving Peyton Manning the ball back with 25 seconds left and less than a 1 score deficit. Besides, The Mathlete has this beat covered.
Another method is to use 1st Down Probability, the likelihood that a team will convert a new set of downs given the current down and distance. I think this is more appropriate to the microcosm of a play because the goal of a play is not necessarily to score it is to keep the ball and move it forward, in that order. Scoring is the goal of an entire drive. To calculate 1DP, you do the same thing you would to derive EP, except you keep track of first downs instead of points.
Whenever you have a mountain of data, you need a way to focus your attention on what matters while still maintaining the value of having so much data in the first place. For this study, I’ve filtered on the following criterion:
- Exclude plays involving a penalty of any kind.
- The game must be close. My arbitrary definition is: all plays in the first and second quarter, third quarter plays where the lead is less than 17, and fourth quarter plays where the lead is less than 10. These values are arbitrary, but there are so many plays available that the sample sizes are still large enough that any additional precision is of negligible value. Also, any unimportant plays are swarmed by a large number of plays that are important, then math deals with the noise.
- Results of the play are limited to –10 and +25 yards. The logic here is two fold. On the negative side, the average sack is good for about 6 to 8 yards, anything bigger than that is a fluke play (botched snap for example). On the positive side, most plays aren’t designed to go for huge gains. However, there are instances when an OC calls a play like that in order to exploit an advantage and not necessarily as part of a base strategy. Though relatively infrequent, both types of plays happen with enough regularity that they significantly shift the averages even though they are vastly outnumbered by more typical gains. This filter only excludes about 0.5% of all plays to the negative side and about 5.3% to the positive side.
Each play in the database has been assigned a 0 or 1 depending on whether or not it was part of a first down series, touchdowns are counted as first downs in this survey. Essentially, every play in a four down sequence is counted as a being part of a 1st down unless a punt or turnover occurs before a new set of downs is achieved. Filtering the plays that made the cut (over 105k) by down results in the following scatter plot:
Every point on the chart above has at least 15 samples, most have several hundred, some have several thousand, and 1st and 10 has almost 42,000 samples. The trends are self evident and really, really, strong. A few comments on other decisions I’ve needed to make here:
- The small black dots represent 4th down plays. They are essentially overlaid with the 3rd down plays which makes sense, the objectives in both cases is the same, convert to a 1st Down. If you’re in a 4th down decision, use the 3rd down line.
- The curves for 1st and 2nd Down were both pegged to 100% probability of converting a new set of downs at zero yards to go; pretty obvious as to why, it’s the rules. On 3rd Down however, I opted not to peg it to y3 = 1 at x = 0 because even though the R-squared value doesn’t suffer by much (0.005 lower), the resulting curve significantly over estimates 3rd down success inside of 3rd and 5. Also, I think the gap could be real; how much error is there in spotting the ball (especially on QB sneak type plays)? To me this data implies that the ball is mis-spotted to deny a 1st Down conversion approximately 9% of the time. The incremental error of spotting the ball doesn't matter until you end up at 4th and inches.
- For 1st down plays, I intervened on behalf of noise reduction by only including plays where the distance was in multiples of 5. The reason is that the rules say you start at 1st and 10 and the only way you end up with 1st and something other than a multiple of 5 is A) you’re inside the opponents 10, and B) multiple penalties or 1st down repeats after spot fouls. Plays that were rejected are largely noise; the legitimate plays (ex. 1st and X inside the opp. 10) act like 2nd down plays, so use that in those cases.
Generating Hard Targets
Now that we have a survey, we can use the information to answer the question I asked “what makes a successful play”? The question has been tackled before in the seminal tome The Hidden Game of Football. The DVOA system developed by Football Outsiders is based in concepts discussed in Hidden Game. Hidden Game presents the following goal schedule:
On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.
So, the goal schedule by down should be 4-ish yards on 1st Down10, 3 yards on 2nd and 6, and 3 yards on 3rd and 3. I haven’t read Hidden Game but this doesn’t look right, particularly in short yardage situations. For example, 2nd and 1 is a failure if you do not convert a new set of downs. Sure, the consequences of that failure are small because you are virtually guaranteed another chance to convert but gaining zero yards (we only have whole yard resolution) is failure by definition.
Brian Brown of Advanced NFL Stats fame has a better definition: a play is a success as long as your chances to convert a new set of downs are not hurt by the result of a play. The great thing about this definition is that it considers the opportunity cost of running a play. This simple idea probably explains why a lot of OC’s call conservative plays on 1st and 10, if you don’t advance the ball by about 4 yards, you’re worse off than you started. Brown focuses his work on the NFL and has done this work for the League but he stopped at the first chart leaving the answer to the question abstract-don’t hurt your chances of getting a new set of downs. OK, but how do you avoid that?
Running an optimization routine on our curves gives us the concrete answer, a goal schedule by down and distance in chart form.
- 3rd down is obvious, you need to gain all of the yards remaining or you’ve failed. Fourth down decisions should be avoided.
- The 1st down requirement is virtually flat at a 37% yield, lower than what Hidden Game suggested.
- The 2nd down requirement is asymptotic to 65% yield but reaches a requirement of 80% yield by 5 yards to go. Essentially, you need at least 4 yards on 2nd and 5 to not have wasted the down.
First down is all business, you must move the ball 37% of the way or you’re screwing yourself. Third down is also all business, you need to convert or risk deciding which poison tastes the best. Second down however, depending in the situation, that’s a down you can get jiggy with.
On a generic 1st and 10, there’s a 64% chance of converting a new set of downs. So, as long as you end up with about a 64% chance of converting on 3rd down, you can do whatever you want on second down as long as you don’t lose yards or give the ball away. That means, you need to end up at 3rd and 3 or better. On 2nd and 3 or better call in the B2s and Outkast, baby, ‘cause it’s time to drop bombs (over Baghdad).
Hello everyone, Six Zero here with the latest installment of:
SIX QUESTIONS WITH BLUE IN SOUTH BEND
Inspired by the official site’s “Two Minute Drill” series and TomVH’s famous Q&A segments with potential recruits, this weekly feature highlights some of the more famous personalities here at MGoBlog. Without pulling back the infamous veil of blog anonymity, we’ll get to know some of your favorite posters better and possibly shed some light on their definition of why it’s so darn Great, To Be, A Michigan Wolverine.
South Bend, Indiana, where both the pride of all Hibernians and the shame of NBC Sports budget specialists call home. We’ve all suffered for our love of the maize and blue, but some do more than others, and our man Blue In South Bend is certainly a capable testimony of what some of us endure to cheer on the Wolverines. Before we begin, I’d also like to acknowledge that any bit of pressure I felt over my vacation week (I’m looking at you, Sgt. Wolverine) was quickly tempered when Blue not only
did his part, but graciously some of mine as well. Here’s the
exclusive MGoProfile interview:
1. Alright, Blue, first things first—one thing I’ve noticed is that you are routinely among the quickest responders on the blog. At the time of this writing, you’re typically among the first 8 entries of any given blog you respond to. And that’s not a bad thing (unless we might ask your wife or g/f.) So is it safe to say that you spend a lot of the workday connected to MGoBlog? (Not that the rest of us don’t, naturally)
I'd like to say that I respond quickly because I think faster than everyone else, but in reality I just spend way too much time on MGoBlog. I'm a law student, so I spend most of my day sitting in front of a computer screen anyway. It's way too easy to tab over to check on the status of Sean Parker or the latest on the Jihad. So when you combine that with my instinctive need to share my opinion about absolutely everything, the result is a bunch of comments from yours truly.
As for my wife... well, let's just say that if things ever go south in our marriage, Brian Cook may be deposed by my wife during the divorce proceedings.
2. And yet no blogs. You’re a great example of how someone can be a solid contributor without developing too much of your own content… Have you ever had the desire to start your own blog entries, and if so, what would they be about?
I've considered developing my own content, but I've realized that I'm not really the local expert on anything. I can speak intelligently about most things Michigan... but Magnus knows more about football, FA knows more about baseball, EVERYONE knows more about hoops, and THE KNOWLEDGE knows more about the rest of the Universe. My areas of expertise are law and politics. Politics, as you may have heard, is frowned upon on this here blog, and law is rarely very interesting. Also, despite the appearance to the contrary, I usually don't have the time to develop in-depth content. My free time comes in 2-minute spurts at random points throughout the day, and I don't have the attention span to put something comprehensive together, let alone on a regular basis.
What I do bring to the table, I think, is the ability to remain fairly level-headed, and to further the debate when people get a little too high (OMG 13-0 TATE WINS TEH HEISMAN) or low (RR CAN'T WIN 5-7 IS UNACCEPTABLE LULZ). Plus, someone has to post the occasional timely youtube clip or random picture of Fat Boren, so I think I have a niche.
3. Now, the name… obviously you’re one of the Wolverine faithful deep in enemy territory. Explain how that affects your life on an almost daily level, and describe your thoughts on the ND-UM rivalry as a whole.
I sometimes feel like Jane Goodall. The Irish allow me to live among them, and while they do not embrace me, they tolerate my presence as a sort of novelty. Truth be told, it isn't that bad here. I lived in East Lansing a little over 3 years after I graduated from Michigan in '05, and the Irish are far more hospitable than the Spartans. To a certain extent, I think it reflects the different complexes of the two fan bases. Where the Sparties evince an inferiority complex that would make Canada blush, the locals are more worried about how to defend their next National Championship. Their concerns far exceed the Michigan game; they don't want one win, they want a dozen. Besides that, the Irish are actually very knowledgeable, quite accommodating to visitors and opposing fans, and as supportive of their team as any fans I've encountered. So in that respect, it's hard to generate the same hatred for the Irish as for the Spartans or Buckeyes.
Personally, I will very much miss having Charlie Weis on the opposing sideline next year, but all in all my animosity for the Irish is fairly low—I’d definitely rank them the lowest of the three primary rivalries. What else can you tell us about the Irish fan base?
The interesting thing about the Notre Dame fan is that he truly believes in his
delusional highly optimistic take on the University, and on ND football in particular. He really, truly believes that Notre Dame should have competed for a National Title last year, and that Charlie Weis was the ONLY reason they didn't do so. I can't tell you how many conversations I've had with locals who think that ND's is in line for a 10-2 regular season this year, with an outside chance to go 11-1 or 12-0. I've tried to explain the parallels between Michigan '08 and Notre Dame '10 (new coach, new system, suspect defense, replacing a multiple year starting quarterback and several offensive linemen, etc.), but they will have none of it. The caricature we have created of the Irish isn't just based on reality. It IS reality.
Living in enemy territory is hard, though. No one around here wants to talk about Michigan sports. My wife is a Spartan, and she doesn't care much about football anyway. MGoBlog is my one link to people who care whether Troy Woolfolk has been getting more reps at corner or safety, and whether Devin Gardner should redshirt (he should). That, and South Bend is ungodly boring.
4. Which leads us right into our next question: What do you like to do for fun on your own time? And, as always, can you describe the perfect meal?
For a restaurant meal, I'd say it the Bang Bang Shrimp and Diablo Shrimp Fettuccini at Bonefish Grill. For a home-cooked meal, my wife makes a great white chicken chili. Either way, I have a fairly prominent sweet tooth, so desert is a necessity.
As for my free time... thanks to this blog (damn you, Brian Cook) and my various responsibilities, I don't have all that much free time anymore. I spend most of my day reading legal things, writing about those things, and reading some more legal things. I'm also searching for post-graduation employment (HIRE ME PLZ... k thx), which is annoyingly time consuming and thus far unproductive. Beyond that, I enjoy golfing, running, basketball, and most other athletic activities. I like spending time with my wife (we just celebrated our first anniversary) and my dog.
Yes... the dog. Your avatar definitely works—When I see one of your posts I always know it’s you without having to even read the name. Is this your dog? Is there a story behind your avatar?
He isn't my dog. I have a year-old American Bulldog named Gus, but Gus has too much self-respect to dress up in a costume just to satisfy my blogging needs. He's great, though; I'd highly recommend the breed. Truth be known, my avatar is just one of the first Google Image search results for "football dog."
5. Can you explain why you are a Michigan fan?
I actually grew up in a Spartan household. Both of my parents went to State, so it's hard to say how or why I became a Michigan fan. It may have had something to do with Tim Biakabutuka running like a possessed water buffalo against Ohio State in 1995. I loved my four years at Michigan, and really wanted to come back for law school, though it was not to be (Demar and I are starting an "I love Michigan admissions" club. Dues are $10, and that includes the t-shirt). To me, Michigan has always been the total package. First rate academics, excellent athletics, great town, great people. That, and we control space, bitches. SPACE.
6. Finally, the staple last question-- who's your all-time favorite Wolverine?
I don't have a single favorite. I've always loved the workhorse running backs, so my favorites are probably Mike Hart and Chris Perry. But after the events of a cool October evening in 2004, I will always have a soft spot for Braylon as well. Honorable mentions goes to Phil Brabbs and Jason Avant.
/Wife in South Bend takes the computer.
Give me back my husband, damnit. Dinner is getting cold. Besides, the season is still two months away. What could you people possibly need to talk about right now?
/Blue in South Bend takes back the keyboard
Sorry about that. Guess that means it's time to go.
The man’s name says it all: Blue in South Bend. There are several of us,
myself included, who reside behind enemy lines, and are forced to suffer for our fandom, our religious devotion to the block M, and yes, for our wardrobe. Here in Pennsylvania it’s not so bad—at least, not until I break out anything with an #86 on it, which tends to work the more diehard Nits fan up into a seething hatred fairly quickly. (“One second,” or “Lloyd always got what we wanted from the refs,” etc.) Certainly South Bend has their own opinion of both Michigan and its fans, and thanks to the modern miracle of technology known as MGoProfile, we no longer have to guess. I encourage all of you Michigan Men in unfriendly territory to stay the course, hold the line, and wear the block M cap every damn time you leave the house. Thanks again and I’ll see you guys next week for another edition of MGoProfile!
Michigan baseball received, to my knowledge, their second commitment from the freshman class of 2011-12 in Fairfield, OH centerfielder Will Drake. Drake picked Michigan over Cincinnati and UNC-Asheville.
Drake, a 6-foot-1, 165-pound speedy center fielder, verbally committed to the University of Michigan on Monday, June 28, a day after he visited the campus […]
“When I got to Michigan it was a no-brainer for me,” Drake said. “They’re the fourth winningest program in college baseball history. I was surrounded by great coaches and a great facility. I really loved it up there.”
Drake called Michigan coach Rich Maloney on Monday and accepted a 50 percent scholarship offer. […]
Drake was a first-team, All-Greater Miami Conference selection as a junior after hitting .379 with a home run and 23 RBIs. He also had 13 stolen bases.
Drake is another speed outfielder, and may be looking to take over for current Michigan centerfielder Patrick Biondi if Biondi is a 3-and-done, a reasonable assumption based on his freshman year.
Drake is the second commitment in this class that I've seen, and he's also the second outfielder of the class with Zach Fish. Michigan is VERY light in outfielders right now, with only Biondi being a true outfielder returning from last year's team. The remaining two outfield slots next year are up for grabs between Kevin Krantz (former short stop), Garrett Stephens (former first baseman and likely DH/1B next season), Tyler Mills (pitcher), and two true freshmen in Adam Robinson and Michael O'Neill.
Drake will add immediate depth his freshman year, but at this point, I'm not sure he'll be a contributor until at least his red shirt freshman season. His overall numbers appear behind Zach Fish, but Drake still has time to grow and develop.
(HT: TomVH for the tip)
The weekly update is a little slim today, with two commits dropping last week, and the holiday weekend. Here's the latest on this week's happenings.
6'4, 285 lbs.
Cyrus is a big offensive line prospect with major offers to his name. Hobbi currently holds around 17 offers including Michigan, Alabama, ASU, Nebraska, Oklahoma, and USC. As we've found out with other prospects in Arizona, it's sometimes hard to get them out for a visit, especially when it's an unofficial on their own dime. Well, the Hobbi family just happens to be taking a cross country trip to New York, and they will be stopping by Ann Arbor on the way:
We're coming up next week, on Tuesday (July 13th). We're just coming for the day on our way to New York. I don't know a lot about Michigan, so this visit will help me decide if I'm really interested, or not.
That seems fair. Cyrus told me he played against Taylor Lewan and Craig Roh as a sophomore, but doesn't know them well enough to call them up and talk about Michigan. Craig actually tried his patented spin move on Hobbi and was shut down. He's also planning on stopping by Notre Dame. Those plans have changed:
I'm not going to Notre Dame anymore, they haven't returned any of my calls. I guess they filled up, but they won't call me back, either. I'll just enjoy myself at Michigan instead.
Take that! This visit is big for Michigan, who otherwise probably wouldn't have been able to get him on campus before the season. This will give him and his family a chance to see everything without a group of other recruits there, without any distractions.
6'6", 315 lbs.
Tony Posada recently named Michigan his leader after his visit up to Ann Arbor. That still holds true, and it may just be a matter of time before he ends his recruitment:
We're working out a date with my family and coaches to make a decision. I'd like to do it as soon as possible, it could be next week, it could be longer; we're not sure yet. Michigan is still my leader, though.
So... ya know, there's that. It seems like Michigan should be getting the call soon. I hate making predictions off of information that seems obvious, but an upcoming decision with a declared leader is almost always a decision that's already been made privately.
5'11", 184 lbs.
I spoke with Walls recently about his interest in Michigan. He played it close to the vest, but let out a little insight on how this will play out.
Michigan is a team that I am considering very highly. I will be paying close attention to the beginning of their season.
As we've seen with a number of other recruits. If Michigan wins early, they'll get real shot him. If they get off to a slow start, then it will be an uphill battle. Those first six or seven games are going to be CRUCIAL to Michigan's recruiting efforts.
- PA DB Kyshoen Jarrett will have his narrowed down list today, once he clears it with his coach. He told me that Michigan was on it, and he's very interested.
- Instate OL Jake Fisher says he wants to make his decision in the next few weeks. It's between Michigan and MSU at this point. I think Michigan has the slight edge. He does plan on going back up to MSU soon. and things can change quickly, but we look to be in the driver seat.
- PA safety Dondi Kirby tore his ACL, and will be out for the season. It's an unfortunate time to have that happen, as a football recruit. Not that there's a good time for it to happen, but you know what I mean.
Inspired by all of the great statisticatin’ done by such MGoUsers as Misopogon, the Mathlete, and, most recently (and in the past), MCalibur, I decided to look into something I’ve been wondering about for a while. Well, four somethings really, all related to the importance of yards on first down in determining eventual success at getting first downs and sustaining drives:
How much do varying numbers of yards on first down affect the probability of getting a first down (or a touchdown) in that series?
Here I was simply wondering about the “how much?” question. It goes without saying that losing 10 yards on first down (or starting first and 20) reduces the probability that you will get a first down, but by how much? Similarly, obviously the more yards you get on first down, the better your chances are of getting a first down on the series, but by how much? Are there thresholds beyond which your probability of getting a first down increases appreciably, or is it more or less a linear relationship, where every additional yard on first down increases your probability of getting a first down by the same amount?
How much variation is there between teams in their ability to recover from bad first down plays?
My assumption here was that with principally running teams like the RR-era WVU, it is harder to overcome a bad first down play than with more balanced teams like the Lloyd-era UM or the Vest’s OSU teams. Conversely, I assumed that when the RR-era WVU or UM teams got at least four yards on first down, a first down on the series was virtually a lead pipe cinch. But, as these were only assumptions, I was interested in doing some analysis to check this out.
How much variation is there across games in the ease with which first downs are gotten, and in the effects of various numbers of yards on the probability of getting a first down?
For example, you would think that statistically it would be easier to get a first down on any given series in a home game, other things equal, right? Or, it would be harder to get a first down on any given series in a game against an opponent with a better defense, right?
Based on my data, the answer to both of these questions is NSFMF. I’ll explain later.
How much do things other than the focus of the analysis, like field position and penalties, affect first down probabilities?
When I started, I knew I wanted to compare Lloyd-UM with RR-UM and RR-WVU, since I wanted to see how the spread n’ shred in its mature form would compare with the more anemic version (UM 2008-2009) and with the DeBordian “rock, rock, rock, rock, rock, ICBM, rock, rock, rock (also rock)” approach. In compiling the sample, I made several choices:
- I did not look at UM in 2008 because I thought it would unfairly penalize Michigan and/or RR, and also I’m pretty sure we invaded Grenada that year and they called off the season.
- I added another comparison team, the 2006-2009 OSU juggernaut. Damn, those guys won a lot of games in those years. Fuckers…
- I omitted all Baby Seal U games (e.g., OSU vs. Youngstown State, UM vs. Delaware State, and WVU vs. Eastern Washington) except
- which I included. I debated about this latter non-omission because I didn’t want to unfairly stack the deck against Lloyd, but I figured (1) omitting Baby Seal U from the other coaches actually (slightly) stacked the deck in favor of Lloyd, and (2) there are Baby Seal Us and then there are Appy State Us.
This is a picture of an actual baby seal.
As for the unit of analysis (or “record” or “case,” depending on your disciplinary background), you may or may not know that ESPN.com publishes the play by play for each game, with pretty detailed information on each play.
At the game level, the sample consists of 122 games played from 2005 to 2009 by three schools (OSU, WVU, UM) and three coaches (Tressel, Rodriguez, Carr). For each game, I recorded:
- the game number in the season (i.e., first, second, …, thirteenth);
- the opponent’s total defense ranking (from NCAA.org); and
- whether it was a home game or not (away- and neutral-field games were coded the same. In retrospect I probably should have distinguished between these, but it didn’t end up mattering anyway).
At the play/series level, the sample consists of 3,529 first down plays and the series these plays began. For the teams of interest (i.e., not the opponents), I recorded the following data for each first down play:
The dependent variable was whether the series ended in a first down.
The primary independent variable was the number of yards gained on first down.
The control variables were:
- the field position on first down;
- the yards to go on first down;
- whether there was an offensive or defensive penalty (or both) on the series (penalties on first down, where first down was repeated, figured into the “yards to go” variable);
- whether there was a turnover on the series;
- whether there was low time (less than a minute) in the second or fourth quarters;
- whether there was a pass or run on first down;
- the quarter the series took place in; and
- the number of previous first downs for the drive in which the first down took place (so, if it is the first first down play in a drive, this variable would be scored 0; if a team makes a first down, this variable would be scored 1 for the second first down play in the same drive).
Table 1 below shows the sample by season and team.
Hierarchical Linear Models
My initial plan was to run two-level hierarchical linear models (HLM), in which first-down plays/series are nested within games. Briefly, HLM allows you to calculate how much of the variation in the dependent variable is due to level-1 (play/series-level) factors like yards on first down, field position, etc., and how much is due to level-2 (game-level) factors like opponent defensive strength, home/away game, etc.
Essentially, HLM would calculate the average probability of getting a first down, as well as the effect of the level-one independent variables on that probability, for each of the 122 games, and then those parameters would be the dependent variables to be predicted as a function of level-2 (game-level) variables.
Fortunately for those of you who are about to stop reading, one of the things I discovered is that there is not significant variation from game to game either in the probability of getting a first down, nor in the effects of the level-1 independent variables, to support an HLM analysis.
This does not mean that, for example, UM had exactly the same average success in getting a first down against OSU as they did against Eastern Michigan. What it does mean is that there is not so much variation from game to game in this average probability that it makes sense to predict that scant amount of variation with game-level factors.
The Probit Binary Response Model
Hence, the following is just a play/series-level analysis, which is probably more intuitive for the reader anyway. Because the dependent variable is dichotomous (0 if no first down on the series, 1 if first down or touchdown), I used the probit binary response model (PBRM). For those of you not steeped in this method, the PBRM is one of several regression-like methods for binary dependent variables.
Probit coefficients are in the metric of the standard normal cumulative distribution function (CDF), also known as z-scores. When you evaluate the standard normal CDF at a given value, it tells you the probability of scoring a “1” on the dependent variable.
The sign and magnitude of probit coefficients are interpreted in the standard way: a negative effect means that the variable lowers the probability of scoring a “1” on the dependent variable, positive coefficients mean that the variable increases the probability, and larger coefficients (in absolute value terms) mean stronger effects.
Except for Table 3 below, I have transformed all coefficients into probabilities, so you don’t have to worry about the metric of the coefficients.
Several Words on Sampling Error
You may remember from some statistics course that it is generally good practice to report not just the point estimates from any statistical analysis, but also an estimate of sampling error. This is why when networks report polling data, they usually say something like “Candidate X is leading Candidate Y by 5 points [the point estimate], with a margin of error plus or minus 3 points [the sampling error estimate].”
Virtually all statistical software packages (I used Stata/SE 10) assume that the data were gathered via a simple random sample, in which all samples of a given size have an equal probability of selection. Clearly, my choice to non-randomly sample three teams and five seasons, and then take a census of all games (except for Baby Seal U games) and first down plays violates this assumption. Hence, this analysis isn’t necessarily representative of the nation-wide effects of first down yards (and other variables) on first-down probabilities. You should interpret all of these findings as merely relating to UM, OSU, and WVU for the years specified.
Figures 1 and 2 below show, respectively, the number of yards gained on first down and the starting field position for any particular series. Recall that there can be multiple series within a drive, so Figure 2 should not be interpreted as the starting field position for the drive.
Note from Figure 1 that the modal number of yards gained on first down is zero. Obviously, this can occur via an incomplete pass, a completed pass for no gain, or a rush for no gain. The distribution is right-skewed, although fairly normally distributed (excluding the zero yards bar) within a range of about a loss of 10 yards and a gain of about 20 yards.
Note from Figure 2 that the modal starting field position is 80 yards from the opponent’s goal line (or the offensive team’s 20). This is largely due to touchbacks on punts or kickoffs, of course.
Table 2 below shows the descriptive statistics by team for the variables used in the analysis. Note that the percentage of first down plays where the series ended in a first down or touchdown ranges from 66% for the 2009 UM team to about 76% for the 2006-2007 WVU teams. This should explain in part the 5-7 record of the former team and the shredding of opponents achieved by the mature WVU teams. Interestingly, OSU and Lloyd-era UM had about the same overall probability of getting a first down.
Time will tell if the RR UM teams can recapture that glory, or whether the spread n’ shred was simply more effective (1) in the Big East, (2) with Pat White/Steve Slaton, or (3) both (1) and (2).
One bit of hopeful evidence comes from the opponent total defense rank (near the bottom of Table 2). It doesn’t appear as though WVU played an appreciably easier average schedule than OSU, and if anything, WVU’s opponents finished their seasons with, on average, better-ranked defenses than either Lloyd-era or RR-era UM.
In terms of the primary independent variable of interest, Figure 3 shows the distribution of yards gained on first down, by team. Note that RR-UM was more likely than the other teams to lose from 1 to 4 yards on first down, less likely to gain from 3 to 5 yards, more likely to gain 6 or 7 yards (there may be a small sample size problem here), and less likely to hit a big play on first down (10 or more yards) than OSU or WVU.
Interestingly, RR’s WVU teams were less likely to gain 0 to 2 yards on first down, which is probably largely due to the lower percentage of passing plays on first down for WVU (17% vs. about 32-34% for the other three teams. This should demonstrate that RR/Magee understand that when you have Pat White, you run the ball on first down (and most downs thereafter). When you have Tate, you have to be more balanced. Say, maybe these guys do know about football…
Other points of interest from Table 2:
- Lloyd’s teams were more disciplined on offense with respect to penalties than the Vest’s teams--about 4.7% of OSU’s series had at least one post-first down offensive penalty (recall that the first down penalties were folded into the “yards to go” variable), compared to 2.8% for Lloyd-UM. RR’s teams fall in between.
- On the other hand, the Vest’s teams drew more post-first down defensive penalties than RR’s teams. Perhaps the passing attack invites more encroachment/pass interference calls than a more ground-based attack?
- Turnovers! About 7.6% of RR-UM’s series ended in turnovers, compared to 4.0 to 4.7% for the other teams. Yikes.
Figures 4-6 show some results from the regression analysis. First, Figure 4 shows the probability of getting a first down after selected numbers of yards on each first down play, assuming (1) it was first and 10, and (2) there was no penalty on the series.
Note that losing five or more yards on first down gives you about a 0.25-0.30 probability of getting a first down, whereas, obviously, gaining 10 or more yards is by definition a first down (on first and 10 at least).
In between these extremes, the first down returns to yards on first down is basically linear, though there are fairly noticeably inflection points between losing 5 or more and losing 1 to 4 yards (the first two points in the curves) and between gaining 3 to 5 and gaining 6 or 7 yards. By the way, I chose these categories based on exploratory analyses that showed that there was no statistically significant difference between gaining, say, 0, 1, or 2 yards.
Finally, notice the similarity between the OSU and Lloyd-UM curves. This shouldn’t be particularly surprising, since those teams pursued fairly similar offensive strategies--lots of off tackle to Hart/Wells interspersed with daggers to Manningham/Ginn.
I was interested to see that WVU dominated the story, at all categories of yards gained on first down. That is, it isn’t true that the WVU offense bogged down especially on small losses or gains on first down. A great offense will overcome.
Figure 5 shows the probability of getting a first down by field position on first down, in 10-yard increments. There are basically four points here:
- Being inside your own 20 reduces your probability of getting a first down, probably because of more conservative play calling;
- There is basically no difference between the 20 and the 50;
- Probabilities go up between the 50 and field goal range (a field goal attempt was coded 0 on the dependent variable, since there was no first down or touchdown);
- The probability goes way down in field goal range, probably because coaches elect to take the 3 points instead of going for it on 4th (see the Mathlete’s excellent diary on this).
Figure 6 shows basically the same trends, broken down by teams. There isn’t much to see here, except that WVU was awesome, RR-UM sucked, and OSU/Lloyd-UM were basically indistinguishable. It looks like a good rule of thumb is that WVU had a 10-percentage point better probability of getting a first down than RR-UM and a 5-percentage point advantage over the Vest and Lloyd.
Table 3 shows the full regression results. There isn’t much new here, but just to recap:
- Yards on first down matters a lot (duh I);
- WVU kicked ass;
- It’s harder to get a first down on first and 20 than first and 5 (duh II);
- Field position doesn’t matter as much as you might think;
- Offensive penalties make it harder to get a first down; defensive penalties make it easier (duh III); and
- Ceteris paribus, passing on first down increases the probability of getting a first down on that series (though in analysis not shown here, I found that, not surprisingly, it increases the chances of a turnover [see Hayes, W.]).
One other thing: in the note to Table 3, it says that the “Pseudo R2” is .3008. This is a statistic calculated in the PBRM that is analogous to the R2 (r-squared) statistic in linear regression, which is interpreted as the percentage of the variation in the dependent variable that is explained by the model. It’s hard to say whether 30% is a lot or a little; all I know from the coding is that there were lots of series in which a team would lose 10 on first down and still get a first down, and others where they would gain 9 on first down and fail to get a first down. So, there is still a large stochastic component to the process.
Stuff You’d Think Might Matter but Didn’t, Statistically
Statistically, variables that had no significant (but see “Several Words on Sampling Error” above) effect on the probability of getting a first down (net of the other variables included in the model shown in Table 3) included:
- Home vs. not home game;
- Which game of the season it was;
- The quarter of the game;
- The drive number (these last two suggest that there is not a robust effect of either “bursting out of the gate,” nor of “starting sluggishly.” Sometimes teams start strong and finish weak, other times the reverse happens);
- Number of previous first downs on a drive. This was interesting to me, because one often thinks, I think, that teams get “hot” on a drive. In other words, each first down makes it successively easier to get the next first down. My analysis suggests this is not true, at least in these data. There are a couple of explanations for this: one is that it does get slightly easier to get a first down the closer you get to your opponent’s goal line (though not in the field goal zone), so the two effects are collinear--the more first downs you get on a drive, the better your field position is, and it is that latter issue that affects first down probabilities. The second goes back to the stochastic component--there are just as many drives where a team will gain 3 first downs and then stall as ones where they will gain 3 first downs and then 2 more.
I have few beyond the things I’ve already mentioned. Basically, yards on first down are incredibly important, but not in any surprising way. The more yards you get, the better your chances are of getting a first down. However, there is a large random component to getting first downs, so yards aren’t everything.
In terms of UM football, it is clear that the mature spread n’ shred is lethal. But you already knew that. The question is whether UM can recapture that WVU magic. I guess I’m optimistic, for several reasons:
- The RR offense requires experienced, athletic players, really at all offensive positions. This we now have, and/or are quickly cultivating.
- A heavily run-based offense is slightly less likely to turn the ball over and much less likely to suffer no gain on first down (due to the lack of incomplete passes). This bodes well for sustained drives.
- WVU played, on average, slightly better defenses (at least if you think total defense rank at the end of the season is a good indicator of defensive strength) than UM on average, and defenses that were as good as those played by OSU, on average. So, at least by this figuring, there is no reason to think that UM’s current schedule is too good for us to be successful.
Obviously, the $1M unanswered question is whether the RR offense will be as successful at UM as it was at WVU. The analysis I have done can’t really speak to this question, but neither does it suggest obvious reasons why it won’t be successful. It does show how powerful the WVU version was, and I for one support giving RR enough time to have a reasonable chance to put that offense into place.
Comments, suggestions, critiques? Let's have ‘em.
In the loosely adapted ways of Dante, I present to you the second canto of Formerly's Football Inferno. I promise nothing when it comes to grammar, punctuation, logical plots, or anything that normally goes into story writing.
For those of you unfamiliar, Dante walks through each region of hell to learn the sins and punishment by talking to those souls trapped. In the second circle of Dante's hell, home to those committing Lust, the souls that blurred the line between love and sexual desire are tortured until their desires are purged.
Time flies by strangely here. For what seems like endless hours of walking, there is no sun to point my direction. When I look up to the sky, the only thing you can make out is a godzillatron reaching endlessly with small screens of Michigan updates being played.
There is a problem with these updates. It appears those damned to hell can't see them or make them out. Only I, the living, can see them. As we walk, I can see that Rich Rodriguez has signed a top ten recruiting class in his first year. That's welcome news. There's some crazy looking kids as I peak up at the big screen to see that varsityblue.blogspot.com is highlighting the class. It's a definitely a different air coming into Michigan, and that somehow comforts me as I cross down the cliff and trek across the plain of the damned in the second circle of hell.
As we hike forward, the weather becomes torrential. Great wind storms, like infinite tornadoes plague this level of hell. As we seek shelter in a path that runs through a ditch, I notice souls being thrown around in the wind, being held restless.
"Who are these people, tormented by the endless storm?" I asked Crockett.
He replied, "These are the pundits that either set aside their fandom in order to be objective journalists. These few men, and a couple women, mostly covered other teams, but some of them even covered Michigan. Like that guy there, that's Grantland Rice."
"Wow, he's one of the most quoted sportswriters of all time!" I exclaim.
"Sure is. I remember reading him as a ki… I mean, I remember hearing about his writing long after I had died."
"So you are John W-" I started.
Crockett cut me off, proclaiming, "Watch your tongue kiddo or I'll up and leave you here!"
"Yes, sir," I unleash remorsefully. What's with this guy?
"Anyway," Crockett says, "you want to talk with Rice? As a mortal, you have the ability to call on souls here, as long as you don't distract too many."
"Mr. Rice!" I call as his soul fades slowly out of the wind, as if all of a sudden it's not affected by the howling winds at all.
"Good day to you, sir. I'm Grantland Rice. I was once a great sportswriter covering the Four Horseman of Notre Dame. Now I am but a simple soul, ravaged by famine, pestilence, destruction, and this savage wind. I'm stuck in the blue-gray October sky, in a crest of the cyclone before the second circle of hell, peering down upon the bewildered panorama spread out upon the green plain below."
"That sounds vaguely familiar. Did you just rip off yourself?"
"Yes, I am he who penned the story of the four horsemen of Notre Dame."
"I thought so," I say. "So here's a question for you, why was Michigan the team that you never seemed to cover? It was always Notre Dame, Babe Ruth, Red Grange, Jack Dempsey. It seemed like you never covered Fielding Yost and the point a minute offenses."
"You see, I was one of a dying breed. Despite my lifting of athletes up to the level of demigods, I always wanted to be objective in my poetic tales. It just so happened that I was snatched up by Notre Dame during their glory filled years. You know, the ones they keep claiming are bound to echo back. Ha, unlikely."
"Yeah, ha. I've noticed that most of the spirits down here don't know what's happened recently in the mortal world. Did you hear about their yakety-sax 3-9 season? What a bunch of losers!"
Rice let out a sigh and responded, "No, I hadn't heard, but that doesn't surprise me either. I heard that Tyrone Willingham guy really did a number on them though. Or that's at least what I heard when Holth entered hell a few years back."
The intentional inclusion of Holtz's lisp made me laugh. "But Holtz isn't dead?" I ask. "I saw him on game day before I left."
"Souls can leave the earth before the body dies. It's not uncommon. The body is left without a soul to drift until it expires."
"That explains a lot, actually."
"Indeed it does… Alright, well, I feel I must return to punishment. I have been here with you long enough. May you have safe travels, and may you overcome the despair that you have not even seen yet."
Rice slowly backed away out of the ditch, and before I could even question the cryptic fortune, he was whisked away in the fury of the wind. Davy looked at me solemnly, with a look that I knew meant don't lust after the answer. So I didn't, and we continued on our way.