things go poorly
Rerum omnium magister usus.
Experience is the teacher of all things.
Julius Caesar, Commentarii de Bello Civili, Book 2, Chapter 8
Setting. As the 2013 football season rolls on, the problems in Michigan’s run game have become more and more glaring. This has led to much ballyhooing and debate as to the main causes of Michigan’s ground game woes. The most basic argument is whether our problems are caused by weakness on the line or at the running back position. Brian’s UFRs come into play here, and while Fitz and the gang haven’t been perfect by any means, the play-by-play breakdowns seem to suggest that the problem lies with our offensive line. A phalanx our line is not.
Identifying the line as the problem, however, has not really made anyone very content. Rather it’s sparked a debate between the baby blamers – those who see Michigan’s youth as the source of their problems – and the crappy coach contingent – those who find fault with our coaches development of our o-line talent, not to mention play calling.
Previous Work. The MGoCommunity has already produced some solid work on this topic. The Mathlete’s preseason study looked at other teams who had offensive lines with an 1st round NFL pick combined with 2+ freshmen. Although he eventually admitted that we were still probably a year away, his comparison grouped us with the likes of Alabama, Oregon, and Stanford. GuloGulo’s diary from back in September looked at the relationship between average o-line experience in the Big 10 and success mainly defined as yards per play. After the first four weeks of the season, he concluded that we were about average in both experience and success. Gameboy’s recent diaries have shown that Michigan’s line is relatively young whether you take a “number of years in the program” or “number of previous starts” approach.
Questions. In this study I want to delve a little deeper into what we mean by “experience,” what we mean by “success,” and how those two variables are actually related. I will attempt to answer four questions:
- Can offensive line experience explain run game success?
- Are years or starts a better measure of experience?
- Is interior line experience more important that tackle experience?
- Is average experience a better measure than the weakest link?
Definitions. Neither experience nor success have single and obvious definitions. With regard to the o-line, success could be defined by wins, yards per play, yards per rush, sack percentage, or play-by-play results a la UFR. For the first part of this study, I use yards per carry as my metric for success. Experience, likewise, can be defined in a variety of ways, including the number of years in the program, the number of starts, or the number of snaps. This analysis primarily uses the number of years in the program as its measure for experience. This isn’t because it’s necessarily the best measure – we’ll test that in a bit – but rather because it’s the measure that’s easiest to find information about. Redoing this study with a start- or snap-focused measure of experience would be a worthwhile endeavor. In the graphs below, the year of the players are equated with numbers, so that freshman = 1, red shirt freshman = 1.5, sophomore = 2, red shirt sophomore = 2.5, and so on.
Data. The data for this study are drawn from this 2013 season. All 125 FBS teams are included. The YPC stats come from ESPN and the experience info comes from the a scouting site. Because this isn’t necessarily about Michigan’s o-line this year, but rather about the general relationship between offensive line experience and success, data from any recent season should apply though. Giving this thing some time depth would probably improve its efficacy. The stats are current as of 11/6/2013. All the images are from the MGoBlog flickr account; Bryan Fuller gets the credit, I believe.
This is a primarily quantitative study, but I’m in no way a statistician. My background is in Classics, as in Greek and Roman studies, so although I’ve tried brush up on my stats, there’s certainly the possibility that these metrics aren’t employed or interpreted perfectly. Feel free to correct me.
With that said, it’s probably useful to give a brief overview of the statistical measures in an attempt to describe what they actually tell us. I’m looking at 4 main metrics: correlation coefficient, r-squared, p-value, and slope of the linear trend line.
Correlation coefficient: The correlation coefficient quantifies the degree of linear relation between two variables. The coefficient ranges from -1.0 to 1.0, and the larger the absolute value of the coefficient, the stronger the relationship. This will provide a single number for the strength of the relationship between o-line experience and yards per carry.
R-squared: The r-squared provides a measure for the amount of variance in one variable that can be explained by another variable. This will be used to assess how much of the variance in yards per carry can be explained by o-line experience. It’s important to note here that there are obviously many other factors other than experience that govern running game efficiency (coaching, scheme, running back skill, etc.). A low r-squared doesn’t necessarily mean that experience is unimportant, just that other factors are also important.
P-value: The p-value let's us know whether our results are statistically significant; more specifically it provides a measure to assess whether we should discard the null hypothesis. In this case, the null hypothesis is that there is no relationship between o-line experience and running game success. The p-value ranges from 0 to 1. A small p-value, < 0.05, suggests that we reject the null hypothesis, while a large p-value suggests we retain it. If p-values are low, we should have faith in the relationship between experience and success; if they are high we should feel less confident about that relationship.
Slope of linear trend line: The trend lines in the graphs below show the linear relationship between experience and success. The slopes of that lines indicate the extent to which we’d expect YPC to change as a result of a change in experience. For example, if the slope was 0.5, the data would suggest that an extra year of average o-line experience is worth ½ of a yard per carry.
Question 1: Can offensive line experience explain run game success?
Good habits formed during youth make all the difference. - Aristotle
The scatter plot below depicts the relationship between average offensive line experience in years and yards per carry. Click for enlarged scatterplot with all BCS teams labeled.
The data broadly confirm what we’d expect. This is good! This means that we’re right in claiming our youth is (partially) the problem. The older your offensive line is, on average, the more yards per carry that team produces. The correlational coefficient is 0.16 for this data set, indicating that there’s a slightly positive correlation between offensive line experience and yards per carry. The r-squared is small, however, suggesting that only about 3% of the variation in teams’ yards per carry can be attributed to the experience of the offensive line. A p-value of 0.07 is marginal, meaning that it’s not particularly clear whether we should interpret these results as significant or not. Let’s start by giving experience the benefit of the doubt though, and for the time we can conclude that experience does indeed influence ground game success. The slope of the linear trend line suggests that an extra year of average experience is worth about 1/3 of a yard per carry.
At first glance, there does seem to be a positive correlation between o-line experience and YPC, although there is still a lot of variance in YPC that cannot be explained by experience.
Question 2: Are years or starts a better measure of experience?
One of the arguments against the approach taken in question one is that an offensive lineman’s number of previous starts is a better measure of experience than the number of years he’s been in the program. Let’s take a look; the graph below plots this alternate measure of experience against yards per carry. Click to enlarge and see Oregon and Wisconsin put up 6.7 YPC despite having less total starts along the o-line than Michigan.
The relationship between the number of previous OL starts and yards per carry generally mirrors the pattern produced when the number of years in the program is taken into consideration. The correlation coefficient is actually slightly higher (0.23 compared to 0.16), suggesting that starts is indeed a slightly better measure than years in the program for the purpose of predicting o-line success. The r-squared suggests that previous starts can explain about 5% of the variance in yards per carry, and a p-value of 0.01 indicates that these results are indeed significant. The slope of the line suggests that each extra start is worth about 1/100 of a yard per carry, meaning that 50 extra stars is worth about ½ a yard, and 100 extra starts is worth about a full extra yard per carry.
Now that the data show that “number of starts” is probably a better measure of offensive line success, I’m going to revert to “number of years in the program” as my main metric of experience. This is simply due to the convenience of the data. If someone can get number of starts for all the programs, that should improve things. Perhaps another day.
Question 3: Is interior line experience more important than tackle experience?
Why doesn't Lewan make everything okay?
One of the most common arguments against using the average or total experience of the entire offensive line is that all spots along the line are not created equal. Lewan being an awesome LT doesn’t help our RG Mags getting crushed by the NT. Essentially, interior line experience is more important than tackle experience. But does it really matter whether your experience comes on the interior or exterior of the line?
Let’s start with tackle experience first. The graph below shows the relationship between the average experience of each team’s tackles and their YPC.
Check out Michigan and Purdue with their bookend fifth year senior tackles. This doesn’t bode well for a positive relationship. Looking across the entire spectrum of the FBS, there appears to be no correlation between the experience of a team’s tackles and their ability to run the ball successfully. Once again, this is good news for us. It’s not that we’re not taking advantage of our great tackles, it appears that on the whole, tackle experience just doesn’t influence ground game success all that much. The correlation coefficient is a measly 0.02, the r-squared is <0.01, and the p-value is 0.81, which is incredibly high. The slope of the trend line suggests a very, very slight decrease in YPC as tackles increase in age, which doesn't make any sense at all.
This is really interesting actually, as all metrics suggest there is essentially no connection between tackle experience and yards per carry. If tackles aren’t the cause of the correlation between total experience and YPC, then it must be the interior of the line, right? Click to enlarge and see us at the children's table with UCLA and Purdue.
It appears as though the “our interior line is full of infants” excuse is actually a pretty good one. With a correlation coefficient of 0.22, the relationship between these two variables is stronger than when offensive line experience as a whole is averaged (in years) and an order of magnitude stronger than the correlation between tackle experience and YPC. The r-squared indicates about 5% of YPC variation can be explained by experience along the interior of the line, and a p-value of 0.01 suggests these results are significant. The slope of the trend line suggests an extra year, on average, is worth about 1/3 of a yard per carry.
If you extrapolate that out over the course of a season, that’s about 150-200 extra yards of rushing per year (Michigan had 502 rushing attempts in 2012 according to ESPN). Interior line experience does seem to be a big deal. Also, we’re one of the 3 youngest teams out of 125 FBS teams in terms of interior line experience. That is young indeed.
Question 4: Is average experience a better measure than the weakest link?
The foundation of every state is the education of its youth. - Diogenes
Thus far the data have shown that interior line experience is a better predictor of running game success than total offensive line experience. The next question is whether average interior line experience is a better predictor of success than the “weakest link” along the line. In this case we’ll call the youngest person on the interior of the line the weakest link. This really has nothing to do with their ability, it’s just a measure of their experience in the program. Click to enlarge and see Auburn averaging 6+ YPC while starting a true freshman interior lineman.
It looks as though there is something to the “weakest link” argument. The correlation coefficient in this case is 0.29, which stands as our strongest correlation yet between some measure of experience and yards per carry. The r-squared indicates that this measure can explain about 8% of YPC variation, and a p-value of 0.01 suggests that these results are indeed significant. The slope here once again suggests that an extra year is worth about 1/3 of a yard per carry.
The fact that the age of a team's youngest interior offensive lineman is a better predictor of run game success than its average offensive line experience, or even the average experience of just the interior line, is rather unexpected. This should bode well for Michigan's future along the line as we gain experience and depth in future seasons.
First off, offensive line experience leaves a lot of the variance in yards per carry unexplained. So even though this study supports the conclusion that offensive line experience does indeed influence success in the running game, there are clearly many other factors that also play a role.
In this study, experience has been measured in in two ways, both as “years in the program” and as “number of starts.” While both serve as decent predictors of success in the running game as judged by YPC, number of starts seems to be the better measure. Unfortunately, it’s also the measure that is more time-intensive to track. When looking at the outer vs. interior line, the data suggest that success on the ground is much more closely tied to the experience of the interior line than it is to either the tackles or even the average experience of the line as a whole. Surprisingly, tackle experience seems to be completely irrelevant as a predictor of run game success. Finally, the level of experience of the least-experienced person on the interior line serves as an even better metric for predicting running game efficiency. The “weakest link” argument appears to hold water.
Unit of Measurement
|R-Squared||P-Value||Effect on YPC|
|Total Experience||Years||0.16||0.03||0.07||Extra year = +1/3 yard|
|Total Experience||Starts||0.23||0.05||0.01||Extra 10 starts = +1/10 yard|
|Interior Line Experience||Years||0.22||0.05||0.01||Extra year = +1/3 yard|
|Youngest Interior Lineman||Years||0.29||0.08||0.01||
Extra year = +1/3 yard
What does this mean for Michigan? As Gameboy showed us in his diaries, Michigan is young along the O-Line, whether you’re judging by years in the program or by number of starts. What I hope to have demonstrated here is that (a) being young really does matter, and (b) we’re especially young where it matters most (i.e., tied for 2nd youngest on the interior OL out of 125 FBS teams).
Borges and Funk in happier times
There’s been a lot of heat on Borges and Funk recently, and it’s appropriate to ask whether this study indicts or absolves them. Unfortunately, I think the data tend to side step the question. The fact that o-line experience does seem to influence YPC, and especially the finding that interior line experience seems to be of utmost importance, combined with Michigan’s position with regard to these measures (i.e., they fall almost exactly along the linear trend line in both the interior line experience graph and in the weakest link graph), would initially suggest that the line is performing about as expected.
This doesn't let the staff off the hook. The relatively low r-squared values would indicate that there is a lot more than just experience that goes into producing a successful running attack. Coaching, both in terms of scheme and player development, is probably one of the most influential factors in governing run game success, and this study doesn’t attempt to measure or control for that aspect of the game. Moreover, this study doesn't account for talent along the offensive line, which would probably suggest Michigan is underperforming relative to the recruiting rankings. Strength of schedule is also omitted. Having played CMU, Akron, UConn, Indiana, and Nebraska, adding this variable could also raise our team's expected YPC, and in doing so lower our performance relative to expectations.
According to the eyeball test, the apparent regression along the offensive line would seem to indicate that there are some seriously problematic coaching issues. There are several BCS programs with similar youth-related issues on the interior line, both when experience is averaged (e.g., UCLA) and when experience is defined by the youngest interior lineman (e.g., Notre Dame, Arkansas, and Auburn), and these programs still manage to perform significantly better than us in terms of yards per carry. When viewed within the context of the entire FBS, however, the data suggest that Michigan’s youth is a real and influential issue.
On the bright side, this should give us hope for future seasons. As our interior line matures, both in terms of average experience and in terms of its weakest link, we should improve. This only holds, of course, if all the other factors that go into producing a successful offensive line – namely coaching scheme and player development – are on par with the rest of college football. That, unfortunately, is not guaranteed.
Rome wasn’t built in a day, but it did progressively grow bigger and better until it reached a point where it dominated at the point of attack. Let’s hope our offensive line can do the same.
Coming in part 2: Shouldn’t our veteran tackles at least make us better at pass protection?!?
Awww, come on.
 Actually, upon further review, I’m not so sure this is accurate. Over the course of 8 games, Fitz has 5 positive UFR games, 2 negative, and 1 around zero, while the OL has 5 positive UFR games, 2 negative, and 1 around zero. Obviously RB and OL numbers aren’t perfectly commensurate, but this probably suggests the blame should be shared.
Is it youth or coaching?
As Brian pointed out in the Minnesota game, we were very close to a few huge running plays. Yet, PSU was a tire fire. Part of it was losing Lewan, another part was the heavyside formation (which we ran PA out of to see AJ Williams get burned...).
EDIT: and Coaching is your winner!
While everyone is debating the job Borges is doing, why isn't anyone talking about the absolute tire fire that is our offensive line performance? The OSU game was as bad as anything I've seen all year.
Our guys aren't undersized anymore, and are "Wellman-ized," so they aren't weak. So those memes no longer hold true.
I rewatched all of the run plays again in slo-mo to see exactly why we couldn't get even a few meager critical yards when necessary. It jumped out at me time and time again.
Missed blocks, confused linemen linemen not even engaging anyone, while the ballcarrier behind them was getting tackled. The fullbacks were bad, too.
With all the mental errors, it looked like a really poorly coached unit.
So incredibly, my biggest fear going forward isn't Borges. It's the offensive line play. Charlie Weiss proved that even with superior talent, if the guys aren't properly coached up it won't matter how many blue chippers you've got.
I hope to God what we're seeing is a product of the coaching change. But as far as I'm concerned, Darrell Funk just joined Spain as officially On Notice with me!