"He's a hard worker, and he watched me and Tim (Hardaway Jr.) and Nik (Stauksas) put work in to become (first-round picks), and I'm just happy he's getting better," Burke said. "It's great for the program, too. It shows what type of program the University of Michigan is and the direction it continues to go in."
Setting the Scene
no animals were harmed in the making of this diary
Apology. I am sorry to all those who are cringing at seeing another experience diary. I had originally conceived of this as being a two part study, with the first looking at the running game and the second looking at the passing game. Despite the other diaries, both of which were useful in their own ways, I think there is still some horse meat to be gleaned from the carcass of o-line experience as it relates to pass protection. The horse might be dead, but that doesn’t mean it’s useless. Oh, and there's another one since I originally wrote this thing up.
Previous Work. In the first diary, I attempted to demonstrate that o-line experience does indeed play a role in governing a team’s ability to run the ball. R-squared values ranged between about 0.05 and 0.10 depending on how we defined “experience,” suggesting that about 5-10% of the variation in YPC across FBS can be explained by a team’s experience along the o-line. Clearly other factors are also important to running the ball well, but experience isn’t meaningless. Further manipulations to the data set suggested that interior line experience is more important than tackle experience and that the “your line is only as good as its weakest link” argument does hold water.
Some of the comments from that diary questioned why we don’t move one or both of our experienced and talented tackles to the interior if that is where it really seems to matter. Transaction costs of moving around linemen aside, the question is valid in general terms. Why not put your best linemen on the inside if those are the most important positions? A variety of answers could be given to this question – for example, exterior and interior line positions could have different ideal body types with regard to height, weight, strength, and agility – but the most obvious response is that tackles are more influential in the passing game. Our best linemen play at tackle because they protect our quarterback.
Questions. This is a proposition that can be tested statistically, and that’s what I aim to do in the second part of this study. My metric for o-line success in the passing game is sack percentage (i.e., the percentage of pass attempts on which your QB is sacked), since it’s the o-line’s job to keep the QB clean. Using the same essential methodology as the last study, I aim to answer four questions:
- Does o-line experience help prevent sacks?
- Is tackle experience or interior line experience more important in protecting the quarterback?
- Does the “weakest link” theorem hold for the passing game?
- What else could influence sack percentage?
Data. This study looks at 123 FBS teams (Georgia State and UTSA are omitted since their info wasn’t on ESPN). Sack percentage stats come from ESPN, and the experience data comes from the scouting site Ourlads. Star rankings that come at the end of this study are taken from Rivals. Photos come from MGoBlog's flickr account and are attributable to Bryan Fuller. Check out the previous diary for basic definitions of the statistics that are used.
This is long, so buckle up. Feel free to jump to the conclusions if you don’t want the nitty gritty. All the data are summarized in a…chart? Chart.
Questions and Answers
Probably cropped out: massive amounts of backside pressure
Question 1: Does o-line experience help prevent sacks?
Let’s start by taking a broad look using average experience in years of the offensive line. The relationship between experience and sack percentage is plotted below. Click on the graph to see the same sack percentage data plotted against total number of starts. The plot is oriented so that up is good (i.e., your QB isn't getting sacked that often) and down is bad (i.e., you're looking like Michigan against MSU and Nebraska).
Although the trend line makes it look as though there is an inverse relationship between sack percentage and experience (i.e., sacks go down as experience goes up), which is what we’d expect, the r-squared is relatively low (0.02) and the p-value (0.10) suggests the trend may not be statistically significant. If we plot the same relationship but use starts as our metric for experience, the relationship becomes even more spurious with an r-squared of 0.01 and a p-value of 0.28 (click on above graph to see scatter plot). On the whole, total or average o-line experience doesn’t seem to be a great predictor of the o-line’s ability to keep the quarterback from getting sacked.
Question 2: Is tackle experience or interior line experience more important in protecting the quarterback?
I don't care who matters more as long as we keep it under 7 sacks a game
We saw in the previous study that interior line experience was more important to run game success than tackle experience. We’d likely expect the opposite to be true for the passing game based on the premium put on left tackles both in college and in the pros. Average tackle experience and sack percentage is plotted below.
This is unexpected. The correlation is spurious. The r-squared is less than 0.01, and the p-value is 0.67, both of which suggest there is no correlation between tackle experience and sack percentage. The trend line actually rises slightly, which would indicate that sack percentage rises as tackles get more experienced, which makes no sense at all, even if the correlation was statistically significant. On the whole, tackle experience does not appear to be a good predictor of your team’s ability to not give up sacks.
Could the interior of the line be more important in the passing game as well? Click for enlarged scatterplot with teams divided by conference and all BCS teams labeled.
Now we’re getting somewhere. The trend line makes intuitive sense. The percentage of sacks you give up goes down as your interior linemen become more experienced. The r-squared is 0.05, implying interior line experience can explain about 5% of the variance in sack percentage. The p-value is 0.02 suggesting that results are statistically significant. The slope of the trend line suggests that an extra year of average interior line experience is worth a drop of almost 1 percent in sack percentage. If you extrapolate that over the course of a season, that’s about 3 to 5 fewer sacks. Not a huge difference, but if your team matures over the course of several years from starting freshman to starting seniors, that adds up to reducing sacks by about 1 per game.
One possible critique here is that “average” tackle experience is not the correct measure. Teams often put one of their better run blockers at right tackle and their best pass protector at left tackle. Thus instead of looking at the average, we should just look at the correlation between left tackle experience and sack percentage.
We don’t even need a graph here. We get almost the exact same trend as when the tackle experience data are averaged, and the slope of the trend line suggests that sack percentage slight increases as tackles get older. That is intuitively incorrect. A low r-squared value (<0.01) suggests left tackle experience doesn’t matter very much and a high p-value (0.31) implies statistical insignificance. This is admittedly somewhat baffling and definitely unexpected.
Question 3: Does the “weakest link” theorem hold for the passing game?
weakest link only in age, not awesomeness
When looking at the run game, the data suggested that the youngest member of the interior line was a better predictor of success than average experience of the interior line. In the passing game, the “weakest link” is a little less weak.
Unlike with the run game, average interior line experience appears to serve as a better predictor of sack percentage than does the “weakest link” along the interior of the line. The r-squared here is 0.02 and the p-value is 0.12, suggesting that the significance is marginal at best. It's not that the weakest link is a terrible predictor, just that the average experience of the entire interior line serves as a slightly better indicator of sack percentage.
At this point we can draw some basic conclusions from the first three questions. Total or average o-line experience only seems to be a marginal predictor of a team’s ability to keep their quarterback from getting sacked. Tackle experience, whether averaged or just taken as the left tackle, appears to have no relationship whatsoever with sack percentage. Just like with the run game, interior line experience seems to be the most salient characteristic with regard to o-line experience for predicting success in pass protection.
Question 4: What else could influence sack percentage?
One of the main critiques from the last study was that we’re living in a multivariate world and other potentially influential factors should be included in the analysis. I’m still working on getting myself up to speed regarding multivariate analysis, so I’m tentative to try and do too much with that now. We can, however, look to see how some other variables correlate with sack percentage.
Offensive Line Talent
4 star, 4 star, 5 star...sack?
Talent is one obvious potential factor in governing pass protection success. The chart below shows the correlation statistics for the star rankings of the o-line with regard to sack percentage. I’ve omitted the graphs here because none of these produce any correlation of statistical significance.
|Average O-Line Star Ranking||0.01||0.30||Low|
|Average Tackle Star Ranking||<0.01||0.52||Low|
|Average Interior Line Star Ranking||0.01||0.22||Low|
|Left Tackle Star Ranking||0.01||0.32||Low|
Surprisingly, star ranking of offensive linemen doesn’t seem to correlate very strongly with sack percentage. My guess is that this is due to star ranking of offensive linemen correlating closely with the difficulty of defense that a given team plays. For example, Alabama has talented linemen, but they play against tough defenses in the SEC. Toledo, on the other hand, has crappy linemen, but they play against week defenses in the MAC. Moreover, there tends to be very little variation in star rankings with non-BCS schools – almost everyone is a 2 star – so this may be obscuring some of the impact that talent (i.e., star rating) has on pass protection.
This can be accounted for, to some extent, by looking at only the BCS schools, where there will be more variation among offensive line star ratings and more consistency in the level of teams played. The chart below shows the correlation statistics for offensive line talent and pass protection success.
click to zoom with all BCS teams labeled
|Average O-Line Star Ranking||0.03||0.18||Low|
|Average Tackle Star Ranking||<0.01||0.64||Low|
|Average Interior Line Star Ranking||0.05||0.07||Marginal|
|Left Tackle Star Ranking||<0.01||0.87||Low|
Once again, it appears as though the interior of the line is the most crucial for preventing sacks. This corresponds well with the experience data presented in the first three questions. If we’re operating under the hypothesis that the interior of the line is more important than the tackles with regard to pass protection, which the experience data suggest, then we’d expect talent to matter more on the interior than it does at the tackles as well. It turns out that this is exactly the result we get. Whether looking at experience or talent, the interior seems to be the key to success.
I don’t know much about multivariate regression, but when you take both experience and talent of the interior of the offensive line into account for predicting sack percentage, an r-squared of 0.09 is produced. This is almost double the r-squareds produced by regressing experience and sack percentage and talent and sack percentage, and it suggests that these two factors work in tandem to determine the success of the offensive line regarding pass protection.
Unleashing the Dragon
A team’s tendency to throw deep, thus necessitating a longer drop and more time in the pocket, could be another influential factor governing sack percentage. I thought that yards per completion would be the best measure of a team’s tendency to throw deep, since yards per attempt could be equally as high for teams that throw quick, short passes, but complete a high percentage of them. Either way, we can look at both metrics.
click to see scatterplot of ypa and sack percentage
|Passing Depth Metric||R-Squared||P-Value||Significance?|
|Yards per completion||0.01||0.31||Low|
|Yards per attempt||0.01||0.20||Low|
These measures do not correlate particularly well with sack percentage. Yards per completion gives us the trend we’d expect – that sacks go up as yards per completion go up, but the explanatory value is weak as the p-value suggests insignificance. When doing a multivariate regression with yards per completion and interior line experience against sack percentage, the r-squared only rises to 0.056 from 0.05. It doesn’t add much explanatory value. Using YPA instead of yards per completion actually produces a trend where it appears that increased yards per attempt facilitate a decrease in sack percentage. That doesn’t make a lot of sense and the correlation and statistically insignificant anyway.
4 star talents, 5 star smiles
A third possible factor governing sack percentage is the skill of the quarterback. Perhaps sacks are less a matter of how good the line is and more a matter of how good the QB is. To measure this I look at each BCS quarterback’s star rating and their Rivals rating (4.9-6.1) to see whether their high school talent correlates with how often they end up sacked. You can’t really use any college stats as a measure of their talent, because those can be directly influenced by the play of the offensive line, and we’re trying to isolate QB talent as a separate and independent variable here.
click to zoom
There’s really not much here. Whether you go by star ranking (2-5 stars) or by the more precise Rivals rating (4.9 – 6.1), there’s no significant relationship between a quarterback’s talent and his ability to remain upright. R-squareds for both metrics are <0.01, and p-values are 0.75 – 0.80. On an individual level, the skill of a single quarterback might help him avoid sacks, but taken broadly across all BCS schools, quarterback talent doesn’t seem to be a factor.
all 10 linemen on FSU's 2-deep are upperclassmen /Miami Herald
Probably the most common critique in the previous diary was that depth should be taken into account. You can do this different ways: average or total experience on the 2-deep, the oldest player at each position on the 2-deep, or the percentage of upperclassmen on the 2-deep. For this study I'm using the last of these definitions, the percentage of upperclassmen. I'm defining "upperclass" as students who have been on campus for two years prior to this season. So redshirt sophomores and true juniors are both considered "upperclassmen," while true sophomores are not. The graphs below show the trends for both the line as a whole and the interior of the line.
click to see all teams labeled - Duke also has an all-upperclass 2-deep
The correlation is unexpectedly poor. The graphs above show line depth both across the entire o-line and just the interior of the line. In both cases, the trend line suggests that the more upperclassmen you have, the more sacks you give up. This doesn't pass the common sense test, and r-squareds for both are low (0.01) and p-values are high (>0.30) implying that the correlation is not statistically significant. It doesn't appear to be a matter of defining "upperclassmen" either. If you run the same regression using average depth on the line, you get the same spurious results. While line depth might be an excuse for any given team, across the entire FBS the experience of your starters seems to matter much more than the experience on the entire depth chart.
Modest but significant. Despite using a completely different metric for o-line success, sack percentage instead of YPC, the conclusions of this study are eerily similar to the previous one. Let’s begin with the (hopefully) obvious caveats. Offensive line experience explains a modest, though significant, amount of the variation in sack percentage across all FBS schools. We’re talking about 5% of the variation here, so there are clearly a lot of other factors that go into determining how good a team is at protecting their quarterback.
In a way, this study is much less about Michigan than it is about college football in general. The success or failure in pass protection for a single team can often be explained by factors that are specific to that team. For instance, Devin Gardner is essentially the Ben Roethlisberger of college football, refusing to throw the ball away or to be tackled until approximately half the other teams defenders are draped all over him. This certainly contributes to Michigan’s high percentage of sacks, but it is a difficult variable to account for and measure across all of college football.
That being said, offensive line experience does stand out as a particularly salient characteristic for explaining a team’s sack percentage. Although we’d assume that experience at the tackle positions would be more important in the passing game, the results of this study suggest that once again the interior of the line is what matters most. In contrast to the previous study, the “weakest link” (i.e., the youngest interior linemen) is not as good of a predictor as the average experience of the interior of the line.
Taking a comparative approach by looking at experience alongside other potentially influential factors provides some context for how important experience actually is. The chart below plots each of the metrics I’ve looked at in this study along with their r-squared and p-values.
|Independent Variable||Unit of Measurement||Data Set||R-Squared||P-Value||Significance?|
|Avg. O-Line Experience||Years||FBS||0.02||0.10||Marginal|
|Total O-Line Experience||Starts||FBS||0.01||0.28||Low|
|Avg. Tackle Experience||Years||FBS||0.01||0.67||Low|
|Avg. Interior Line Exp.||Years||FBS||0.05||0.02||High|
|Left Tackle Experience||Years||FBS||<0.01||0.31||Low|
|Average O-Line Talent||Rivals Stars||FBS||0.01||0.30||Low|
|Average Tackle Talent||Rivals Stars||FBS||<0.01||0.52||Low|
|Avg. Interior Line Talent||Rivals Stars||FBS||0.01||0.22||Low|
|Left Tackle Talent||Rivals Stars||FBS||0.01||0.32||Low|
|Average O-Line Talent||Rivals Stars||BCS||0.03||0.18||Low|
|Average Tackle Talent||Rivals Stars||BCS||<0.01||0.64||Low|
|Avg. Interior Line Talent||Rivals Stars||BCS||0.05||0.07||Marginal|
|Left Tackle Talent||Rivals Stars||BCS||<0.01||0.87||Low|
|Throwin' Deep A||Yards per Completion||FBS||0.01||0.31||Low|
|Throwin' Deep B||Yards per Attempt||FBS||0.01||0.20||Low|
|QB Talent A||Rivals Stars||BCS||<0.01||0.80||Low|
|QB Talent B||Rivals Rating||BCS||<0.01||0.78||Low|
|Total Line Depth||Upperclassmen %||FBS||0.01||0.33||Low|
|Interior Line Depth||Upperclassmen %||FBS||0.01||0.43||Low|
This provides some much needed perspective. IME this really highlights the importance of experience, and especially the importance of the interior line. Interior line experience correlates more strongly with sack percentage than does a team’s tendency to throw the ball deep (at least when measured by yards per completion), and it serves as a better predictor than average talent of an entire offensive line (at least when measured by star ranking). This is really interesting! If I was a betting wizard, and I am, I would have bet on average o-line talent being a much better predictor of success than experience. Also, although the experience of the starting interior linemen does correlate significantly with sack percentage, depth along the offensive line does not.
The factor that comes closest to interior line experience in terms of predicting sack percentage is the talent of the interior of the line. This should strengthen our confidence in the conclusion that the interior line is the more crucial than the tackles in keeping the quarterback clean. As previously touched upon, when we combine interior line experience and interior line talent as two predictors of sack percentage and run a multiple regression, the r-squared returned is approximately 0.09. This isn’t huge by any means, but it serves as a better measure of success in pass protection than any single metric we’ve looked at so far.
Why don’t the best linemen play on the interior? This was one of the main questions raised during the last study, and the assumption was that teams play their best lineman at tackle in order to protect their quarterback. This study suggests that the interior of the line is more influential in accomplishing that task. There are a couple potential explanations. QB injuries and fumbles could still be most common from blind side hits, and team’s put their best guy there in order to mitigate these disasters. I haven’t tested this but I imagine it’s something that could be done statistically. It could also be that the best linemen play at left tackle because that’s the most important line position in the NFL, where one might assume that tackles matter more (hence their bloated salaries). If you look at the relationship between left tackle talent in the NFL (as measured by salary) and sack percentage, however, you get a pretty spurious correlation.
The line does trend up a bit suggesting that higher paid left tackles allow fewer sacks, but the r-squared is only 0.01 and the p-value is 0.66. It appears that left tackles aren’t much better at predicting pass protection success in the NFL than they are in college. (This is obviously more complicated than I’ve presented here. For example, teams could spend more on left tackles to fix problems that are inherent in the rest of their line or in their offensive system, thus producing a trend where teams with higher paid left tackles actually have higher sack percentages. This study is about college though, so I’m just leaving this for now).
I guess I just don’t know, man. The argument about protecting the quarterback from taking blind side hits makes intuitive sense to me, but the data all suggest that the interior is a more important factor in pass protection. If anyone’s got any quantitative study on why it makes more sense to play your best lineman at left tackle, or that tackles are, in fact, more crucial to pass protection, I’d be interested to see it.
What does this mean for Michigan? Let’s reemphasize that the experience data explain a relatively small proportion of the variance in sack percentage and that for any single team, and for any given team, team-specific explanations probably outweigh the statistical ones suggested by youth or talent. That being said, Michigan is very young where it appears to matter most. They are, however, talented – at least according to their star rankings. If these players develop at an average rate, then our line should make some serious strides by the time it’s full of talented upperclassmen on the interior. This is somewhat disheartening for this year but should provide some hope for the future.
This hope, of course, is based on the expectation of reasonable player development. We don’t need the best coaches in the world, since they tend to recruit already talented and physically gifted players, but we do need to develop those players on par with the rest of college football. I have no idea whether Borges and Funk have histories of successful o-line development, but it might be something worth looking into. The potential is there, however, to have a very successful o-line with regard to both the running and passing game as these kids become upperclassmen.
This study isn’t meant to indict or absolve any of the coaches, and it really does say more about college football as a whole than Michigan in particular. I do, however, think it’s interesting to see how Michigan’s production compares to other schools given a specific level of experience. We’re pretty far below the trend line even when experience on the interior is accounted for, and especially when talent along the interior line is taken into consideration. I think that Devin Gardner’s inner Ben Roethlisbergerness has something to do with this, as does Al’s predilection for two man routes where both receivers go deep. Experience, especially on the interior, does seem to play a role though. I don’t think it’s really possible to accurately assign percentages of blame (it’s really just a guessing game), but until we get that sack percentage out of the FBS basement, rest assured, there will be plenty of blame to go around.
the past the future (let's hope)
Happy MGoThanksgiving to all!
the 36th image that comes up when you google "turkey football"
yes, I am taking this as a sign we beat Ohio
Amidst all of Michigan's offensive struggles this year, the relative youth of our offensive line, particularly along the interior, has been a constant concern. Regardless of which side you fall on in the "Execution vs. Play-Calling" debate, you probably agree that having 6 combined years of experience between our three interior linemen is part of the issue. While the interior will hopefully improve next year with another year under each of their belts, Michigan will be losing it's two 5th year senior tackles.
With this in mind, I attempted to quantify the impact that an offensive line's experience has on a team's offensive success. Can Michigan expect to improve it's offensive output next season? How painful will it be to lose Lewan and Schofield?
I used depth charts from Rivals' database (they seemed more or less up to date) to get average experience (defined by academic standing) for each team in the Big 10's OL as a whole as well as split up by subgroup (T, G, C). I then drew scatter plots comparing those ages to overall YPA and rushing/passing YPA.
After seeing the initial results, I had to remove Purdue since they have the oldest OL and yet the worst offense by nearly two standard deviations. Some other teams were lesser outliers (Michigan, Indiana) but with such a small sample I didn't want to remove them too.
Michigan has a young, but not absurdly so, offensive line. Of course, they are helped tremendously by Lewan and Schofield. They have a very young interior though, with 6 combined years of experience. Iowa's interior is second youngest at 8 combined years, and their offense isn't exactly instilling fear across the nation either.
Purdue sucks. Wisconsin, OSU, and Indiana are really good. The rest of the Big 10 is mediocre. No news here.
Below are a series of scatterplots comparing the age of each Big 10 team's OL (and it's subgroups) to the offense's YPA (passing and rushing.
Overall, the age of the offensive line seems to have little to no impact on overall yards per play. In the Big 10, it has a slight positive impact on rushing yards but actually has a slight negative impact on passing yards. The negative impact on Passing YPA is mainly due to Michigan and Indiana, two teams with young OLs but high Passing YPA. In general though, these are pretty low R-squared values and don't seem to show much.
Tackles seem to have almost no impact on YPA, whether in the air or on the ground. This is probably largely a function of the sample, where 7 of the 11 teams have tackles with 7 combined years of experience. If I remove Indiana, the impact on Passing YPA becomes more meaningful, which makes sense given tackles' roles in pass protection. Michigan is wholly responsible for dragging down the Rushing YPA graph.
Again, guards' ages don't show much impact on YPA according to R-squared, although you can see an upward trend here at least. Even removing Michigan's young guards results in guard age having a negative correlation with passing YPA which is surprising. The rushing YPA graph makes sense at least, showing a relatively strong R-squared value.
Center experience shows by far the strongest relationship between age and YPA. This makes sense both because of center's importance to the OL as well as the complexity of the position requiring some experience to learn. The entirety of that influence comes on the ground, with center's age meaning nothing to passing YPA.
Before attempting to draw any conclusions, I think a few caveats must be stated:
- This analysis looks strictly at academic standing as a measure of experience. That means that a RS senior starting for the first time is viewed as more experienced than a true junior in his third year of starting.
- This analysis ignores all positions other than OL. As we saw last season at Michigan, for example, a dynamic QB can make up for a youthful offensive line. It also overlooks TEs, which in a system like Michigan's are also a vital part of the OL.
- The sample size is pretty thin. If I had more time, I'd like to do this for a large group - maybe the top 25 teams or something.
The above aside, I still think there are some interesting takeaways from this analysis.
- Offensive line experience matters in the run game. OSU and Wisconsin have the top two offenses in terms of YPA and 2 of the 4 most experienced lines. In particular, their rushing attacks average over a yard more per attempt than any other team.
- Offensive line experience does not seem to matter as in the air. While Indiana's passing YPA may be a function of it's system, Minnesota and Illinois have respectable averages with relatively young lines.
- Michigan's rushing game should improve next year (how can it get any worse...). Both guard and center experience correlate with rushing YPA. Losing their tackles doesn't appear to have much impact based on this analysis.
- Michigan's passing game is not necessarily doomed next year. The data doesn't show much one way or another, but tackle experience at least is not strongly correlated with offensive success in this sample.
Any thoughts/feedback are welcome.
Rerum omnium magister usus.
Experience is the teacher of all things.
Julius Caesar, Commentarii de Bello Civili, Book 2, Chapter 8
Setting. As the 2013 football season rolls on, the problems in Michigan’s run game have become more and more glaring. This has led to much ballyhooing and debate as to the main causes of Michigan’s ground game woes. The most basic argument is whether our problems are caused by weakness on the line or at the running back position. Brian’s UFRs come into play here, and while Fitz and the gang haven’t been perfect by any means, the play-by-play breakdowns seem to suggest that the problem lies with our offensive line. A phalanx our line is not.
Identifying the line as the problem, however, has not really made anyone very content. Rather it’s sparked a debate between the baby blamers – those who see Michigan’s youth as the source of their problems – and the crappy coach contingent – those who find fault with our coaches development of our o-line talent, not to mention play calling.
Previous Work. The MGoCommunity has already produced some solid work on this topic. The Mathlete’s preseason study looked at other teams who had offensive lines with an 1st round NFL pick combined with 2+ freshmen. Although he eventually admitted that we were still probably a year away, his comparison grouped us with the likes of Alabama, Oregon, and Stanford. GuloGulo’s diary from back in September looked at the relationship between average o-line experience in the Big 10 and success mainly defined as yards per play. After the first four weeks of the season, he concluded that we were about average in both experience and success. Gameboy’s recent diaries have shown that Michigan’s line is relatively young whether you take a “number of years in the program” or “number of previous starts” approach.
Questions. In this study I want to delve a little deeper into what we mean by “experience,” what we mean by “success,” and how those two variables are actually related. I will attempt to answer four questions:
- Can offensive line experience explain run game success?
- Are years or starts a better measure of experience?
- Is interior line experience more important that tackle experience?
- Is average experience a better measure than the weakest link?
Definitions. Neither experience nor success have single and obvious definitions. With regard to the o-line, success could be defined by wins, yards per play, yards per rush, sack percentage, or play-by-play results a la UFR. For the first part of this study, I use yards per carry as my metric for success. Experience, likewise, can be defined in a variety of ways, including the number of years in the program, the number of starts, or the number of snaps. This analysis primarily uses the number of years in the program as its measure for experience. This isn’t because it’s necessarily the best measure – we’ll test that in a bit – but rather because it’s the measure that’s easiest to find information about. Redoing this study with a start- or snap-focused measure of experience would be a worthwhile endeavor. In the graphs below, the year of the players are equated with numbers, so that freshman = 1, red shirt freshman = 1.5, sophomore = 2, red shirt sophomore = 2.5, and so on.
Data. The data for this study are drawn from this 2013 season. All 125 FBS teams are included. The YPC stats come from ESPN and the experience info comes from the a scouting site. Because this isn’t necessarily about Michigan’s o-line this year, but rather about the general relationship between offensive line experience and success, data from any recent season should apply though. Giving this thing some time depth would probably improve its efficacy. The stats are current as of 11/6/2013. All the images are from the MGoBlog flickr account; Bryan Fuller gets the credit, I believe.
This is a primarily quantitative study, but I’m in no way a statistician. My background is in Classics, as in Greek and Roman studies, so although I’ve tried brush up on my stats, there’s certainly the possibility that these metrics aren’t employed or interpreted perfectly. Feel free to correct me.
With that said, it’s probably useful to give a brief overview of the statistical measures in an attempt to describe what they actually tell us. I’m looking at 4 main metrics: correlation coefficient, r-squared, p-value, and slope of the linear trend line.
Correlation coefficient: The correlation coefficient quantifies the degree of linear relation between two variables. The coefficient ranges from -1.0 to 1.0, and the larger the absolute value of the coefficient, the stronger the relationship. This will provide a single number for the strength of the relationship between o-line experience and yards per carry.
R-squared: The r-squared provides a measure for the amount of variance in one variable that can be explained by another variable. This will be used to assess how much of the variance in yards per carry can be explained by o-line experience. It’s important to note here that there are obviously many other factors other than experience that govern running game efficiency (coaching, scheme, running back skill, etc.). A low r-squared doesn’t necessarily mean that experience is unimportant, just that other factors are also important.
P-value: The p-value let's us know whether our results are statistically significant; more specifically it provides a measure to assess whether we should discard the null hypothesis. In this case, the null hypothesis is that there is no relationship between o-line experience and running game success. The p-value ranges from 0 to 1. A small p-value, < 0.05, suggests that we reject the null hypothesis, while a large p-value suggests we retain it. If p-values are low, we should have faith in the relationship between experience and success; if they are high we should feel less confident about that relationship.
Slope of linear trend line: The trend lines in the graphs below show the linear relationship between experience and success. The slopes of that lines indicate the extent to which we’d expect YPC to change as a result of a change in experience. For example, if the slope was 0.5, the data would suggest that an extra year of average o-line experience is worth ½ of a yard per carry.
Question 1: Can offensive line experience explain run game success?
Good habits formed during youth make all the difference. - Aristotle
The scatter plot below depicts the relationship between average offensive line experience in years and yards per carry. Click for enlarged scatterplot with all BCS teams labeled.
The data broadly confirm what we’d expect. This is good! This means that we’re right in claiming our youth is (partially) the problem. The older your offensive line is, on average, the more yards per carry that team produces. The correlational coefficient is 0.16 for this data set, indicating that there’s a slightly positive correlation between offensive line experience and yards per carry. The r-squared is small, however, suggesting that only about 3% of the variation in teams’ yards per carry can be attributed to the experience of the offensive line. A p-value of 0.07 is marginal, meaning that it’s not particularly clear whether we should interpret these results as significant or not. Let’s start by giving experience the benefit of the doubt though, and for the time we can conclude that experience does indeed influence ground game success. The slope of the linear trend line suggests that an extra year of average experience is worth about 1/3 of a yard per carry.
At first glance, there does seem to be a positive correlation between o-line experience and YPC, although there is still a lot of variance in YPC that cannot be explained by experience.
Question 2: Are years or starts a better measure of experience?
One of the arguments against the approach taken in question one is that an offensive lineman’s number of previous starts is a better measure of experience than the number of years he’s been in the program. Let’s take a look; the graph below plots this alternate measure of experience against yards per carry. Click to enlarge and see Oregon and Wisconsin put up 6.7 YPC despite having less total starts along the o-line than Michigan.
The relationship between the number of previous OL starts and yards per carry generally mirrors the pattern produced when the number of years in the program is taken into consideration. The correlation coefficient is actually slightly higher (0.23 compared to 0.16), suggesting that starts is indeed a slightly better measure than years in the program for the purpose of predicting o-line success. The r-squared suggests that previous starts can explain about 5% of the variance in yards per carry, and a p-value of 0.01 indicates that these results are indeed significant. The slope of the line suggests that each extra start is worth about 1/100 of a yard per carry, meaning that 50 extra stars is worth about ½ a yard, and 100 extra starts is worth about a full extra yard per carry.
Now that the data show that “number of starts” is probably a better measure of offensive line success, I’m going to revert to “number of years in the program” as my main metric of experience. This is simply due to the convenience of the data. If someone can get number of starts for all the programs, that should improve things. Perhaps another day.
Question 3: Is interior line experience more important than tackle experience?
Why doesn't Lewan make everything okay?
One of the most common arguments against using the average or total experience of the entire offensive line is that all spots along the line are not created equal. Lewan being an awesome LT doesn’t help our RG Mags getting crushed by the NT. Essentially, interior line experience is more important than tackle experience. But does it really matter whether your experience comes on the interior or exterior of the line?
Let’s start with tackle experience first. The graph below shows the relationship between the average experience of each team’s tackles and their YPC.
Check out Michigan and Purdue with their bookend fifth year senior tackles. This doesn’t bode well for a positive relationship. Looking across the entire spectrum of the FBS, there appears to be no correlation between the experience of a team’s tackles and their ability to run the ball successfully. Once again, this is good news for us. It’s not that we’re not taking advantage of our great tackles, it appears that on the whole, tackle experience just doesn’t influence ground game success all that much. The correlation coefficient is a measly 0.02, the r-squared is <0.01, and the p-value is 0.81, which is incredibly high. The slope of the trend line suggests a very, very slight decrease in YPC as tackles increase in age, which doesn't make any sense at all.
This is really interesting actually, as all metrics suggest there is essentially no connection between tackle experience and yards per carry. If tackles aren’t the cause of the correlation between total experience and YPC, then it must be the interior of the line, right? Click to enlarge and see us at the children's table with UCLA and Purdue.
It appears as though the “our interior line is full of infants” excuse is actually a pretty good one. With a correlation coefficient of 0.22, the relationship between these two variables is stronger than when offensive line experience as a whole is averaged (in years) and an order of magnitude stronger than the correlation between tackle experience and YPC. The r-squared indicates about 5% of YPC variation can be explained by experience along the interior of the line, and a p-value of 0.01 suggests these results are significant. The slope of the trend line suggests an extra year, on average, is worth about 1/3 of a yard per carry.
If you extrapolate that out over the course of a season, that’s about 150-200 extra yards of rushing per year (Michigan had 502 rushing attempts in 2012 according to ESPN). Interior line experience does seem to be a big deal. Also, we’re one of the 3 youngest teams out of 125 FBS teams in terms of interior line experience. That is young indeed.
Question 4: Is average experience a better measure than the weakest link?
The foundation of every state is the education of its youth. - Diogenes
Thus far the data have shown that interior line experience is a better predictor of running game success than total offensive line experience. The next question is whether average interior line experience is a better predictor of success than the “weakest link” along the line. In this case we’ll call the youngest person on the interior of the line the weakest link. This really has nothing to do with their ability, it’s just a measure of their experience in the program. Click to enlarge and see Auburn averaging 6+ YPC while starting a true freshman interior lineman.
It looks as though there is something to the “weakest link” argument. The correlation coefficient in this case is 0.29, which stands as our strongest correlation yet between some measure of experience and yards per carry. The r-squared indicates that this measure can explain about 8% of YPC variation, and a p-value of 0.01 suggests that these results are indeed significant. The slope here once again suggests that an extra year is worth about 1/3 of a yard per carry.
The fact that the age of a team's youngest interior offensive lineman is a better predictor of run game success than its average offensive line experience, or even the average experience of just the interior line, is rather unexpected. This should bode well for Michigan's future along the line as we gain experience and depth in future seasons.
First off, offensive line experience leaves a lot of the variance in yards per carry unexplained. So even though this study supports the conclusion that offensive line experience does indeed influence success in the running game, there are clearly many other factors that also play a role.
In this study, experience has been measured in in two ways, both as “years in the program” and as “number of starts.” While both serve as decent predictors of success in the running game as judged by YPC, number of starts seems to be the better measure. Unfortunately, it’s also the measure that is more time-intensive to track. When looking at the outer vs. interior line, the data suggest that success on the ground is much more closely tied to the experience of the interior line than it is to either the tackles or even the average experience of the line as a whole. Surprisingly, tackle experience seems to be completely irrelevant as a predictor of run game success. Finally, the level of experience of the least-experienced person on the interior line serves as an even better metric for predicting running game efficiency. The “weakest link” argument appears to hold water.
Unit of Measurement
|R-Squared||P-Value||Effect on YPC|
|Total Experience||Years||0.16||0.03||0.07||Extra year = +1/3 yard|
|Total Experience||Starts||0.23||0.05||0.01||Extra 10 starts = +1/10 yard|
|Interior Line Experience||Years||0.22||0.05||0.01||Extra year = +1/3 yard|
|Youngest Interior Lineman||Years||0.29||0.08||0.01||
Extra year = +1/3 yard
What does this mean for Michigan? As Gameboy showed us in his diaries, Michigan is young along the O-Line, whether you’re judging by years in the program or by number of starts. What I hope to have demonstrated here is that (a) being young really does matter, and (b) we’re especially young where it matters most (i.e., tied for 2nd youngest on the interior OL out of 125 FBS teams).
Borges and Funk in happier times
There’s been a lot of heat on Borges and Funk recently, and it’s appropriate to ask whether this study indicts or absolves them. Unfortunately, I think the data tend to side step the question. The fact that o-line experience does seem to influence YPC, and especially the finding that interior line experience seems to be of utmost importance, combined with Michigan’s position with regard to these measures (i.e., they fall almost exactly along the linear trend line in both the interior line experience graph and in the weakest link graph), would initially suggest that the line is performing about as expected.
This doesn't let the staff off the hook. The relatively low r-squared values would indicate that there is a lot more than just experience that goes into producing a successful running attack. Coaching, both in terms of scheme and player development, is probably one of the most influential factors in governing run game success, and this study doesn’t attempt to measure or control for that aspect of the game. Moreover, this study doesn't account for talent along the offensive line, which would probably suggest Michigan is underperforming relative to the recruiting rankings. Strength of schedule is also omitted. Having played CMU, Akron, UConn, Indiana, and Nebraska, adding this variable could also raise our team's expected YPC, and in doing so lower our performance relative to expectations.
According to the eyeball test, the apparent regression along the offensive line would seem to indicate that there are some seriously problematic coaching issues. There are several BCS programs with similar youth-related issues on the interior line, both when experience is averaged (e.g., UCLA) and when experience is defined by the youngest interior lineman (e.g., Notre Dame, Arkansas, and Auburn), and these programs still manage to perform significantly better than us in terms of yards per carry. When viewed within the context of the entire FBS, however, the data suggest that Michigan’s youth is a real and influential issue.
On the bright side, this should give us hope for future seasons. As our interior line matures, both in terms of average experience and in terms of its weakest link, we should improve. This only holds, of course, if all the other factors that go into producing a successful offensive line – namely coaching scheme and player development – are on par with the rest of college football. That, unfortunately, is not guaranteed.
Rome wasn’t built in a day, but it did progressively grow bigger and better until it reached a point where it dominated at the point of attack. Let’s hope our offensive line can do the same.
Coming in part 2: Shouldn’t our veteran tackles at least make us better at pass protection?!?
Awww, come on.
 Actually, upon further review, I’m not so sure this is accurate. Over the course of 8 games, Fitz has 5 positive UFR games, 2 negative, and 1 around zero, while the OL has 5 positive UFR games, 2 negative, and 1 around zero. Obviously RB and OL numbers aren’t perfectly commensurate, but this probably suggests the blame should be shared.
After watching the UMass game I went through a series of strange feelings:
- Relief. Game never in question, easy opponent dispatched, spread covered.
- Anxiety. The D-Line didn't look that good. Come to think of it, neither did the run-blocking.
- Relief. 2013 isn't far away, and we'll be transitioning from Borges-Denard Fusion to Manball.
- Anxiety. 2013 isn't far away, and we'll be transitioning from Borges-Denard Fusion to Manball.
Yes, the last two are the same. No, it's not a mistake. Here's what happened in my brain:
- I sure will miss Denard next year, but we'll be back to Michigan Manball without him. And our offensive line will be much better at run-blocking.
- Wait, why will it be better at run-blocking? Who will even be playing O-Line for us next year?
- SHUT-UP! SHUT-UP! SHUT-UP! Enjoy the win. Just pretend that Hoke will sprinkle magical Manball dust (wait, that doesn't sound right) and everything will turn out fine.
- No, you shut-up. I have to research this so I can sleep. Or so I can't sleep. AAARRRGGGGH!
Suddenly, I'm lying awake in bed, and going through the depth chart in my head. The one guy that seems to be playing good football on the O-Line is Taylor Lewan, and it's no secret that he's projected to get a 1st round NFL draft grade. We have to assume he's gone. Also gone from the line is...EVERYONE. Except Schofield. He'll stick around for his fifth year. Right? RIGHT?!!! And it's good that he stays...right? RIGHT?!!!
Let's just assume the whole line, except Schofield, is gone. Our new O-Line looks like this:
LT Erik Magnuson; LG Chris Bryant; C Jack Miller; RG Kyle Kalis; RT Michael Schofield
Maybe Joey Burzynski sneaks in there, but we're likely going with four (FOUR!) players with
little or no playing experience. And Schofield.
And I think, if we're geing honest with ourselves, that line doesn't look much better than this year's version. Sure, Bryant and Kalis should be better people-movers than Barnum and Omameh, but they'll be learning the position as they go. And a redshirt freshman left tackle? Ummmm...
Oh, and did I mention that Jack Miller is currently 6-4 and 288 lbs? Not exactly the size we're looking for at center.
My point is that our O-Line will be far from "Manball-Ready" and that we have much to fear about their abilities. Maybe we'll get some better natural push, but we're trading that for what will likely be a drop-off in pass-blocking. Erik Gunderson is currently listed ahead of Ben Braden, so I'm not holding my breath there. Blake Bars isn't currently listed on the depth chart.
On the other side of the ball, things don't look much better. BWC and Roh are gone. Pipkins should be seasoned, and hopefully in shape. Brink, Washington, Black, and Ash are all back, but like, seriously.
Our hopes will be tied to Pee Wee (NT) and Frank Clark (WDE), with Heitzman/Brink/Wormley/Strobel at SDE and Black/Washington/Ash/Brink at DT needing to just not screw it up.
The only logical conclusion is that next year will require the same patience with our lines as this year. This issue will be exacerbated by the lack of Denard...whether it's Bellomy or Gardner running the offense, it will be an offense that is trying to be pro style, and won't have a premier QB to run it. Gardner's growth will be stunted by his time at WR, and Bellomy looks like a solid-but-not-spectacular player. Navarre Light, if you will, with more mobility and a not-as-good arm.
I guess what I'm saying is that we'll really have to wait until 2014 before we have our Manball lines in place, and maybe then we can go back to complaining about the secondary.