Experience and Success on the Offensive Line - Part II: Pass Protection

Submitted by Gandalf the Maize on November 28th, 2013 at 1:54 PM

Setting the Scene

  

no animals were harmed in the making of this diary

Apology. I am sorry to all those who are cringing at seeing another experience diary. I had originally conceived of this as being a two part study, with the first looking at the running game and the second looking at the passing game. Despite the other diaries, both of which were useful in their own ways, I think there is still some horse meat to be gleaned from the carcass of o-line experience as it relates to pass protection. The horse might be dead, but that doesn’t mean it’s useless. Oh, and there's another one since I originally wrote this thing up.

Previous Work. In the first diary, I attempted to demonstrate that o-line experience does indeed play a role in governing a team’s ability to run the ball. R-squared values ranged between about 0.05 and 0.10 depending on how we defined “experience,” suggesting that about 5-10% of the variation in YPC across FBS can be explained by a team’s experience along the o-line. Clearly other factors are also important to running the ball well, but experience isn’t meaningless. Further manipulations to the data set suggested that interior line experience is more important than tackle experience and that the “your line is only as good as its weakest link” argument does hold water. 

Some of the comments from that diary questioned why we don’t move one or both of our experienced and talented tackles to the interior if that is where it really seems to matter. Transaction costs of moving around linemen aside, the question is valid in general terms. Why not put your best linemen on the inside if those are the most important positions? A variety of answers could be given to this question – for example, exterior and interior line positions could have different ideal body types with regard to height, weight, strength, and agility – but the most obvious response is that tackles are more influential in the passing game. Our best linemen play at tackle because they protect our quarterback.

Questions. This is a proposition that can be tested statistically, and that’s what I aim to do in the second part of this study. My metric for o-line success in the passing game is sack percentage (i.e., the percentage of pass attempts on which your QB is sacked), since it’s the o-line’s job to keep the QB clean. Using the same essential methodology as the last study, I aim to answer four questions:

  1. Does o-line experience help prevent sacks?
  2. Is tackle experience or interior line experience more important in protecting the quarterback?
  3. Does the “weakest link” theorem hold for the passing game?
  4. What else could influence sack percentage?

Data. This study looks at 123 FBS teams (Georgia State and UTSA are omitted since their info wasn’t on ESPN). Sack percentage stats come from ESPN, and the experience data comes from the scouting site Ourlads. Star rankings that come at the end of this study are taken from Rivals. Photos come from MGoBlog's flickr account and are attributable to Bryan Fuller. Check out the previous diary for basic definitions of the statistics that are used.

This is long, so buckle up. Feel free to jump to the conclusions if you don’t want the nitty gritty. All the data are summarized in a…chart?  Chart.


Questions and Answers

Probably cropped out: massive amounts of backside pressure

Question 1: Does o-line experience help prevent sacks?

Let’s start by taking a broad look using average experience in years of the offensive line. The relationship between experience and sack percentage is plotted below. Click on the graph to see the same sack percentage data plotted against total number of starts. The plot is oriented so that up is good (i.e., your QB isn't getting sacked that often) and down is bad (i.e., you're looking like Michigan against MSU and Nebraska).

Although the trend line makes it look as though there is an inverse relationship between sack percentage and experience (i.e., sacks go down as experience goes up), which is what we’d expect, the r-squared is relatively low (0.02) and the p-value (0.10) suggests the trend may not be statistically significant. If we plot the same relationship but use starts as our metric for experience, the relationship becomes even more spurious with an r-squared of 0.01 and a p-value of 0.28 (click on above graph to see scatter plot). On the whole, total or average o-line experience doesn’t seem to be a great predictor of the o-line’s ability to keep the quarterback from getting sacked. 

 

Question 2: Is tackle experience or interior line experience more important in protecting the quarterback?

 

I don't care who matters more as long as we keep it under 7 sacks a game

We saw in the previous study that interior line experience was more important to run game success than tackle experience. We’d likely expect the opposite to be true for the passing game based on the premium put on left tackles both in college and in the pros. Average tackle experience and sack percentage is plotted below. 

This is unexpected. The correlation is spurious. The r-squared is less than 0.01, and the p-value is 0.67, both of which suggest there is no correlation between tackle experience and sack percentage. The trend line actually rises slightly, which would indicate that sack percentage rises as tackles get more experienced, which makes no sense at all, even if the correlation was statistically significant. On the whole, tackle experience does not appear to be a good predictor of your team’s ability to not give up sacks.

Could the interior of the line be more important in the passing game as well? Click for enlarged scatterplot with teams divided by conference and all BCS teams labeled.

Now we’re getting somewhere. The trend line makes intuitive sense. The percentage of sacks you give up goes down as your interior linemen become more experienced. The r-squared is 0.05, implying interior line experience can explain about 5% of the variance in sack percentage. The p-value is 0.02 suggesting that results are statistically significant. The slope of the trend line suggests that an extra year of average interior line experience is worth a drop of almost 1 percent in sack percentage. If you extrapolate that over the course of a season, that’s about 3 to 5 fewer sacks. Not a huge difference, but if your team matures over the course of several years from starting freshman to starting seniors, that adds up to reducing sacks by about 1 per game.

One possible critique here is that “average” tackle experience is not the correct measure. Teams often put one of their better run blockers at right tackle and their best pass protector at left tackle. Thus instead of looking at the average, we should just look at the correlation between left tackle experience and sack percentage.

We don’t even need a graph here. We get almost the exact same trend as when the tackle experience data are averaged, and the slope of the trend line suggests that sack percentage slight increases as tackles get older. That is intuitively incorrect. A low r-squared value (<0.01) suggests left tackle experience doesn’t matter very much and a high p-value (0.31) implies statistical insignificance. This is admittedly somewhat baffling and definitely unexpected.

 

Question 3: Does the “weakest link” theorem hold for the passing game?

weakest link only in age, not awesomeness

When looking at the run game, the data suggested that the youngest member of the interior line was a better predictor of success than average experience of the interior line. In the passing game, the “weakest link” is a little less weak.

Unlike with the run game, average interior line experience appears to serve as a better predictor of sack percentage than does the “weakest link” along the interior of the line. The r-squared here is 0.02 and the p-value is 0.12, suggesting that the significance is marginal at best. It's not that the weakest link is a terrible predictor, just that the average experience of the entire interior line serves as a slightly better indicator of sack percentage.

At this point we can draw some basic conclusions from the first three questions. Total or average o-line experience only seems to be a marginal predictor of a team’s ability to keep their quarterback from getting sacked. Tackle experience, whether averaged or just taken as the left tackle, appears to have no relationship whatsoever with sack percentage. Just like with the run game, interior line experience seems to be the most salient characteristic with regard to o-line experience for predicting success in pass protection.

 

Question 4: What else could influence sack percentage?

One of the main critiques from the last study was that we’re living in a multivariate world and other potentially influential factors should be included in the analysis. I’m still working on getting myself up to speed regarding multivariate analysis, so I’m tentative to try and do too much with that now. We can, however, look to see how some other variables correlate with sack percentage. 

Offensive Line Talent

4 star, 4 star, 5 star...sack?

Talent is one obvious potential factor in governing pass protection success. The chart below shows the correlation statistics for the star rankings of the o-line with regard to sack percentage. I’ve omitted the graphs here because none of these produce any correlation of statistical significance.

Correlations between offensive line talent and sack percentage for all FBS teams
Talent Metric R-Squared P-Value Significance?
Average O-Line Star Ranking 0.01 0.30 Low
Average Tackle Star Ranking <0.01 0.52 Low
Average Interior Line Star Ranking 0.01 0.22 Low
Left Tackle Star Ranking 0.01 0.32 Low

Surprisingly, star ranking of offensive linemen doesn’t seem to correlate very strongly with sack percentage. My guess is that this is due to star ranking of offensive linemen correlating closely with the difficulty of defense that a given team plays. For example, Alabama has talented linemen, but they play against tough defenses in the SEC. Toledo, on the other hand, has crappy linemen, but they play against week defenses in the MAC. Moreover, there tends to be very little variation in star rankings with non-BCS schools – almost everyone is a 2 star – so this may be obscuring some of the impact that talent (i.e., star rating) has on pass protection.

This can be accounted for, to some extent, by looking at only the BCS schools, where there will be more variation among offensive line star ratings and more consistency in the level of teams played. The chart below shows the correlation statistics for offensive line talent and pass protection success. 

click to zoom with all BCS teams labeled

Correlations between offensive line talent and sack percentage for BCS teams
Talent Metric R-Squared P-Value Significance?
Average O-Line Star Ranking 0.03 0.18 Low
Average Tackle Star Ranking <0.01 0.64 Low
Average Interior Line Star Ranking 0.05 0.07 Marginal
Left Tackle Star Ranking <0.01 0.87 Low

Once again, it appears as though the interior of the line is the most crucial for preventing sacks. This corresponds well with the experience data presented in the first three questions. If we’re operating under the hypothesis that the interior of the line is more important than the tackles with regard to pass protection, which the experience data suggest, then we’d expect talent to matter more on the interior than it does at the tackles as well. It turns out that this is exactly the result we get. Whether looking at experience or talent, the interior seems to be the key to success.

I don’t know much about multivariate regression, but when you take both experience and talent of the interior of the offensive line into account for predicting sack percentage, an r-squared of 0.09 is produced. This is almost double the r-squareds produced by regressing experience and sack percentage and talent and sack percentage, and it suggests that these two factors work in tandem to determine the success of the offensive line regarding pass protection.

Unleashing the Dragon

A team’s tendency to throw deep, thus necessitating a longer drop and more time in the pocket, could be another influential factor governing sack percentage. I thought that yards per completion would be the best measure of a team’s tendency to throw deep, since yards per attempt could be equally as high for teams that throw quick, short passes, but complete a high percentage of them. Either way, we can look at both metrics.

click to see scatterplot of ypa and sack percentage

Correlations between passing depth metrics and sack percentage
Passing Depth Metric R-Squared P-Value Significance?
Yards per completion 0.01 0.31 Low
Yards per attempt 0.01 0.20 Low

These measures do not correlate particularly well with sack percentage. Yards per completion gives us the trend we’d expect – that sacks go up as yards per completion go up, but the explanatory value is weak as the p-value suggests insignificance. When doing a multivariate regression with yards per completion and interior line experience against sack percentage, the r-squared only rises to 0.056 from 0.05. It doesn’t add much explanatory value. Using YPA instead of yards per completion actually produces a trend where it appears that increased yards per attempt facilitate a decrease in sack percentage. That doesn’t make a lot of sense and the correlation and statistically insignificant anyway.

Quarterback Talent

 

4 star talents, 5 star smiles

A third possible factor governing sack percentage is the skill of the quarterback. Perhaps sacks are less a matter of how good the line is and more a matter of how good the QB is. To measure this I look at each BCS quarterback’s star rating and their Rivals rating (4.9-6.1) to see whether their high school talent correlates with how often they end up sacked. You can’t really use any college stats as a measure of their talent, because those can be directly influenced by the play of the offensive line, and we’re trying to isolate QB talent as a separate and independent variable here.

 

click to zoom

There’s really not much here. Whether you go by star ranking (2-5 stars) or by the more precise Rivals rating (4.9 – 6.1), there’s no significant relationship between a quarterback’s talent and his ability to remain upright. R-squareds for both metrics are <0.01, and p-values are 0.75 – 0.80. On an individual level, the skill of a single quarterback might help him avoid sacks, but taken broadly across all BCS schools, quarterback talent doesn’t seem to be a factor.

Depth

all 10 linemen on FSU's 2-deep are upperclassmen /Miami Herald

Probably the most common critique in the previous diary was that depth should be taken into account. You can do this different ways: average or total experience on the 2-deep, the oldest player at each position on the 2-deep, or the percentage of upperclassmen on the 2-deep. For this study I'm using the last of these definitions, the percentage of upperclassmen. I'm defining "upperclass" as students who have been on campus for two years prior to this season. So redshirt sophomores and true juniors are both considered "upperclassmen," while true sophomores are not. The graphs below show the trends for both the line as a whole and the interior of the line.

 

click to see all teams labeled - Duke also has an all-upperclass 2-deep

The correlation is unexpectedly poor. The graphs above show line depth both across the entire o-line and just the interior of the line. In both cases, the trend line suggests that the more upperclassmen you have, the more sacks you give up. This doesn't pass the common sense test, and r-squareds for both are low (0.01) and p-values are high (>0.30) implying that the correlation is not statistically significant. It doesn't appear to be a matter of defining "upperclassmen" either. If you run the same regression using average depth on the line, you get the same spurious results. While line depth might be an excuse for any given team, across the entire FBS the experience of your starters seems to matter much more than the experience on the entire depth chart.


Conclusions

Modest but significant. Despite using a completely different metric for o-line success, sack percentage instead of YPC, the conclusions of this study are eerily similar to the previous one. Let’s begin with the (hopefully) obvious caveats. Offensive line experience explains a modest, though significant, amount of the variation in sack percentage across all FBS schools. We’re talking about 5% of the variation here, so there are clearly a lot of other factors that go into determining how good a team is at protecting their quarterback.

In a way, this study is much less about Michigan than it is about college football in general. The success or failure in pass protection for a single team can often be explained by factors that are specific to that team. For instance, Devin Gardner is essentially the Ben Roethlisberger of college football, refusing to throw the ball away or to be tackled until approximately half the other teams defenders are draped all over him. This certainly contributes to Michigan’s high percentage of sacks, but it is a difficult variable to account for and measure across all of college football.

That being said, offensive line experience does stand out as a particularly salient characteristic for explaining a team’s sack percentage. Although we’d assume that experience at the tackle positions would be more important in the passing game, the results of this study suggest that once again the interior of the line is what matters most. In contrast to the previous study, the “weakest link” (i.e., the youngest interior linemen) is not as good of a predictor as the average experience of the interior of the line.

Taking a comparative approach by looking at experience alongside other potentially influential factors provides some context for how important experience actually is. The chart below plots each of the metrics I’ve looked at in this study along with their r-squared and p-values. 

Independent Variable Unit of Measurement Data Set R-Squared P-Value Significance?
Avg. O-Line Experience Years FBS 0.02 0.10 Marginal
Total O-Line Experience Starts FBS 0.01 0.28 Low
Avg. Tackle Experience Years FBS 0.01 0.67 Low
Avg. Interior Line Exp. Years FBS 0.05 0.02 High
Left Tackle Experience Years FBS <0.01 0.31 Low
Average O-Line Talent Rivals Stars FBS 0.01 0.30 Low
Average Tackle Talent Rivals Stars FBS <0.01 0.52 Low
Avg. Interior Line Talent Rivals Stars FBS 0.01 0.22 Low
Left Tackle Talent Rivals Stars FBS 0.01 0.32 Low
Average O-Line Talent Rivals Stars BCS 0.03 0.18 Low
Average Tackle Talent Rivals Stars BCS <0.01 0.64 Low
Avg. Interior Line Talent Rivals Stars BCS 0.05 0.07 Marginal
Left Tackle Talent Rivals Stars BCS <0.01 0.87 Low
Throwin' Deep A Yards per Completion FBS 0.01 0.31 Low
Throwin' Deep B Yards per Attempt FBS 0.01 0.20 Low
QB Talent A Rivals Stars BCS <0.01 0.80 Low
QB Talent B Rivals Rating BCS <0.01 0.78 Low
Total Line Depth Upperclassmen % FBS 0.01 0.33 Low
Interior Line Depth Upperclassmen % FBS 0.01 0.43 Low

This provides some much needed perspective. IME this really highlights the importance of experience, and especially the importance of the interior line. Interior line experience correlates more strongly with sack percentage than does a team’s tendency to throw the ball deep (at least when measured by yards per completion), and it serves as a better predictor than average talent of an entire offensive line (at least when measured by star ranking). This is really interesting! If I was a betting wizard, and I am, I would have bet on average o-line talent being a much better predictor of success than experience. Also, although the experience of the starting interior linemen does correlate significantly with sack percentage, depth along the offensive line does not.

The factor that comes closest to interior line experience in terms of predicting sack percentage is the talent of the interior of the line. This should strengthen our confidence in the conclusion that the interior line is the more crucial than the tackles in keeping the quarterback clean. As previously touched upon, when we combine interior line experience and interior line talent as two predictors of sack percentage and run a multiple regression, the r-squared returned is approximately 0.09. This isn’t huge by any means, but it serves as a better measure of success in pass protection than any single metric we’ve looked at so far.

Why don’t the best linemen play on the interior? This was one of the main questions raised during the last study, and the assumption was that teams play their best lineman at tackle in order to protect their quarterback. This study suggests that the interior of the line is more influential in accomplishing that task. There are a couple potential explanations. QB injuries and fumbles could still be most common from blind side hits, and team’s put their best guy there in order to mitigate these disasters. I haven’t tested this but I imagine it’s something that could be done statistically. It could also be that the best linemen play at left tackle because that’s the most important line position in the NFL, where one might assume that tackles matter more (hence their bloated salaries). If you look at the relationship between left tackle talent in the NFL (as measured by salary) and sack percentage, however, you get a pretty spurious correlation.

The line does trend up a bit suggesting that higher paid left tackles allow fewer sacks, but the r-squared is only 0.01 and the p-value is 0.66. It appears that left tackles aren’t much better at predicting pass protection success in the NFL than they are in college. (This is obviously more complicated than I’ve presented here. For example, teams could spend more on left tackles to fix problems that are inherent in the rest of their line or in their offensive system, thus producing a trend where teams with higher paid left tackles actually have higher sack percentages. This study is about college though, so I’m just leaving this for now).

I guess I just don’t know, man. The argument about protecting the quarterback from taking blind side hits makes intuitive sense to me, but the data all suggest that the interior is a more important factor in pass protection. If anyone’s got any quantitative study on why it makes more sense to play your best lineman at left tackle, or that tackles are, in fact, more crucial to pass protection, I’d be interested to see it.

What does this mean for Michigan? Let’s reemphasize that the experience data explain a relatively small proportion of the variance in sack percentage and that for any single team, and for any given team, team-specific explanations probably outweigh the statistical ones suggested by youth or talent. That being said, Michigan is very young where it appears to matter most. They are, however, talented – at least according to their star rankings. If these players develop at an average rate, then our line should make some serious strides by the time it’s full of talented upperclassmen on the interior. This is somewhat disheartening for this year but should provide some hope for the future.

This hope, of course, is based on the expectation of reasonable player development. We don’t need the best coaches in the world, since they tend to recruit already talented and physically gifted players, but we do need to develop those players on par with the rest of college football. I have no idea whether Borges and Funk have histories of successful o-line development, but it might be something worth looking into. The potential is there, however, to have a very successful o-line with regard to both the running and passing game as these kids become upperclassmen.

This study isn’t meant to indict or absolve any of the coaches, and it really does say more about college football as a whole than Michigan in particular. I do, however, think it’s interesting to see how Michigan’s production compares to other schools given a specific level of experience. We’re pretty far below the trend line even when experience on the interior is accounted for, and especially when talent along the interior line is taken into consideration. I think that Devin Gardner’s inner Ben Roethlisbergerness has something to do with this, as does Al’s predilection for two man routes where both receivers go deep. Experience, especially on the interior, does seem to play a role though. I don’t think it’s really possible to accurately assign percentages of blame (it’s really just a guessing game), but until we get that sack percentage out of the FBS basement, rest assured, there will be plenty of blame to go around.

 

the past                                         the future (let's hope)

Happy MGoThanksgiving to all!

the 36th image that comes up when you google "turkey football"

yes, I am taking this as a sign we beat Ohio

Comments

massblue

November 28th, 2013 at 2:55 PM ^

Unfortunately, they all suffer from multiple sample selection problems.  In your case, you have a sever case of survivorship bias.  Actually, I am surprised that you are not getting stronger results given the significant amount of bias that you have.  Let me explain, if you look at mutual fund managers who have been around 15 years or more, they have outperformed their peers by about 3% per year.  So, do these fund managers get smarter as they get older? No. These are the ones who survived either because they were either good or lucky.  We cannot tell.

In your case and many other such studies I have seen here, you are looking at OL members who have 3-4 years of experience.  They fact that they have played so long, it means they have survived competition from lower classmen. They could be good to begin with or could've gotten better through time.  We do not know.

One way to correct for suvivorship bias in this case would be to look the experienced OL that have good performance and see what the peformance was when the same players were playing with 1-2 years of experience. If they have the same performance as their peers, but have better performance in years 3-4 you can claim (rather weakly) that experience pays off and you might be able to measure its marginal impact as well.

Also, looking at the charts, I see most of the results driven by one or two outliers.  You have to adjust the results for those.

 

Gandalf the Maize

November 28th, 2013 at 4:00 PM ^

I think I get what you're saying. Basically that older o-linemen are better not because they are older, but because they are just more talented or lucky (and thus fend of younger guys). I guess it's possible that this influences the results to some extent. But shouldn't that survivorship bias show up at the tackle position as well as the interior? 

Also is there evidence showing that mutual fund managers don't get better with experience? It seems logical to me that OL would get better with experience, and so if mutal fund managers don't and OL do, then I'm not sure the analogy applies.

And it's certainly possible that outliers drive some of the trends. I would tend to doubt it with the fairly large sample sizes. But it's possible. 

Honestly, I've beaten the dead horse about as much as I care to. But I'd be interested in seeing the results if you carry out your proposed study. Not sure how you would measure "how good" an OL was after year 1 or 2 and then "how good" they are now. 

 

Yeoman

November 28th, 2013 at 5:01 PM ^

In particular, I don't see the parallel with money managers, where everyone's tossed in for year one and those that fail are discarded.

That's not how it works with college football players. They're given a four-year scholarship and the majority stick around for those four years (ok, there's some adjusting for redshirts and the occasional early departure for the draft). Each of those four years they compete for starting positions. If there wasn't any improvement from year to year, the freshmen would be just as likely to win those competitions as the upperclassmen. That's not the case; freshman starters are relatively rare.

I suppose there are a few players each year that drop out of football altogether, or move down to a lower division, because they've despaired of ever seeing the field at this level. But those players never see the light of day in these studies. They're irrelevant.

Maybe that five-star chart I did confused matters--it might make someone think starting in year 1 or 2 is common. It's only common among the very top recruits; the much more common trajectory is to work one's way up the depth chart, finally starting in year 3 or 4 if ever at all. We don't see their performance in year 1--how do you propose we evaluate it, other than to assume it probably wasn't as good because if it was they would have been starting?

 

massblue

November 28th, 2013 at 9:33 PM ^

has more severe case of survivorship bias than mutual fund managers because it takes place with replacement.  Failing mutual fund managers are not, typically, replaced with another one -- the fund is closed.  In case of a OL players they are replaced by another one.  So the bias is more severe.

Suppose you are a 1st year OL and you are good enough to start your freshman year.  However, you are replaced with another freshman next year, because he is better than you.  In fact, he is so good that he survives the four year period.  That line is good because of experience or because it had good players to being with?  The only way you can figure out the importance of experience is to have an OL that is made up of senior players not because of their abilities but because the coach wants to have senior OL and then compare the performance of this team with the same team 4 years ago when they are started as freshmen.

Now, I think we can all agree that there is at least some physical development with age that its benefit cannot be denied.  But the question is the importance of age or experience? Given the bias that I described and the fact that regression results are so weak, it seems that experience per se is not that big a factor.  A bad young line will be a slightly less bad line 3 years later.

Yeoman

November 29th, 2013 at 12:33 AM ^

...if the scenario you describe was at all common. But I don't think I saw a single example of someone starting as a freshman and then being replaced later. Starters being replaced for reasons other than injury are fairly rare; the usual career progression is that, barring injury,  once you establish yourself as a starter you typically remain one until you graduate (or are drafted). That's consistent with the idea that, at least during the portion of a career we're dealing with here, players tend to improve with age. (Obviously there's a point later on when skills begin to decline, but that's a postgraduate problem.)

It occurs to me that there's a strong parallel to a sabermetric controversy from thirty years ago. Early studies of the typical trajectory of a player's career done by Pete Palmer seemed to show that player performance was essentially constant from age 23 to 40. His studies averaged all players of a given age, then compared those averages.

The problem with that approach was that at the extremes--say age 18-20 at one end, 38 and up at the other end--only very exceptional players were included in the sample. At the peak ages, near 28 or 29 as it turned out, the sample had expanded to include a lot of players who weren't nearly good enough to play at 19, or to stick around after their skills had begun to decline. For age 18 the sample basically consisted of Robin Yount, whose batting line at 18 looked a lot like the average major leaguer at 28...but why were we comparing Robin Yount to the average major leaguer, instead of comparing Robin Yount to his peers?

The result was to wash out the effects of age. A properly-designed study, one that used the same set of players throughout (sometimes including examination of minor league statistics once it was understood that with proper park and league adjustments those were every bit as good as major league statistics for assessing performance) showed a much stronger effect. The first of these was published by Bill James in the first commercially-available Abstract; there have been many since. His particular findings on peak age have been refined but the overturning of Palmer's finding of little age effect at all has been confirmed again and again.

There's a very brief discussion of this in section 9 of this pape r;  (which doesn't even mention Palmer because by that time the refutation was so final it was no longer worth a comment); I wish I could find James's own article on line (it was in the 1982 Baseball Abstract if anyone actually feels like looking it up--as I recall it had a very nice visual representation of what I'm trying to describe and if I could find that diagram I'd link to it).

I suspect there is a very real problem with the sample here, one that you've identified when you point out that an ideal study would be longitudinal within a given group of players. The diaries so far have all suffered from the same flaw as those early baseball studies--the only freshmen included in the study are those that have already won jobs long before their skills and bodies have matured. If I'm right, a properly designed study (there may, unfortunately, be no such thing because of the difficulties of measuring the individual performance of an offensive linemen) would show the effect of age to be much greater than what we've been seeing in these studies.

Gandalf the Maize

November 29th, 2013 at 2:56 AM ^

I'm trying to think through this, and I'm still not sure I quite have my head wrapped around it. I mean I get that survivorship exists. But my initial reaction though is that this would actually decrease the r-squared that we get when we look at experience and success, meaning that there are probably a good number of freshman and sophomores in there because they're good and not because they have to be. If they're good they should help produce more YPC or fewer sacks, and if younger people are producing fewer sacks or more YPC, then that should lessen the correlation that we see.

Interesting analogy to the Bill James stuff. I think that's essentially what we see with a lot of the freshmen who are starting. Reflecting on this, I think that perhaps the best metric for experience along the o-line might be the oldest person on the 2-deep at a given position. So if Bosch is backed up by Bryant, the position gets a 2.5 (RS Soph) instead of 1.0 (FR). 

Yeoman

November 30th, 2013 at 11:05 AM ^

Maybe the reason Massblue and I are having trouble understanding each other is that we're looking at different parts of the study.

I was thinking about the "years" metric for experience, and there the selection process is favorable to the less-experienced group for the reasons I explained.

But for the "starts" metric it's the reverse--the only players that ever get to 30-40 starts are the ones so good they were starting right out of the box, and the selection process favors the more-experienced groups. Weaker players have graduated before they make it into these groups and Massblue's suggestion is on point.

Yeoman

November 29th, 2013 at 9:06 AM ^

"Attrition" isn't the relevant concept here. I think you're imagining a situation where you start with some sample group in year 1 and players drop out over time.

That's not what this is. We have basically the same universe of players throughout, moving progressively from year 1 to year 4 or 5 and then graduating (or not). Each year those players compete, and the winners of the competitions become our sample.

Hardly anyone leaves the universe of players being considered until their time is done, and with rare exceptions those few that do weren't winners of the competitions and never entered any of our samples in the first place.

massblue

November 29th, 2013 at 7:54 PM ^

We just do not observe their performance.  That is, they are no longer part of the universe from which we select the sample. Would have been nice if no OL player ever stopped playing.  Then we would have data on all OL players and the impact of experience could be estimated.

Yeoman

November 29th, 2013 at 10:31 PM ^

Can you provide examples of this attrition you claim? Players that were starters in year one but then were passed later in their career by younger players?

There are 625 starting offensive linemen in FBS:

  • 25 first-year players
  • 90 second-year players
  • 158 third-year players
  • 193 fourth-year players
  • 159 fifth-year players

The year 5 class is somewhat smaller because not everyone redshirts; the first four classes are all roughly the same size. The percentage of starters increases dramatically with experience, especially over the first three years.

How do you explain this, if players do not improve significantly over time?

You've hit on something important here. But it's the early years that are underrepresented in the sample, and thus overrated because weaker players have been selected out. It's a sort of time-reversed attrition I suppose, if you want to look at it that way.

ca_prophet

November 28th, 2013 at 6:28 PM ^

One of the things this study assumes is that you run the five most competent linemen you have. At the start of the season, putting him at guard meant displacing Kalis/Glasgow, and starting Magnusen or Braden instead. That would pretty much mean we could never run right, and would likely have been even worse.

At this point, though, it's impossible to talk about this rationally. Logic and reason have left the blog, so the cause is whatever you want to believe and post about.

Yeoman

November 28th, 2013 at 6:59 PM ^

Schofield would move back to guard; Magnuson would start at a tackle. Maybe it was even tried at some point in practice? Who knows?

I don't know whether they would have put Schofield next to Lewan to have someplace to run effectively, or just move him inside on the right so there wouldn't be three raw guys side by side. Maybe the former, since they tried something similar with tackle-over.

Maybe it would have been better, maybe it would have been worse. Again, how could anyone know? I think we tried enough trial-and-error under fire this year; I don't regret missing this one.

MGoStrength

November 28th, 2013 at 5:53 PM ^

So, it seems we have established that the youth of our interior line is the root of our problem and the experience of our tackles can't help us that much.  With that in mind, will loosing two senior tackles and getting another year of experience for our interior guys be a help or a hindrence?  Basically what I want to know is will we still be dealing with this next year, 2 years from now, etc?  Assuming we run out Braden, Bosch, Glasgow, Kalis, and Mags next year, we will still have a young interior (Soph, RS Jr, RS Soph), but now also young tackles (RS Soph, RS Soph).  Does that equate into a step forward, more of the same, or worse yet a step backwards.  I can't imagine it could get any worse but I also have a hard time believing that line will anything better than average.

Yeoman

November 28th, 2013 at 6:47 PM ^

I did something on that last week and I think Seth quoted it on the front page. All else being equal, just looking at age across the line, it looks like we'd expect to move up 10-20 spots nationally on offense. That'd be Michigan State this year, or maybe Iowa.

Livable, hopefully. Still a young line though.

MGoStrength

November 28th, 2013 at 7:10 PM ^

I really feel like Gardner is a little shell shocked this year.  With an experienced, older QB in Gardner that can use his feet (hopefully with a little better strategy in the pocket), a 5-star RB in Green, and Funchess, I think we have enough weapons on offense that if we can simply establish an average offensive line we can be good enough to be a quality team that doesn't fear playing the OSU's and MSU's defenses of the world.

Gandalf the Maize

November 29th, 2013 at 2:49 AM ^

Yeah, all else being equal, I think this would say would should improve on the line despite losing some good tackles. There are enough Michigan-specific factors that it's really impossible to say with any certainty, but I'd be the experience on the interior helps us more than losing the tackles hurts us. One can hope, right?

ca_prophet

November 28th, 2013 at 6:40 PM ^

Every team in the NFL has a DE or LB that can threaten the edge - they have the 30-40 best edge rushers in the world, in fact. Most college teams don't have that, so a terrific LT isn't as necessary until you get to the Clowney games.

The other thing is that salary is a poor proxy for talent, because salaries rise over time: a 1st round rookie and a just-before-free-agency veteran will have similar salaries. Years of experience doesn't quite work either, since you get worse after a certain point. Maybe by experience "bins" - 0-3, 4-6, 7-8, 9+ or something?

Gandalf the Maize

November 29th, 2013 at 3:02 AM ^

Yeah, I agree that salary's a pretty poor proxy. Tough to measure "talent" though without it being tautological. I was just trying to think of ways to to explain teams' best OLs playing LT, and I thought one reason might be because that is what preps them to become the best pros (i.e., LT is the most important position in the pros).

I was thinking of the Lawrence Taylor thing too, but any idea if there has ever been a quantitative study on where most QB injuries come from? I was really suprised that sacks tend to correlate the best with the interior of the line, so I wonder if a few anecdotal weak side hits have somewhat entrenched this as a fact. No idea how you'd prove it without some serious work, but it's interesting to think about.

ChiBlueBoy

November 29th, 2013 at 9:46 AM ^

Here's a great article on centers in the NFL: http://www.slate.com/articles/sports/sports_nut/2007/01/a_center_shall_… (sorry, iPad won't embed).

A couple thoughts: LTs are your best "athletes" because how we normally think of "athlete" speaks to what is required for a tackle--agility, length, etc. Interior line is more critical but what we think of as important there: figuring out assignments, low center of gravity, IQ, etc, don't strike us as being as rare or athletic.

My theory is that there's a bias and LTs are over-valued and IL (especially centers) are undervalued.

Yeoman

November 28th, 2013 at 6:52 PM ^

That experience doesn't matter a lot at a position doesn't mean that the position itself isn't important. I think maybe the reason experience matters less at tackle is that there are physical requirements at the position--size, enough lateral quickness to protect the edge on pass protection--that outweigh experience to some extent. You've got to have that size and quickness and if it's a younger player that has it, he has to play even if he hasn't mastered the finer points yet.

That would explain NFL salaries at tackle, especially left tackle where you're protecting the quarterback's backside and you don't have a tight end next to you. Players that fit the physical requirements are rare and you have to pay to get them.

newtopos

November 28th, 2013 at 7:09 PM ^

Doesn't surprise me to see Indiana, with OL Coach Frey, well above the trendline in all of your charts.  I hope we have an excellent OL Coach and OC next year to coach up all the talent we have on our roster.

Yeoman

November 29th, 2013 at 9:15 AM ^

You're assuming we would have the same talent level if we ran that offense.

I suspect that isn't true. I think one of the reasons we've had more success lately on the recruiting trail, especially on the offensive line, is the promise of a return to a style of offense similar to what players expect to see in the pros. Linemen of that quality aren't as likely to want to spend four or five years doing pass protection on an endless series of quck throws--they want to push people around. And if you look at the linemen going high in the NFL draft the last few years you see a whole lot of Alabama, Wisconsin, Stanford.

UMgradMSUdad

November 30th, 2013 at 6:45 AM ^

Kevin Wilson is the one responsible for the tempo.  He was OC at Oklahoma before he took the job at Indiana. At Oklahoma, it all goes back to Bob Stoops hiring Mike Leach as his OC.  When Stoops was DC at Florida, Leach was OC at Kentucky and created the offense that Stoops has said gave him the greatest difficulty preparing for. Oklahoma has been running variations of the offense Leach installed ever since.

MichiganMan24

November 28th, 2013 at 7:32 PM ^

Interested to see whether the experience/talent of the primary running back has any correlation with qb protection. My guess would be that a more experienced/talented back means the team hypothetically has a better running game and keeps the defense from teeing off on pass rushing. Regardless, this was a really interesting diary. Thanks for taking the time to analyze all this stuff

Gandalf the Maize

November 29th, 2013 at 3:14 AM ^

No problem, glad you enjoyed it! I agree that it'd be useful to look at RB experience/talent, especially with the way our RBs seem to get blown up running and passing the ball. 

Fitz gets destroyed in pass protection and he's a senior. But so do the young guys and they're, well, young. I guess that's why the large FBS-wide analyses are useful, since they can help point a general finger towards either youth, talent, or bad coaching being the problem.

hfhmilkman

November 29th, 2013 at 10:35 PM ^

Greetings.  Here is my explaination for possibly why the tackles appear to not have an impact on sack rates.  The basic premise of most coaches in any sport is to put their players in a position to succeed and not fail by utilizing their strengths and covering their weakness.  Thus a smart coach who knows he has an inexperienced line will build an offense to minimize sacks.  For example a coaching staff may shorten routes in anticipation that a longer route is not possible.  Thus a team with a great passing offense and great Oline might have the same sack percentage.  In fact the great passing offense might have inferior sack numbers because the experience and talent of passing the ball deep generates the greatest return.  Using an NFL example I would look at some of Aaron Rogers years where Green Bay was one of the leaders in sacks allowed.  Yet it was worth it because of the YPA.

To really determine the impact of tackle experience I believe you need to look at YPA.  You would count the negative yardage of a sack.  If you have the exact data you would also want to account for offensive holds.  Flip side defensive pass interference and defensive holding would be accounted as positive yardage.  Lastly you would want to account for the conferences YPA.  I would be surprised if there was no correlation between YPA and tackle experience. 

Overall, I would not even bother with looking at running verses passing as some coaches have an emphasis on  style anyway.   Just figure out yards per play and see what happens.  I find it really hard to believe that every coach in football has been blatently wrong in searching for a great left tackle when the data suggests I can throw the water boy there and not impact by offense.