On Turnovers and Randomness

(tl;dr? Skip to the Conclusion at the bottom)

After a lot of discussion on this site about how random turnovers are, I decided to look at them in more detail. My hypothesis was that, while turnovers as a whole may appear very random, individual components of turnovers might be much less random. For example, as has been discussed before, once a fumble is on the ground it appears to be very random who recovers it. But what if causing a fumble is not random at all? The randomness of recovering the fumble might still obscure that fact if you only look at turnover margin.

I decided to look at five components of turnover margin: interceptions gained, interceptions lost, fumbles when on offense, fumbles when on defense, and fumble recovery rate.

Analysis

I used whole-year statistics and compared the change from one season to the previous, using a total of 6 seasons worth of data. In college football there are, of course, many factors that change from year-to-year, but if there’s very little luck involved, I would still expect to see a decent correlation from year to the next. For simplicity, I assumed a linear relationship between stats from one year and the following year, so the analysis used linear regression, a simple but reasonably robust model.

The R-Squared statistics (simply the correlation squared) gives us an understanding of how much the variability is accounted for by our model. In simpler terms: how much is success in the stat from one year accounted for by the success from the year before?

All data was obtained from www.teamrankings.com. Data was always rounded to the nearest tenth by the source—because interception and fumble frequency are fairly low, this rounding may have a larger-than-ideal impact on the results; if anything, I would expect that should in general impact the results negatively (make them appear more random).

Results

I next look at those five components of turnover margin: interceptions gained, interceptions lost, fumbles when on offensive, fumbles when on defense, and fumble recovery rate. The R-Squared values are:
Interceptions Gained: 0.057
Interceptions Lost: 0.049
Fumbles on Offense: 0.016
Fumbles on Defense: 0.001
Fumble Recovery Rate: 0.003 (the correlation is actually negative)

The first result that jumps out at us is that interceptions appear much more repeatable/less random than fumbles. Interceptions gained and lost in the previous year account for about 5% of the success the following year, compared to under 2% for the number of fumbles on offense. Fumbles on defense and fumble recovery rate appear almost completely random.

(For those still unconvinced that fumble recovery rate is almost completely random, the best team from each year did decently at best the following year--Michigan was 1st in 2006 but 47th in 2007, which tied for the best performance by a returning #1.)

Providing Context by Analyzing the Optimum

The previous analysis tells us a lot about how repeatable results from one year are, but it doesn’t really tell us about how much is skill vs luck: after all, stats in a college sport ought to vary a lot from year-to-year: there is player development, new players, potential coaching changes, different strengths of schedule, and many other factors.

To provide an optimal baseline, I also looked at offensive and defensive yards per game. Intuitively, that’s a statistic that should be very greatly influenced by skill (though there are certainly amble sources of other influence, including luck). This will provide context by helping us understand what kind of change we should expect to see due to year-to-year variance (player or coaching changes, player development, changes in strength of schedule, etc.) instead of due to randomness.

For offensive yards per game, the R-Squared value is 0.243, while for defensive yards given up per game, the R-Squared value is 0.275.

Roughly 25% of success being accounted for by the previous year’s success is not very high, but that’s not a surprise—again, there is lots of change from one year to another. What this is helpful for is to provide context: even if turnovers are very skill-based, we would still only expect an R-Squared of .25.

Conclusion

There are two ways to view the turnover margin numbers: the first is viewing them in isolation. Even the best component of turnover margin, interceptions gained per game, is not very repeatable: success one year accounts for under 6% of success the following year.

The second way is to view them in comparison to the yards-per-game stats. With this perspective, interception rates on both sides of the ball are a little under one-quarter as repeatable as yardage. If we assume that yards-per-game is heavily impacted by skill, interception rates are likely fairly impacted by skill as well. I would hypothesize that within a given season, teams that are good with interceptions on either side of the ball will be likely to continue being good.

Fumbles seem to be much less skill-oriented. Fumbles lost is less than 1/15th as repeatable as yards-per-game. Fumbles forced on defense and the fumble recovery rate are almost completely random. (What this really says is that almost all teams are roughly equally good, not actually that there’s no skill in forcing or recovering fumbles.)

With total yardage, since 75% of success remains unaccounted for by the previous year’s success, we’d expect that it’s made up of two different things: randomness and other factors. If offensive yards per game is, indeed, not very random, that means outside factors (returning starters, returning coaches/schemes, different strengths of schedule) will have a large influence. This is important—we may be able to take some of these into account to improve the prediction we’d get based on just the previous year’s results.

Likewise, while interceptions are not very repeatable overall, they’re still about one-fourth as repeatable as our optimum. In a very rough estimate, we might then guess that outside factors also have one-fourth the strength with interceptions. Thus, if up to 75% of yardage success is outside factors, then up to 18% of success in interceptions is accounted for by those factors (this is very rough, since the factors may be quite different, or at least have different impact). That would leave roughly 76% of even the most skill-based category as random (100% - 6% based on previous year – 18% based on outside influences). The same rough calculation gives 5% of offensive fumbles based on outside factors, and 93% based on chance.

In summary, there is definitely some repeatability in three of the five turnover factors, but even the best of those still has under 6% repeatability, and by a very rough estimate, is still 76% random.

Implications for Michigan

In 2011, Michigan was 34th overall in turnover margin per game, with +0.4. That’s good, but not amazing, despite Michigan’s stellar fumble recovery rate.

There are three factors I’ve identified and tried to account for: repeatability based on the previous year, outside factors, and chance.

Statistical repeatability bodes poorly for the Wolverines, unfortunately: the two most repeatable categories were the two at which Michigan did worst: Michigan was 82nd and 89th in interceptions gained and interceptions lost, respectively. Michigan’s best two categories, fumbles on defense (28th) and fumble recovery rate (1st) are basically random. In the fifth category, fumbles on offense, Michigan was a decent 42nd.

The second factor is really a category: outside factors, which probably impact interceptions the most. This seems positive for the Wolverines: returning Denard and the defensive backs, plus a coaching staff (an outside factor that would have pulled last year’s number down). Michigan’s biggest loss is Junior Hemingway, who certainly bailed Michigan out a few times last year.

The last category is randomness, which appears to have a very large impact on even the most skilled category, and complete control over a couple of them, meaning any real prediction is fairly foolish. To be a little foolish, then, I’d guess that interception categories improve to above average (say low 40s), but overall turnover margin gets worse, dropping to the 50s. However, I have only slightly more confidence than I do when calling a coin toss.