Using UFR to analyze offensive play-calling and outcomes

Submitted by EastCoast_Wolv… on October 10th, 2019 at 2:49 PM

I thought about calling this Diary "What the hell are you doing Josh Gattis?" but that seemed like it might be a little harsh. I actually started this out as a way to avoid BPONE and hopefully find a bright light in Gattis's play-calling, but the more I dug the more I fell into the BPONE. If you're looking for the optimistic take, it's that Michigan is actually quite good at running a spread offense. They're just not actually running a spread offense all that often.

Full disclosure: I'm a researcher and data analyst but have never coached football. So this is coming from a purely analytics stand-point.

My analysis here is based on the premise that Josh Gattis and Jim Harbaugh ultimately want this offense to be a spread offense. But given the lack of offensive success and discussion of transition costs, I wondered 1) How much has Michigan been running a spread offense? and 2) How well have they been running a spread offense?

To do this I scraped the play-by-play tables from Brian's UFR film analysis, in particular focusing on the personnel on the field, play-call, and outcome.

There are lots of ways that spread offenses try to put get their athletes in space, but for the first part of this analysis I decided to focus on two fundamental aspects of spread offenses (if you have suggestions for others I'm happy to include them):

1. Percentage of non-goal-line plays with 3+ WRs
2. Percentage of non-goal-line plays that utilize a read option, QB run, or run-pass option

The figure shown below shows the percentage of total plays for run plays (left panel) and pass plays (right panel) where Michigan used a spread concept with either less than 3 WRs on the field (maize bars) or 3+ WRs on the field (blue bars). Here I'm counting a play as using a spread concept if it includes at least one of the keywords "RPO", "read", "option", "QB", "keeper", "waggle", or "boot".

Michigan does actually use spread personnel quite frequently (67% of the time), but the interesting thing to focus on here is that they are only using spread concepts (rpo, read-option, waggle, etc.) 15% of the time! If we drill down a little further, they're only running RPOs on 8% of plays, which is especially crazy when you look at a team like Illinois, who Seth just charted in Illinois' FFFF as running RPOs 26% of the time.

So Michigan is putting personnel on the field in order to spread the defense out, but not using play concepts that are designed to exploit that fact.

If you look at the yards/play, success rate, and percentage of plays that gain at least 10 yards, you'll also see that Michigan is a much better offense when they have 3+ WRs on the field AND incorporate actual spread concepts into their play design.

Michigan is WAY better at passing the ball when they spread the field with 3 WRs and use RPOs, waggles, or boot legs. They're also better at running the ball with 3 WRs on the field, a stat that I'm sure makes Harbaugh shed a tear. Spread concepts don't really help the running game that much, but that is mostly because the QB hands the ball off on the vast majority of read-options, which negates any advantage spread concepts give the running game.

Don't believe me? For the next analysis, I broke down Michigan's running game into three types of plays: plays with a  designed QB run where the QB actually keeps the ball (plays with keywords "QB", "keeper", "read", or "option" where Patterson, McCaffrey, or Milton were the ball-carriers; maize bars below), plays where a RB ran the ball that came less than 10 plays after a designed QB run (blue bars below), and RB carries where there wasn't a recent QB run (gray bars below).

First, Michigan QBs have only run the ball 22 times all year in non goal-line situations, and only 14 of those plays have been designed QB runs (the other 8 were QB scrambles).

Those 14 designed QB runs (maize bars below) have actually been pretty effective on their own (5.5 yards/play, 39% success rate, 21% gained at least 10 yards), but what is more important is what they've done for the rest of the running game. Michigan has averaged 6 yards/play on the 39 RB carries that happened less than 10 plays after a designed QB run (blue bars below), with a 43% success rate and 17% gaining at least 10 yards, which is way better than RB carries that didn't follow a QB run (gray bars below), which only got 4.4 yards/play with a 36% success rate and 15% gaining at least 10 yards. 

What does this mean for the rest of the season?

So what can we take away from this? For starters, Michigan hasn't done a good job actually running a spread offense. But, they're actually quite good at running spread plays, and doing things like running the ball with your QB make the rest of the offense better too. Michigan only has to have the QBs keep 2-3 more times per game than they currently are to keep the defense honest. So the optimistic take is there are a number of relatively small changes that Michigan could do that would dramatically improve the offense. But, it's already week 6 of the regular season, so why haven't they done these things? Hopefully it's true that they weren't running the ball with Patterson because he was hurt. And hopefully as the season progresses they can implement more spread concepts into their offense. How much you believe that will happen depends on how deep in the BPONE you are.

 

 

 

 

Comments

smwilliams

October 10th, 2019 at 4:46 PM ^

Fascinating because it’s got me intrigued by the cause rather than the outcome. I can’t imagine Gattis is wavering on his desire to run a true spread which means either:

A) Harbaugh gave him the keys the way a father gives his son the keys - with a bunch of caveats

or

B) This is the cost of the transition. They can’t run a lot of spread concepts because the players don’t really know what they should be doing on them.

It does, however, reinforce my belief that if they don’t want to run a ton of RPOs, that getting Patterson out of the pocket could be very beneficial. I get it, rollouts cut down on the field and make you easier to defend. But, if you have a QB that’s struggling with his reads and identifying open receivers, it makes the most sense. It also can help lessen the pressure on the OL. 

Ultimately, I do think the run game isn’t the fault of the line or the RBs. If your play design is based on having the defense react to a potential QB run, and you never run the QB, it’s gonna be hard.

EastCoast_Wolv…

October 10th, 2019 at 5:04 PM ^

Yeah I'm not sure which of those options it is either. I also think the combination of QB injuries and QB fumbles probably jointly reduced the overall number of QB runs. Transition costs could also be hurting the use of RPOs. One of the most disappointing things about the Iowa game to me was that I didn't see any of the new stuff they tested out in the Rutgers game. The waggle plays and stretch plays were reasonably effective in the Rutgers game (Rutgers caveat of course) but they didn't even really try them in the Iowa game (UFR only shows 1 waggle and 1 stretch play). It felt like that was Michigan trying to add spread elements to replace RPOs and read-options that attack the perimeter of the field, and then they ran almost none of that. So I have no idea what the Iowa gameplan was, or how it related to past gameplans.

DrMantisToboggan

October 10th, 2019 at 5:59 PM ^

Cool stuff. 

I'd like to see the comparison of frequency of option plays - both RPOs and option runs - to the best offenses right now.

I'd guess OSU, Penn State, Alabama are much closer to Michigan in terms of RPO frequency than they are to Illinois. OSU actually is probably even less frequent than Michigan is. Running RPOs 26% of the time is a crazy high number.

EastCoast_Wolv…

October 10th, 2019 at 6:18 PM ^

That's a very good question, and a good point that Illinois might not be the team we want to emulate haha.

What percentage of the time do you think a team should be running RPOs? 

It's not as easy to find play-calling data for other teams, and Brian almost never does a UFR for Ohio State, but I could at least look at the option play frequency for the past few Penn State teams.  

DrMantisToboggan

October 10th, 2019 at 11:18 PM ^

Oh I know, I say that knowing it's an impossible ask. Maybe some dedicated soul at The Athletic will watch 1000 hours of film and run these numbers in the offseason. 

I'd guess the best offenses nowadays run an RPO every 10 plays or so. Michigan at 8% might be tad low, but I would think anything over 15% is pretty high. Just a guess.

MGlobules

October 15th, 2019 at 9:53 AM ^

I think this conversation gets at the nub of the thing. If we assume that Shea was pretty badly hurt in the first game, some of these stats may make better sense. Yes, we have won with considerably less panache than hoped for, but we have won (Wisconsin wasn't in the freaking cards). If they CAN use more of the new offense in the upcoming serious tilts, with a greater element of surprise, there would be silver linings.

We're not beating OSU, and probably not Wisconsin again, but PSU and ND should be games where we have decent chances. Clearly, Shea has to be healthy, because they obviously don't see Dylan managing whole games or extended series.

Mongo

October 10th, 2019 at 6:56 PM ^

The OL is not any where close to supporting the Gattis read option run plays.  Also, the WRs are whiffing on blocks when the OL does its job.  Compounding those issues is Charbonnet and Wilson are not 100%.  The lack of run game is killing us. 

This lack of "spread" execution is not 100% on Shea like fans believe.  Could he be better ?  Heck yes, but the lack of execution is a team thing.  No QB could bail out the errors being made across the board.  

Each guy has to take responsibility and do their jobs, 100% go on every play.  Are they close?  Hard for fans to see but the coaches body language is positive.  

jbuch002

October 11th, 2019 at 8:33 AM ^

I feel confident that the applicable coaching staff have these kinds of analytics. After all, M has one of the deepest support staffs - including analysts that do this kind of work - in CFB.

The analysis is interesting, though. I would tend to go with (B) as well. I've also come to the conclusion that this really isn't the ZR/RPO offense we expected Gattis would run. It still has a lot of IZ/OZ Power concepts in it - a hybrid, I guess. 

We probably should have understood this when, over the summer, Gattis in the interviews he had, was pretty clear that his offense wasn't going to abandon the things that M did well in 2018.

I would add when considering "transition costs" that if you compare how M has performed in this circumstance to how other teams going through similar transitions have performed, specifically LSU, you can argue this coaching staff hasn't excelled. 

Ryan Day at osu is a bit different but, nonetheless, osu's offense has excelled in incorporating the offense he is running and it is a departure from ufm's offense - and of course, Day has had a prior year to work on this. You have to wonder though, why is M's offense so Janky?    

EastCoast_Wolv…

October 11th, 2019 at 11:24 AM ^

Mostly a sample size thing at this point. Once you exclude penalties and goal-line plays, there are only ~290 offensive plays. That might sound like a lot, but when you start to separate out plays based on 2, 3, or 4 different variables, there are lots of bins I'm looking at that have only 10-15 plays in them. Probably need 2-3 more games before you could start thinking about significance testing.

Leatherstocking Blue

October 11th, 2019 at 8:35 AM ^

Really great content. Thank you for the effort and analysis. It seems like Michigan has left 1-2 touchdowns on the field per game with missed receivers and turnovers - which I don't believe is on Gattis. Hit the open guys and the run game opens up and suddenly were talking about how we have an offense that can move the ball. Maybe not a playoff team, but one that is competitive in all games.

MadMatt

October 11th, 2019 at 10:55 AM ^

Between Harbaugh and Gattis, one of these guys is telling Shea not to pull the ball so much. The other needs to have a "come to Jesus" talk with him about the spread needing the credible threat of a QB run to be successful. If you want to transition to a new offense, you need to actually TRANSITION TO A NEW OFFENSE! Yes, I'm sure they both know that intellectually, but it's not reflected in the offense's actual behavior.

Hab

October 11th, 2019 at 12:42 PM ^

Every part of your analysis screams to me that we are intentionally hiding our spread packages, either to keep things off film because they're great, or to keep them off film because they expose a weakness. 

BPONE dictates I assume the latter.  But fuck the BPONE, I shall assume the former. 

The real conclusion for me is that this confirms that this is a transitional year and that, even if we are hiding the good things, it is because there isn't much to back it up once it's found out... yet.

Thanks for the great insights.

EastCoast_Wolv…

October 11th, 2019 at 2:02 PM ^

I mean those are possible, but I don't think likely. The spread stuff that we're talking about is basic stuff, and it's not like they're not running it at all, they're just not running it enough. And when you don't use that stuff when you can't move the ball and you're tied with Army in the 4th quarter, getting blasted by Wisconsin, or in a one score game with Iowa in the 4th quarter, what are you really saving that stuff for?

I also am skeptical that they aren't running it because it exposes a weakness. The numbers shown above suggest that they are actually quite good at running spread plays. What weakness are you exposing if you're getting 8+ yards/play on spread passes and getting 5.5+ yards/play on QB runs or RB carries that followed a QB run? Michigan's averaging almost 1.5 yards/play MORE on spread-style passes and QB runs than when they try to run non-spread stuff.

I really hope the answer is injury, fumbles, and transition costs led them to avoid spread stuff, but that healthier QB, slowly improving confidence in Shea not fumbling, and more overall comfort in the offense will lead to improvements over the next few weeks. Good news is Illinois' defense sucks so they have another week to fine tune the offense. Bad news is Penn State is the 15th ranked defense according to FPI and has held three (admittedly mediocre) Power 5 offenses in Pitt, Maryland, and Purdue to a combined 17 points.

Sambojangles

October 11th, 2019 at 2:03 PM ^

The Army and Iowa games were within one score the entire game. Wisconsin was a loss. I can't believe that they're not using as many plays as they can to win. I think the problem is more likely that the plays we're not currently running are the ones that the staff doesn't trust the team to execute based on what the see in practice. To a certain extent I think this happens every season, but it feels like the learning curve is longer and steeper than usual with the changes that happened.

EastCoast_Wolv…

October 11th, 2019 at 2:48 PM ^

Agreed that the learning curve is longer and steeper than I would have hoped for. But to me this analysis shows that Michigan is successfully running some subset of spread plays 7-9 times per game with decent success. So I'm not sure why you couldn't just run those plays 12-14 times per game instead. If every week they made sure they were able to effectively run 1-2 RPOs, 1-2 arc read plays, 1-2 waggle plays, and the speed option play they've already used multiple times, they would still be able to utilize spread plays way more often than they currently are. I mean you don't want to use the same play too frequently, but you should be able to run it at least twice before the defense adjusts.

Blue Middle

October 13th, 2019 at 4:15 PM ^

This is good stuff but has some serious error sources. Waggles and boots are NOT spread concepts. They are extremely old, very flexible concepts that have been part of pro style offenses for almost as long as the forward pass has been around. Michigan has been running naked bootlegs as a staple far longer than I’ve been alive. That said, plays that move the QB or threaten to move him appear to be far better for us, which is no surprise since Shea can’t seem to run through progressions in the pocket and the run game is predicated on his threat to run. 
What is much more interesting is the formation (WR) data. The good news is that injuries will likely continue to force us into 11 personnel. 

All that said, there are plenty of spread offenses that aren’t using their QB move plays to create yards. 
We just aren’t that good at this offense yet. Hopefully we get there.