SP+ Week 14: M #4, Georgia still #1
From Bill Connelly:
Yes, Georgia remains No. 1 after its demoralizing loss to Alabama in the SEC championship. That was all but guaranteed because of the Bulldogs' enormous advantage over the field heading into the weekend. The Dawgs were 5.5 points ahead of Ohio State, 7.3 ahead of Alabama, 9.2 ahead of Michigan and 11.5 ahead of Cincinnati a week ago, and this deep into the season one game simply isn't going to make that big of a difference.
As you see, however, the gap closed pretty significantly. The Dawgs' advantage over the field shrank by about three points against each top team, which is as big a shift as you'll see at this point in the year. And as you would expect, Alabama closed the gap on Ohio State too.
In résumé SP+, which looks at actual results vs. the average top-five team, the top 5 was Georgia, OSU, Alabama, Michigan, Cincinnati. In loss-adjusted SP+, the top 5 was Georgia, Alabama, Cincinnati, Michigan, OSU.
Top 10 below. Other top 50 B1G teams: OSU 2, Wisc 9, PSU 16, Minny 20, Iowa 32, MSU 35, Nebraska 37, Purdue 40.
December 6th, 2021 at 8:48 AM ^
Well again, I'm not a math major and I really don't understand this formula, but I can see, and from watching a ton of football. Georgia's offense is not #3 and their defense is not #1. They have inflated stats based on SEC bias. They looked like a Don Brown defense getting exposed in that game.
December 6th, 2021 at 9:00 AM ^
Is it SEC bias or ain't played nobody? I'd lean toward the latter, but they still have a shit ton of talent and it'll be a tough game.
December 6th, 2021 at 9:20 AM ^
Yeah, I kinda doubt that Connelly built in some 'bias' in his strictly mathematical formula... But that 'ain't played nobody' thing is well represented!
December 6th, 2021 at 11:19 AM ^
The idea that bias can't be built into mathematical formulas...lol.
December 6th, 2021 at 11:36 AM ^
The point is it's very unlikely Connelly has incorporated an explicit weighting favoring different conferences. And furthermore, by the end of the season the algorithms for SP+ etc. have eliminated any dependency on preseason information.
Georgia is #1 because they had been utterly dominant until Saturday. Meanwhile teams like Alabama, UM, Cinci have struggled against mediocre teams. The catch is dominance against mediocre teams is not a perfect proxy for performance against elite teams, and that's what we saw Saturday.
December 6th, 2021 at 11:44 AM ^
Bill has come out and explicitly said there are some adjustments for conferences. Namely the Big Ten and SEC. Or even the AAC when compared to other G5 conferences. It's not a big enough adjustment to be the reason Georgia is #1 but it exists to an extent. Here's a tweet from him this year referencing conference adjustments for a weekly ranking.
December 6th, 2021 at 11:51 AM ^
Important to note that any conference adjustments, when done in service of tweaking a formula to make it more accurate/predictive, aren't really "bias" -- at least not nearly in the same sense that human polls/committees are susceptible to SEC talking points literally dictated to TV partners. Math that aspires to be accurate will not be biased by those talking points
December 6th, 2021 at 11:59 AM ^
I see there are some manual knobs, but his description of his tuning is adjusting conference strength to better match previous outcomes. It's just a results based correction. He's not subjectively saying B10 gets this many points and SEC gets this many.
"Basically, I craft the ratings as normal, then use them to 'project' each game that has already happened and look at the differences. B1G teams are overachieving those projections by >3 PPG, so each team gets a retro adjustment of sorts."
December 6th, 2021 at 9:27 AM ^
It's both. They haven't played anyone, but they(and Alabama) get credit as if they have due to SEC bias.
December 6th, 2021 at 10:50 AM ^
While I am on Team Fancystats, this glitch in the matrix has come up with Kenpom and some other models too. Gonzaga basketball, which is outstanding, but also has been #1 or #2 in kenpom for 5 of the past 6 years to date is the best example. When you dominate mediocre competition by extraordinary margins after overwhelming them athletically this can be modeled as superior performance relative to that of teams that win by smaller margins against strong competition. The performance doesn't seem to hold up as well when such teams go against similar caliber competition. This issue seems to be particularly true for teams with stifling defenses FWIW.
December 6th, 2021 at 11:49 AM ^
Right. I said something similar above. Dominance vs. mediocre teams will not necessarily to scale proportionally vs. elite teams in the real world. Maybe some years for Bama or Clemson it has, but not always.
Meanwhile, struggling against a couple mediocre teams (like Bama and Mich have) doesn't necessarily indicate poor performance against elite teams (Georgia and OSU). But what else is an algorithm supposed to do if the only data it has is dominance vs. mediocre teams?
December 6th, 2021 at 12:01 PM ^
While I agree that it's extremely difficult to compare Gonzaga to a good power-conference team, I don't think KenPom was necessarily wrong for making them #1, even if Gonzaga hasn't won a championship. (And they have made two championship games, so making them #2 is hardly a crime either.)
The NCAA basketball tournament is notoriously chaotic, with the best-team-during-the-season losing more often than not. If Pomeroy could accurately predict the winner of that tournament with any reliability, I'd ask if I could borrow his time machine.
December 6th, 2021 at 11:56 AM ^
Well that's the bias. They get credit for beating "good" SEC teams and a Clemson team that isn't any good. 6 SEC teams were ranked in the preseason polls. There aren't that many teams currently ranked.
December 6th, 2021 at 9:02 AM ^
It kind of sounds like you just watched one game
December 6th, 2021 at 9:27 AM ^
I’ve watched tons of their games and they’ve played absolutely no one. Majority of their quality opponents were at home. I’ll say it again their LB play is awful. A lot of it is cleaned up in the wash because of their DT’s but their ends aren’t that great either. They don’t appear to be 100 times better than anyone else.
December 6th, 2021 at 10:42 AM ^
Wake's offense is much better than UGAs. Agree they don't have the #3 offense. I'm all for stats and modeling, but something isn't right in the formula if they have the #3 offense.
December 6th, 2021 at 9:17 AM ^
I'm no sportsman analyst myself, but isn't the point of a statistical formula like SP+ to show how the teams compare independent from any kind of "eyeball assessment"? Meaning, that's SP+ was kinda created so people aren't left with "Well they don't look like number 3 to me!"
December 6th, 2021 at 11:20 AM ^
Double post.
December 6th, 2021 at 11:23 AM ^
But someone still has to create a formula for it and assign weights to various categories. There is inherent bias there. How much do you weigh yardage? How much scoring? How much do you weight for schedule strength? Do you cut off leads at a certain point to account for running up the score?
There aren't objectively correct answers to these. It's up to the creator to decide.
Our 2010 offense was ranked #2 in the country by either SP+ or FEI despite not having a great scoring average. That team moved the ball well up to the red zone but had trouble punching it in. Whichever formula that ranked us #2 evidently didn’t value red zone scoring that much but that seems like a design flaw IMO.
December 6th, 2021 at 11:40 AM ^
Any good formula (and I think Bill's is pretty good) has only one standard: how predictive is it?
The weights of measures you mentioned should be tweaked for maximum predictive value, which is what Bill always appeals to when explaining the long-term creation of his formula.
December 7th, 2021 at 2:32 AM ^
A good statistic is good at one of two things (ideally, both):
1. Prediction. Do past values accurately predict future values, and to what degree?
2. Description. Does this statistic accurately reflect what happened (and hence better explain how it happened)?
As an example, Football Outsider's DVOA and related statistics are optimized for prediction, which occasionally cuts their descriptive value. A canonical example of this is the Steeler's James Harrison's 100-yd INT return for a TD at the end of the half in the Super Bowl against Arizona. That play was critical to describing, and hence understanding, what happened, because the expected points swing was more than 10 points (Arizona expected 3-4 points from that series). Yet DVOA downplayed it, because long INT returns are so rare giving them "full" value reduces the statistic's predictive value. (End-of-half also makes it less predictive, since a 99-yard return would have been worth the same as a 0-yd return; time had run out.)
All of which is to say that one should be careful about looking at a predictive stat and using it for description, or vice versa.
December 6th, 2021 at 12:24 PM ^
Yeah, I've never really been a "global relativist." I think we can go into all human-based interactions with the core assumption that humans created it, therefore there is humanity to it. I'm not sure that that particular argument works for me here or really even takes away from what was my base point...
A statistical approach to analysis, is an attempt to use the relevant data that exists to quantify the subject, or in the case of SP+ to quantify and be predictive. They're created to remove the "eyeball" of it all, so you're just looking at data and not basing it on how you "felt" watching the game.
ldeveon's response was to give his eyeball thoughts to something intended to be the opposite. And honestly my reply to that wasn't really intended to be critical of his response, not like I negged him or went "strong" with a take. I was simply pointing out that the two were incongruous.
December 6th, 2021 at 4:06 PM ^
yes, but the main issue with fancystats wrt CFB is both the limited sample size and the clustering of the interactions. Both of which make the results less than stellar overall.
December 6th, 2021 at 9:17 AM ^
The exact formula for SP+ is proprietary, but Bill C developed it to be predictive, and has spent years refining its predictive power. So no, there's no baked-in SEC bias conspiracy. It's just that this is the formula that Bill C has found most accurately predicts future results in CFB. And before you say "but M beat OSU!", no, it's not perfectly predictive, but it is better than any other ratings system at predicting the future (which isn't necessarily saying much).
December 6th, 2021 at 10:53 AM ^
Brian Fremeau Has a predictive model as will. Football Outsides uses his system for their FEI rankings. The top is largely the same,
He too has Georgia as comfortably the best team.
- Georgia 11-1 (1.60) Offense 3 (1.56) Defense 1 (1.78)
- Alabama 11-1 (1.24) Offense 2 (1.74) Defense 6 (0.98)
- Ohio State 10-2 (1.21) Offense 1 (2.70 Defense 40 (0.31)
- Michigan 12-1 (1.05) Offense 4 (1.26) Defense 11 (0.74)
If OSU's offense is number 1 and their D number 40. Yet they are over all tanked 3rd. So they rate their offece as significantly better than anyone else. They do the same with Georgia's defense.
Bama designed the blueprint to beat Georgia. Their secondary is average, If Cade is crisp we have the horses to get it done.
December 6th, 2021 at 11:48 AM ^
Right, no formula is perfectly predictive. "Why we play the games" etc
The common retorts ("LOL but U-X beat U-Y by z points, how can they be ahead, your ratings are garbage") are based on years of socialization into poll-thinking, where 'deservedness' is assessed, in a backward-looking manner, to form rankings -- whereas SP is about ratings more than rankings, and forward-looking/predictive.
Imagine using only known final score results to predict future games. ("Well Oklahoma State beat Baylor by 10 when they first played, so I can only assume Oklahoma State will win by 10 every time they play...")
December 6th, 2021 at 9:18 AM ^
Georgia has beaten up some pretty good teams. I dont think michigan is the same team that struggled against rutgers and lost (got screwed) against a MSU team that is much worse than 10-2, and recency bias is hard to overcome. My point is that computer rankings dont have recency bias, and a close to objective look at the season as a whole isn't a bad thing.
December 6th, 2021 at 9:28 AM ^
They’ve beaten a bunch of slightly better than average teams, mostly at home.
December 6th, 2021 at 10:37 AM ^
I am inclined to believe that they have a better defense than Wisconsin or Iowa, and certainly Ohio State. We need to be ready to play our best game. They've pitched a few shutouts, and a few near shutouts this season. They are a really good defensive team.
Now, maybe we can still abuse them because we have become a REALLY good offense. I am hoping that Georgia's D is like our revenge tour year where we dominated a bunch of teams defensively and then got wiped out when we faced an offense that picked on our weaknesses.
December 6th, 2021 at 11:39 AM ^
You talk about recency bias, but a system that looks at a season in total with equal weight (not sure if Connelly does this, fwiw) runs its own risk of bias compared to a system that gives more weight to more recent results. If the offense is different now than it was early in the season, does it make the system more or less accurate to equally weight the first games with the more recent games? Do you think Georgia expects us to look like the Rutgers game or the Iowa game?
And it may be true that in average something closer to equal weighting produces the most accurate results on a whole, while also being true that for some teams this introduces a bias that makes it less accurate.
December 6th, 2021 at 1:19 PM ^
Yeah, it would be interesting to see a version of SP+ that only looks at the back half of the season.
December 6th, 2021 at 9:23 AM ^
Hot take: there is no formula that can account for a team that is physically/athletically superior to teams that it bulldozes, but then struggles tactically when it goes up against teams with equal (or closer to equal) talent. See OSU, Georgia.
I think we beat Georgia because we have mastered the bend don't break defense and our offense is FINALLY able to wear down opponents, but also has the athletes to throw in a lot of misdirection. It means defensive players have to run more and play harder and then you get endings like the last drive against OSU.
December 6th, 2021 at 9:56 AM ^
I’m guessing the math does suffer from only 12-13 outcomes for almost all teams. Compared to the advanced stats in baseball which have 162 won-loss outcomes and the other stats, like plate appearances get much bigger from there.
I wonder if there could be a Monte Carlo analysis run from S&P+ info that might predict how often the results of future games might match the rankings? I mean S&P+ is the “most predictive” is that 75% accurate? 90%? Less?
December 6th, 2021 at 11:45 AM ^
Yes, this is the real problem 13 data points is a joke in terms of sample size.
December 6th, 2021 at 4:10 PM ^
There is also the distinct lack of interconnection between conferences.
Also there are websites that track the predictive ability of the various computer polls. They've been ~70-75% with a best case of 78% so far. SP+ has been ~70% this year iirc.
December 6th, 2021 at 9:33 AM ^
SEC bias is a factor in human polls. Not in computer programs written by some dude who only cares about things like yards per play and other measurable stats.
December 6th, 2021 at 11:29 AM ^
It's funny seeing the tweets every week at Bill, with fans claiming he "hates" their school/conference.
I don't think the math has changed in several years.
December 6th, 2021 at 9:41 AM ^
I agree most on the offense. For all of their talent, it seemed once they were in a bind, all they could do was throw to the same TE over and over again. Now that guy is really good, but it's not the death machine OSU was for 80% of the games this year where they could throw to any WR at any place on the field while having an RB that can break huge gains against light boxes.
I really like the use of advanced metrics, but I do question how quantitative SP+ really is - meaning the final rankings are probably reasonably accurate but getting into the number of "points" that separate each team might be putting a bit too much faith in an undisclosed formula. I'm also not a big fan of the post game win expectancy formulas, because game situation and turnovers are huge in football and minimizing them/taking them out just doesn't make sense to me.
December 6th, 2021 at 12:43 PM ^
A formula doesn't hold the truth, but it can be invalided by the truth.
December 6th, 2021 at 8:48 AM ^
OSU was the scariest offense on the schedule and M survived. UGA's is used to playing with the lead. Hopefully that means something
December 6th, 2021 at 9:29 AM ^
Hopefully Georgia players are ill as well when we play them
December 6th, 2021 at 9:48 AM ^
3M TA3
December 6th, 2021 at 8:49 AM ^
There are so many variables to this game. Using only a statistical analysis is foolhardy-to put it kindly.
December 6th, 2021 at 9:58 AM ^
Well what do you suggest?
should we get a field at a predetermined time and actually play the two teams against each other?
I would want that type of thing observed by as many experts as possible.
December 6th, 2021 at 8:49 AM ^
Man, that headline triggered me - can't wait to see Michigan give back-to-back ass kickings to the SEC powers.
December 6th, 2021 at 9:02 AM ^
Lets hope, but that's a tall order
December 6th, 2021 at 9:12 AM ^
On any given Saturday, or Friday or Monday in the case of the playoffs, anything can happen. Many of us were skeptical that Michigan would be able to compete with Ohio State. And we came away looking like fools with Michigan completely dominating that corrupt but highly talented football team. I’m a believer that this team is different and will not give up instead of folding as it has in the past.
December 6th, 2021 at 10:07 AM ^
No argument from me...... you just have to string (by that time) 4 of your best games of the season together, without a hicup.... not impossible, but no small task.
The good thing is, we've seen two near perfect game plans in the last two weeks and we know we have the talent to make it happen.
LFG.... GO BLUE
December 6th, 2021 at 10:21 AM ^
It absolutely is. If Michigan wins this year it will likely be a 4 game stretch of OSU, Big Ten West Champ Iowa, Georgia, and Bama. That would be the hardest path to the CFP championship ever.
December 6th, 2021 at 11:54 AM ^
And the thing is, Michigan could do that and still wind up being ranked 4th in SP+.