Résumé S&P - A Different Way of Ranking Strength of Schedule
October 16th, 2018 at 9:00 AM ^
This is actually promising.
Take this, the "who has played their schedule best" in the first half of the year and use it as the basis for the strength of schedule element in ranking teams in the last half.
That seems a heck of a lot more accurate than keeping pre-season assumptions as such a huge part of November rankings.
October 16th, 2018 at 9:05 AM ^
PSU at 4 immediately loses all credibility.
Polls, fancy stats and ranking articles are good for one thing - getting clicks.
October 16th, 2018 at 9:23 AM ^
Except for the fact that, polls, ranking articles and stats are literally used to decide the college football playoff, which ultimately determines the national championship. Except for that one small thing, they're no good.
October 16th, 2018 at 10:04 AM ^
That's delusional. Politics and potential game viewership is used to assign the playoff spots.
Otherwise, UCF would have been in if record and fancy stats had a lot of weight.
October 16th, 2018 at 9:37 AM ^
S&P+ also is over time more accurate than the Vegas line by about 54%.
October 16th, 2018 at 10:20 AM ^
Small correction in your wording: It's not 54% more accurate; it beats the Vegas line 54% of the time.
October 16th, 2018 at 9:34 AM ^
My question would be what problem are you fixing? Whenever anybody at work comes up with some elaborate plan that requires lots of organizational time and money to implement that's the first question I always ask and you'd be amazed how many times there are only blank looks and silence in response.
So what problem are you fixing Bill? Which team has gotten left out of the final four that would've made it based on your new fancy stats? Which team got screwed over by the humans that richly deserved to go if only this amazing new way of determining SOS was implemented?
I'll hang up and listen.
October 16th, 2018 at 9:55 AM ^
Well the result seems to be it replaces the over ranking of the SEC in pre season with the over ranking of the B1G now.
Which I'm cool with no matter how wrong it is.
October 16th, 2018 at 9:58 AM ^
I felt that after reading the accompanying article by Bill, it pretty well lays out why he came up with this metric. Basically, with football there's such tiny sample sizes that are widely varied by opponent that there's not great way to compare teams like in other sports where you always play everyone.
But if you're asking in a more philosophical sense why he is doing this, I would ask why are we even talking about sports in general? It has no impact on 99.9% of the people in the world in any tangible way. It's kind of weird that you're critical of a sports writer for writing about sports when you're on a blog talking about sports.
Now if you're asking in an economic/efficiency/systems use type of why is he doing this, I think the answer is simple. That's his job. It is literally his job to look at the numbers in sports and find ways to better predict outcomes, analyze data, and rank teams. As Bill even says, you can always learn something from every game. And I would say that because he is doing this, he is making money doing it, and SB nation is finding value in him doing it. Just because you can't find any use in the data doesn't mean other people can't.
October 16th, 2018 at 10:16 AM ^
He’s fixing that his fancy stats (S&P+) don’t take into account strength of record. So instead of saying “these teams down to down are the best,” which is what S&P tells us, he’s combining that with “who’s played an honest-to-god schedule”
So it’s trying to combine “best” and “most deserving,” which he argues you need both to determine who should be in the CFP.
People are discounting it because of Penn State, but when you take into account that S&P doesn’t necessarily look at the final score, it makes sense. In both their losses they were the better team for the majority of the game, or the numbers would tell you that.
October 16th, 2018 at 12:02 PM ^
I agree that there doesn't seem to be a real problem here.
The goal of college football for the fans is to be fun. Penn State lost to MSU, but would still be in this calculated playoff because they played well while losing? Where's the fun in that?
October 16th, 2018 at 9:50 AM ^
I like Connelly's work. I really like S&P+. But I am less of a fan of "Resume S&P+" and particularly how we he works scoring margin into "Resume S&P+."
To me, your resume, at least when it comes to the College Football Playoff at the end of the year, should be nearly strictly binary. You either won or you lost.
I play around with stats on my own end --- I have something that looks at "what would the 10th (10th is a bit arbitrary admittedly) best team in the country do against your own schedule in terms of a W-L record, and how much better is your W-L record vs. that?" That gives you credit for good wins, minimal credit for wins over Huck-A-Buck State, and doesn't severely penalize losses to good teams. FWIW, my rankings there:
1. ND, 2. Clemson, 3. Ohio State, 4. LSU, 5. Alabama ..... 11. Texas, 12. Michigan .... 36., MSU, 37. Wisconsin, 38. Western Michigan, 39. Penn State, ....... 130. UTEP.
October 16th, 2018 at 10:03 AM ^
I think you're point is the ultimate philosophical debate in college sports post season play where sample sizes of games are so small considering how many teams there are. Should post season rewards be based on what you did, regardless of how good you actually are? Or should they be based on how good you actually are, regardless of what luck may have happened in any one or two moments through the season? Should it be a balance of the two? How much weight on each factor?
I honestly don't have a good answer and I don't think anyone else does. That's why I appreciate multiple ways to look at that same problem.
October 16th, 2018 at 10:31 AM ^
It's definitely the ultimate philosophical debate ---- I definitely appreciate S&P+, because I think that's the ultimate "how good you actually are, regardless of luck" metric.
I don't think, however, "Resume S&P+" is an actual "how good you actually are, regardless of luck" metric.
My philosophical issues with "Resume S&P+" is that scoring margin isn't capped until you get to a fifty (50!) point margin. Take PSU @ Pittsburgh for instance. PSU gets ahead 23-6 and it was really game over at that point. 17-point win, nice win. But PSU gets 28 more points and turns it into a 45-point win. Resume S&P+ gives PSU 4.67 more points credit (28 points/6 games played thus far) for that! So those 28 points are the difference between PSU being #4 and #9.
I'd prefer to keep it at "Penn State won at Pittsburgh, period." A typical 10th-ranked team would do that 63% of the time - PSU w/ their 1-0 record in that game gets 0.37 "points of credit" for that.
October 16th, 2018 at 10:47 AM ^
I agree that the 50 point cap is kind of ridiculous. I do think that there is a difference between a 1-point win, a 7-point win, and a 21-point win, though. Maybe, he should cap it at differential when the team reaches garbage time by his other metrics, i.e. 21 in the 4th or 35 in the 3rd. I could also see using a log, or something similar, of the differential to give diminishing returns when running up the score. Someone also mentioned using buckets of 1-7, 8-14, and 15+, which could work. The point is that there are lots of things he could do. The problem is that he has no measuring stick other than the eye test, unlike S&P+ having Vegas and actual records to compare against.
October 16th, 2018 at 5:38 PM ^
I like the idea of changing the cap to exclude garbage time TDs. If he did PSU would drop a good amount because they have pored it on against inferior teams late in games.
I think Bill will eventually do that and the resume s&p+ will be very good.
October 16th, 2018 at 10:49 AM ^
I'd prefer to keep it at "Penn State won at Pittsburgh, period."
The problem is that there is so little data in college football, and ADs and coaches are intent upon having less and less by playing as many tomato cans as possible. "Penn State lost to Michigan State" tells you a lot less about the game than "Penn State outplayed Michigan State and got unlucky."
I don't like running up the score either, and I'd probably pick a lower cap than 50, but I can't get that angry about the methodology.
Also, to everyone complaining about PSU being #4, and how that should invalidate the system -- if a new statistical tool only confirms what you already believed, it's of limited value. The truly useful tools bring new insights, such as "maybe PSU is better than we've assumed."
October 16th, 2018 at 11:03 AM ^
There is actually a lot of data in college football if every single play is a distinct event.
October 16th, 2018 at 4:04 PM ^
There's still not that much, but, yes, it's a lot more when you look at it that way. The problem is that by looking just at who beat whom, you're reducing the amount of data available by multiple orders of magnitude.
October 16th, 2018 at 10:37 AM ^
"Should post season rewards be based on what you did, regardless of how good you actually are? Or should they be based on how good you actually are, regardless of what luck may have happened in any one or two moments through the season? Should it be a balance of the two? How much weight on each factor?"
I think there is a more fundamental question that needs to be answered first: What is the purpose of the playoff? Is it to have the 4 highest-ranked (a proxy for best or most deserving) teams play each other or is it to find the best team in the country?
People often think that putting the 4 highest-ranked teams in a playoff is going to necessarily result in the best team winning the championship, but there is no guarantee that the best team is one of the 4 highest-ranked teams. If the point is to find the best team in the country, then there should be as much diversity as possible in the playoff and we've never really done it right. If the point is to reward the top 4 teams, then you can start asking the questions that you asked.
October 16th, 2018 at 11:12 AM ^
Dude, you have Alabama at 5, your rankings are flawed.
October 16th, 2018 at 9:52 AM ^
Once he got PSU at 4, he should have realized his formula doesn’t pass the eye test. You can’t ignore a loss just because it was unlucky. Needs a major tweak.
October 16th, 2018 at 11:48 AM ^
I think what he's saying with these rankings, based on S+P and SOS, is that PSU should be ranked 4th. It shows that they are under-performing, which is true. MSU/ PSU was a game where PSU vastly out-performed MSU, but failed... failed to pick off somewhere close to 4 passes that were right there for the picking - including a straight drop on the last drive. This is how you lose a game when you are playing a team close to #50 at home when you are close to #4.
October 16th, 2018 at 2:44 PM ^
Advanced stats can only do so much to adjust first downs and yards according to the opponent at the time. e. g. QBs look great when they have an extra half second or two. When #5 played number #85 the adjustment is less meaningful than the adjustment when #5 played #15 and that pollutes the value of fancy stats to evaluate #5 about to play #12.
I love advanced stats but as most of us know - the refs, a great play or play call, a turnover, and/or a fortuitous bounce can easily swing 7-17 points the wrong way.
October 16th, 2018 at 6:21 PM ^
I’ve always been firmly in the camp that championships, playoff berths, etc. should be awarded strictly on the basis of a team’s accomplishments (I.e., “what you did”) rather than “how good” a team happens to be. If Rutgers goes undefeated, they should be in the playoff even if it’s a function of pure luck, with their opponents spectacularly choking in every game. But how does one objectively compare what team A did vs. what team B did?
It seems like you could just use basic S&P+ (not resume S&P+) to determine “how good” each team is, then just compare teams based solely on their records—but with wins against teams higher ranked per S&P+ counting for more, and losses against teams lower ranked on S&P+ being punished more heavily.