Semi-Objective CFP Ranking System: 2018 Week 11

Submitted by The Maizer on November 6th, 2018 at 11:15 AM

The Preface

This is is the week 11 college football team rankings produced by a semi-objective model as described previously HERE. The model was inspired by Seth's post that proposed a point system to determine bowl eligibility. These rankings aim to be a relatively objective starting point from which to apply considerations such as the eye test, margin of victory, and head-to-head results. The goal is to rank the quality of the resumes of teams from an accomplishment standpoint.

The Rules

  • +3 points for a conference championship.
  • +4 points for a win over a top 10 team.
  • +3 points for a win over a top 25 team (not in top 10).
  • +2 points for a win over a winning P5 team (not in the top 25).
  • +1 point for a win over a winning G5 or a losing P5 team.
  • +0 points for a win over a losing G5 or any FCS team.
  • -1 point for a loss to a top 10 team.
  • -2 points for a loss to a top 25 team (not in top 10).
  • -3 points for a loss to a winning P5 team (not in top 25).
  • -4 points for a loss to a losing P5 or any G5/FCS team.

Top 10 and top 25 status are determined by this ranking system and the model is solved iteratively until it converges. In scenarios where oscillatory states lead to a failure in convergence, the average points for oscillatory states are used. No voter polls are used in any capacity. Ties are broken by head-to-head results when applicable and RPI otherwise.

The Results

This week there were two oscillatory states, hence the 0.5 point increments.

1.png

2.png

3.png

4.png

5.png

6.png

I expect a notably different order from the playoff committee tonight, particularly Alabama being on top and Michigan staying ahead of Georgia. Also I expect LSU to fall a little farther, below Oklahoma and likely West Virginia.

Last week, this model was a remarkably accurate predictor of the committee's selections, getting 100% of the top 4, 90% of the top 10, and 92% of the top 25 correct with the largest discrepancy being only 6 positions off. A comparison to the committee rankings gave a mean squared error of only 7.6 (AP poll was 22.0, FPI was 97.2, S&P+ was 157.8). Shout out to mgoblog user EastCoast_Wolverine2016 for these analyses. I wouldn't imagine this model's accuracy would continue at such a ridiculous level, but it is interesting that this very simplistic approach mirrored the committee so well in the first week.

Comments

J.

November 6th, 2018 at 11:23 AM ^

Heather Dinich posted something similar today on ESPN.  It wouldn't surprise me at all to see Georgia jump Michigan, because somehow all SEC teams are awesome, and Kentucky and Florida can't possibly both be garbage.

I still take issue with the point system, but since my suggested correction last week made it worse, I don't have much of a leg to stand on. :)

TrueBlue2003

November 6th, 2018 at 5:10 PM ^

I commented this last week, but these results are once again almost identical to the ESPN's Strength of Record metric which gets cited frequently as a very reliable predictor of the committees top 4 (15 of 16 and the last 12).

Once again, these results are very close to that metric.  Interestingly, they have ND #1 and Clemson #3 so those are swapped.

But SoR has the exact same 4-6 teams: UGA, M and LSU.

wildbackdunesman

November 6th, 2018 at 11:26 AM ^

It is a nice idea, but I could see a situation where a team makes the playoffs or not based on whether some past opponent wins a game to finish 7 and 5 with a winning record to get an extra point for beating a team with a winning record.

wildbackdunesman

November 7th, 2018 at 7:45 AM ^

If you had a system where you automatically get a point if you beat a team with a winning record.  You could conceivably have a scenario where the last playoff spot is determined by the very late Army versus Navy game...if the right team wins they have a winning record giving a team that beat them earlier in the year an extra point to leap frog someone else.

Next year we play Army.  Wouldn't it be silly to have Michigan's playoff hopes come down to whether or not they beat Navy in the week after the conference championships giving us an extra point in this hypothetical system?

wildbackdunesman

November 6th, 2018 at 11:27 AM ^

It is a nice idea, but I could see a situation where a team makes the playoffs or not based on whether some past opponent wins a game to finish 7 and 5 with a winning record to get an extra point for beating a team with a winning record.

MgoWood

November 6th, 2018 at 12:04 PM ^

ESPN which has numerous financial ties to the sec, which also has many presumable (*opinion is mine) ties to the playoff committee will most undoubtedly preserve their relationship forever. The money induced politics is never ending.

 

Should the playoff commitee be composed of (supposedly) AD's and former coaches? Or should they be other informed individuals?

 

Edit: I'm piggybacking more on Klatt's information because I completely agree with what he has to say on the matter.

The Maizer

November 6th, 2018 at 12:13 PM ^

I like what Klatt says, but I don't agree with his reasoning. His point is that there is shadiness going on in the CFP rankings because those rankings don't agree with the fancy-stats metrics. I think they don't agree with the fancy-stats metrics because the committee is ignoring those metrics in favor of much more simplistic resume considerations. The ramifications are the same, but I don't agree that it's an agenda-driven thing by corrupt committee members.

And I'm not trying to say I agree with the committee's approach; just that the simplest explanation for their rankings is resume over fancy-states, not anything nefarious.

8_team_playoff

November 6th, 2018 at 12:31 PM ^

It's interesting to me that we like to drag the AP and the CFP committee for overrating the ACC, but your model still has some of the "overrated teams" (NC State, BC, Syracuse) in the top 25 without taking those polls into account. 

J.

November 6th, 2018 at 12:55 PM ^

Are the oscillations coming from A&M moving in and out of the 'Top 25'?  I'd argue that it should be Top 10 & Ties and Top 25 & Ties, since I don't see an obvious tiebreaker metric to use.

The Maizer

November 6th, 2018 at 1:11 PM ^

A&M is one of four teams that bounce in and out of the the top 25, yes. I like your idea of everyone who has the same number of points as the 10th team counts as top 10 and same number of points as the 25th team are top 25. I'll try this and if the results are consistently compelling I'll add this to the algorithm. Thanks!

Maize4Life

November 6th, 2018 at 1:47 PM ^

Well..despite the obvious SEC is awesome supergreat fantabuloso and their last place team can beat any conference champion , blah blah blah..I am not worried about any other SEC team other than Alabama..Bama will beat UGA hand them their 2nd loss and thats that...Im not convinced that ND doesnt slip up...FSU, SYR, USC...)  Win out and its a done deal...Duh...but its not going to be UGA that steals our spot...

Ali G Bomaye

November 6th, 2018 at 2:17 PM ^

I just don't get the point of this ranking. Yes, the rules create an empirical number for each team. But the rules themselves are completely arbitrary. So this isn't anywhere close to a "semi-objective" ranking system - on the contrary, it's completely subjective.

The Maizer

November 6th, 2018 at 2:32 PM ^

Well yes, that is precisely the reason I have used "semi-objective" instead of "objective." There is no way to have an objective rankings. Even the choice to base something solely on wins and losses would be a subjective decision to exclude other information.

I think you do get the point of this ranking but have a problem with the semantics. I can't argue that point. The rules here are not completely arbitrary, though they are mostly arbitrary.

Tauro

November 6th, 2018 at 2:40 PM ^

Like last week, I took a look at the top teams to calculate their SOS (adding up the figures for the teams on their schedule).  Why?  Just because I like it.  I gave teams that scheduled an FCS team Rutgers score - just felt right to do that.  Get the following results after last week (I have put them lowest to highest):

Georgia: -43
Clemson: -56
Notre Dame: -59
Michigan: -59.5
West Virginia: -61.5
Oklahoma: -65
Alabama: -80
Ohio State: -98
Washington State: -109.5

Interesting to see that the SEC bias favors Georgia more since Alabama did not have any of the 'good' SEC teams on their schedule (other than LSU).

mfan_in_ohio

November 6th, 2018 at 3:17 PM ^

I really like this idea.  There needs to be more objectivity in ranking teams if those rankings are going to affect who is eligible to win the championship.  

One suggestion I would have (and I don't really know how to implement it) regards "winning P5 teams."  Many of the SEC teams play 3 cupcake-level nonconference opponents because they only play 8 conference games.  It's how they can wind up with 10 bowl-eligible teams, and nobody really notices because they have a few good conference games during the traditional nonconference season, then slip in a game against Lamar in November when no one is paying attention.

SEC teams have a built-in advantage in this metric, as their average team essentially starts with an extra half win by virtue of having one fewer game against other teams in their own conference.  Example: South Carolina will end up (likely) 7-5, with wins over Akron, Chattanooga, Coastal Carolina, Tennessee, Vandy, Missouri, and Ole Miss.  Their 5 losses will all be to ranked teams.  What happens if you replace, say, the Chattanooga win with a game against an SEC West team that they didn't play? Options: MS St, Auburn, LSU, Alabama, Arkansas.  Sure, if they're lucky they get to play Arkansas, but they are probably underdogs to each of the others.  

The SEC is, intentionally or not, gaming that metric, because it helps teams like South Carolina and either Vandy or Tennessee (and maybe Missouri) eventually get to or above .500 by virtue of an easy nonconference schedule.  Since teams like Kentucky and Florida play all of those teams, they are somewhat artifically getting 1-3 additional wins over "winning teams" that other P5 teams don't get.  

I'd like to see wins against losing G5 and FCS teams excised from the records completely, and then have the "winning teams" calculated based on the remaining games.  It's not like the cupcake games are anything other than glorified scrimmages anyway, so they shouldn't serve to boost the teams' resumes.  

The Maizer

November 6th, 2018 at 4:03 PM ^

This is an interesting idea; however, I think there may be some unplanned consequences. For example, G5 conferences may have very few teams to evaluate against to determine "winning" status. If the MAC only has 5 teams with a 0.500+ record, then really one or two conference losses puts them in serious danger of being a "losing" team. Also, I think you probably want to adjust your method since you probably don't want losses to losing G5 or FCS teams to be ignored when determining "winning" status.

mgowill

November 6th, 2018 at 3:29 PM ^

I ran 6 teams through S&P+ data and came up with this information as a result.  The first line under the team is average opponent rank as of this week and their top 25 wins per S&P+ as of today.  The second line is what it would look like if all of these teams won their next 3 games.

ALABAMA, 9-0

CURRENT AVG OPP RK - 59.7, 3 TOP 25 WINS

PROJECTED AVG OPP RK - 51.2, 5 TOP 25 WINS

 

CLEMSON, 9-0

CURRENT AVG OPP RK - 64.6, 1 TOP 25 WIN

PROJECTED AVG OPP RK - 59.5, 1 TOP 25 WIN

 

NOTRE DAME, 9-0

CURRENT AVG OPP RK - 70.3, 1 TOP 25 WIN

PROJECTED AVG OPP RK - 67.4, 1 TOP 25 WIN

 

MICHIGAN, 8-1 LOST TO #6 ND

CURRENT AVG OPP RK - 48.0, 2 TOP 25 WINS

PROJECTED AVG OPP RK - 54.2, 3 TOP 25 WINS

 

GEORGIA, 8-1 LOST TO #22 LSU

CURRENT AVG OPP RK - 49.8, 1 TOP 25 WIN

PROJECTED AVG OPP RK - 52.2, 2 TOP 25 WINS

 

OKLAHOMA 8-1 LOST TO #42 TEXAS

CURRENT AVG OPP RK - 63.3, 0 TOP 25 WINS

PROJECTED AVG OPP RANK - 60.5, 2 TOP 25 WINS

 

So by what I'm looking at in data, Georgia is definitely a team to keep an eye on.  Notre Dame is the first pretender to fall off this list.  

 

 

Red is Blue

November 6th, 2018 at 4:15 PM ^

As a general rule, I don't like things with few significant step changes.  Is beating the #25 team really that much different than beating the #26 such that it warrants a full point difference.  Whereas beating the #24 team and the #25 team are the same?  Wouldn't it make more sense to have a more gradual decline, something like points = 2 + 0.04 * (50 - opponents rank)?

This might also help in dampening oscillations (or maybe not?)

The Maizer

November 6th, 2018 at 4:33 PM ^

Your point is valid, but I think both methods are arbitrary. Is beating the #25 team a point more relevant than beating the #26 team? I don't know, probably not. Is beating the #17 team a third of a point more relevant than the #25 team? Same answer. It's a bin size argument and I've chosen to use the (admittedly arbitrary) bin sizes which match common discussions and perceptions that surround college football.

Anyway, I basically agree with you, but I think the value of a simplistic approach outweighs your concern.

Benoit Balls

November 6th, 2018 at 9:13 PM ^

Something Ive always thought should be worked in somehow, and something I always thought was kind of messed up about the whole "top 25 wins" metric is that credit should be given for where teams were ranked at the time the game was played, not where the teams are ranked at the time of the computation.  I think this for 2 reasons:

First- roster status/injuries.  Say a team wins a game against a team ranked #10 on the day of the game.  As the season rolls on, perhaps that team suffers injury attrition to one or several key players and falls out of the top 25 by week 9.  At the time of the win, it was a quality win in the eyes of (most) everyone, it wasn't a win against the injury depleted roster missing the marquee players.  

Second:  Rankings have an impact on player mentalities (at least on some level).  Whether that gets lower ranked teams hyped up or psyched out depends on a number of variables, but it DOES have an impact.  

Now, it could be incredibly hard for laypeople to keep track of what team was ranked where during indivudual weeks, but not so much for computers.  I suppose an argument could be made that it could also go the other direction, where a team wins early in the season aganst a team ranked 14th, and that team eventually rises higher in the polls.  At the end of the day, I thin if we're going to use complicated formulas as part of the winnowing process, those formulas might as well be as accurate as possible, and give credit for wins/losses as they happen, and not how they look a month or two down the road.

OTOH, its entirely possible that Im crazy

The Maizer

November 7th, 2018 at 9:15 AM ^

This week's accuracy was slightly less impressive than the first week:

Top 4: 75% with the one miss (5: Michigan) being 1 spot out.

Top 10: 90% with the one miss (11: Ohio State) being 1 spot out.

Top 25: 88% with the three misses (36: Iowa, 27: Iowa State, 34: Fresno State).

Still pretty good for such a simplistic model. The biggest miss was Iowa which was off by 15 spots. I think it's interesting that the 13-15 spots are the same three teams as the CFP rankings and are the teams that general opinion says are all over-ranked (Syracuse, NC State, Florida).