Semi-Objective CFP Rankings System: 2016 Week 10 Results

Submitted by The Maizer on November 10th, 2016 at 12:52 PM

At the conclusion of last season, I posted the results of a fun exercise that took a relatively objective look at ranking teams based on the quality of their wins and losses. It was inspired by Seth's post that proposed a point system to determine bowl eligibility. The premise was to create a ranking as a starting point from which to apply considerations such as the eye test, margin of victory, or head-to-head results.

The feedback I received from last year's post led me to adjust the model such that top 10 and top 25 wins/losses would refer to the model's own rankings and iteratively converge to a stable ranking. This was an improvement over using the end-of-season AP poll as the top 10 and top 25 rankings. In addition, wins over FCS teams are no longer considered to determine a team's "winning" status. Below you will find the point system description and the rankings resulting from this model as of the conclusion of week 10.

I would love to have feedback from the community again, especially with regards to:

  • The weight of points for each category of win or loss.
  • The weight of points for conference championship.
  • Should there be a bonus for division winners?
  • Should Notre Dame and/or BYU be considered power 5 teams? (They were not, here.)
  • What's the best way to break a tie in points? (So teams do not have the same rank.)
  • What to do when the system does not converge?

The rules for this proposed system are:

+3 points for a conference championship.

+4 points for a win over a top 10 team.

+3 points for a win over a top 25 team (not in top 10).

+2 points for a win over a winning P5 team (not in top 25).

+1 point for a win over a winning G5 or losing P5 team.

+0 points for a win over a losing G5 or any FCS team.

-1 point for a loss to a top 10 team.

-2 points for a loss to a top 25 team (not in top 10).

-3 points for a loss to a winning P5 team (not in top 25).

-4 points for a loss to a losing P5 or any G5/FCS team.


2016 Week 10 Results

Rank Team Points
1 'Alabama' 18
2 'Michigan' 17
3 'Clemson' 14
3 'Ohio State' 14
5 'Washington' 10
6 'Penn State' 9
7 'Texas A&M' 8
8 'Auburn' 7
8 'Louisville' 7
8 'Wisconsin' 7
11 'North Carolina' 6
12 'Nebraska' 5
12 'Tennessee' 5
12 'West Virginia' 5
15 'Arkansas' 4
15 'Colorado' 4
15 'Oklahoma State' 4
15 'Virginia Tech' 4
19 'Boise State' 3
19 'Florida' 3
19 'Florida State' 3
19 'Minnesota' 3
19 'Oklahoma' 3
19 'Washington State' 3
19 'Western Michigan' 3
26 'Stanford' 2
27 'LSU' 1
27 'USC' 1
27 'Troy' 1
27 'Utah' 1
31 'East Carolina' 0
31 'Wyoming' 0
33 'Baylor' -1
34 'Pittsburgh' -2
35 'Appalachian State' -3
35 'Houston' -3
35 'San Diego State' -3
35 'South Florida' -3
35 'Tulsa' -3
40 'Georgia' -4
40 'Iowa' -4
40 'Maryland' -4
40 'Northwestern' -4
44 'Kentucky' -5
44 'Miami (FL)' -5
44 'Navy' -5
44 'South Carolina' -5
44 'Wake Forest' -5
49 'Georgia Tech' -6
49 'Indiana' -6
49 'Kansas State' -6
49 'Ole Miss' -6
49 'TCU' -6
49 'Toledo' -6
55 'BYU' -7
55 'Temple' -7
55 'Texas' -7
55 'Western Kentucky' -7
59 'Arizona State' -8
59 'Lousiana Tech' -8
59 'New Mexico' -8
59 'Ohio' -8
59 'Vanderbilt' -8
64 'Colorado State' -9
64 'Middle Tennessee' -9
64 'Mississippi State' -9
64 'NC State' -9
64 'Syracuse' -9
64 'Texas Tech' -9
70 'Air Force' -10
70 'Boston College' -10
70 'California' -10
70 'Memphis' -10
74 'UCF' -11
74 'Idaho' -11
74 'Illinois' -11
74 'Old Dominion' -11
74 'Oregon' -11
79 'Army' -12
79 'UCLA' -12
79 'Eastern Michigan' -12
79 'North Texas' -12
79 'UTSA' -12
84 'Arkansas State' -13
84 'Southern Miss' -13
86 'Hawaii' -14
86 'Oregon State' -14
86 'Purdue' -14
86 'Rutgers' -14
90 'Akron' -15
90 'Central Michigan' -15
92 'Charlotte' -16
92 'Duke' -16
92 'SMU' -16
95 'Ball State' -17
95 'Georgia Southern' -17
97 'Cincinnati' -18
97 'Kent State' -18
97 'Louisiana-Lafayette' -18
97 'Miami (OH)' -18
97 'South Alabama' -18
102 'Louisiana-Monroe' -19
102 'Missouri' -19
102 'New Mexico State' -19
102 'Notre Dame' -19
102 'Utah State' -19
107 'Arizona' -20
107 'Michigan State' -20
107 'Virginia' -20
110 'Texas State' -21
111 'Northern Illinois' -22
111 'UTEP' -22
113 'Iowa State' -23
113 'Tulane' -23
115 'Georgia State' -24
115 'Kansas' -24
115 'Marshall' -24
115 'Nevada' -24
115 'UNLV' -24
120 'San Jose State' -25
121 'FIU' -26
121 'Florida Atlantic' -26
123 'Buffalo' -27
123 'Connecticut' -27
125 'Bowling Green' -29
125 'Massachusetts' -29
127 'Rice' -31
128 'Fresno State' -33




November 10th, 2016 at 2:49 PM ^

It would complicate calculations, but I wonder about a multiplier effect along the lines of: 1-8 point win (loss): 100% of your scale; 9-16 point win (loss) worth 110% of your scale; 17+ point win (loss) worth 120% of your scale. Could tinker with the values and the percentages, but the idea is to reward 2 score margins a little bit, and more than 2 score margins a little more than that.

Sent from MGoBlog HD for iPhone & iPad

Blue Hokie

November 10th, 2016 at 3:16 PM ^

To account for margin of victory, you could add half of the points awarded for a win greater than 3 scores (>17 points).  This would award blowout wins over good teams more than blowout wins over average teams.


For example: Michigan's win over Penn State would gain them 6 points (4 for a Top 10 win + 2 for a blowout win); Michigan's win over UCF would gain 1.5 points (1 for Group of 5 w/ winning record + 0.5 for blowout win); Michigan's win over Rutgers would still net 0 points :(


I like the rankings so far.  They certainly pass the eye test.


EDIT: I misread your point system.  The Rutgers win would gain 1.5 points.  Hawaii would have been a better example of a 0 point blowout win.

The Maizer

November 10th, 2016 at 3:30 PM ^

I like this idea (even though I'm not totally convinced margin of victory should come into this ranking), and Kzoo's idea above it. However, I'm still going to leave margin of victory out for now because I don't want to scour games to add in margin of victory data to the model (we're talking nearly 600 games so far). If someone has an easy way to pull this data (each team's margin of victory/defeat against each of their opponents), I'd love to hear it.

The Maizer

November 21st, 2016 at 4:45 PM ^

Thanks again! Unfortunately for the older ones, even after you delete the dates you still have different column widths so you can't easily separate scores from team names (especially because some team names have spaces so you can't use a space as a delineator). Could probably make a script to read it in but can't justify putting the time into it.

Lan Jiao

November 10th, 2016 at 9:56 PM ^

Major rugby tournaments integrate a margin-of-victory aspect into pool play. 4 points for a win, 2 for a draw, and 0 for a loss. However, a team can earn an extra point by scoring 4 tries (my Aussie/Kiwi friends still laugh at me when I say touchdown because they LITERALLY have to touch the ball down to the ground) and teams losing by less than 8 points get somewhat of a consolation point.


This wouldn't necessarily work for your hypothetical system because teams aren't attempting to hit these "bonus point thresholds," but this system seems to have worked in rugby. Yes, it was a change, but teams adjusted. You'll see a team up by 25 points working to score a try in the waning minutes or a team down 9 trying desperately to score a kick to get the extra point.


Biggest detraction to this is that it's in pool play where all teams play each other within the same pool, which isn't the case for NCAAF. Only works if everyone is playing the same slate of games.


November 10th, 2016 at 2:47 PM ^

So each week the teams are placed in a certain order.  Then each team that beat one of the top 25 on that ranking system gets extra points?  Won't that change the order again?  Do the teams that beat that new list of top 25's get additional points now? When does it end? Am I over thinking this?

The Maizer

November 10th, 2016 at 3:00 PM ^

Yes, it does change the order and the calculations are done again. And again and again. It ends when the calculation is redone and there was no change to the rankings. This week it only took 6 iterations before the model converged on a constant ranking for each team.


November 10th, 2016 at 3:43 PM ^

What about doing using the S&P+ and doing an inverse rating for wins, so 128 pts for beating the #1 team and 1 point for beating the #128 team, and subtracting loses with the actual rank, minus 1 point for losing to the #1 team and minus 128 points for losing to the #128 team. Zero points for an FCS team, minus 128 points for a loss. Maybe add in a home/away point modifier. Maybe this is too simple or is something that is already taken into account by S&P+. I'm not much of a statistics guy.

I did the top 6 teams in the CFP poll in this manner. I didn't add a home/away or margin of victory modifier.

1. Alabama - 817pts

2. Clemson - 695pts

3. Michigan - 606pts

4. OSU - 576pts

5. Washington - 421pts

6. Louisville - 410pts



November 10th, 2016 at 4:30 PM ^

it's just the old bowl system with a plus-one game.

Committee decisions at the point it matters:

  1. P5 conf winners minus one -- who is the minus one?
  2. P5 conf winners minus two with one P5 conf #2
  3. P5 conf winners minus one with a G5 undefeated -- who is that deserving G5 undefeated?
  4. P5 conf winners minus two with one P5 conf #2 and one G5 undefeated.

I guess multiple multiple-loss conf winners would make this bollocks, but it's still hard for me to see more than two P5 conf winners getting locked out.  Haven't looked at the history, but my admittedly poor memory doesn't bring a lot to mind.

The old bowl conference commitments are probably no worse than seeding 1-4, and adds the benefit of tradition/fan planning on locations.  

The championship game?  Well, that's special now isn't it?

All the lead-up is woo-woo-ESPN-makes-money. 

Not a fan of the lead-up.  You win your conference, you get a shot.  You don't and you are among the hopers.

The Maizer

November 11th, 2016 at 8:53 AM ^

I'm sure there is, but they already have zero positive points here since I didn't count Notre Dame as a P5 school. Maybe I can add a losing streak penalty: -10 points for losing 7 in a row. Or maybe a -10 points for losing to Rutgers penalty, if that comes into play tomorrow.

Blue Durham

November 10th, 2016 at 9:32 PM ^

some relevance to the discussion:

Michigan, not only has won every game to go 9-0, but has given EVERY TEAM THEY HAVE FACED THEIR WORSE LOSS EXCEPT MICHIGAN STATE.

Eight opponents out of nine, their worse game the entire season (so far, anyway) has been against Michigan. That is incredible.

That is a little different than scoring margin and to me speaks volumes.

I have been watching Michigan football since the mid 1970's. This is the best Michigan team I have ever seen, including the 1997 team.


November 11th, 2016 at 6:18 PM ^

Conference championship should be at least equal to, or greater than in my opinion, a top 10 win. Also, Note Dame absolutely should be considered a P5 team, they recruit like one and their schedule mostly consists of P5 teams. I'D include BYU as well, but understand them not being included. I love this idea though, and think the rankings look great!

The Maizer

November 14th, 2016 at 9:33 AM ^

First, remember that winning the conference championship means you also got an extra win over a probably top 25 or top 10 team if you're in a P5 and at least a winning G5 team if not (Big 12 excluded). Last year if I waited the conference championship more than this, it looked as though even bad division winners if they won the championship game would make the playoffs.

Second, I think you're right about Notre Dame, I'm going to make them count as P5.

The Maizer

November 14th, 2016 at 9:38 AM ^

Okay, I will consider this. Right now it would be too onerous to enter the data but I'll make a script to automate it and then it should be relatively trivial.

Curious, would it be best to compare to results before the bowl games (i.e. last CFP/BCS rankings)? Or after bowl games (would have to use AP poll)?