Introducing MOVE Ratings

Submitted by 1464 on

For the past few years, I have attempted to create an objective look into conference superiority.  I was sick of the SEC love, and felt that I could develop a metric which allowed for an accurate indication of how the conferences stacked up, sans TV contracts, media bandwaggoning, and regional affiliation. 

Before, I used a method that assigned points to each conference based on the W/L percentage of the conference they beat, which I called CPR (Conference Power Ranking).  The more I picked over it, the more I realized that the CPR had one fatal flaw - beating Purdue was exactly equal to beating Ohio State or Michigan.  It assigned the same number of points.  My argument was that over the course of a season, those would balance out, but that was a pretty hollow argument.

This year, I've come up with what I consider a better method of tracking conference power, which I have dubbed the MOVE Rating.  Sounds sweet, right?  That's because a metric is only as good as it's acronym (Margin of Victory Evaluation).  Has a nice ring, right?

So what is MOVE?  Because of a small sample size (10-20 out of conference games against a BSC Qualifier for each conference), I set out to attempt to make every game an average vs. average scenario.  I feel I have achieved this by using the following formula to handicap the games:

-(Team CMARG-Opponent CMARG*) + AM = MOVE POINTS

*expressed as the EM or expected margin

In this formula, CMARG represents a team's "conference margin" (margin of victory, but a negative number represents an average loss) in that school's conference.  So To give an example, Michigan's CMARG over B1G schools last year was 16.38.  This means that Michigan beat the "average" B1G team by 16.38 points.  That formula is simple, add up all the margins of victory, including negatives, and divide by the number of conference games.  Instant CMARG!  So since Michigan won by 16.38 over the average B1G team and Alabama won by 24.12 over the average SEC team, the EM (expected margin) of that game was Alabama -7.74.  The final tally saw Michigan lose by 27. 

That is represented by this for Michigan:

-(16.38-24.12) + -27 = -19.26

and this for Alabama:

-(24.12-16.38) + 27 = 19.26

What that boils down to is that the AVERAGE SEC team was 19.26 better than the AVERAGE B1G team, according to the results of that match.  This also accounts for bad teams.  Por ejamplo, Illinois lost to Arizona State by 31 points.  The MOVE Rating on that game saw Illinois lose 0.40 points for the B1G, as Illinois was expected to lose by 30.6 points, the EM on that game.  Their CMARG was -23.38, while Arizona State's CMARG was 7.22.

So now that you see a couple of games worth of MOVE ratings, all you have to do is throw all of a conference's MOVE scores in a pot and divide by the total number of games to receive a MOVE rating for the conference.  It's important to note that I am only evaluating the 5 auto-qualifying conference at the time being.  I may expand my data to the entire FBS if I have enough time.

Now, there are still some flaws to this system.  It does consider each conference to be equal, so if your conference plays a bunch of ACC schools, there will be a bit of a uptick in your MOVE as compared to if your conference plays a bunch of SEC schools.  I plan to mitigate that in one of two ways - either take all the conference vs. conference MOVE ratings and divide by 4, or by comparing the MOVE rating for each game compared to the opposing conference MOVE rating, find the difference, then assign a "MOVE2" rating.  How much did you beat a team by MORE than the average team beat that conference?  For the time being, we will just allow the MOVE rating to stand on it's own.

Ready to see some numbers?  I decided that to test my system, I would go back to 2012 and plug in all the data.  Let's just say I was disappointed with the results.

 

Here's your first look at actual data.  It's listed in decending order by the MOVE scores.  What it says is that the SEC is, on average, two touchdowns better than the average AQ team.  Yikes.  Also notice that aside from the dismal ACC, the B1G did not do well.  Not well at all.  What happened to me disproving the superiousness of the SEC or the baditude of the B1G?  I'll go conference by conference, but first a couple of notes.

GAMES = Number of games played against AQ schools, including bowls.
W% = Win percentage in those games.
MARG = Average margin of victory (or loss) in said games.
MOVE = Average MOVE score in those games.

- Having a MARG that is noticeably higher than your MOVE indicates that, on average, you are sending out your better teams to play against inferior opponents.  For the B1G, think the opposite of "Rose Bowl, Illinois vs USC".

- If you add up all the MOVE scores in this chart, it will not equal 0, however if you multiply the GAMES by the MOVE, then divide by the total number of GAMES, it will be close.  It does actually 0 out for auditing, but the fractions are rounded, so the number is a bit off.

On to the conferences...

 

- The ACC looks worse than they actually are, as more than half of their games are against the SEC.

- Their best performance was actually a 9-point Boston College loss to Northwestern.  BC was a 22-point dog, as Nortwestern was good and BC lost to a weak ACC by an average of 15.25.  They gained 13 points in that matchup, despite walking away with a loss.

- The most out of whack stat?  A 7-point Clemson win over Auburn netted an ugly -36.  That's because Auburn was a 43-point dog, after being smashed by the SEC and playing a Clemson team that went 7-1 in the ACC for an average CMARG of +19.13.

 

(As the B1G is our conference, I will go team by team.  It's... not pretty.)

Illinois - As mentioned before, Illinois lost by 31, and yet still almost broke even against an undermatched Arizona State team.  They netted -0.40 MOVE on the year.

Indiana - Did not play an AQ school all year.  Sadly, this made them the B1G's third best performing team, as 9 B1G teams scored a negative MOVE score.

Iowa - Netted a 0.04 for the year for losing by 3 to an Iowa State team that performed only slightly better in the Big 12.  They finished second in the entire B1G in MOVE.  With a 0.04.  Maybe the ESPN talking heads were right...

Michigan - With great power comes... a 16.38 CMARG.  This caused us to lose 19.26 points to Bama, and 13.88 points to South Carolina.  We were actually 8.88 point favorites in the SCar game, as their CMARG was only 7.5 in the SEC.  The problem with being the big boys in a conference is that you have to produce.  We did not, even in a close loss to SCar.  Our MOVE for the year, fourth worst in the conference with -16.57.

Michigan State - A 1-point win over TCU in their bowl game netted a -1.25 on the year.  Both MSU and TCU were very close to average, with MSU gaining a 1.25 CMARG and TCU holding a -1.00 CMARG for the year.

Minnesota - Conference MOVE champion!  Minnesota represented the B1G better than any team, by averaging a -10.38 CMARG, while falling to Texas Tech by only 3 points.  This gave Minnesota a 1.82 MOVE rating on the year.

Nebraska - In a word, bi-polar.  How else do you explain a CMARG of -0.33 while going 7-2 in the B1G?  Oh yeah, giving up 70 to Wisconsin and  63 to OSU will do that.  But it is worth repeating, Nebraska went 7-2 in the B1G last year and STILL managed to have a negative margin of victory.  That's amazing.  Overall, they perfomed to expectations in their OOC schedule, losing to UCLA by 6 but gaining a fraction of MOVE (EM was -6.03) and giving a fraction of MOVE to Georgia by losing the bowl game by 14 (EM was -13.56).  They finished at -0.21.

Northwestern - Won all three of their games, but due to an un-NW like 5-3 record in conference, gave up -1.33 MOVE.  NW exceeded expectations against Vandy and Miss State, but lost -13 MOVE points to BC in their 9-point victory.

Ohio State - The expected margin of the Cal game was 25.25, but they only won by 7.  Good for the third worst MOVE in the league, at -18.25.

Penn State - Speaking of bad, PSU gave -21.75 MOVE to Virginia in the 1-point loss.  The EM on that game was 20.75.   The -21.75 was their total on the year, good for second worst in the B1G.

Purdue - Bad.  They managed to go -25.72 on the year by losing to Oklahoma State by 44.  Worst in the conference.

Wisconsin - Charitable to the PAC-12.  In two losses close losses, they gave double digit MOVE points to both Oregon State and Stanford.  Finished with a -10.67 MOVE.

 

- Second only to the SEC in MOVE.  They actually outperformed every conference they went up against, even though they had the average of a 3-point loss to the PAC-12.

- Baylor and Texas led the way with 28.03 and 24.34 MOVE ratings, respectively.

- The low point of the year saw Oklahoma State, a 2-TD CMARG favorite lose to Arizona by 21, good for a -35.11 beatdown.  Oklahoma State, clearly concerned about how this would affect their MOVE, then throttled Purdue by 44.

 

- The MARG was higher than the MOVE for the PAC-12 in each conference they played.  This is because bottom feeders Colorado, Washington State, and Utah all played no AQ schools. 

- A 2-8 PAC-12 team in Cal lost to OSU by 7, gaining 18.25 points for the PAC-12.  The aforemetnioned  Arizona was the big winner though, getting 35.11 points for their 3-TD victory over Oklahoma State.

 

- All (begrudgingly) hail your power conference.  The SEC was 12-5 against AQ schools and on average, an SEC school is worth 2-TD more than their non-SEC equivalent.  That really hurts me to write.  The good news is that the SEC is looking far weaker this year.

 

So... 2013?

I won't be releasing any MOVE data this year until November, as the stats don't mean much until we get deeper into conference play.  The good news is that the B1G has already gone 3-0 against AQ schools.  Last year, it was 5-11 (the Lions special) all year.  So going 2-11 will be a push type thing.  OSU whould beat Cal, from there we only need 1-2 wins to exceed last year.  The SEC will also see a big dropoff, as heavyweights Georgia and Florida have already lost OOC.  As they are expected to do well within the SEC, that will translate to losing points as well.

Comments

One Inch Woody…

September 9th, 2013 at 5:18 PM ^

Nice work.

Everyone knows last year was a down year for the Big 10. It appears that we finally have some coaching stability at the major programs and enough middling talent to win against middling teams in other conferences. I expect to see the Big 10 be the 3rd best conference this year sandwiched between the ACC and the Pac 12 now that we finally have some coaching stability and the recruits gleaned from said coaching stability are starting to make some impacts.

Your analysis of 2012 undoubtedly shows that your metric is flawed, however. By failing to differentiate between in-league tiers, you get inflation or depression of the metric depending on how bad the bad teams are in the league and how good the best teams are in the league. The SEC is a regular offender in inflation of SOS because there are enough bottom tier teams, which, while not being abhorrent, are simply bad. In addition, the top tier teams were the ones billed for OOC games in 2012 and they were far better than their opponents. However, if for example Michigan were to play Alabama's schedule, one would expect Michigan to win nearly the same number of conference games that Alabama did (Arkansas, Ole Miss, Missouri, Tennessee, Mississippi State, and Auburn). There needs to be a way to account for the parity or lack thereof in a league. Maybe take total offense/defense into account?

Keep up the good work!

1464

September 10th, 2013 at 1:15 PM ^

I think that the CMARG adjusts for this.  A team plays 8-10 in-conference games a year, which represents at least 70% of a conference.  Their scoring margin is probably pretty representational as to where they fit into the conference as a whole.  I think that the issues here are due to small sample size and the fact that I don't yet have a metric for scoring MOVE points on a sliding scale, depending on the relative strength of the opposing conference.  I'm hoping to have this solved when I release this year's preliminary ratings.  The small sample size, though, will always be a problem.