What do the numbers say?
The Bradley-Terry method applied to college football.
A couple of notes regarding the calculations. I use the Bradley-Terry method for determining the ratings. This is an iterative, statistical rating that computes a hypothetical round-robin winning percentage if all teams played each other. Clearly, that's not the case in college football, and this method gives infinite results if teams are undefeated. This problem is 'solved' for the sake of comparison by adding a fictitious tie to each team's record.
- Game results are pulled from the NCAAFootball.com.
- Blogpoll results are pulled from SBNation.
- There are a lot of explanatory notes and links; I put those at the end of the post so people who don't care about them can skip them and get right to the results. There is also a link to all my results.
- For brevity, I only listed the top 20 here. For those who are interested, I also listed Michigan's position, FYI.
- I release this after all the major polls come out to avoid 'influencing' anybody's vote.
To the numbers...
|Through games of 2010.11.06|
Auburn is pulling away as the number one team. Oregon has finally caught up with everybody's ranking due to strength of schedule, and Boise St. has begun to fall for the same reason. The bonus for being undefeated this far into the season is starting to balance out with perceived strength of schedule, as LSU is nipping at the Broncos heels.
As a point of comparison, it lines up really well with the blogpoll. Two outliers: blogpollers really like Ohio State, and they really dislike Virginia Tech (see discussion of limitations, below).
There's a pretty significant drop-off from #5 LSU to #6 Stanford, and another notable drop-off from #10 Wisconsin to #11 Utah. I think this supports the notion that a 16-team tournament would be sufficient to include all the top teams. If you're in the muddle around 16, there really isn't all that much to complain about if you're left out.
The conference breakdown in the top 10 and top 16 (non-BCS conferences in parentheses):
|Conference||Top 10||Top 16|
The top 10 has roughly equal representation from the BCS conferences. Looking at the top 16, a bias toward the SEC begins to emerge, with 5 teams, including those in spots 14-16. Not surprisingly, the Big East is absent, and the ACC is, well, underrepresented. TCU, Boise St. and Utah are ranked above all comers from these conferences, including Virginia Tech, who lost to Boise St. and, as we all know, James Madison. The nagging question remains: how do you compare the relative strengths of conferences when they don't play each other?
The next few weeks should be interesting.
Discussion of limitations
That said, there's always a bit of resistance when I post this rating. It's one additional data point. It's not even my opinion, and it doesn't mean your team is better or worse than you think it is. It attempts to look objectively at how teams would fare, should they play every other team. There are some limitations, namely the infinite results, and incomparability, of undefeated teams. As with any statistical calculation, sample size is important; while there are only ~12 games per team, there are ~120 teams. One could argue the merits of using any sort of statistical calculation on said sample. Also, it should be pointed out that games against FCS teams are ignored. This is a double-edged sword: teams don't get credit for beating up on FCS teams, but Virginia Tech effectively gets a pass for losing to James Madison.
Brian pointed out another interesting anomaly (it's the double star at the very bottom) in last year's end-of-season college hockey KRACH. A similar effect can be seen in this rating, as discussed above. So, why does this happen? Like college football, there's little overlap between conferences, teams tend to get compartmentalised.
As with any tool, it's only as good as its user; we can't blindly take the results as fact. One possible solution is to take the top 30 teams at the end of the season and run a KRACH on only those teams. Although, for any hypothetical tournament, I would strongly support the inclusion of all conference champions.
What if I want to see the entire rating, and results for each week?
All the results are available, if you'd like to see the numbers yourself. As I said last year, John Whelan freely gave me the perl script in 1998 to calculate KRACH for ACHA club hockey teams, so I'm happy to share the script and input data if you don't want to write it yourself. And I am fallible. There's a lot of data to crunch, and I copied and pasted from the NCAA site; there may be errors. If you find one, please bring it to my attention and I'll make the fix posthaste.
- Texas overtakes TCU
- Oregon and Georgia Tech swap
- LSU jumps 7 spots after a similar fall last week.
- USC jumps 7 spots after beating UCLA
- BYU is up 10 spots after knocking off Utah
- Pittsburgh drops 10 spots after losing to West Virginia
Interestingly, I don't think there's any way Florida or Alabama falls further than 4th; and they end up with the top 2 SOS rankings by a significant margin (the SEC takes 6 of the top 7 spots here). EDIT: If all the underdogs win this week, Alabama and Florida end up 1 and 2, respectively.
This week's top 20:
|Team||Pvs||+/-||Rank||KRACH||RRWP||Record Rank||W||L||T||Win %||SOS Rank||SOS|
Apologies for the tardiness of this post - I meant to post this when the blogpoll came out, but I missed that, then it was the weekend. And there's not much change at the top.
I'll post this week's results after the blogpoll comes out.
Here's the top 20 from last week (through games of 2009.11.22):
|Team||Pvs||+/-||Rank||KRACH||RRWP||Record Rank||W||L||T||Win %||SOS Rank||SOS|
I put a bit of effort in cleaning up my code to address some of the problems from last week:
- Undefeated and winless teams: I added the fictitious tie. Now there's a path from every team to every other team. While this is a hack, it's reasonable and about as far as I really care to go with it. I also calculated the round-robin winning percentage (RRWP) and KRACH strength of schedule (SOS). These are useful metrics for comparing teams that don't line up very well, as per John Whelan's KRACH site. Now the undefeated and winless teams can be compared with multiple data elements.
- Rasmus: Open source: The original Pairwise and KRACH code was freely given to me years ago by John Whelan. This code is my own, based only on the information given in his KRACH site; given that, I'm comfortable sharing my code, so long as the user gives due credit to John and Ken Butler.
- joeyb: Undefeated teams will always rate better than teams with losses: To investigate this, I created a fictitious team that was 10-0 with wins against only the bottom ten teams. It rated out at #37, well below several 1- and 2-loss teams.
- SpartanDan: Top teams are ranked backwards: You are correct, sir. I mistakenly assumed that a larger deviation from predicted meant a less accurate KRACH. This has been fixed, as in 1) above, with the RRWP, SOS and fictitious tie.
- Seth9: The rating doesn't apply to college football: You may be right on this. It's still curious, and now trivial, for me to crunch the numbers. EDIT: But is this any less applicable than any of the computer ratings used in the BCS? And it certainly does not have the bias and politicking associated with opinion polls.
Again, this rating includes all D-IA games through 15 November 2009:
|Team||BlogPoll||BCS||Rank||KRACH||RRWP||Record Rank||W||L||T||Win %||SOS Rank||SOS|
|Middle Tennessee State||64||0.943||0.490||8||7||3||0||0.700||102||0.440|
|North Carolina State||84||0.510||0.385||22||2||6||0||0.250||50||1.326|
|San Diego State||92||0.303||0.303||20||3||6||0||0.333||93||0.562|
|New Mexico State||113||0.079||0.141||23||2||7||0||0.222||118||0.236|
|San Jose State||116||0.048||0.101||29||0||8||0||0.000||75||0.815|
A few years ago I applied Ken's Rating for American College Hockey (KRACH), or Bradley-Terry statistics, to ACHA club hockey teams. At the time, participants for the national tournament were determined by an opinion poll, and there wasn't enough interplay for that to be meaningful (sound familiar?).
In an earlier post here, I intimated that I'd like to see someone crunch the numbers as a mechanism for rating Division I-A college football teams. It's something I've been thinking about for quite some time, just to see what would happen. So tonight I threw something together.
"The KRACH rating system is an attempt to combine the performance of each team with the strength of the opposition against which that performance was achieved, and to summarize the result as one number, a "rating", for each team. The higher the rating, the better the team."
"Interpreting the ratings
The ratings are given on an "odds scale": that is, if team A is rated at 400 and team B at 200, team A is reckoned to have odds of 2 to 1 of defeating team B when they meet (since 400 is twice 200). Equivalently, team A is reckoned to have probability 2/3 of defeating team B (since 400/(400+200) is 2/3).""There are two things we need to check, to make sure that the rating system is sensible:
- If you win more against the same opposition as another team, your rating will be higher.
- If you have the same record as another team, but against tougher opposition, your rating will be higher."
So, I took the season results from the official NCAA page. I excluded results against FCS competition, as a matter of principle.
One caveat here - I haven't yet worked out what to do exactly with undefeated and winless teams. This will become meaningful at the end of the season if there are multiple undefeated teams (I'm not sure I really care about the winless teams). While I sort that out, I've done the following:
- Verified my calculated rating by calculating the predicted number of wins;
- Determining a percentage difference between the predicted and actual number of wins;
Without further ado, the first KRACH rating for Division I-A college football:
|64||North Carolina State||0.561||2.000||2||0.017|
|73||Middle Tennessee State||0.363||5.999||6||0.022|
|88||San Diego State||0.147||2.999||3||0.019|
|111||New Mexico State||0.009||2.000||2||0.017|
|119||San Jose State||0.001||0.006||0||0.000|
i've liked the RR hire since i heard of it. i just missed the bo era, but the predictable and underachieving teams since the early 90s were just maddening. sure, we won big ten titles. and once in a while we knocked off an sec team in a bowl game. but for the most part, we lost big (or worse, little) games it seemed like we should have won. there are a couple of anomalies - i don't know where we pulled that nc from in 97, nor the defeat of florida in 2007.
don't get me wrong - i have a lot of respect for lloyd carr, and i'm proud to have had him as coach at michigan for so long. he brought an air of responsibility and respectability to a fan base that really believes we do things the right way.
which brings me to my point. there's no denying these players spend upwards of 40 hours a week on football. how many of those are "voluntary" or mandated, honestly, i don't think there's a distinction. a virtual distinction, sure, but in reality, they're all mandated. i'm ok with that. to win, you have to be willing to make the sacrifices and put in the time.
if we're guilty, i'll take my lumps. ok, we broke the rules. give us our punishment...
but then go out and punish the other 100+ programs who do exactly the same thing. i doubt that michigan is unique in their interpretation and application of the "rules" regarding practice time. to keep up with the joneses, you have to work as hard as the joneses. so i do believe that everybody is doing it.
but what about the student side to these student athletes? is everybody going to stop doing "it"? or are the rules archaic enough to be changed? and if the rules are changed, what should they be changed to?
this is not really a new suggestion. maybe it's not particularly original. but i haven't seen it anywhere since this thing exploded. give these students a "professional sports" major. throw in some football (or hockey, baseball, basketball, rowing, golf, etc.) as part of their class load, and give them credit for it. that's quality learning that could (should?) lead to the attainment of a degree and definitely leads to the development of important life skills. throw in some finance and money management courses. maybe something on how the professional sports operate as businesses; what it means to have and/or be an agent; and, if you want, throw in the rest of the general ed requirements that all the rest of us have to take. during the season, let the sport be 8/12 units they're enrolled in. if you don't want to take up the whole year eg hockey, go to a quarter system that may more directly line up with the seasons.
not all these students will go pro. their degree will still serve them well. brian has raised many examples of athletes who used their "general studies" degrees to great advantage. and this is no different than art students, or architecture, music, english, history, and even some engineering students (not saying there's anything wrong with any of those majors - my sister is an art major and very happy; i have degrees in engineering and biology, and am working on a degree in education).
i don't like the idea of paying student athletes - let them go pro if they want to get paid. to me, college athletics is great because of the players and their allegiance to their schools (most of them, at least). but in this day and age of the huge cash grab at the expense of these students, give them a break.
i, for one, appreciate the culture change that's come with coaches rodriguez and barwis. i don't understand how suddenly coach lost his integrity when he left wvu, but wasn't a bum at wvu, clemson, tulane or glenville state. i don't see it. from the outside, and where i'm sitting, all the negative press has been largely vindictive, with little substance. and i think he's doing an outstanding job with our student athletes.
in summary, i think some ncaa policies are a bit outdated and could use some refreshing. while the ideas presented here may seem ridiculous to some, i would argue that some creative thinking would go a long way to reconciling big-time college athletics with the concept of student athlete that so many people (including me) cling to.
scott quakkelaar '93