Tracking KenPom, Round 2
Or, why my bracket is a mess.
To follow up on the previous KenPom charts and graphs, I decided to pick my NCAA tourney bracket based on Ken's predictions and see how accurate he is. The way I used the data is as follows: I assumed that M = (AdjOffence - AdjDefense)* (AdjTempo)/100 gives an average margin of victory for the dataset. Then, M1 - M2 = margin of victory difference between the two team playing. To apply to the Michigan/Ohio State games gives
Michigan = (107.0 - 92.7) * (62.7/100) = 8.99
Ohio State = (118.9 - 89.8) * (65.8 / 100) = 19.1
Which predicts a 10.1 point margin of victory for OSU, pretty close to the actual KenPom prediction.
I'll save you all the eye chart of the data table. If you're interested, it's here:
http://spreadsheets.google.com/ccc?key=0AtVjC1g8WVJfdFFJR0F4ZEhRNGQyVTZ…
The relevant data:
Calculating the probable winners in this fashion gave a win/loss of 24/8. And, four of those that are wrong were predicted to be 3 point games, and ended up +/- 3. Here's the corresponding chart.
I calculated the total average margin of error (absolute value) for all games at 7.44, and margin of error in games correct at 7.07, and margin of error in games wrong at 10.6.
I next calculated the distribution of error. Since I used absolute value in the previous calculation, I ended up with half a bell-curve distribution. Data:
Chart:
What's interesting is that this is a better prediction than just using KenPom as a relative rating. By picking solely based on the higher ranked team, the record is 23/9.
Conclusion:
If you can draw any conclusion from all this, it is that Ken is pretty accurate, except when he's not. I didn't expect to be 100%, because I don't think any system out there will predict Georgetown, or Kansas or Villanova to lose, based on the numbers. But, by this point in the season, the system is remarkably accurate in predicting probable outcomes. It has some margin for error in predicted close games, but I don't think there's any system that would be able to predict close games, either. They just come down to the luck of the draw.
To follow up on the previous KenPom charts and graphs, I decided to pick my NCAA tourney bracket based on Ken's predictions and see how accurate he is. The way I used the data is as follows: I assumed that M = (AdjOffence - AdjDefense)* (AdjTempo)/100 gives an average margin of victory for the dataset. Then, M1 - M2 = margin of victory difference between the two team playing. To apply to the Michigan/Ohio State games gives
Michigan = (107.0 - 92.7) * (62.7/100) = 8.99
Ohio State = (118.9 - 89.8) * (65.8 / 100) = 19.1
Which predicts a 10.1 point margin of victory for OSU, pretty close to the actual KenPom prediction.
I'll save you all the eye chart of the data table. If you're interested, it's here:
http://spreadsheets.google.com/ccc?key=0AtVjC1g8WVJfdFFJR0F4ZEhRNGQyVTZ…
The relevant data:
| predicted | actual | |||||
| team | adjusted tempo | adjusted offense | adjusted defense | difference | rd1 | |
| Duke | 66.6852 | 121.47903 | 85.90015 | 23.72584729 | 28.71596632 | 29 |
| Kansas | 68.3863 | 121.44865 | 86.07246 | 24.19246742 | 24.84369214 | 16 |
| Wisconsin | 60.2126 | 116.45837 | 87.28726 | 17.56468378 | 11.29768539 | 4 |
| Ohio St. | 65.8254 | 118.95206 | 89.81885 | 19.17705202 | 17.99644835 | 17 |
| Syracuse | 70.746 | 117.86636 | 89.08539 | 20.36138504 | 17.3322846 | 23 |
| Kentucky | 69.4046 | 115.499 | 87.67961 | 19.30793635 | 17.37462094 | 29 |
| Brigham Young | 72.8212 | 117.39741 | 89.58698 | 20.25188885 | 8.889108347 | 7 |
| West Virginia | 63.4144 | 117.45049 | 90.03022 | 17.3883997 | 15.9712738 | 27 |
| Kansas St. | 71.0506 | 115.80342 | 88.85456 | 19.14732672 | 18.92650121 | 20 |
| Maryland | 70.1779 | 119.08432 | 91.72191 | 19.20236473 | 14.6032751 | 12 |
| Georgetown | 66.568 | 117.55443 | 90.86621 | 17.76581429 | 12.0774709 | -14 |
| Baylor | 65.821 | 119.55356 | 92.74282 | 17.64709718 | 12.01782068 | 9 |
| Purdue | 67.1526 | 109.82044 | 86.43841 | 15.70164108 | 5.331953813 | 8 |
| California | 68.4634 | 121.03346 | 95.61323 | 17.40355375 | 4.205805859 | 15 |
| Villanova | 72.8674 | 118.6973 | 93.87151 | 18.0899077 | 19.06129835 | 3 |
| Clemson | 69.0581 | 110.39498 | 87.65073 | 15.70674691 | 0.24014592 | 8 |
| Texas | 72.7178 | 113.39859 | 90.23584 | 16.84344222 | 5.30461363 | -1 |
| Temple | 61.2054 | 107.49016 | 85.70485 | 13.33378613 | 4.035245019 | -13 |
| Florida St. | 66.8561 | 105.16445 | 83.88638 | 14.22568776 | 2.918960477 | -7 |
| Utah St. | 61.5388 | 116.42159 | 92.9556 | 14.44068865 | ||
| Missouri | 70.5163 | 109.76675 | 87.83338 | 15.46660099 | ||
| Xavier | 70.7384 | 115.82011 | 92.69146 | 16.36083695 | 1.677395019 | 11 |
| Texas A&M | 65.6301 | 111.89438 | 89.90554 | 14.43129768 | -0.009390973 | 16 |
| Michigan St. | 67.1182 | 112.03972 | 90.26292 | 14.61619618 | 10.01710655 | 3 |
| Minnesota | 66.6113 | 114.14477 | 92.1013 | 14.68344193 | ||
| Butler | 65.1637 | 109.60831 | 88.44321 | 13.79196227 | 0.483029584 | 18 |
| Georgia Tech | 69.6931 | 109.49437 | 88.67489 | 14.50974102 | 2.127220831 | 5 |
| Marquette | 63.7768 | 114.2881 | 92.58042 | 13.84446366 | -1.604015104 | -2 |
| Washington | 73.212 | 112.62146 | 91.52044 | 15.44847876 | ||
| Pittsburgh | 61.8641 | 111.69304 | 90.87344 | 12.87985816 | 10.70287965 | 23 |
| Virginia Tech | 69.215 | 107.40448 | 87.82223 | 13.55385434 | ||
| Northern Iowa | 59.905 | 107.33922 | 87.87319 | 11.66112527 | -0.983365598 | 3 |
| Old Dominion | 64.0632 | 107.91845 | 88.5319 | 12.4196443 | ||
| Texas El Paso | 69.45 | 107.47143 | 88.3081 | 13.30893269 | ||
| Tennessee | 69.4899 | 106.15591 | 87.3393 | 13.07564347 | 0.873802425 | 3 |
| Vanderbilt | 69.7518 | 114.01593 | 94.1034 | 13.8893481 | 3.130584581 | -1 |
| Nevada Las Vegas | 67.3486 | 109.46974 | 90.69505 | 12.64449087 | ||
| Notre Dame | 63.7293 | 119.82189 | 99.29155 | 13.08384197 | 0.66419767 | -1 |
| Louisville | 67.2914 | 114.92586 | 95.31303 | 13.19774789 | ||
| San Diego St. | 64.6273 | 110.97383 | 92.09351 | 12.20184105 | ||
| Mississippi St. | 67.3041 | 109.28839 | 90.90574 | 12.37227714 | ||
| Arizona St. | 62.2639 | 110.10876 | 91.62218 | 11.51046568 | ||
| St. Mary's | 67.99 | 114.92716 | 95.70062 | 13.07212455 | ||
| Oklahoma St. | 68.0078 | 112.20736 | 93.99986 | 12.38252019 | ||
| Dayton | 66.2161 | 105.97277 | 88.83869 | 11.34551955 | ||
| Miami FL | 66.4123 | 113.37386 | 95.10575 | 12.13227202 | ||
| New Mexico | 68.6401 | 114.31427 | 95.96488 | 12.59503965 | 7.920566861 | 5 |
| Richmond | 64.1081 | 108.36048 | 91.18129 | 11.0132523 | -2.058872242 | -9 |
| Florida | 65.4107 | 112.27696 | 94.90552 | 11.3627805 | ||
| Wake Forest | 70.2826 | 106.582 | 90.16424 | 11.53882859 |
Calculating the probable winners in this fashion gave a win/loss of 24/8. And, four of those that are wrong were predicted to be 3 point games, and ended up +/- 3. Here's the corresponding chart.
I calculated the total average margin of error (absolute value) for all games at 7.44, and margin of error in games correct at 7.07, and margin of error in games wrong at 10.6.
I next calculated the distribution of error. Since I used absolute value in the previous calculation, I ended up with half a bell-curve distribution. Data:
| +/-5 | +/-10 | +/-15 | +/-20 | +/-25 |
| 14 | 9 | 4 | 4 | 0 |
Chart:
What's interesting is that this is a better prediction than just using KenPom as a relative rating. By picking solely based on the higher ranked team, the record is 23/9.
Conclusion:
If you can draw any conclusion from all this, it is that Ken is pretty accurate, except when he's not. I didn't expect to be 100%, because I don't think any system out there will predict Georgetown, or Kansas or Villanova to lose, based on the numbers. But, by this point in the season, the system is remarkably accurate in predicting probable outcomes. It has some margin for error in predicted close games, but I don't think there's any system that would be able to predict close games, either. They just come down to the luck of the draw.
engineers
I was a film major so I'm not even capable of digesting your (excellent, I'm sure) chart. But here's my Ken Pom experience.
I stopped following basketball closely a couple of years ago and only really follow M until tourney time. Then, because I'm an American dammit, I throw a little money into a "bracket" pool. This year, I decided to just put myself in Ken Pom's hands and used his rankings straight up. In my pool, extra points are awarded for correctly picking upsets so in a couple of cases if the teams were closely ranked I went with the 'dog. In the first round I blew away the competition.
Then came the weekend. In round 2, everything just went to shit and after going 3 for 8 on Saturday I just stopped paying attention.
I don't blame Ken Pom. But round two just seemed in a lot of ways kind of random.
If you look at KenPom ranking history, no one outside the top 5 in his rankings has won the NCAA tournament. The majority of the time it's the 2 ranked team that takes it down. Last year MSU became the lowest ranked team to make the finals at the number 8 slot.
KenPom does a pretty good job of "getting it right", he may just take the "wrong" way to get there.
Comments