Ken Pomeroy rankings not very useful in predicting men's hoops
I went back and looked at all of Dylan's previews, noting the Pomeroy predictions for each Big Ten game. They predicted:
Michigan by 11 over PSU(check)
Michigan by 4 over Minnesota(check)
Indiana by 12 over Michigan(check)
Wisconsin by 7 over Michigan(nope...79% chance of badger win)
Michigan by 5 over Northwestern(check)
Michigan by 4 over Iowa(nope)
MSU by 4 over Michigan(nope)
Purdue by 4 over Michigan(nope)
You might as well flip a coin.
If you look at the actual scores, they're all over the place, ranging from Wisconsin(underpredicted by 25 pts) to Iowa (overpredicted by 20) you come up with Pomeroy underestimating Michigan by about 3.5 points/game. (I expected a larger number too, but that Iowa game really knocked the average down).
January 25th, 2012 at 4:22 PM ^
we sucked it up in the iowa game and surprised everybody by spanking wiscy. you might think both of those would have been pretty hard to predict. and then the msu and purdue misses were pretty damn close.
but like i said on brian's purdue preview, i hope he keeps using kenpom to make his predictions, because they've been working out for us so far.
January 25th, 2012 at 4:24 PM ^
I fixed the title. Thanks.
January 25th, 2012 at 4:22 PM ^
Can't really fault them for the Wisconsin or the Iowa game, nobody saw those scores coming.
January 25th, 2012 at 4:22 PM ^
doesn't really seem like a very comprehensive study. yours is a pretty small sample size. the question is how well he does when applying his prediction to hundreds of games. also, you have to keep in mind that if his simulator predicts that team A will win by 1 and team B wins by 1, it still was really close despite the fact that he predicted the wrong winning team. for the most part, his stats do a good job.
January 25th, 2012 at 4:41 PM ^
I guess I am not trying to say its not useful for the entire population of college basketball, it just doesn't seem very useful in predicting Michigan's games(and in truth, these are all I really care about).
I guess this was a bit of a response to all the people who are saying that Michigan cannot win the big ten because their statistical profile isn't what it should be. There's no number in that formula for will, and I dont think we've seen everything that those sophomores have in store for us yet. I'm not saying its going to happen, but after OSU Michigan has as good a chance as anyone.
January 25th, 2012 at 4:23 PM ^
Well... stop gambling on the games according to K Pom predictions and you won't be so mad. Bro.
January 25th, 2012 at 4:23 PM ^
that 8 game sample is absolutely enough to draw that conclusion.
January 25th, 2012 at 4:41 PM ^
Come on man. You just put a man's life work on trial with an 8 game sample?? You are using .0006 % of the available data**. Based on this logic I could prove that the Detroit Lions win their home games by and average of 30pts a game.
I was hoping for a minimal amount of analysis which would be this years total games vs the spread. That total would be in the thousands for just a half year.
Just lock this thread. It offers nothing.
** Yes I did the math and it comes out to exactly that number.
/s
January 25th, 2012 at 8:01 PM ^
Is this Ken I'm talking to? Usually threads are locked when someone says something really offensive. I had no idea you'd be this sensitive...Is there anything you want to talk about?
I had considered labeling this thread, "Ken Pomeroy predictions not useful in predicting B1G Michigan's men's basketball games", but then I thought that might take up 3 or 4 lines of the topics board...Next time I will be more specific so as not to offend your delicate sensibilities.
January 25th, 2012 at 8:06 PM ^
Brilliant response. Made me laugh out loud at Ziffy's peculiarities
January 25th, 2012 at 5:00 PM ^
I did a Wilcoxon signed-rank test on Michigan's 8 Big Ten games to estimate the probability that the Kenpom predictions differ significantly from the final scores of the games. The set of differences between the Kenpom predictions and actual final scores is {-7,-1,-10,-25,+3,+20,-5,-6}. Based on those differences, I get a critical value of S=9, which corresponds to a probability of about 20% that the median score margin predicted by Kenpom is not the same as the actual median score margin. This is not statistically significant, so I won't reject my null hypothesis that Kenpom is doing an OK job.
Based on most of the numbers in that set of differences being negative, it wouldn't be unreasonable to wonder if Kenpom is underrating Michigan with respect to the rest of Big Ten, but there's not enough data to prove of disprove it.
January 25th, 2012 at 8:04 PM ^
I thought it might reach significance, but then again I had to take Stats 402 twice...
January 25th, 2012 at 9:08 PM ^
The lesson here is that if you don't do your stats homework on MGoBlog, five people are going to jump in and do it for you.
January 25th, 2012 at 4:26 PM ^
Ken Pom does a good job, but basketball is a very unpredictable game. A team can shoot 65 percent one game and 25 the next. Refs can call the games loose or tight, and often decide that one team is a bunch of thugs while the other has odorless feces, even though they are both playing the same.
If basketball was easy to predict, a lot of people would be making a living out of gambling on it. I would imagine that not too many do unless they are "fixers."
If anyone gets to be better than fifty percent against the spread on a regular basis and are just writing about it, they are in the wrong business.
January 25th, 2012 at 4:31 PM ^
it'd be interesting to see kenpom's record against the vegas spread--i.e., whether or not it's a (historically) winning strategy to make bets in vegas by assuming kenpom has predicted it correctly. actually, i wonder if he's got this sort of thing up on his website somewhere already.
January 25th, 2012 at 4:38 PM ^
The more data aggregated by KenPom, the more useful it gets.
Tempo free analysis is relatively meaningless without playing conference opponents as statistics usually vary widely from cupcakes early on in the season at home vs. away contests in Bloomington, etc. You'll see that as more B1G contests go on, there will be more value in his method.
January 25th, 2012 at 5:00 PM ^
Perhaps an analysis of how far his prediction deviated from the score v. other methods of prediction could prove useful. All you've demonstrated is that he went 4-4 in predicting the winners of the game. While he was wrong about MSU and Purdue, he was only wrong by 6 points in terms of predicted margin of victory in those games.
I have no dog in the fight, as I don't really use Pomeroy or rely on his stats at all, but your analysis is kind of half-assed for us to arrive at any conclusions.
January 25th, 2012 at 8:10 PM ^
I didn't really put a lot of time and effort into it. Maybe I'll take the whole season as a sample...maybe the whole of the B1G to date and put together a diary. I wont be able to run ANOVA's like the guy earlier. Some of the Pomeroy predictions just seem really unrealistic. Outside of Nebraska @ OSU any win probability over 60% seems really counterintuitive in this conference.
January 25th, 2012 at 5:19 PM ^
As a statistician, I'll throw in my 2 cents....Pomeroy is predicting scores/winners using a model of some sort. That model's output for these games is essentially the average outcome. So just because his model said "Purdue by 4" doesn't mean that Purdue should win by 4, it means that his model is saying if these teams played 100 times, 1000 times, 1000000 times, the average outcome of those games would be "Purdue by 4"....statistics are not and never will be a be all end all "answer", they are a helpful guide. And really the only outcomes that really raise your eye about how strong his model is are the Iowa and Wisco games, but even those have a chance of happening in his model....even if it was less than 1% chance, there's still a chance. When you have a sample size of 1, your chance for error can be pretty large obviously.
January 25th, 2012 at 8:24 PM ^
I definitely see what you are saying. Just looking back, when people say "take these predictions with a grain of salt" I suppose I was just surprised by the size of that grain.
Looking at Michigan's Pomeroy profile is rather depressing, and doesn't fit with the product that I see on the court. http://kenpom.com/
I find it comforting to believe that this profile doesn't come close to telling the whole story, and I believe that the remainder of the season will bear that out.