Michigan's Tourney fate/ranking- Lunardi's accuracy
EDIT: This post is not saying Lunardi is a bad source. He's been great at predicting who will make it into the tournament and who wont. It was written to point out that his predictions should be taken with a grain of salt beyond bubble-predictions- seedings, regions, etc.
Since our beloved men's basketball team first played itself onto the bubble in the last few weeks, countless threads have been started citing Joe Lunardi's picks for Who's in/who's out, rankings, and even the regions teams will be placed in. What has earned Lunardi the "Bracketologist" title? More importantly, does his "wisdom" deserve so much attention within the mgoblogosphere? Like most of you, I assumed it would be his proven track record that allows him this McShay-like authority. Unfortunately, the following data argues otherwise. He accurately predicts the bubble, but beyond that you might as well have your girlfriend set up the bracket.
[The statistical data in this post is courtesy of ssreports.wordpress.com. I was going to compile the data myself for my first diary, but some time spent on google showed me that someone beat me to the punch.]
The article is worth a read but here are the main points:
Lunardi Accuracy 2010
Who's in who's out- 64/65
"Saying Lunardi got 64 out of 65 teams correct is no big accomplishment because over 50 of those teams either received auto-bids or were locks."
Teams correctly seated- 27/65 (41.5%)
2009- 31/65 (47.7%)
Teams correctly placed in each region- 17/64 (26.5%)
2009- 29/64 (45.3%)
but who cares about whether he gets the seeds or regions right?
I agree with you on regions, but disagree on seeds. Lunardi is always touting his accuracy in predicting who is in the field, but as has been pointed out countess times, he's really only judging himself on correctly predicting 4-8 teams (essentially the last 4 in, last 4 out), Anyone else could get the rest of the teams correct.
I care about how accurate he is on the seeds, because that tells me how good he is at deciphering the various criteria used and placing teams accordingly. After all, whether the committee is deciding on who would be 12 seeds vs. teams out of the bracket, or possible 1 seeds vs. 2 seeds, they are using the exact same criteria. If Lunardi can't reasonably predict seedings, how much faith should we put in his predictions on who is in and who is out?
I guess he gets a it "wrong" if he mixes up the 8 and 9 seeds, or has a team as a 14 when they ended up being a 15. We would need to know how off he was to say whether it was truly material.
But I agree with you that I would care how accurate his seedings were more than the regions.
I was just going to edit my post to make clear that I don't expect anywhere near perfection in predicting seeds. And I'm not even sure his accuracy is good or bad.
A better stat would be how many teams were within one slot of projected seeding. As you point out, it shouldn't be counted against him to seed a team at 8 only to have the committee give them a 9. But, a two seed difference means that he's off by at least 5 on the "S curve." To me, that should count as a miss as much as incorrectly predicting at eam as in/out.
I'd like to know his percentage of predicting teams within 1 seed (+ or -) of where they end up...but don't care enough to do the work myself. That would be a meaningful stat.
Lunardi's respected because his final bracket does a good job of predicting the field. For the most part though, he's fairly useless.
<br>Also predicting the seeding and placement is a crapshoot. We can have an idea of what teams will be in but no one outside the committee can know exactly how they want to seed/ place teams. Cant hold that against him.
I'm ok with
We only care about getting in, not where we are seeded. 64/65 is good enough for me. Jerry Palm at CBS was 65/65 last year and he has us as an 11 too. Folks on this blog are citing the bracket matrix most often which also has us at an 11 seed.
Citing stats about correct seeds and regions (which are mostly arbitrary) is pointless without any comparison anyway. How does he perform compared to other "experts" and random people like you for instance? You'd need to know that before you can say he's not a good source.
64/65 is great and admittedly much better than I would do. Yes, I'm glad he has michigan in because that bodes well for our future. My point was that beyond who's in/who's out, his predictions are nearly useless. Yes it is a "crapshoot", yet we still cite him for where we'll be ranked and who we might play.
Give yourself a little credit, if you stacked all the criteria for the past few years of all teams on the bubble, and then used that information to predict this year's field, you'd be pretty close to 64/65.
And if this were literally your job - to watch a bunch of college basketball games, figure out what the committee looks at and make educated guesses about they'll do - you could probably perform similarly.
I'm not sure what goes into getting placed in a particular region but if its random then its at 25%, right? As far as seeds go, 1s and 2s probably are fairly easy but the end of the spectrum has to be tough because because its a judgement call I would assume not just your exact spot in the RPI. Did they happen to post the std dev on the correctly seated?
I dont care where we end up as long as we are in the tournament. In or out, he is excellent, therefore I tend to believe what he says.
For real, man. If Michigan's in the wrong region, I'm not watching.
Why do they even keep up the pretense of "regions" when you have West region games in D.C. and such?
You can't make much of Lunardi's stats without comparing them to others' predictions. Obviously the man isn't just pulling names out of his ass, I mean, he gets paid to do this, but I have no idea if correctly predicting 42% of the seeds is good or not.
And anybody that pays even a little attention to this stuff is going to get 63/65 a decent amount of the time. 64 is good but I'd expect a pro bracketologist to be getting 64+ almost every year.
Bracketology 101 came up with something called the 40/60 club for correctly seeding at least 40 teams and correctly picking at least 60 of the teams. They claim to be the only site to have ever done this. Correctly picking 60 teams seems pretty easy, but 40 seeds is obviously pretty difficult. But Lunardi's 42% does seem a little low.
What I am more interested in is someone's percentage of predicting at-large bids.
As the OP pointed out, 50 or so of the bids are either automatic or guaranteed. For example, I would bet one million dollars that Ohio State, Wisconsin, and Purdue will make the tourney. Congrats to me! I'm a genius.
I'm guessing bracketologist's at-large prediction rates are much lower, especially if they're made much in advance.
March 10th, 2011 at 10:09 AM ^
The bracket matrix also comes up with a ranking of the bracketologists. Lunardi comes in at 23rd. The 3rd ranked bracketologist is one of the few that has us out (1st 4 out).
http://bracketproject.50webs.com/rankings.html
Bottomline: Don't lose tomorrow and should be safe.