Fab Five. Wolverine Historian continues to feature Fab Five games that officially may not exist anymore:
The inside scoop. Seth Davis did one of those ask-coaches-off-the-record articles that always feature a mix of insight and bitchiness and make for quality reading. The take on Michigan (emphasis mine):
Michigan: The Wolverines are dangerous because they shoot the ball so well and stay within their sets, but they can also lay an egg because they rely so much on threes. You almost have to play small with them because they force you to. If you have a big man, it's hard to guard them because everybody will step out and score. I don't think Tim Hardaway Jr. is a tough kid. He just wants to shoot jumpers. If you have a dominant person inside, you can go right at them because they're not real big. Hardaway has not had the kind of year we were all expecting, but he has an uncanny ability to make threes late even when he's not shooting well. Trey Burke is the best guard in our league, and Jordan Morgan is much better offensively than he was last year. They don't scare you defensively. They'll get after you and compete, but you can run your stuff and score on them.
The section on Ohio State also mentions that they're "probably kicking themselves a little for not taking Trey Burke," and the Wisconsin bit is all about how terrible and awful and disrespectful they are.
Maybe this whole standards thing isn't a huge deal. Remember when some guy said that unconfirmed thing about Brandon saying that Michigan wasn't going to compete with the SEC for things and stuff and would have standard like things and everyone was all like boo boo boo we want to recruit Manninghams even if they like smoking pot, like, forever and ever?
Yeah, that was in the long long ago when Michigan was striking out late in the 2012 class and hadn't secured a top five 2013 class like two weeks into that recruiting cycle. But, like, you know who we lost out to for a couple important guys? Stanford. This Stanford:
Haskins points out that just because a guy plays football doesn't necessarily mean he's physically tough. From a mental side, Shaw maintains the Cardinal's rigorous academic requirements forces the program to get determined people. "To be honest, it's built in for us," he says. "We can look [at] the physical toughness when you watch a kid play, but we're also finding out about that stick-to-it-iveness when we're asking them to re-take tests, take AP courses and make tough decisions to try and get admitted here. That shows dedication, toughness and perseverance."
That's from a long Bruce Feldman piece on Stanford's ridiculous-not-just-for-Stanford recruiting. The Cardinal is proving that you can avoid the flakes and still bring in monster classes. Michigan seems to be doing the same, and as long as Notre Dame isn't swooping in on the guys they want they seem like they'll be able to maintain that over the long haul.
First one, then the other. I've been pining for Urban Meyer's shovel option for a while now. You know, this thing:
It seems like a natural fit for Michigan for multiple reasons: it's just power blocking, which Hoke loves. It forces the defensive end to either cheat down on the pitch or potentially let Denard outside. If Denard makes a bad decision the potential for disaster is low—either he is running around for a small loss (or gain!) because he kept or he's throwing an incomplete pass. The main issue is finding a tight end who can run it, but if Michigan's throwing Hopkins on the field as an H-back sort he's got the chops to make that a viable option.
Once you've got that in the book, you could add bells and whistles like a quick cover-two beater on the edge to give that corner a problem he can't fix:
Michigan did run some run-plus-short-pass concepts like this last year…
…so this might be something to keep an eye on as Borges tries to get the most use out of Denard's legs in year two. Borges loves to add new stuff on the regular; it's 50-50 we see something like the above in 2012.
Speaking of Borges. He talks with Howard Griffith:
Money quote: "I don't want to have an offense with a name" because then people start running clinics on how to defend it.
Unintended consequences. The NCAA's recent adjustment of kickoff rules smacks of a public relations effort to assure people concerned about concussions that football is also concerned. The net impact of slightly changing 2% of a football game is going to be statistically zero when it comes to long term health outcomes, but it says to the world that the NCAA is Doing Something, so it passes.
It won't do much. It might not do anything since the NCAA made a change that seems counterproductive to its goals: it's changed kickoff touchbacks to the 25. This is supposed to encourage returners to take a knee. Instead it may encourage kicking teams to not put it in the endzone.
Florida State has one of the best kickoff specialists in the country, Dustin Hopkins. Last year his 29 touchbacks were a victory. This year some back of the envelope calculations by Tomahawk Nation suggest the Seminoles' optimal strategy on kickoffs from the 35 will be this:
LET'S RECAP - If FSU does indeed ask Hopkins to kick it just a little higher and a little shorter, we can realistically expect him to average the ball around the 2-3 yard line with a hangtime of around 4.6 seconds. This is enough time that the majority of the coverage team will be inside the 25 yard line, with the faster players being somewhere around the 20. One can expect first contact to be made somewhere inside the 15 yard line on average. If the return man dances or does not immediately run full speed after the catch, it could be even worse. It may be a common occurrence for many returns to fail to exceed the 10 yard line. That is epic.
85% of TN readers think that's the way to go. The NCAA probably just made kicking for a touchback a mistake. There's a good chance these new rules go the way of the Hated Clock Rules from about five years back.
Two options: idiot or fabulist. Good lord, Phil Birnbaum points out that the Berri study-type substance on NFL quarterback draft positions…
- Uses a regression to determine "expected" draft position instead of using, you know, draft position.
- Their regression on expected performance does show a correlation between draft position and performance, but it's not statistically significant, so they use that to say "there is no relationship between draft position and performance."
- Tom Brady alone accounts for 14% of the plays from quarterbacks drafted from 150-250.
David Berri is the worst statistician on the planet.
BONUS OHIO STATE SCHOLARSHIP SIGN UPDATE! With Jordan Whiting's transfer to Louisville the only scholarship business major on the team is a kicker.
Etc.: Another rat is poised to jump off Dooley's sinking ship. He's their recruiting coordinator and would be the seventh assistant to leave this offseason if he takes an equivalent position at Nebraska. Michigan NFL combine recap. Molk says things, people take offense, Molk seethes, repeat.
Release the constructions. Stage N of the everlasting Crisler revamp has begun. Behold pictures of construction.
The renovations will be completed just in time for no one to be able to afford tickets.
Combine crushage. Mike Martin may not have put up as many reps as he wanted in the bench press but he still finished second amongst DTs. In everything else he was exceptional:
Bench press: 36 repetitions, where he tied for second
40-yard dash: 4.88 seconds
Vertical jump: 33.5 inches, where he tied for 13th
Broad jump: 113 inches
3-cone drill: 7.19 seconds, where he tied for 15th
20-yard shuttle: 4.25 seconds, tied for sixth
That is at 306 pounds. He's a riser amongst DTs.
Meanwhile, Junior Hemingway put up two 4.5 40s and killed the agility drills:
In the other events, Hemingway really stood out:
- Three-cone: 6.59 seconds, first out of 26 receivers
- 20-yard shuttle: 3.98 seconds, tied for first out of 26 receivers
- 60-yard shuttle: 11.16 seconds, 2nd out of 13 receivers
"There aren't many receivers who did more for themselves than that guy," an AFC scout said of Hemingway. "He wasn't even on our radar going into this thing. He is now.
Therein is the inexplicable YAC knack. I wonder why it seemed like he could never get separation if he's putting up those numbers. A 225 pound guy who can change directions that fast should be open all the time.
For his part, David Molk put up 41 reps in the bench, second only to Memphis freak of nature Dontari Poe. He is furious about this, because David Molk is furious about everything.
The bust. Rivals puts out a list of recruiting class busts highlighted by Kiffin's single year at Tennessee and three consecutive Florida State classes in the dying days of the Bowden era. Michigan's '05 class checks in 7th. Michigan finished sixth in the class rankings that year and got very little from that class.
BUSTS – TALENT DIVISION
- Kevin Grady (#22 overall)
- Jason Forcier
- Brandon Logan
- LaTerryal Savoy
- Mister Simpson
- Andre Criswell (not that anyone expected a ton from him)
BUSTS – CHARACTER EVALUATION DIVISION
- Marques Slocum (#37)
- James McKinney (#98)
- Eugene Germany
- Chris Richards
- Johnny Sears
- Justin Schifano (not a bad guy but didn't want to play football)
- Carson Butler
- Antonio Bass (#49)
- Cory Zirbel (#83)
- Chris McLaurin
ACTUALLY USEFUL PLAYERS
- Mario Manningham (#45)
- Terrance Taylor (#96)
- Brandon Harrison
- David Moosman
- Zoltan Mesko
- Tim McAvoy (sort of)
- Mark Ortmann
That's brutal. You've got a couple of okay linemen, Manningham, Taylor, and Harrison. The next year's class was decent (Brandon Graham, Steve Schilling, Jonas Mouton, Steve Brown, Brandon Minor) but didn't produce anything past the four stars save Perry Dorrestein; 2007 had Mallett and Warren as five star headliners and was then mostly junk, which set Rodriguez up for failure from the start. Rodriguez then helped matters along, of course.
This is all so clear in retrospect. One of the things I'll be looking for in the first couple Hoke classes is how many guys we see burn out for character/grade issues. It certainly seems like that number is going to be a lot lower than we saw at the tail end of the Carr regime. So far Hoke's only got Chris Barnett, the surprise tight end with red flags galore who burned out halfway through fall camp and took Kellen Jones with him. (Jones ended up at Oklahoma. QED.) That's an understandable misstep in the midst of a chaotic final month before signing day with a new roster. Since then he hasn't approached a guy with a whisper of an issue.
Popcorn time. Sports statistics fabulist David Berri is still plugging his ridiculous notion that draft status is not at all predictive of NFL QB performance. This was one of the claims that caused me to write a long screed about how useless Berri is a couple years ago in which I collected Berri debunkings in the four major sports to point out that his claims are almost always either easily proven false or simpleminded simplifications of incredibly complex questions.
This remains the case if he's still pumping his inane NFL QB study. This time Phil Birnbaum has noticed:
They argue -- as does Gladwell -- that we should just assume the guys who played less, or didn't play at all, are just as good as the guys who did play. We should just disregard the opinions of the coaches, who decided they weren't good enough.
That's silly, isn't it? I mean, it's not logically impossible, but it defies common sense. At least you should need some evidence for it, instead of just blithely accepting it as a given.
And, in any case, there's an obvious, reasonable alternative model that doesn't force you to second-guess the professionals quite as much. That is: maybe early draft choices aren't taken because they're expected to be *better* superstars, but because they're expected to be *more likely* to be superstars.
He promises future posts examining the subject. I promise they'll leave Berri's study shattered at the bottom of a ravine.
Erp? TTB interviews Ben Braden and comes back with this:
Early playing time: Right now, the plan is that I will not be redshirting. I think I'll be a second stringer, and then just go from there. But right now that's the plan between me and the coaches. . . . [I'll be playing] right tackle.
I'm not exactly stressed that Michigan might miss out on a redshirt senior year from Braden what with the 2013 OL class, but if I had to bet I'd say Braden ends up redshirting anyway.
There's a difference between Jack Miller redshirting—the world will end before you see the field this year—and Braden's spot on the depth chart. It's not too hard to envision a situation in which he's forced onto the field. Even assuming Kalis is what he's reputed to be Braden's certainly in the running to be the second guy off the bench in the event of injuries at tackle. I bet we'd see Elliott Mealer in before Braden in the event a tackle goes down with a minor injury a la Lewan last year, as they'll want to preserve that redshirt if possible.
Good news for people who like boring news. There is a webcam of Michigan taking down their new scoreboards. You can watch it, or you can look at this picture. They are basically equivalent:
Yes, they left the Big Chill lingo up.
Womp-rats? Yesterday at about 7 PM Yahoo released its latest article that terrifies and thrills, and it's a doozy:
Tressel knew of gear scheme last April
If true, that would expose Ohio State to the worst kind of NCAA justice. Cover-ups are very bad. They got SMU the death penalty and are soon to terminate the job of Bruce Pearl.
Can Yahoo/the NCAA prove it, though? The Robinson/Wetzel piece relies on one anonymous source who said Tressel was "troubled by the information" and promised to investigate. I don't think OSU can reasonably suggest they investigated and found nothing since it didn't take the NCAA long to confirm the story, but previous Yahoo gotchas came with paper trails—as of now there isn't one.
The worst-case scenario here is that this gets rolled into an investigation of Terrelle Pryor's perpetual loaner and it turns out that—surprise—zealous OSU boosters are funneling massive amounts of impermissible benefits to players. It's getting to the point where it's hard to downplay everything that comes to light as an isolated incident, especially when Antonio Pittman tweets that cats have been getting hookups on tats since 2001.
I don't think anyone knows where this is going but if Yahoo can produce paper a major violation, an actual one not about stretching, is in the offing. Eleven Warriors just tweeted that they are hearing Tressel will admit wrongdoing(!) and sanctions/suspensions are "possible."
No serious harm done. According to Mike Spath, Carl Hagelin and Billy Powers expect Louie Caporusso to return for next weekend's CCHA finals at the Joe. Presuming Michigan can get by Bowling Green, by far the worst team in the league this season, without him they won't be short in their quest for a one-seed.
Word. Best NFL draft evaluation ever on one Justin Boren:
Plays angry on the field but his mental makeup is in question after a transfer from Michigan. Day 3 prospect.
Love to bits. The SBN Oilers blog goes off on semi-regular rants about how numbers are just not understood, man, that I love to tiny bits. Their latest is about the Avalanche and their fluky run last year. According to hockey's advanced metrics last year, the Avs were a terrible team. According to the standings midway through the year they were pretty good. They managed to survive a massive late slump to squeeze into the playoffs and fans thought this was sustainable and numbers were stupid. This year they're pretty much the same team except they're not nearly as lucky, so they're just above the Oilers in the standings and fans are discussing whether they should fire the coach they were pumping for the Jack Adams last year.
Avalanche fans are not alone in ignoring, even denying the evidence behind the performance of the team. In an article entitled "When the scientific evidence is unwelcome, people try to reason it away" in The Guardian, author Ben Goldacre explores what happens when people are "...confronted with scientific evidence that challenges their pre-existing view." His conclusion? "Often they will try to ignore it, intimidate it, buy it off, sue it for libel or reason it away." Goldacre references a 1979 paper from Lord, Ross and Lepper. From the paper's abstract:
People who hold strong opinions on complex social issues are likely to examine relevant empirical evidence in a biased manner. They are apt to accept "confirming" evidence at face value while subjecting "disconfirming" evidence to critical evaluation, and, as a result, draw undue support for their initial positions from mixed or random empirical findings.
Goldacre goes on to discuss a second group of people - those who attack the science behind the evidence presented to them.
When presented with unwelcome scientific evidence, it seems, in a desperate attempt to retain some consistency in their world view, people would rather conclude that science in general is broken.
This line of thinking is similar to that used by fans who argue in favor of shot quality. Shot quality has become the great foil used by those arguing against possession metrics as a basis of hockey analytics. The ever-increasing mountain of possession data, evidence and studies means little to the shot quality folks. Arguments abound in favor of shot quality with no evidence to back it up, so lacking so Desjardins challenged the world to prove the existence of shot quality. There were no takers.
When presented with unwelcome scientific evidence, it seems, in a desperate attempt to retain some consistency in their world view, people would rather conclude that science in general is broken.
What's that on the horizon? It's getting closer! It's getting closer very fast!
This is why numbers are important—they at least force you to consider things that conventional wisdom holds are ridiculous, like Derek Jeter being a pretty crappy defensive shortstop. The advanced metrics said the Avs were due to regress badly and they did. This would be just another guy who loves numbers accepting confirming evidence while some other team that defied the numbers would be seized upon by the Joe Morgans of the world as their confirming evidence… except for the fact that you can collect big sets of numbers and show they are accurate more often than not. We had a discussion about this before college football season when I predicted Iowa wouldn't do so hot and Iowa fans were like "numbers are stupid."
The other end of the spectrum from Joe Morgan is David Berri, who's just as wrong as Morgan and relies on a just-as-irrelevant credential ("I was the greatest second baseman of all time"/"I went to Princeton") in his quest to reduce everything in sports to a regression. I'm not arguing for that, either. The numbers gathered by football and basketball box scores are witheringly insufficient to hope to explain anything.
In reality, numbers are insufficient to fully explain anything but baseball for a lot of reasons. Baseball's easier and there are orders of magnitude more data—Pitch FX is insane. But in all sports advanced metrics can at least provide a much better answer for "what," if not how and why. An example: about a week ago LaVall Jordan tweeted that Michigan had the fourth best defense in the Big Ten. That's true on a pure counting number basis but if you do something like divide they were ninth*. That's a huge difference and the tempo-free number is indisputably better. There's a huge difference between talking about why Michigan has an above average defense or why they have a below-average one, and anyone who would prefer to talk about the former is just wasting people's time.
*[The MSU game moved them up to seventh.]
Hardaway explosion. Rod Beard's latest in the News has a wide array of quotes on the emergence of Tim Hardaway Jr. Vitale is involved, but don't let that phase you. Here's the most interesting bit on his recent blowup:
"When he was shooting a lower percentage earlier in the year, I called him in and we just talked a little about getting a better shot than he was taking," Beilein said. "(I told him) you're probably going to take just as many shots, but the ball will come back to you again.
"He did it immediately and his shooting percentage has gone way up."
Beilein has repeatedly praised Hardaway's coachability, which suggests he will continue to improve over the duration of his career at Michigan. Dad is also impressed:
"He's developed very well and the whole team has, from November to today," Tim Sr. said. "You can see a lot of confidence in them and you can see their swagger. They're playing well, they believe in the system and they believe in the coach."
Random offer thought. Michigan continues to litter the nation with offers, but a Q: could this be a more general pattern? The NCAA just implemented a rule that prohibits schools from sending written offers until August. In the past there was the verbal offer, which was more of an indication of interest, and the written offer, which was as close to official as something that says "we can revoke this at any time" gets. Now there are no written offers, nothing to distinguish between the two, and kids who may have waited to declare they had an offer until they had the actual paper in their hands now have nothing else do go on.
In any case, the universal predictions that this rule would lead to confusion and would do nothing to slow down the breakneck pace of recruiting have come true, like it was obvious they would.
Etc.: Posnanski writes something about the "joy of rooting against Lebron" that expands on yesterday's trash-talk assertions. According to Ira at WTKA via Brandon, Michigan's club seats and suites are sold out. Evolving Evan Smotrycz. Big Ten wrestling details.
The last straw for Run of Play proprietor, Slate contributor, and Dirty Tackle blogger Brian Phillips were two articles on consecutive days citing Franklin Foer's assertion that dictatorships led to good soccer. Many of the nations that have been super good at soccer over the years have been run by dictators if you lump Vichy France in with them and think Hitler and Mussolini have anything to do with anything in the 21st century. The first problem with this piece of intellectual noodling is that the percentage of teams who have won the World Cup during or after a period of dictatorship (86%) is almost equivalent to the percentage of countries that have undergone periods of dictatorship since 1930. Twenty-five of the 32 teams in this year's edition have done so, 78%.
The second is that the statement means nothing. Phillips on the Kuper/Szymanski book Soccernomics, which endeavors to be a Freakonomics for the beautiful game:
You want to say that money is the secret behind soccer success, so you break down international games by GDP and find that, yeah, it matches up fairly well. But it doesn’t work as a theory, because China is terrible at soccer and the US is only okay at it. So you invent a variable called “tradition” and add it into the formula, which helps (now Brazil’s looking really strong), but you’re still left struggling to explain why, say, England doesn’t do better. So you add in population size, and on and on and on. Eventually, you have a delicately balanced curl of math that correctly reproduces the results of most recent matches (even if it accidentally predicts that Serbia will reach the current World Cup final). So you go to a publisher, but no one wants to buy a book about how GDP is covariable with national-team success 40% of the time, or whatever; they want a book that claims to have Uncovered the Secrets of Soccer© Using Funky Mathematical Techniques™. And so you’re led into making grand claims for the predictive power of research that really only demonstrates correlation. And there’s enough data swirling around a complex event like the World Cup that you could get the same results by collating fishing exports, number of historic churches, and percentage of authors whose names include a tilde.
You have no mechanism. Your correlation is extraordinarily weak. You have just wasted everyone's time.
The very same day, Slate (et tu!) published an article by a guy who studies a particular brain parasite claiming a correlation between soccer performance and infection rates of Toxoplasma gondii, a bacteria whose raison d'être is to get in a cat's stomach so it can make babies. An R-squared was not mentioned, but it was gestured to. Regression rules everything around me. This is why most published research results are false.
Soccer is not the only sport suffering from pseudoscience obsessed with elevating correlation above all else, mechanism be damned, and elegant curls of math that prove little other than the academic's talent for obfuscation in the name of publishing. Kuper and Syzmanski actually got to the party late. Princeton economist and Malcolm Gladwell fave-rave David Berri's been here for years, and he's packing the platonic ideal of delicately balanced curls of math that end up ludicrous on further inspection. Behold the best (and sixth-best) players of the 1999 NBA season:
the emperor's clothes are fine indeed.
Berri made a splash in the sports world when he released a transparently silly book that purported to show that Dennis Rodman was responsible for more wins than teammate Michael Jordan. This drew the ire of the basketball statistics community and anyone with a damn lick of sense. People set about showing that Berri was peddling snake-oil. I even had a go at it in one of the erratic Pistons posts that showed up around here a couple years ago, noting that after Ben Wallace left the Pistons' rebounding changed not one percent on either end of the floor. Ben Wallace got his rebounds from his teammates. (It turned out that Wallace's major skill was an ability to keep opponents off the free throw line.)
This did not take, unfortunately, and Berri has been permitted to say silly things about all sports that apparently intelligent people take seriously because he has "Princeton" next to his name. He moved on from basketball to "show" that NFL teams don't care how well their quarterbacks perform, only how high they're drafted…
Aggregate performance and draft position are statistically related. But as Rob and I argue, this is because in the NFL (like we see in the NBA) draft position is linked to playing time. And this link is independent of performance.
…that NHL goalies are indistinguishable from each other…
... there simply is little difference in the performance of most NHL goalies.
…and has returned to state basketball coaches don't understand who their best players are:
"... the allocation of minutes suggests the age profile in basketball is not well understood by NBA coaches."
Berri's at least had the common sense to stay away from baseball, where a horde of men with razor-sharp protractors wait for him to make a false move. (We will see later that collaborator JC Bradbury has not.) The statistical communities in football, basketball, and hockey are considerably more unsure of what the hell is going on in their chosen sport and are thus vulnerable to suggestion from an economist, even if it's one who seems to have never watched a sport of any variety.
The problem with all of Berri's outlandish theories is that they are wrong. Not because of old guys who peer into the soul of Andre Ethier and see a ballplayer, but because of other, more careful numbers from people who are looking for things that are true instead of things that are impressive to Malcolm Gladwell.
Berri's study actually shows that amongst quarterbacks who play a lot, draft position is not a strong factor in their performance. This is his magnificent leap:
For us to study the link between draft position and performance, we can only consider players who actually performed. It’s possible that those quarterbacks who never performed were really bad quarterbacks. But since they never played, we don’t know that (and Pinker also doesn’t know this).
Low draft picks who don't play only find the bench because of bias. A coach's decision to start one player over the other is a worthless signal. Coaches are dumb.
When you restrict your regressions to the top 20 goalies in terms of minutes, about half of the variation in save percentage appears repeatable. A standard deviation of talent is worth around ten goals. These days, a unit of five skaters who finished +50 at the end of the season would be heroes on the league's best team. Berri's undisclosed approach to the data set apparently takes goalies with far fewer than starter's minutes. A quick correlation run by Phil Birnbaum shows radically different r-squared values than those Berri finds just by upping the sample size. Maybe Birnbaum's numbers aren't dead-on—he doesn't use even strength save percentage, for instance—but he's not the one claiming a massive inefficiency. He's just showing that throwing a small r-squared out doesn't actually mean anything:
I don't know how the authors got .06 when my analysis shows .14 ... maybe their cutoff was lower than 1,000 minutes. Maybe there's some selection bias in my sample of top goalies only. Maybe my four seasons just happened to be not quite representative. Regardless, the fact that the r-squared varies so much with your selection criterion shows that you can't take it at face value without doing a bit of work to interpret it.
Age in the NBA
In the NBA, 23 and 24 year old players net more minutes than any other age bracket, and while the average age of an NBA minute is 26.6 this year there's a blindingly obvious explanation for this:
Berri and Schmidt think that NBA minutes peak later than 24 because coaches don't understand how players age. It seems obvious that there's a more plausible explanation -- that it's because players like Shaquille O'Neal are able to play NBA basketball at age 37, but not at age 9.
In sum: wrong, wrong, wrong, and wrong.
So what's going on here?
When you've got a hammer, everything looks like a nail. Berri's hammer is regression analysis, and he goes about hitting everything he can find with it until he finds something that seems vaguely nail-like from a certain angle. Then he proclaims a group of extremely well-paid subject matter experts dumb. When challenged about this, he says things like "regressions are nice, but not always understood by everyone." He calls the protestors dumb.
This is more than a logical fallacy: it's a worldview. In a post on a cricket study by another set of authors, Birnbaum points out the assumption built into a lot of economics studies. It, like most of Berri's work, runs a regression on some data and reports back that something fails to be statistically significant:
The authors chose the null hypothesis that the managers' adjustment of HFA [home field advantage] is zero. They then fail to reject the hypothesis.
But, what if they chose a contradictory null hypothesis -- that managers' HFA *irrationality* was zero? That is, what if the null hypothesis was that managers fully understood what HFA meant and adjusted their expectations accordingly? The authors would have included a "managers are dumb" dummy variable. The equations would have still come up with 4% for a road player and 10% for a home player -- and it would turn out that the significance of the "managers are dumb" variable would not be significant. Two different and contradictory null hypotheses, both which would be rejected by the data. The authors chose to test one, but not the other.
Basically, the test the authors chose is not powerful enough to distinguish the two hypotheses (manager dumb, manager not dumb) with statistical significance.
But if you look at the actual equation, which shows that home players are twice as likely to be dropped than road players for equal levels of underperformance -- it certainly looks like "not dumb" is a lot more likely than "dumb".
The goalie example is the most illuminating here: by adjusting the parameters of your study you can arrive at radically different conclusions. I'm not sure if Berri is intentionally skewing his results to get shiny Moneyball answers, but given how dumb his justifications are for the NFL study that's the kinder interpretation. Running around saying that we don't know that the average sixth rounder isn't John Elway waiting to happen because they can't get on the field is obtuseness that almost has to be intentional. On the other hand, he does blithely state he's "not sure there is much to clarify" about his assertion that NFL general managers are on par with stock-picking monkeys when it comes to identifying quarterbacks, so he may be that genuinely clueless. (The Lions tried a stock-picking monkey. It didn't work out.)
There's often a kernel of truth in a Berri study. When the Oilers were casting about for a goalie, smart Oilers bloggers were noting the glut of basically average goalies available and jumped off a cliff when they signed a mediocre 36-year-old to a four year, $15 million dollar deal when they could have signed two guys for something around the league minimum and expected about the same performance. That's something close to the criticism Berri levels with the volume turned way down. Hockey and football and basketball are not baseball. It is incredibly difficult to encapsulate performance in any of these sports in statistics. So when Berri makes a proclamation that NHL goalies are basically the same based on plain old save percentage—which isn't even the best metric available—he ascribes more power to a stat than it deserves and simultaneously ignores a raging debate about one of the most difficult questions in sports statistics to get a handle on.
At the very least, the questions Berri attempts to tackle with really complicated regressions are murky things best delivered with a dose of humility. Instead Berri and colleagues say there is "simply" no difference, that his research is "not understood by everyone," that a formula that declares Jeff Francouer worth 12 million a year is justifiable and that protestors are making "consistent basic errors in logic, economics and statistics" when any minor league player making the minimum could replace his production, and that David Berri went to Princeton. If he bothers to respond to what's admittedly a pretty shrill criticism, he will undoubtedly state that if only I had managed to understand his papers the many ludicrous conclusions easily disproved by competing studies (QBs, save percentage), simple facts that blow up the idea being presented (NBA minutes), or common sense (Rodman, Francouer) would have come to me in an epiphany.
These things are all ridiculously complicated and it's obvious with every response to another Berri study that declares someone dumb that different views on the data produce different results. Berri's overarching thesis is that subject matter experts make huge errors because they refuse to look at data from all possible angles. Stuck in their ruts, they robotically bang out decisions like their forefathers. Statistician, heal thyself.
There may be some social utility in distracting economists from theorizing about the economy, but there's no utility in the domain they're actually tackling.