is there such a thing as an etsy genuis? if so, this is it.
Joe Paterno's death was a hugely misreported fiasco of the sort that is inevitable given the speed of information in the internet age. This post is an attempt to provide a framework for existing in a world of uncertain information.
This is what happened: Onward State, a blog/online newspaper run by PSU students, reported Paterno's death based on an email sent to Penn State players that turned out to be a hoax. This was good enough for a local radio station and StateCollege.com. It hit twitter and was then picked up without attribution by CBS Sports. It took off from there once the imprimatur of a major news agency was on it. Black Shoe Diaries has a detailed chronology of the mass screwup if you're interested in details. Shirtless Mark Twain isn't sure if he approves of this whole business or not, but would like you to know that rumors of his rippling pecs have been sorely undersold.
It's a story about the internet screwing up in very understandable ways. Onward State had what seemed like reliable information, and it passed their threshold for reporting. It is not a good threshold, but not everyone has one these days. CBS's Adam Jacobi did something unwise and sloppy. Pagewhoring Huffington Post saw an opportunity for views and cares about nothing else.
We've seen this happen before when a newspaper intern replicates an internet rumor on one of the dingy blogs shuffled off into the corner of large metro papers: as soon as a rumor gets paired with header graphics associated with a real newspaper, everyone else is confirming it via "sources." In this instance, CBS's screwup was compounded because they didn't even provide a link to the primary source; Huffington Post did the same thing, but that's just their MO. Jacobi is a BHGP founder and should have known better.
I've screwed these things up myself. Earlier this year I erroneously reported that Kaleb Ringer had been booted from his high school team based on information that seemed solid but obvious was not. By contrast, a couple years ago I had the sense not to run anything about the serious car accident that Jon Bills and Mark Moundros were in despite having a ton of solid sources telling me about it. That seemed like a place to let journalists be journalists.
As I go along here that realm has steadily expanded. I probably won't report something like the Ringer thing again for a lot of reasons. Michigan playing Alabama is one thing to be wrong about; a high school kid's problems or lack thereof is another. This leaves windows open for crass opportunists like Ace Williams, but it's the internet. There's always going to be a bottom of the barrel.
Anyway, these things evolve naturally. As this site expands it has more at risk and becomes more cautious. People just starting out have little to lose and have not experienced the backlash from being wrong—or the frightening period between your post and official confirmation of it. Also some of them are total idiots.
From the user's perspective, the thing to do is maintain a Bayesian approach. Phil Birnbaum explains what that is:
Generally, Bayesian is a process by which you refine your probability estimate. You start out with whatever evidence you have which leads you to a "prior" estimate for how things are. Then, you get more evidence. You add that to the pile, and refine your estimate by combining the evidence. That gives you a new, "posterior" estimate for how things are.
You're a juror at a trial. At the beginning of the trial, you have no idea whether the guy is guilty or not. You might think it's 50/50 -- not necessarily explicitly, but just intuitively. Then, a witness comes up that says he saw the crime happen, and he's "pretty sure" this is the guy. Combining that with the 50/50, you might now think it's 80/20.
Then, the defense calls the guy's boss, who said he was at work when the crime happened. Hmmm, you say, that sounds like he couldn't have done it. But there's still the eyewitness. Maybe, then, it's now 40/60.
And so on, as the other evidence unfolds.
That's how Bayesian works. You start out with your "prior" estimate, based on all the evidence to date: 50/50. Then, you see some new evidence: there's an eyewitness, but the boss provides an alibi. You combine that new evidence with the prior, and you adjust your estimate accordingly. So your new best estimate, your "posterior," is now 40/60.
So if some guy with 50 followers claims Armani Reeves is headed to Michigan because Urban was late for his in-home visit, you might increment your 50% to 51%. If Mike Farrell says its 52-48 you might bump it to 52%, but if Farrell said he thought Reeves was definitely headed to Michigan you could push it up further. You base your confidence in the opinion on previous accuracy, with a list like this…
…and change your baseline confidence based on the information and your confidence level in it. This is something people do naturally, but too often the weight they put on the information is either 0 or 1 when it should be somewhere in between.
For purveyors of information, it's time to put an explicit confidence level on what you're relaying. My mistake with the Ringer thing, other than mentioning it at all, was saying something was the case when I should have said something less certain. When I got tips about the Michigan-Alabama game I erred by saying with certainty a contract would be signed on a certain date when the people involved with the thing probably didn't know that.
I try to follow a policy of revealing as much as I can about the nature any information I pass along without exposing a source, and that added transparency is necessary in an age when information—valid information—can come from anywhere or anyone. I still make mistakes. That's inevitable. I'm trying, though.
However, not even linking to the original report is a mortal sin. If you are going to run something based on someone else's reporting it is vital that you explicitly tell readers that. Otherwise one report from a little-known online news source turns into multiple reports, some of them from organizations with people paid to do reporting, and the echo chamber starts going exponential. If you do not link, you are telling people that you are reporting it, and when it turns out to be wrong you can't point the finger at anyone but yourself.
Do hyperintelligent tacos wear pants?
Brian - Have you read The Black Swan* by Nassim Taleb? It well worth the read if you get chance.
*For good or for bad, it has nothing really to do with Natalie Portman.
Why doesn't he just tell us. Plus the 2013 class just to get it out of the way.
With the Internet age upon us--SOPA and PIPA be damned--there are more sources out there than ever. Some are of course more reliable than others, but with so many new faces on the scene, e.g., Aquaman, it's getting more difficult to keep track of them. To me that's why having MGoBlog around is essential as everything out "there" gets put the grist mill here. And while you can fool me some of the time, it's not likely that the MGoBlog borg will be deceived once the chicken shit has been separated from the chicken salad.
BTW, I also appreciate the enlightenment on the varying intelligence of tacos which I'd formerly viewed as one delicious group. <There must be a bad Taco Charlton joke out there that I don't have the wit or courage to make.>
That is one you can report early and often, especially if the guy keeps coming to work. P.S. There could be a chance he is raping more kids Joe.
Is Paterno still alive? Or did I miss something?
Paterno's death was reported a day early in the manner that Brian describes above.
JoePa died on Sunday Morning.
Saturady afternoon errant reports of his death proliferated quickly before being shot down. JoePa passed away (for real) the next day.
Erik and Ben, I think you guys need to adjust your sarcasm meters. Or, alternately, the poster you're responding to has been on the moon since last Saturday morning. Or in Amsterdam. Or at a whorehouse in Bangkok. Or...you get the picture.
Yes, that's right. I'll believe what a Klingon has to say before I'll believe anything Ace Williams has to say.
Who is Ace Williams?
Have some honor.
It infuriates me that many places just say "sources" and not "XXXXXX Is Reporting". I follow Bruce Feldman on twitter and he reported the Mike Leach to WSU story. His next tweet or so was "Oh, I guess ESPN is referring to me as 'sources' now" because ESPN never credits anyone who doesn't work for them.
I also appreciate that you have a filter, and for the most part, you are right. When you are not right, you make a front page correction (which is more than most outlets, including the NYT can say. Corrections hidden in a corner of a page?).
Based on my previous 2 points, I am reporting that MGoBlog is more reliable than both ESPN and the NYT. I'd say you're doing a damn fine job.
The avatar of "infotainment" come to earth.
is the man. Just felt I needed to throw that in there.
Ace Williams certainly takes it on the chin, though he deserves this special "below dumb tacos" ranking.
There is only one correct answer. Right?
Like the sky is up?
So how can a fucking journalist get that one wrong regarding the most important person in the history of Penn State U?
Getting that one correct is easy. Getting it correct before everyone else is hard.
Don't ever leave me
I'm glad Brian tackled this. An anonymous source should never be enough to report on someone's death.
That said, where do we rank Dandy Don in the Internet reliability taxonomy? Above or below Ace Williams?
60/40 above Ace Williams
Correct with the Bayesian therory, however, when you are a juror you must presume the defendant is innocent (or are at least instructed to do so) until proven guilty by the prosecution; thus, in reality you should start out 0/100 until evidence and argument is heard. Just sayin'...
This is 'MURICA where we let Nancy Grace tell us our base line. And that base line is GUILTY, MAH-FUCKAHS!
Interesting point. The perfect estimate would involve figuring out the historic conviction rate for all cases that go in front a jury and then refine your estimate based on the type of case and so forth. That would be your prior estimate and then of course conditional probability takes over and you form your Maximum Likelihood estimate(P(guilty|evidence1,evidence2...)) based on information presented. Statistics vs Human Nature.
Each defendant gets a zero baseline, and then the sovereign has to "prove up" each element of a given crime to get a conviction on that charge. That a given proportion of defendants are convicted of a given crime is supposed to have zero bearing on the particular case the juror is hearing. Your way might be an effective method for an observer to use when predicting an outcome, but it's still not the way a juror is supposed to be operating.
Just to interject: you guys are talking about the legal standard in a criminal case, only. The standard in a civil case is a "preponderance of the evidence," which, depending on the state you're in, can/cannot be described as 50.1 percent one way = you decide that way. (Re: "depending on the state you're in," the standard doesn't change, but whether or not you can actually describe it to a jury in numerical terms such as that 50.1 percent is reversible error in some states; you're always safe telling the jury that they should weigh the evidence like it is on two scales, and whichever is "heavier" is the side they should cast their vote for). Also, there was/is a part of me that read the explanations with respect to the presumptions jurors are supposed to have and wanted to add far more than this; suffice to say, lawyers fight over the instructions to be given the jury, which include what you guys are discussing, before every trial, civil or criminal. Rarely are the instructions ever agreed upon; the judge has to decide which party's proposed instruction to use on each and every issue where there's disagreement on which instruction to use.
Which is funny, because I'm quite confident that juries don't always listen too hard to the instructions. (Which is excusable; sometimes it takes the judge 45 minutes to finish reciting the instructions, and it can be quite boring to non-anal lawyers...so, everyone in society)
Yeah, juries probably don't listen to instructions, and the burden is different in civil cases. Doesn't change the nature of what a burden is and is not. The plaintiff still has to be the one to get the jurors up to a preponderance. Maybe it works when allocating liability among multiple defendants (haven't really thought about that since school), but I suspect not.
Otherwise the posterior odds cant change. They wold remain 0/100. Let's get our application of Bayes theormem straight.
That is one damn sexy Mark Twain.
So dapper, so regal.
saying, "the reports of his [Paterno's] death were slightly exaggerated."
Brilliant, poetic choice of Twain photos here.
is the new objectivity.
Nice article. It was a good read and very refreshing. Love the hierarchy of "intel" sources. To add more brown-nosing, like the above poster said you (et. al.) do a damn good job overall here. Similar to the New York Times matter, this place is more credible than most anything else today.
Taco Charlton notwithstanding, I find your enumerated list to be full of win.
Does that sweater come in a v-neck?
"information that seemed solid but obvious was not."
I can never tell if these kind of things are typos or just some cutesy internet speak I'm not hip to.
Brian is a genius! :)
Hahaa great post!
I suppose people would FREAK OUT if you dared mention a certain former coach. But I for one am thinking back to the AD source who told a Detroit Paper that RR had been fired--some 12 or so hours before it actually did happen. WTF was that about.
Does anyone actually care who the FIRST person to report something is? Seriously, I'm asking. Because it seems like it's a matter of life or death (no pun intended here) to journalistic types, and I just can't fathom why. Quick, who first reported the death of Michael Jackson? I'll give you a minute...
It was gossip website TMZ. Ok, so does knowing that change your opinion of TMZ even one iota? Assuming you had even heard of it previously? Whether or not TMZ got there first, I'd still rather read about it on a site that can pair the news with worthwhile context or analysis - something that reporting it extra quickly actually works against, since context and analysis take time to add. Does it make me weird that I care about who reports best than about who reports fastest? I'm not the only one, right?
I remember how 9/11 unfolded with my co-workers all on different websites and radio stations. It was kind of crazy, how many people wanted to be the first to pass on the latest rumor. A few of us stuck with public radio. We never were the "first to know" or pass on any terrible new detail, but neither did we end having to retract ridiculous false crap like other people did.
Sorry to burst your bubble, but NPR is no better than CBS, NBC, ABC, or FOX at this sort of thing. Here's one example:
Uhh, I don't think you're "bursting his bubble," as you're responding to a statement the poster never made. You're responding to what you perceive to be something like the following: "NPR is a better source of 'hard news' than internet sites and radio." The problem with your premise: he never said that. Further, your comparison to CBS, NBC, ABC or FOX is only relevant if the people the poster was referring to were checking any of those sites and/or listening to some sort of sanctioned news update via radio station in 2001 that was affiliated with one of those networks, and the information there derived came from one of the hourly/half-hourly network updates. For example, the Howard Stern show was, at that time, carried by stations all across the nation, including those with affiliations with ABC News, CBS News, etc for their half-hourly news updates. I am certain that it was the Howard Stern show being broadcast live on those affiliates and not some sort of pre-emption for the "home office" of CBS/ABC/NBC, whomever, during that morning. So attributing false reporting to any of them- CBS/ABC/NBC/FOX/CNN (which also does radio updates on the news, nationwide, and I believe did in 2001 as well) isn't really germane to the topic.
Additionally, assuming for the sake of argument the poster was making the comment that you're responding to, your story is from 10 years after 9/11. I don't think you can logically make your claim regarding NPR's credibility in September, 2001 based on a misstep made a decade later. Finally, i wish to point out that the link is from NPR itself, engaging in self-policing.
For what it's worth, I don't even listen to NPR except when in a taxi and it's already on.
Ok. I'll grant that NPR was a better source of news during 911 than Howard Stern. The link I provided shows that someone listening to NPR during an important breaking news story is going to be misinformed sometimes, just like all the other major news sources. Sure NPR is superior as a news source to Howard Stern. Anyone depending upon Howard Stern types for news is an idiot in the first place. And if you're going to extend the notion to not just the news that these networks carry but all the commentators, and all the programming, well, yeah NPR gets a lot wrong, too, on a regular basis, sometimes by omission, just like the others.
Yeah, ending on that point made my post seem like it was all about promoting NPR. But what really came to mind was what that day was like, with everyone wondering WTF was happening and news outlets (legit and not-so-legit) racing and clawing to get the next piece of unconfirmed information blurted out. 20,000 people dead in the WTC. The Capitol building has been hit. A van packed with explosives was found in DC... those are just a few of the ones I can recall.
Clearly some news outlets put great stock in being first. As someone else stated, part of it may be the ad revenue you can command when you claim to be the place everyone seems to turn to. There's also journalistic ego in there too.
I'll stand by my recollection that on that day, I heard less bizarre, untrue crap via public radio than did some of my colleagues who had other stations at hand. But this is less about NPR and more about the race to be first at all costs, even the cost of integrity.
Yeah, I see your point. I was probably too quick to jump on NPR because I have a colleague who is nice and fairly bright, but she can be insufferable when it comes to news. We work with young adults, and often the easiest way to connect is knowing a little about sports, popular music, or any other aspect of popular culture. We often discuss, in passing, the news of the day. When I bring up a topic she's not familiar with (usually dealing with popular culture of some sort) her response is something to the effect of "I don't know about that; I get my news from NPR." It comes across as elitist and self righteous.
You're right, of course, about the race to be first. I wonder with Paterno how far and long the story would have gone on were it not for Jay Paterno tweeting, and the fact that life and death really is a black and white situation (except, perhaps for Miracle Max). So having a first hand account from someone that everyone knows is close to the situation being able to refute information where there's not a gray area, quickly stemmed the spread of the "news" to all the other networks (unlike what happened with the "news" about Giffords' death).
The only reason I learned what TMZ is was because they got the scope on the death and then cbs, nbc, etc picked it up from them, so as was said in the post, small outfits can make big names for themselves by reporting one gigantic headline.
I've always wondered this myself. I just don't get what the big hoop-la is about being the FIRST to report on a matter. Journalism these days is not what one would call up to par and this doesn't mean much anymore but reporting should be about getting the facts and getting the story correct. Instead we have "journalists" jumping at the bit to report who took a dump where and being the first one to do so, instead of getting the facts straight and giving a solid report that they really only pissed in the alley. Bad analogy I know but, do they receive higher wages and earnings or something for being the first? Even if their initial reporting is flawed? I just don't get it.
No you're not alone in caring about quality versus "punctuality"(?).
From the days of at least pre-tv. When your "Lois Lane types" could get the "scoop" on everyone else by uncovering something, and it would only appear in your newspaper. And it would at least be an afternoon edition to 24 hours away before anyone else could cover it, so people would buy you paper. Even TV, when there were just 3 news networks, and not 24 hour coverage, it could take time before people could cover it.
Now, that seems alien to people who grew up with the Internet and 24 hour news, and they're right. And it's become ridiculous in a Twitter age, where things become widespread almost instantly. Look at the Fielder signing...it was becoming public before everyone in the organization even had heard about it. I think i heard the (false) tweet that Rich was fired before the people who had told me Rich was hired had heard. It's a different world. Everybody has become a disseminator of news, and being first by a couple seconds doesn't carry the cache' it used to.