Never Be The First To Report Someone's Dead

[ED: Meta. I was just going to put a bullet in a UV about this but things got a little crazy and I ended up with a full post.]

Joe Paterno's death was a hugely misreported fiasco of the sort that is inevitable given the speed of information in the internet age. This post is an attempt to provide a framework for existing in a world of uncertain information.

This is what happened: Onward State, a blog/online newspaper run by PSU students, reported Paterno's death based on an email sent to Penn State players that turned out to be a hoax. This was good enough for a local radio station and StateCollege.com. It hit twitter and was then picked up without attribution by CBS Sports. It took off from there once the imprimatur of a major news agency was on it. Black Shoe Diaries has a detailed chronology of the mass screwup if you're interested in details. Shirtless Mark Twain isn't sure if he approves of this whole business or not, but would like you to know that rumors of his rippling pecs have been sorely undersold.

It's a story about the internet screwing up in very understandable ways. Onward State had what seemed like reliable information, and it passed their threshold for reporting. It is not a good threshold, but not everyone has one these days. CBS's Adam Jacobi did something unwise and sloppy. Pagewhoring Huffington Post saw an opportunity for views and cares about nothing else.

We've seen this happen before when a newspaper intern replicates an internet rumor on one of the dingy blogs shuffled off into the corner of large metro papers: as soon as a rumor gets paired with header graphics associated with a real newspaper, everyone else is confirming it via "sources." In this instance, CBS's screwup was compounded because they didn't even provide a link to the primary source; Huffington Post did the same thing, but that's just their MO. Jacobi is a BHGP founder and should have known better.

I've screwed these things up myself. Earlier this year I erroneously reported that Kaleb Ringer had been booted from his high school team based on information that seemed solid but obvious was not. By contrast, a couple years ago I had the sense not to run anything about the serious car accident that Jon Bills and Mark Moundros were in despite having a ton of solid sources telling me about it. That seemed like a place to let journalists be journalists.

As I go along here that realm has steadily expanded. I probably won't report something like the Ringer thing again for a lot of reasons. Michigan playing Alabama is one thing to be wrong about; a high school kid's problems or lack thereof is another. This leaves windows open for crass opportunists like Ace Williams, but it's the internet. There's always going to be a bottom of the barrel.

------------

Anyway, these things evolve naturally. As this site expands it has more at risk and becomes more cautious. People just starting out have little to lose and have not experienced the backlash from being wrong—or the frightening period between your post and official confirmation of it. Also some of them are total idiots.

From the user's perspective, the thing to do is maintain a Bayesian approach. Phil Birnbaum explains what that is:

Generally, Bayesian is a process by which you refine your probability estimate. You start out with whatever evidence you have which leads you to a "prior" estimate for how things are. Then, you get more evidence. You add that to the pile, and refine your estimate by combining the evidence. That gives you a new, "posterior" estimate for how things are.

You're a juror at a trial. At the beginning of the trial, you have no idea whether the guy is guilty or not. You might think it's 50/50 -- not necessarily explicitly, but just intuitively. Then, a witness comes up that says he saw the crime happen, and he's "pretty sure" this is the guy. Combining that with the 50/50, you might now think it's 80/20.

Then, the defense calls the guy's boss, who said he was at work when the crime happened. Hmmm, you say, that sounds like he couldn't have done it. But there's still the eyewitness. Maybe, then, it's now 40/60.
And so on, as the other evidence unfolds.

That's how Bayesian works. You start out with your "prior" estimate, based on all the evidence to date: 50/50. Then, you see some new evidence: there's an eyewitness, but the boss provides an alibi. You combine that new evidence with the prior, and you adjust your estimate accordingly. So your new best estimate, your "posterior," is now 40/60.

So if some guy with 50 followers claims Armani Reeves is headed to Michigan because Urban was late for his in-home visit, you might increment your 50% to 51%. If Mike Farrell says its 52-48 you might bump it to 52%, but if Farrell said he thought Reeves was definitely headed to Michigan you could push it up further. You base your confidence in the opinion on previous accuracy, with a list like this…

1. TomVH/Sam Webb
2. Established message board posters
3. National analysts
4. Random message board posters
5. Raving lunatics
6. People who don't know what football is
7. Fictional races from another galaxy
8. Hyperintelligent tacos
9. Regular tacos
10. Tacos that aren't too bright even for tacos
11. Ace Williams

…and change your baseline confidence based on the information and your confidence level in it. This is something people do naturally, but too often the weight they put on the information is either 0 or 1 when it should be somewhere in between.

For purveyors of information, it's time to put an explicit confidence level on what you're relaying. My mistake with the Ringer thing, other than mentioning it at all, was saying something was the case when I should have said something less certain. When I got tips about the Michigan-Alabama game I erred by saying with certainty a contract would be signed on a certain date when the people involved with the thing probably didn't know that.

I try to follow a policy of revealing as much as I can about the nature any information I pass along without exposing a source, and that added transparency is necessary in an age when information—valid information—can come from anywhere or anyone. I still make mistakes. That's inevitable. I'm trying, though.

However, not even linking to the original report is a mortal sin. If you are going to run something based on someone else's reporting it is vital that you explicitly tell readers that. Otherwise one report from a little-known online news source turns into multiple reports, some of them from organizations with people paid to do reporting, and the echo chamber starts going exponential. If you do not link, you are telling people that you are reporting it, and when it turns out to be wrong you can't point the finger at anyone but yourself.