What's wrong with documentaries and journalism

Just saw a documentary last night: Rize. It's about the culture of clowning and krumping in LA, which is basically street kids dressing up like clowns and having these energetic dance competitions. And though I enjoyed the movie, it re-affirmed the view I walked in with: Don't trust documentaries.

It's not just documentaries I don't trust-- it's also journalism and, most recently, many books.

The basis of this reasoning is that truth is difficult to get at. It requires careful study, and personally, I believe that science is the only trustworthy way to determine what it's rational to believe about the world. It's not that science always brings up the truth; it doesn't. It's not that non-science can't get lucky once in a while; it does. It's just that looking at current scientific research gives you what is most rational to believe right now, given the world's state of knowledge and theory.

Key to this is peer review, and that's where journalism, documentaries, and many books get into trouble. Peer review is the process of getting published through approval of your scientific peers. If you've made a mistake, it's the reviewer's job to see that. You might have not controlled for a variable. You might have ignored some other force that has been discovered that might render your hypothesis faulty. When you read a paper in a scientific journal, or see it at a scientific conference, you can trust to some degree that it's been screened by about three scientists who can vouch for its quality. That means that if you want to walk away believing what the article concludes, you can feel fairly fuzzy inside with respect to your justification in doing so.

The problem with documentaries and journalism is not that they can't get it right, but since there's no peer review you can't tell if it was done right or not. The fact that you can create polemical, one-sided documentaries means that you can't tell, as a naive observer, whether the documentary you're watching is one of those or not. So when I watch documentaries I have this uncomfortable conflict in my head: believe it or don't? I'd rather not have the struggle and just watch fiction. Another problem with documentaries is that the visual medium is not condusive to reporting quantitative data which is often so key in understanding anything. Here's an example from Rize: They showed a long sequence in which the krumping people were dancing, and similar-looking dances from African tribes were interspersed. What am I to make of this? Did these inner-city kids watch a lot of discovery channel, tape these dances and run with them? Is the filmmaker trying to say there is something preserved through black culture running all the way back to Africa? Or even more ridiculously, that there is something innate about black people to make them dance this way? Of course, this question is not answered in the film. All we have is this sequence that suggests a connection, but doesn't come out and say what it is, let alone try to justify it. So what is the observer rationally supposed to take from this? The verbal nature of writing allows very precise statements, and when you have precise statements, the required evaluation of them becomes much clearer.

[From my friend Daniel Saunders after reading my essay (LaChappelle is the director):
From a salon.com interview with David LaChappelle,
http://www.salon.com/ent/feature/2005/01/27/rize/index3.html:

"They create family where there was none and they create art where there
wasn't any. I was given every art program when I was a kid. They've had
none. They've had no African studies. They've never seen African dance.
They've never seen African face paint. It's in their blood."

Ouch.]

What's my problem with journalism? Well, journalism has one up on documentaries because at least they have fact-checkers. So I think it's fine to trust journalism with getting the facts right. However, there is the problem of which facts are reported, and in what context. Like a documentary, the presentation of facts usually makes a subtle, implied point. It's this point, which is often implicit, that is not a fact that can be "checked." It is, whether they know it or not, a scientific theory that needs empirical evaluation. Even worse, sometimes this theory is not implicit. There's a respected magazine called The Economist. I was reading an article about vehicle safety and in that article they wrote that Americans accept accidents as inevitable. The facts they drew on were that they purchased big cars that had poor maneuverability and bulk. This is in spite of the fact that you're safer in a Jetta, statistically, than in an SUV. But to say a culture accepts accidents as inevitable is a major claim that is underconstrained by the facts. Why on earth did they publish such a thing? It's a plausible explanation, but there are a hundered roads called "Plausible" on the way to truth. It could be that they are buying SUVs purely for style reasons. It could be they like being higher up, and their reasons for buying them has nothing to do with safety. It could be that they don't rellly understand how smaller cars can avoid accidents. As you can see, the Economist's statement assumes a lot. Figuring this out is the job of a scientist.

Now we get to books. The problem with books, even "scientific" books, is that, believe it or not they are not peer reviewed either. Gladwell's awful "The Tipping Point" is a great case in point. He's a great writer, and the book is full of interesting information, but it's terribly unscientific, even though he's arguing for a scientific theory. Yesterday I bought a book for my father for Christmas (you can keep reading Dad; I'm taking it back today) called "The Skeptical Environmentalist," by Lomborg. It looked great! Full of charts, one quarter of the book was endnotes and a bibliography, written by a statistician, quotes of praise from Matt Ridley (author of Gemone, a great book), the Wall Street Journal, and the Times. It looked like a great, data-driven review of the state of the environment. I got on the web later and found out that there was a huge outcry from scientists blasting this book. I could not find a single scientist who praised it (Matt Ridley is a science writer, but not a scientist). Like Diamond's "Collapse," (which is on the opposite side of the environmental adgenda), the science in books is often either poorly reported (Collapse) or poorly done (Lomborg).

So now I'm even skeptical of books. For me to trust a non-fiction book now, it needs to be written by a scientist I trust, with a heavy publication record in the field as the book. See what I mean about getting crotchety?

A caveat: Autobiographies and travel stories have their value. In this essay I am talking primarily about big truths worth knowing. Big things about the world we live in. Little details about the lives of individuals perhaps need not be scientific in their reporting.

Anyway, this is the long-winded reason that if you invite me over to watch a documentary, I might just stay home and watch "Spirited Away." If I'm watching untrustworthy material, it might as well have a good story.