Take-home messages vs. the devil in the details

Submitted by drupaladmin on 15 June 2012.

As scientists, whether we're reading a paper or listening to a talk, we often focus on the take-home message. The main conclusion. The key point. The bottom line. The gist. The summary.

But should we do that? Always?

Because the devil is in the details. And not just sometimes, but pretty much all the time. So if you don't understand the details, if you don't know how the "bottom line" was "calculated", what good does it do you to know it? If you don't know what the summary is summarizing, what's the point of knowing the summary? Indeed, can you even be said to understand the summary?

Now, I actually think those questions have good answers--sometimes. Summaries do have their uses. There are certain times when it's ok to ignore the details and just focus on getting the gist. But details have their uses too, and there are times when it is anything but ok to ignore them. In my experience, many of the most serious mistakes in ecology arise from insufficient attention to detail (see here and here for just two of many possible examples).

So here are some quite specific circumstances in which I think summaries have value even if you don't know the details of what's being summarized:

• The summary is only a starting point; you're going to learn the details. For instance, you read the abstract of an interesting-looking paper, or read a live-tweet of a talk, and decide to go read some papers on that topic. Or, you read a bunch of abstracts, and decide to read whichever papers sound most interesting, thereby using the abstracts as a filter on what details to learn rather than as a substitute for learning the details.
• You're only mildly interested in some topic and have no need to know very much about it.

Conversely, there are other circumstances in which you had better know the details, so that you know exactly what's being summarized. Now, summaries are still valuable in these circumstances--but only because they help you understand the details, not as a substitute for the details.

• You're going to work on the topic.
• You're going to publish something on the topic.
• You're going to apply for a grant on the topic.
• You're going to present on the topic.
• You're going to teach the topic.
• You're going to cite a paper on the topic. Yes, I believe that you should read--carefully, critically, and in its entirety--every paper you cite (rule of thumb: read it as if you're reviewing it). To cite something is to rely on it; you ought to satisfy yourself that you can rely on it. Doing otherwise is how mistakes get perpetuated. Yes, I admit to not always doing this myself. We are all sinners.

And here are some bad reasons and poor excuses for focusing on summaries to the exclusion of the underlying details:

• When reading a theoretical paper, skipping the math and any technical explanation and just focusing on the broad-brush summary, on the grounds that you don't know math or don't like math. No, you don't need to rederive all the math, any more than you have to reperform every experiment you read about. But you can hardly claim to understand the math if you've skipped over the math entirely! Something similar could be said for any technical paper, of course, but in my experience "math phobia" is far more common than "natural history phobia" or "experimental design phobia" or "statistics phobia" or etc.
• Ignoring the details because you're just looking for a way to get a quick paper. Many ecologists are understandably eager to find and use methods that seem easy to apply but yet promise great insight. This doesn't just lead to bandwagons, it also leads to people rushing out to apply these methods without having thought about the details of how the methods work or how the results should be interpreted. In this context, focusing on "the bottom line" to the exclusion of details basically amounts to saying "I don't want to have to think about this method, I just want to be able to 'crank the handle' and use it without thinking."
• Ignoring the details because you think "the big picture" is what really matters. Hey, I'm a big-picture person too. I love the big picture. But if you don't know the underlying details, then you don't know what it's a big picture of. You say you want to see "the forest for the trees"? Well, you'd better be able to tell if that green patch in your "big picture" (your "satellite photo", if you will) is a forest and not grassland or farmland. Heck, you'd better be able to tell if your "big picture" is a satellite photo and not a child's fingerpainting. (oops, that snapping sound you just heard was the sound of the whole "big picture/forest for the trees" metaphor being stretched beyond the breaking point...)

It occurs to me that this is connected to a really old post of mine on hand waving in ecology, in which I illustrated by example, but struggled to articulate, the difference between "good" and "bad" hand waving. One characteristic of good hand waving is that it starts from an appreciation of the details. When someone like Mathew Leibold argues that a very simple food web model "captures the essence" of what's going on in some complex natural community, he's starting from a thorough understanding of what the model assumes and what it predicts. But if you only "get the gist" of that same model, you're in no position to judge whether it is, or is likely to, "capture the essence" of what's going on in your study system.

p.s. Protip: if it's not obvious where a link goes, it probably goes to something funny that's related to the post. This is true of many of my posts.

New ideas

Comments