Advice: weak reasons for choosing a research project

Submitted by drupaladmin on 3 June 2011.

This is the first of what I hope will be an occasional series of posts on ‘random advice for graduate students’. I will make no attempt to be comprehensive; if you want comprehensive, have a look at How to do ecology and the excellent set of resources compiled by Spencer Hall. Instead, I’ll just be tossing out occasional tips that I haven’t seen offered, or sufficiently emphasized, elsewhere.

I emphasize that these tips spring from my own personal values and preferences as a scientist; not all of those values and preferences are universally shared. So treat these tips as food for thought, not as the word of your god of choice.

Today I’m going to talk about weak reasons for choosing a research project. There are lots of reasons for choosing a research project from the infinite universe of possibilities. Some reasons are better than others. Here are some reasons that I’ve encountered often, both as a member of student committees, and as an editor and referee, but that (in my view) are weak or incomplete.

1. Lots of ecologists have long been interested in, or are currently interested in, X. Lots of ecologists are interested in boring and unimportant things (I’m no exception). The fact that lots of people have been, or are, interested in X is not in itself a good reason to study X. Choose your research on the basis of ecology, not based on what ecologists think about ecology. If lots of people study X because X is interesting or important, then you should be able to explain why X is interesting or important. Invoking the authority of other ecologists is not going to convince anyone that X is interesting or important, or demonstrate that you know why X is interesting or important. Because if you don't know that, how do you know you're not just jumping on the latest trendy bandwagon?

2. Not much is known about X. Which means you’ll probably struggle to learn much about it. We learn new things by building on what we already know. Plus, there’s an infinity of things we don’t know much about. Why, out of all those things, do you want to know more about X?

3. People have never studied X in system Y. Ah, the good old “But in my system, things might be different” argument! Sorry, but if that’s your reason for studying X, you’ve immediately raised some questions for anyone who doesn’t already share your fascination with system Y. For instance, is there any reason to think that X works any differently in system Y than in any other system? This is an especially big issue if X is already well-studied in other systems. Note that one common but mistaken response to this concern is to list reasons why system Y is different than other systems, without specifying why those differences would matter for X. For instance, if you say you want to study resource competition in phoenixes, and you say that phoenixes are different than other organisms because they reproduce via spontaneous combustion, you still haven’t explained why you expect resource competition in phoenixes to work differently than in any other organism.

Also, is Y a good model system in which to study X? Maybe nobody’s studied X in system Y because system Y has features that make it difficult to study X. Is there even any reason to think that X occurs, or applies, or matters in system Y? People who are mostly interested in X, as opposed to system Y, will be particularly keen to have these questions answered.

4. People have studied X, and they’ve studied Y, but they’ve never studied X and Y together. There is an infinity of things we’ve never studied in combination. You need some independent reason to study X and Y together (and “X and Y are both important” is not a good reason). For instance, if X and Y are known or thought to be the only two things that affect the response variable of interest, then it make sense to study both together because now you’re exhaustively considering all the possible factors affecting your response variable, rather than just two arbitrarily-chosen ones.

5. Does X or Y have a bigger effect on Z? As stated, this question makes no sense. It only makes sense to ask whether x units of X has a bigger effect than y units of Y. And even though this modified question makes sense, that doesn’t necessarily mean it’s a good question. You still have to explain why your answer isn’t simply a function of your choice of x and y. X may have a weak per-unit effect, and Y may have a strong per-unit effect, but if you use a lot of X and only a little of Y, X will still have a bigger effect on Z. Which seems rather trivial.

6. Do X and Y have independent or interactive effects on Z? I may be in a minority here, but to me, just asking whether two factors have statistically-independent effects seems like a pretty boring question in most circumstances. In general, the effect of anything on anything is going to depend on other things. Now, if X and Y are known to be the only two factors affecting Z, then this question often is a good one, especially if you also have some a priori theoretical reason to expect that X and Y will, or will not, affect Z independently.

There are also technical issues with this kind of question, because whether or not you get a statistical interaction term in your analysis is going to be sensitive to the units in which you measure X, Y, and Z, meaning that (e.g.) otherwise-innocuous data transformations can change the answer. My suggestion would be to frame this kind of question in mechanistic, process-based terms rather than statistical terms. For instance, don’t ask whether predation and competition have statistically-independent effects on diversity, ask if the mechanisms by which competition affects diversity operate any differently in the presence vs. absence of predation, and whether there are any diversity-affecting mechanisms that only operate when both competition and predation are present. This kind of framing is often more informative than a purely statistical framing. Your statistics should be a means to help you answer your scientific question, they shouldn’t define your scientific question.

Advice