On asking tough questions

Submitted by drupaladmin on 14 August 2011.

A previous post on this blog asked why ecologists can't all just get along. Contrarian that I am, I want to ask whether we get along too well. That is, do we ask each other enough tough questions?

I've been thinking about posting on this for a while. I finally decided to do so after a commenter on a previous post asked my view on whether it's appropriate to ask a speaker at the ESA meeting a question that implies the speaker has made an extremely serious mistake (e.g., a pseudoreplicated experimental design, ignorance of basic facts about one's study organism, etc.) Questions implying that the speaker has made a serious mistake aren't the only tough questions, of course. Questions which undermine the motivation for the speaker's entire study, for instance (which I ask more often than many people, because I don't take for granted the speaker's framing of the issues, choice of question, and choice of study system)

Tough questions rarely get asked at the ESA meeting, though anecdotally they're more common in other contexts. In Canada, UBC and McGill both have the reputation of asking tough questions of invited speakers, and at Duke in the 70s and 80s I'm told the graduate students were really tough on each other. Do we ask tough questions as often as we should? I don't know the answer, but in the interests of helping to ensure that the answer is "yes", below are some suggestions on how to ask tough questions.

I should preface my remarks by noting that I have the reputation, at my university and probably elsewhere, of being quite a tough questioner. I invite tough questions as well: as a grad student I found the audience at the graduate student seminar series to be too uncritical, and once asked them to fill out anonymous comment forms after my talk in an attempt to get more critical feedback. Since I'm comfortable getting tough questions, and confident in my ability to answer them, I tend to have little hesitation in asking them and probably don't always ask them in the best way. I think the advice I offer below is reasonable advice, but I admit I'm not perfect about following it myself (though I'm better than I used to be).

First, tough questions aren't the same as rude questions. At his PhD defense seminar, my colleague Steve Vamosi was asked "So Steve, you've shown that sometimes populations go up, and sometimes they go down. What have we learned here?" And I know of an ecologist who once asked, of an invited speaker, "What was the point of this?" That kind of thing is just asinine; it's easy to make anyone's work sound stupid ("So Mr. Darwin, you've shown that things change. What have we learned here?") No matter how bad you think someone's talk was, don't ask a rude question. If the purpose of the question is not to elicit an answer, but simply to let the speaker and the audience know how bad you thought the talk was, that's rude (and redundant, because if the talk really was that bad, everyone in the audience already knows it)

Second, consider the speaker and the setting. If the speaker is a student, or is clearly nervous, or the seminar is some kind of informal or semi-formal practice talk, phrase your question in a positive, helpful way. Prefacing your question with some positive remarks about the talk is one way to do this. I admit I'm probably not as good as I should be about following this particular piece of advice. I asked a pretty challenging question of a student at this year's meeting, indeed so challenging that the student's supervisor (who's a friend of mine) was swivelling in her seat trying to see who was asking the question. In my defense, the student seemed confident and professional during the seminar, and so didn't seem like someone who would just wilt when pushed a little. And he handled the question fine, although at the time his answer didn't really satisfy me. So I went and found the student and his supervisor later to talk further about it, and to make sure there were no hard feelings, and we ended up having a really good conversation about different ways to approach the problem the student was working on. So it worked out in the end, but I was probably over-aggressive at first.

The flip side of this second point is that, if the speaker has a PhD, and if the setting is a formal one like the ESA meeting, I tend not to pull many punches. If you have a PhD, and you're talking in a setting in which speakers are expected to be extremely well-prepared, you should be able to handle any non-rude question anyone asks you. And if the speaker is aggressive about pushing a point with which I disagree, I don't hesitate to push back, taking the view that the speaker is spoiling for a fight (I did encounter one such speaker at the ESA this year).

Third, phrase your question in such a way as to make it unaggressive, but without changing the content. For instance, if you want to question the use of a pseudoreplicated design, you could phrase the question as a request for the speaker to clarify the design, because maybe you misunderstood but it sounded like it might be pseudoreplicated. This kind of phrasing also hedges against the possibility that you really did misunderstand and the experiment really was replicated properly (or whatever). Or, depending on the nature of the question, you might be able to "defuse" it by simply saying right up front that it's a tough question, but it's one you really think is important and are really interested in having answered, so you're going to ask it. This tack can work well if the question is one that's been debated in the literature, so you're basically inviting the speaker to engage in a wider, ongoing debate rather than challenging the speaker as an individual. This tack also can work well if the speaker is a friend, or is someone who you know likes a good back-and-forth debate.

Fourth, if the speaker is not a native English speaker, keep that in mind when deciding how to ask your question.

Fifth, if you really think that there's no way to ask the question without seriously embarrassing the speaker, contact the speaker later and ask the question privately.

Sixth, once you ask the question, wait for the answer. This is something else I'm not as good at as I should be. If the speaker appears not to be answering my question to my satisfaction, I have to fight the urge to cut them off and ask the question more forcefully, which is bad.

So what do you think? Do we ask enough tough questions of each other? And do we ask them well? What's the toughest (or rudest!) question you've ever gotten? Answer in the comments.

Advice