Can we trust field-guides in ecological studies?

How consistent are field-guides and atlases? Enough to be used as sources in ecological research studies? Jay Fitzsimmons has checked and has the answer! Find out in his new Early View paper "How consistent are trait data between sources? A quantitative assessment". Below he tells us what made him conduct the study:

I compared several field guides and atlases to see whether they were consistent in what they said about species’ traits.  The proximate reason why I did this research is that, given the popularity of trait-based research, I wanted to determine how consistent trait data were among authoritative sources.  The ultimate reason why I did this research is that I’m a paranoid city slicker who worries over how little I know about my study species.

Trait-based ecology is increasingly popular, with researchers evaluating whether species’ traits are related to a variety of ecological factors (e.g., extinction risk, invasiveness).  In my PhD I did such an analysis myself, evaluating the relationship between butterfly species’ traits and their rates of northward range shift in Canada over the 1900s (not yet published – it’s only been two weeks since my PhD ended so give me a break).  The advice given to me on how to obtain species’ trait data was “just use an atlas or a field guide.”  This is when my paranoia alarm started ringing.  Which atlas or field guide should I use?  Do they all say the same thing?

 While I love and respect natural history (I volunteer as Journal Manager for The Canadian Field-Naturalist – www.canadianfieldnaturalist.ca), I am not a great naturalist myself.  I cannot even identify most of Canada’s butterflies, never mind critically evaluate the accuracy of their trait data.  This is a serious problem for macroecology that isn’t given the critical attention it deserves: researchers using fancy models and elaborate analyses can miss critical issues if they don’t know the natural history of their study species.  Non-naturalist macroecologists can miss interesting results that merit follow-up, or wonky results that could indicate coding errors.

 So which source should a paranoid, butterfly-ignorant macroecologist use?  All the sources of course!  Ok, not all the sources (not by a long shot), but I used five authoritative sources authored by recognized experts.  I entered data from each source for 22 traits for 263 Canadian butterfly species.  I compared trait data for species across sources: do different sources say the same things about species?  I found some traits to be very consistent across sources, and others worryingly inconsistent.  In general it seems that subjective traits (e.g., habitat association) were less consistent than more clearly-defined traits (e.g., wingspan, over-winter stage).  This suggests results from single-source trait studies may depend in part on which source was used for trait data.

It was a pleasure to do this work, and to vindicate my paranoia (I can put the tinfoil hat away for another day).  I hope others do similar comparisons of field guides and atlases for other taxa and regions to reveal how general my findings are, and what effect such inconsistency has on trait-based research results.