[Coral-List] push for more reliable research in ecology

[Garnet Hooper]

I wanted to broaden this discussion somewhat, to highlight that issues in reliability aren’t just potentially limited to statistical robustness, or even to a single paper or study. There are examples where whole research programs, involving participation by multiple reputable academic and governmental organisations and volunteers, have resulted in outcomes that ‘muddy the waters’ of our understanding because the conclusions are just plain incorrect. In the example below (I appreciate it’s a non-coral example, but still worth raising), poor design of multiple facets of the survey was the culprit, and the likely reason was selection of non-technical (though academically-reputable) specialists to design the study. 

The Eighty Mile Beach and Roebuck Bay intertidal/infaunal surveys in Western Australia (e.g. ‘the long mud’) have been undertaken on several occasions and comprised multiple intertidal transects across beaches across a wide area. The aim was to understand zonation across the intertidal zone. But the problem was that the survey wasn’t designed by infaunal specialists, but shorebird specialists - and that’s important. Key issues with the design resulted from a lack of consideration of drivers of zonation for infauna (e.g. transects commenced at the top of the shore), used standardised sampling sub-units of a pre-determined size across all shorelines, and failed to consider differences in profiles of the different shorelines. The failures in the design meant that identification and mapping of different zones was not feasible in most cases. The conclusions therefore erroneously noted that there was no evidence of zonation at the sites surveyed. Several reports and papers of these findings have been peer reviewed and published. 

To be fair, the most recent repeat survey (i.e. ANNROEBIM 2016) report has avoided some of the issues by re-framing the study to focus on mapping and identifying ecological value for potential listing of the site as a world heritage area. And at least the report recognised that some species preferred the mudflats at the top of the shore, and other species preferred the mudflats in the lower shore. However, the underlying issues in the survey design still exist, and therefore far more could be derived from the data if additions to the survey design were made to allow determination of patterns of zonation.

Season’s greetings and all the best for 2021,

Garnet

> On 27 Dec 2020, at 01:00, Nohora Galvis via Coral-List <coral-list at coral.aoml.noaa.gov> wrote:
> 
> I agree with the need to improve reliability in research to improve coral
> reef conservation effectiveness. Most scientists visit coral reef areas to
> sample 5-10 transects in 3 days and conclude about what is enough to save
> coral reefs.
> 
> The role of Citizen Science involving local community is to detail
> georeferenced informatión of what is happening in each site. Therefore,
> case studies are useful to provide a reliable background about
> anthropogenic factors and the respective ecological impact.
> 
> It is vital that researchers take into account local secondary information
> as well to avoid jumping into conclusions or declaring a discovery that was
> considered decades before.
> 
> Season's Greetings !!
> 
> El jue, dic 17, 2020 14:23, Rupert Ormond via Coral-List <
> coral-list at coral.aoml.noaa.gov> escribió:
> 
>> Hi Doug, and coral-listers,
>> 
>> This is certainly an important issue. Human behaviour is highly variable
>> and can shift in relation to numerous subtle factors. The ecology of
>> many marine organisms can similarly be very variable with time and
>> place. In both fields there has been a tendency for researchers to look
>> for situations or locations likely to provide evidence that will support
>> their fashionable theory - old or new - and then play down evidence
>> pointing the other way.
>> 
>> As the papers you mention highlight, there is also a regular problem in
>> ecology with sample size, so very often researchers or referees reject
>> other ideas that conflict with the prevailing view, when often had
>> larger sample sizes been possible, the data would have supported a more
>> complex explanation.
>> 
>> The behaviour of a given species may also be adaptive and change. As an
>> example, I remember very well decades ago running some experiments on
>> the feeding behaviour of Crown-of-thorns with about 30 animals all kept
>> in separate tanks. We had a very clear statistically significant if
>> surprising result. To check, several months later we repeated the
>> experiment and got exactly the opposite result, equally clearly. It
>> turned out starfish show ingestive conditioning and can alter their
>> feeding behaviour dramatically depending on past and recent experience.
>> 
>> Researchers and journals should both be less ready to reject papers with
>> contrary results.
>> 
>> Rupert
>> 
>> *Prof. Rupert Ormond**
>> *Co-Director, Marine Conservation International
>> Hon. Professor, Centre for Marine Biodiversity & Biotechnology,
>> Heriot-Watt University, Edinburgh
>> 
>> 
>>> On 14/12/2020 08:20, Douglas Fenner via Coral-List wrote:
>>> Psychology's replication crisis inspires ecologists to push for more
>>> reliable research
>>> 
>>> 
>> https://www.sciencemag.org/news/2020/12/psychology-s-replication-crisis-inspires-ecologists-push-more-reliable-research
>>> 
>>> By the way, I believe the problem in psychology has been primarily in
>>> social psychology experiments, a small part of psychology as a whole.
>>> 
>>> I think this fits with the need to do much better at documenting the
>>> identification of species in our research (except in areas of low
>> diversity
>>> or easily identified groups, and in field surveys where it is
>> impossible),
>>> as pointed out for insects in the study I pointed to twice.
>> Interestingly,
>>> there wasn't a single comment online or offline to me about the
>>> implications of the insect study for coral reef ecology.  Makes me wonder
>>> if maybe people don't have an argument against it, but just prefer to
>>> continue doing things the way we always have been, after all it would
>>> involve some extra work.  But it goes directly to the question of
>>> replicability, you can't replicate a study if the species identification
>>> can't be verified and may well be wrong.
>>> 
>>> Survey results suggest that a lot of entomology research could be
>>> impossible to replicate
>>> 
>>> https://www.nature.com/articles/d41586-018-01541-0%20
>>> 
>>>  Packer, L., Monckton, S. K., Onuferko, T. M. & Ferrari, R. R.
>> Validating
>>> taxonomic identifications in entomological research.  Insect Conservation
>>> and Diversity 11, 1–12 (2018)
>>> 
>>> https://skmonckton.com/Packeretal._2018_Validating.pdf
>>> 
>>> Cheers,  Doug
>>> 
>>> 
>> _______________________________________________
>> Coral-List mailing list
>> Coral-List at coral.aoml.noaa.gov
>> https://coral.aoml.noaa.gov/mailman/listinfo/coral-list
> _______________________________________________
> Coral-List mailing list
> Coral-List at coral.aoml.noaa.gov
> https://coral.aoml.noaa.gov/mailman/listinfo/coral-list