[Coral-List] push for more reliable research in ecology

Douglas Fenner douglasfennertassi at gmail.com
Tue Jan 5 08:16:28 UTC 2021


       Personally, I don't think 5000 publications a day for all of science
and all scientists in all countries, is any kind of indictment of science.
The scientific enterprise has grown enormously from the early days several
hundred years ago, primarily because science has proven to be incredibly
good at learning about the natural world, and incredibly useful for
society.  A UNESCO report says that "There were *7.8 million* full-time
equivalent researchers in 2013."  5000 publications a day would be
1,825,000 papers a year.  Divide by 7.8 million that would be 0.23 papers
per year per scientist.  That sounds like a very low rate of publication, I
would think.
       Science is in the reality business, and knowledge of reality is
incredibly useful.  People who believe in a flat earth or the sun revolving
around the earth or crystal spheres in the sky with planets and stars glued
to them rarely are successful at putting up satellites that photograph the
reality of the earth's surface including coral reefs, nor satellites that
relay email messages such as ours.  Not to mention vaccines that conquer
diseases and the medical science that saves vast numbers of lives, the
technology that we rely on for nearly everything, and a few other odds and
ends.  Replication is indeed an important part of our science, and calls go
out from time to time to allow more attempts at replication to be
published.  Surely some sort of balance is needed, if 90% of attempted
replications succeed at replicating findings, then a lot of effort and
money may be argued to be wasted in that effort.  If 90% of replication
attempts fail, then the efforts to attempt replication are vital to our
science, and we need to let that be published and push us into doing better
science.
      This whole argument you point to is built on calculations.  Where is
the empirical evidence that it is true????  At least with social psychology
the claims were based on attempts to replicate findings, which failed,
that's empirical.  Of course, calculations can never go wrong.  Tell that
to the Boeing company, a very large company that does lots of technology,
one of the two major airplane manufacturers in the world, been doing it
since early in commercial aviation.  Mention "737 max" when you ask how
they are doing.  Faith in calculations nearly bankrupted the company.  So
tell me calculations are always right and there is no need for any
empirical tests to see if they are.  Come back and tell me about it when
you have a pile of empirical tests, actual data.  Preferably for coral reef
science not just medical science or social psychology.  I'm NOT saying
every single study is correct.  I'm asking for empirical evidence not just
calculations.  Let's test it.  Isn't that what science is about, as opposed
to pure calculations??  Should we be aware of the possibility that lots of
things are incorrect?  Sure, good.  How about starting smaller, with some
things we know are problems. like the lack of vouchers for species IDs?  In
your favor, until we have empirical data, these calculations may be the
best available evidence currently.  I'm not an expert on the subject and
don't know if that is the case.  But I suggest a healthy dose of scepticism
to go along with it.
     There is also a need to distinguish direct replication, in which every
detail of a study is replicated exactly, and systematic replication, in
which some things are changed and the generality of a finding is tested.  I
think we underestimate the amount of systematic replication that we do.
So, someone finds a particular result in a small plot of reef at Discovery
Bay, Jamaica.  Someone else does the same type of study, but maybe in
Barbados.  The results fit.  As long as systematic replication works, we're
in pretty good shape, and the generality of our results has been increased,
which is an important part of the building of science.
       Take something like the discovery that many corals spawn on one
night on the Great Barrier Reef.  Has it been replicated??  Many times
over, in fact you can go on a paid tourist night dive to see it yourself.
Our coral reef science is full of things that have been replicated.  There
is still much to do.
       I guess I worry about claims that some people may interpret as all
our science is incorrect.  I view this as an open invitation to throwing
the baby out with the bath.  There is a large component of the public, and
some very active leaders and politicians, who would love that.  Science
denial is a growing problem in societies.  There are reports of people
denying the existence of COVID just before they die of it.  On the other
hand, look at the military.  If they don't base what they do on reality,
they will lose conflicts.  They have not joined the deniers of climate
change, ignoring the risks of climate change may raise risks to national
security and people's lives.  They take the science seriously.
Evidence-based, and the best currently available evidence.  The military
does lots of planning based on the best available science about the
reality that global warming is happening and will get worse.
       I think it is foolish to expect perfection and the "final truth"
from any one study, and I think it is easy to confuse some of these
criticisms with that expectation.  Science is a human endeavor, and like
most of what we do is not perfect at any one stage.  But we contribute and
make progress, and sometimes "stand on the shoulders of giants" as I
believe Darwin once said.
        There are those that point out that science is the best way of
understanding the natural world that has ever been devised.  It has been
devised over hundreds of years by a very large number of people, who
quarrel nearly endlessly about many things.  Look at the difference between
statistics now, and the way it was just 50 years ago, let alone 200 years
ago.  To my knowledge, Newton didn't use any statistics, did he?  Lots of
math, but not statistics.  How about Darwin, any statistics there?  I dare
say none.  Almost all of the great scientists that founded science and many
of which discovered many things that are now so well replicated that they
are rarely questioned, never used any statistics.  Doesn't mean it is bad
to use statistics.  But statistics aren't everything.
     It's easy to say "most scientific findings are false" if you ignore
the fact that essentially NO studies are able to jump directly from where
we are to "final, complete, absolute truth."  Science is largely iterative,
it mostly progresses in relatively small steps.  Some philosophers of
science such as Karl Popper, argue that nearly all hypotheses are probably
incorrect, and we actually make progress when our results disprove
(falsify) hypotheses so we weed out things that are incorrect, rather than
when the results fit with our hypotheses, which is affirming the
consequent.  Affirming the consequent doesn't prove the hypothesis correct
because many other possible hypotheses can predict the same results.  Data
that falsify a hypothesis disprove it, at least in the exact form it was
stated.  And that produces more progress some argue than positive results
that affirm the consequent.
      I DO think that when we find out that there is a better way to do our
science, we need to start applying that, and lifting our game.  Not
sufficiently documenting species identifications for taxa that are not
trivially easy to ID makes replication uncertain, unreliable and more
likely to be incorrect.  I think that's the lesson from the insect article,
and that it applies to all sorts of organisms, particularly in our field,
corals, sponges, etc.  Do we still do multiple t-tests, or have we lifted
that game??  I think we know the weakness there.  We need transparent,
verifiable evidence trails for coral species identification, such as
voucher specimens.  How many DNA sequencing studies done on corals today
leave such a trail of evidence of the ID of the corals they claim to have
such and such a name??  I don't know a survey of that yet, but it appears
to me to not be all the studies.  This is sloppy science.  It doesn't mean
their results per se are wrong, just that the name applied may well be
wrong, and thus it is essentially impossible to replicate the studies
because you don't know which corals to do the replication on.
     I'd also suggest that not all science is done the exact same way as
the medical science this article is talking about, is done.  Try this on a
physicist.  Try it on a naturalist.  Do we deny the photos that show
craters on the moon and insist it is made of green cheese because there is
no statistical test?? Do you really think that over half of all the science
of coral reefs is "wrong"?  There are plenty of things we debate.  But
there are loads of things that have been repeated so many times they are
well accepted.  Some or many of which have some exceptions.  We document
exceptions, we increase accuracy, we come up with new interpretations, new
kinds of data, all part of normal science progress.
     You can't be serious about conspiracy theories.  One of the things I
said offline to a climate change denier once is that just because some
theories that have come to be accepted because they have mountains of
evidence supporting them, even though initially they were not accepted by
hardly anyone, does not mean that every crackpot theory around that is not
currently accepted is actually true.  How is "room temperature fusion"
working these days?  But claiming that most scientific studies are false
WILL give comfort to science deniers and conspiracy theorists
everywhere, who are eager to throw all science out ("it's all fake!!").
Society will suffer if they get their way, and they have made serious
inroads in some countries in recent years.  They are a serious threat to
science and society.  Doesn't mean all criticisms of science are wrong or
should be ignored or discarded.  If I thought that, why would I bring up
the insect ID problem?
     Cheers, Doug


On Sun, Dec 20, 2020 at 11:07 PM David Blakeway <
fathom5marineresearch at gmail.com> wrote:

> I agree this is a major problem for science. Here's a link to an
> influential 2005 article concluding that most published research is false
> <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/>. The argument is
> based on the probability distribution of false positives in statistical
> tests. We should not expect the outcome to be any better in research that
> does not involve statistical tests. There are just so many ways to be
> wrong. Although science is supposed to be self-correcting, correction will
> only happen with the type of concerted effort that is barely possible in
> today's 'hyper-productive' science (current estimates of >5000 publications
> per day, increasing at 8% per year)*.
>
> Charles Darwin had a relevant opinion:
>
> "*False facts are highly injurious to the progress of science, for they
> often endure long; but false views, if supported by some evidence, do
> little harm, for everyone takes a salutary pleasure in proving their
> falseness; and when this is done, one path towards error is closed and the
> road to truth is often at the same time opened*."
>
> Fair enough, but what happens when science's self-correction mechanisms
> are overwhelmed? That would have to increase the likelihood of false views
> becoming false facts without ever being fully tested.
>
>
> The *Acanthaster *example is instructive. A significant result in a test
> with n=30 would be considered pretty solid; write it up and move on. But
> caution revealed something deeper. If the expected result had come first
> though? Well, I don't think I'd be repeating the experiment in that
> situation (especially as it's December 21 already and the lab's only
> published 15 papers this year!!)
>
>
> *from a quick search, these numbers are so astounding I'm not sure they're
> right
>


More information about the Coral-List mailing list