[Coral-List] Reliable coral reef stats

John McManus jmcmanus at rsmas.miami.edu
Sat Sep 21 10:29:33 EDT 2013

I have seen a similar problem. For identifying the impacts of lionfish
invasions, a funding agency insisted that the more appropriate AGRRA
methodology I had recommended was too complex (an extra two days of training
to pay for) for the local scientists to use, and that the Reef Check
approach be used, even though that left that country with far less
analytical variables with which to identify what the lionfish were doing to
the ecological communities and under what circumstances (such as when there
were lots versus few places for the prey to hide). That was at a national
level, and there was even less interest in applying a statistical design to
the broad area study. As has been pointed out by others, the RC method is
reasonably good for large area summaries of certain indicators. It is just
not always the right approach. 

We really do have to start moving toward appropriate statistical design for
single reefs or systems of reefs. The sample units one uses could be based
on the Reef Check procedures, the AGRRA procedures, the ASEAN-Australian
procedures, the excellent new ReefBudget procedures or any of a myriad of
others. One can simply apply them in an appropriate statistical design
(stratified random, the adaptive approach of Thompson, the widely useful
replicated systematic approach of Hayek and Buzas, and any combination of
designs in a nested manner such as the approach by Smith et al 2011 in
Environmental Monitoring and Assessment). Often, a 'haphazard' approach will
suffice for the lower level of a nested design (such as where to place 4
quadrats at a location along a long transect), provided that bias is
minimized. In some cases, additional data may need to be obtained, as via
the point-centered quadrant or other approach to obtain minimally biased
information by colony. In all cases, it is crucial that the number of
sampling units be determined in advance (whether by pre-sampling or from
other data sets or studies) so that one actually learns what one needs to
know. In the end, one will actually gather the information one needs with
known uncertainty, and still have valid data for those larger databases.
And, of course, all that data should be made publicly easily available so
the world benefits as much as possible from the work.

The main reason for improving the statistical approaches is that some
scientists seem to be self-delusional. For example, I often hear that
someone has identified a 10% change in bottom cover or fish numbers. Yet, it
is highly unlikely in most cases that the sampling design and number of
sample units that the person used could possibly provide an estimate within
20% of the actual values to an 80% probability. So, one can either live in a
delusional world, or one in which one properly estimates how well something
is known. 

Note that formal sampling design is always about scaling up. If one is
trying to understand interrelationships among species, or many other things
one investigates in reef research, the focus may not be on scaling up to
mean values for a reef or set of reefs. Observations, modelling exercises
and small experiments  are often quite appropriately done without worrying
about being representative of the reef as a whole. It is only when one
intends to extrapolate something to one or more entire reefs that one fails
in the absence of a statistical design. The saving factor in large surveys
is the ability to group and weight data to improve levels of certainty.
However, even there, large scale statistical design could greatly improve
the studies, provided there is a central planning capacity with pre-arranged
funding. It is usually the 'ad-hoc' nature of studies involving 'catch as
you can' funding that limits statistical design for these studies. In those
cases, drive forth, cover as much ground as possible, and use grouping and
weighting methods as best as one can. 



-----Original Message-----
From: coral-list-bounces at coral.aoml.noaa.gov
[mailto:coral-list-bounces at coral.aoml.noaa.gov] On Behalf Of Dan Brumbaugh
Sent: Friday, September 20, 2013 1:58 PM
To: coral-list at coral.aoml.noaa.gov
Subject: Re: [Coral-List] Reliable coral reef stats

Hi John,

As you mentioned below, there is no one-size-fits-all monitoring method and
all have pros and cons. Similarly, David Fisk alluded to the fact that
different reef survey methods have different strengths and weakness. Whereas
some like Reef Check are relatively good at applying an easily implemented,
uniform approach that can be used extensively around the world, and is
therefore good for global comparisons, other methods are often better for
local monitoring programs where the bigger priority, beyond comparing their
sites to others regionally or globally, is detecting significant local
changes to reefs to facilitate adaptive management. David Fisk's critique,
which also resonates with my experience, seemed less about Reef Check
itself, but its potential misapplication (e.g., because of its overly coarse
classification of benthic categories or selection of fish target groups) by
local people to these local monitoring objectives. I've seen similar choices
play out in the Caribbean as well, where the method sometimes appears to
have been chosen simply because it's well known and accessible rather than
because it's the best method to generate the desired information with the
desired sensitivity for detecting changes.

Of course, many many reef scientists decry the fact that reef monitoring
data has been and is being collected in too many different ways at different
sites, and that clearly has been a problem for scaling up to larger-scale
assessments. But it's also important to keep in mind that local managers or
other conservation practitioners usually have limited monitoring resources
that need to be devoted to getting the best information possible to meet
their specific management needs (rather than contributing to the global
database of reef change). Ideally, we might be able to devise "local"
methods that are tailored to local needs that could also be nested within
globally more uniform methods at little additional effort and cost, but
given the seemingly unavoidable tradeoffs in effort and money, I wonder
whether that's practically possible and that we may really be talking about
apples, oranges, and even bananas when it comes to the diversity of
objectives that different people have for reef research or monitoring

So while I applaud the Reef Check program for developing and promoting a
method that is easy to use and gets a lot of people looking at reefs in ways
and places they otherwise wouldn't (I recall an inspiring video of Gregor
Hodgson getting people in post-earthquake Haiti, many non-swimmers, into the
water to look at their marine ecosystem), I also find myself wondering to
what degree many management agencies have considered the best fit between
their management needs and their monitoring options. To help managers with
these decisions, it would be great if reef scientists would be more
thoughtful and specific when they talk about the need for universal survey
methods that may be mainly useful for scientists wanting to scale-up
geographically or otherwise do comparative analyses versus approaches that
may be more useful for local managers who are likely more interested in
detecting particular changes over time at the sites they manage.


Dan Brumbaugh, Ph.D.
Senior Conservation Scientist
Center for Biodiversity and Conservation American Museum of Natural History
Central Park West at 79th Street New York, NY 10024-5192 USA

tel: (212) 496-3494; fax: (212) 769-5292
email: dbrumbaugh at amnh.org
Skype: dan.brumbaugh

Research Associate
Institute of Marine Sciences
University of California
Center for Ocean Health, Long Marine Laboratory
100 Shaffer Road
Santa Cruz, CA 95060-5730

tel: (831) 480-5157; fax: (831) 459-3383

> A few years ago, we were asked to look into existing coral reef 
> sampling protocols to determine what is most appropriate as use as a 
> baseline for a future mitigation concern. We studied every standard we 
> could find from the standpoint of its utility in identifying and 
> quantifying what ecological 'functions' would be lost over time after 
> the damage (as per federal regulations, where in these functions are 
> not well-defined -- but we did what we could in that regard). The answer
was: none.
> The closest was a methodology in which coral colonies were identified 
> and measured as part of the sampling, combined with remote sensing 
> bottom classifications. From this one can obtain rough estimates of 
> potential rates of recovery from previous rates of recruitment and 
> mortality, and quantify the recovery spatially. However, aside from 
> the obvious problem in defining a 'colony' in some circumstances, the 
> approach involved measuring only the longest dimension. This was fine 
> for the purposes of the original methodology designers, but for us, it 
> added a systematic error -- comparable to that of putting transect 
> lines on places with the highest local cover, as sometimes encountered 
> in other protocols. Still, because the bias is known and systematic, 
> prior data could still be used with a correction factor, based either 
> on rationalization or, ultimately, on comparative sampling (something 
> that would be good for those 'highest cover' surveys). Of course, we
recommended that multiple colony measures be used for mitigation cases.
> One point of this is that there is no 'one size fits all' sampling
> If someone needs to identify change in a specific reef, one should 
> match a sampling unit to the purposes of the study, select an 
> appropriate statistical design, choose the appropriate number of 
> sample units from which to obtain data based on an analysis of 
> estimated or pre-sampled local variance, and try to minimize bias 
> throughout the process. Although some people use experimental design 
> procedures for this, it is much better to actually use sampling design 
> principles (see books by Hayek and Buzas, Thompson, or the old Cochran 
> book, or better, get a statistician to help early on!).
> Usually, formal statistical design is not practical for sampling 
> across many reefs (though given the appropriate prior funding 
> commitments it would be feasible), so we must trade off intensiveness 
> for extensiveness. Every methodology has pros and cons for this. Reef 
> Check's approach is very 'extensive', gathering less data to lower 
> cost and permit very wide, rapid sampling. All existing methods have 
> biases one way or another. I think the worst bias we see in regional 
> and global surveys is that of reef selection
> -- we sample primarily convenient, well-known reefs. We have seen 
> specific efforts to gather information from less-known and distant 
> reefs (including efforts from Reef Check, AGRRA, NOAA, Cordio, etc.), 
> and so this bias can increasingly be adjusted for via appropriate data 
> groupings and weightings
> -- assuming that all this data are made publicly and easily available .
> I am not associated with Reef Check, but it clearly is the best thing 
> we currently have in terms of global coverage using a uniform standard.
> However, as global concerns about reef issues increases, there may be 
> new opportunities to improve sampling protocols. So, I encourage 
> discussion on how assessments at national, regional and global scales 
> can be improved. So, please do suggest better ways to obtain this 
> vital information. Critique without suggestion is not terribly helpful.
> Cheers!
> John
> John W. McManus, PhD
> Director, National Center for Coral Reef Research (NCORE) Professor, 
> Marine Biology and Fisheries Coral Reef Ecology and Management Lab 
> (CREM Lab) Rosenstiel School of Marine and Atmospheric Science (RSMAS) 
> University of Miami, 4600 Rickenbacker Causeway, Miami, 33149
> jmcmanus at rsmas.miami.edu      http://ncore.rsmas.miami.edu/
> Phone: 305-421-4814
> "Far better an approximate answer to the right question, which is 
> often vague,
>     than an exact answer to the wrong question, which can always be 
> made precise."
>       --John Tukey, Statistician, National Medal of Science and IEEE 
> Medal of Honor
> -----Original Message-----
> From: coral-list-bounces at coral.aoml.noaa.gov
> [mailto:coral-list-bounces at coral.aoml.noaa.gov] On Behalf Of David 
> Fisk
> Sent: Wednesday, September 18, 2013 1:11 PM
> To: coral-list at coral.aoml.noaa.gov
> Subject: [Coral-List] Reliable coral reef stats
> Some time ago a question was asked on Coral List regarding the 
> appropriateness of Reef Check (RC) for monitoring and I seem to 
> remember that no real answer to the question was offered by anybody at 
> the time. So this latest contribution from RC headquarters has offered 
> a chance for me to give my two cents worth that I should have done the 
> first time. What I would like to know is whether there is anybody else 
> out there in Coral-List land who like myself, feels a little 
> uncomfortable whenever we are given the hard sell from Reef Check, 
> particularly in its effort to legitimise its scientific credential 
> claims with what is and will always be, a simple community awareness 
> raising tool. Reef Check was developed solely as a community awareness 
> tool and has been included in the GCRMN list of protocols under that 
> category, and is widely understood to be the least scientifically 
> robust method for most ecological questions. And yet over the years 
> the assertion of its scientific rigour and global usefulness to answer
scientific question has grown beyond its initial purpose.
> RC's main usefulness lies in getting many observers in the water over 
> a wide geographical scale, with the possibility of ocean wide or 
> regional scale phenomenon being recorded, like regional bleaching 
> events which at that scale, does not actually need scientific rigour 
> to be confident that a broad scale disturbance event is occurring.
> My greatest concern is that the Reef Check juggernaut, principally 
> because of its well organised format, gets presented to many 
> developing countries as the basic tool for all reef related issues 
> that a country may be concerned about. This results in the collection 
> of data that are entirely useless for the real local issues that they 
> may be faced with. The monitoring data may be of some use for regional 
> assessments but I remain sceptical and uneasy about the extremely 
> limited resources of these poor countries being used for the good of 
> regional/global assessments only. The reason I say this is that the 
> pro forma list of indicators species and sample sizes and replicates 
> required by RC are rarely useful for smaller scale questions like Pacific
Island fisheries departments (for example).
> Yet many such organisations are led to believe that Reef Check is all 
> they need to monitor their fisheries or to answer specific disturbance 
> issues, or provide an accurate reef status, which it clearly cannot do.
> Furthermore, the interpretation of data from a protocol like RC should 
> be treated with caution for a very wide range of reef organisms listed 
> as health indicators by RC. A good example is the use of humpheaded 
> parrot fish and maori wrass as 'health indicators' in the Pacific. 
> Both species have very large home ranges that probably extend to a 
> kilometer or more of reef tract, so when very small belt transects are 
> used (RC suggests 20m x 5m belt transect replicates), any observation 
> can only be interpreted as the species being being present, and 
> nothing can be said about their true abundance or density as the sampling
is inappropriate with respect to the target species'
> normal home range. Most importantly, the absence of such species in RC 
> survey records does not mean they are not present, just that the 
> sampling regime is inadequate and did not cover the likelihood of them 
> being observed.
> Another significant issue is the choice of sites for RC as to their 
> 'representativeness' with respect to the aims of a project. RC 
> predominantly utilises volunteers and diver/resort type people to 
> establish monitoring sites and to gather data, but from my experience, 
> the choice of sites are highly biased towards habitats and sites that 
> are appealing to such volunteers such as high coral cover sites, drop 
> offs, or sites where tourism operators can safely moor boats.
> In summary, drawing conclusions on the 'health' of large regional reef 
> systems and the management of coastal fisheries using RC data is 
> obviously wrong for any inappropriately sampled species groups and in 
> most cases the RC protocol does not address the real issues of many
> Don't get me wrong, I believe there is a use for RC but I think it is 
> not in the areas that are becoming too frequently claimed by RC.
> Now, where is my crash helmet - I suspect I may need it.  Cheers Dave 
> Fisk
>> Since 1997, Reef Check has carried out a global monitoring program of
> coral
>> reefs using a standardized method based on about 30 indicators 
>> including living coral and recently killed coral, bleached and 
>> diseased coral. Prior to 2005, Reef Check teams were trained and led 
>> by Masters or PhD level scientists who were responsible for field 
>> level Q & A. Since 2005, all
> Reef
>> Check data has been collected by individuals who have been through a
> formal
>> training program, tested and certified to collect data.  In fact 
>> about 75% of the data has always been collected by research 
>> scientists because Reef Check methods have been adopted by many 
>> countries and research institutes as part of their core monitoring
>> Independent researchers such as
> John
>> Bruno et al., have used the database for local and regional meta
>> The database is available to any researcher. Our WRAS online coral 
>> reef database has been taken offline to transition it from a GIS to a 
>> Google Earth platform, and to merge it with our California NED online 
>> rocky reef database. http://ned.reefcheck.org/ An analysis of the 
>> 15-year coral reef dataset is due out in 2014. For a list of peer 
>> reviewed and other
> technical
>> publications based on or related to Reef Check data please see:
>> http://reefcheck.org/about_RC_Reef/Publications.php
>> Reef Check offers regular training programs throughout the world 
>> using our local coordinators. If you think it is important to track 
>> coral reefs during the next 20 years of global warming, please 
>> contact rcinfo at reefcheck.orgto arrange a training or if you would 
>> like to help coordinate surveys in a country where we do not have a
>> I am noticing two problems when researchers try to compare baselines 
>> now with pre-1980s data:
>> 1. there is a shifting baseline in new cohorts of coral reef 
>> researchers who have a hard time understanding what reefs actually 
>> were like 40 years ago, but typically have not read the old 
>> descriptive literature from pre-1970s such as Saville-Kent that 
>> provide exceedingly detailed descriptions of
> what
>> the shallow reefs were like; 2. a problem with meta-analyses that do 
>> not take into account the fact that in 2013, there are no "zero coral
> cover"
>> data from former reefs or zones of reefs that are no longer reefs 
>> because they experienced 100% die off and so researchers no longer 
>> monitor
> them.
>> Some former reef zones that were populated by Acropora palmata can 
>> still
> be
>> seen, as the dead skeletons are still in place, but those formerly 
>> populated by the more fragile A.
>> cervicornis are often simply gone. This means that the current 
>> regional
> and
>> global coral cover estimates are probably underestimating the decline 
>> as researchers no longer include reefs that died and have not recovered.
>> Gregor Hodgson, PhD
>> Executive Director
>> Reef Check Foundation
> _______________________________________________
> Coral-List mailing list
> Coral-List at coral.aoml.noaa.gov
> http://coral.aoml.noaa.gov/mailman/listinfo/coral-list
> _______________________________________________
> Coral-List mailing list
> Coral-List at coral.aoml.noaa.gov
> http://coral.aoml.noaa.gov/mailman/listinfo/coral-list

Coral-List mailing list
Coral-List at coral.aoml.noaa.gov

More information about the Coral-List mailing list