[Coral-List] Reliable coral reef stats

[Dan Brumbaugh]

Hi John,

As you mentioned below, there is no one-size-fits-all monitoring method 
and all have pros and cons. Similarly, David Fisk alluded to the fact 
that different reef survey methods have different strengths and 
weakness. Whereas some like Reef Check are relatively good at applying 
an easily implemented, uniform approach that can be used extensively 
around the world, and is therefore good for global comparisons, other 
methods are often better for local monitoring programs where the bigger 
priority, beyond comparing their sites to others regionally or globally, 
is detecting significant local changes to reefs to facilitate adaptive 
management. David Fisk's critique, which also resonates with my 
experience, seemed less about Reef Check itself, but its potential 
misapplication (e.g., because of its overly coarse classification of 
benthic categories or selection of fish target groups) by local people 
to these local monitoring objectives. I've seen similar choices play out 
in the Caribbean as well, where the method sometimes appears to have 
been chosen simply because it's well known and accessible rather than 
because it's the best method to generate the desired information with 
the desired sensitivity for detecting changes.

Of course, many many reef scientists decry the fact that reef monitoring 
data has been and is being collected in too many different ways at 
different sites, and that clearly has been a problem for scaling up to 
larger-scale assessments. But it's also important to keep in mind that 
local managers or other conservation practitioners usually have limited 
monitoring resources that need to be devoted to getting the best 
information possible to meet their specific management needs (rather 
than contributing to the global database of reef change). Ideally, we 
might be able to devise "local" methods that are tailored to local needs 
that could also be nested within globally more uniform methods at little 
additional effort and cost, but given the seemingly unavoidable 
tradeoffs in effort and money, I wonder whether that's practically 
possible and that we may really be talking about apples, oranges, and 
even bananas when it comes to the diversity of objectives that different 
people have for reef research or monitoring programs.

So while I applaud the Reef Check program for developing and promoting a 
method that is easy to use and gets a lot of people looking at reefs in 
ways and places they otherwise wouldn't (I recall an inspiring video of 
Gregor Hodgson getting people in post-earthquake Haiti, many 
non-swimmers, into the water to look at their marine ecosystem), I also 
find myself wondering to what degree many management agencies have 
considered the best fit between their management needs and their 
monitoring options. To help managers with these decisions, it would be 
great if reef scientists would be more thoughtful and specific when they 
talk about the need for universal survey methods that may be mainly 
useful for scientists wanting to scale-up geographically or otherwise do 
comparative analyses versus approaches that may be more useful for local 
managers who are likely more interested in detecting particular changes 
over time at the sites they manage.

Cheers,
Dan

-- 
Dan Brumbaugh, Ph.D.
Senior Conservation Scientist
Center for Biodiversity and Conservation
American Museum of Natural History
Central Park West at 79th Street
New York, NY 10024-5192
USA

http://cbc.amnh.org/
tel: (212) 496-3494; fax: (212) 769-5292
email: dbrumbaugh at amnh.org
Skype: dan.brumbaugh

AND
Research Associate
Institute of Marine Sciences
University of California
Center for Ocean Health, Long Marine Laboratory
100 Shaffer Road
Santa Cruz, CA 95060-5730
USA

tel: (831) 480-5157; fax: (831) 459-3383

> A few years ago, we were asked to look into existing coral reef sampling
> protocols to determine what is most appropriate as use as a baseline for a
> future mitigation concern. We studied every standard we could find from the
> standpoint of its utility in identifying and quantifying what ecological
> 'functions' would be lost over time after the damage (as per federal
> regulations, where in these functions are not well-defined -- but we did
> what we could in that regard). The answer was: none.
>
> The closest was a methodology in which coral colonies were identified and
> measured as part of the sampling, combined with remote sensing bottom
> classifications. From this one can obtain rough estimates of potential rates
> of recovery from previous rates of recruitment and mortality, and quantify
> the recovery spatially. However, aside from the obvious problem in defining
> a 'colony' in some circumstances, the approach involved measuring only the
> longest dimension. This was fine for the purposes of the original
> methodology designers, but for us, it added a systematic error -- comparable
> to that of putting transect lines on places with the highest local cover, as
> sometimes encountered in other protocols. Still, because the bias is known
> and systematic, prior data could still be used with a correction factor,
> based either on rationalization or, ultimately, on comparative sampling
> (something that would be good for those 'highest cover' surveys). Of course,
> we recommended that multiple colony measures be used for mitigation cases.
>
> One point of this is that there is no 'one size fits all' sampling approach.
> If someone needs to identify change in a specific reef, one should match a
> sampling unit to the purposes of the study, select an appropriate
> statistical design, choose the appropriate number of sample units from which
> to obtain data based on an analysis of estimated or pre-sampled local
> variance, and try to minimize bias throughout the process. Although some
> people use experimental design procedures for this, it is much better to
> actually use sampling design principles (see books by Hayek and Buzas,
> Thompson, or the old Cochran book, or better, get a statistician to help
> early on!).
>
> Usually, formal statistical design is not practical for sampling across many
> reefs (though given the appropriate prior funding commitments it would be
> feasible), so we must trade off intensiveness for extensiveness. Every
> methodology has pros and cons for this. Reef Check's approach is very
> 'extensive', gathering less data to lower cost and permit very wide, rapid
> sampling. All existing methods have biases one way or another. I think the
> worst bias we see in regional and global surveys is that of reef selection
> -- we sample primarily convenient, well-known reefs. We have seen specific
> efforts to gather information from less-known and distant reefs (including
> efforts from Reef Check, AGRRA, NOAA, Cordio, etc.), and so this bias can
> increasingly be adjusted for via appropriate data groupings and weightings
> -- assuming that all this data are made publicly and easily available .
>
> I am not associated with Reef Check, but it clearly is the best thing we
> currently have in terms of global coverage using a uniform standard.
>
> However, as global concerns about reef issues increases, there may be new
> opportunities to improve sampling protocols. So, I encourage discussion on
> how assessments at national, regional and global scales can be improved. So,
> please do suggest better ways to obtain this vital information. Critique
> without suggestion is not terribly helpful.
>
> Cheers!
>
>
> John
>
> John W. McManus, PhD
> Director, National Center for Coral Reef Research (NCORE)
> Professor, Marine Biology and Fisheries
> Coral Reef Ecology and Management Lab (CREM Lab)
> Rosenstiel School of Marine and Atmospheric Science (RSMAS)
> University of Miami, 4600 Rickenbacker Causeway, Miami, 33149
> jmcmanus at rsmas.miami.edu      http://ncore.rsmas.miami.edu/
> Phone: 305-421-4814
>
> "Far better an approximate answer to the right question, which is often
> vague,
>     than an exact answer to the wrong question, which can always be made
> precise."
>                
>       --John Tukey, Statistician, National Medal of Science and IEEE Medal of
> Honor
>
>
>
>
> -----Original Message-----
> From: coral-list-bounces at coral.aoml.noaa.gov
> [mailto:coral-list-bounces at coral.aoml.noaa.gov] On Behalf Of David Fisk
> Sent: Wednesday, September 18, 2013 1:11 PM
> To: coral-list at coral.aoml.noaa.gov
> Subject: [Coral-List] Reliable coral reef stats
>
> Some time ago a question was asked on Coral List regarding the
> appropriateness of Reef Check (RC) for monitoring and I seem to remember
> that no real answer to the question was offered by anybody at the time. So
> this latest contribution from RC headquarters has offered a chance for me to
> give my two cents worth that I should have done the first time. What I would
> like to know is whether there is anybody else out there in Coral-List land
> who like myself, feels a little uncomfortable whenever we are given the hard
> sell from Reef Check, particularly in its effort to legitimise its
> scientific credential claims with what is and will always be, a simple
> community awareness raising tool. Reef Check was developed solely as a
> community awareness tool and has been included in the GCRMN list of
> protocols under that category, and is widely understood to be the least
> scientifically robust method for most ecological questions. And yet over the
> years the assertion of its scientific rigour and global usefulness to answer
> scientific question has grown beyond its initial purpose.
>
> RC's main usefulness lies in getting many observers in the water over a wide
> geographical scale, with the possibility of ocean wide or regional scale
> phenomenon being recorded, like regional bleaching events which at that
> scale, does not actually need scientific rigour to be confident that a broad
> scale disturbance event is occurring.
>
> My greatest concern is that the Reef Check juggernaut, principally because
> of its well organised format, gets presented to many developing countries as
> the basic tool for all reef related issues that a country may be concerned
> about. This results in the collection of data that are entirely useless for
> the real local issues that they may be faced with. The monitoring data may
> be of some use for regional assessments but I remain sceptical and uneasy
> about the extremely limited resources of these poor countries being used for
> the good of regional/global assessments only. The reason I say this is that
> the pro forma list of indicators species and sample sizes and replicates
> required by RC are rarely useful for smaller scale questions like Pacific
> Island fisheries departments (for example).
> Yet many such organisations are led to believe that Reef Check is all they
> need to monitor their fisheries or to answer specific disturbance issues, or
> provide an accurate reef status, which it clearly cannot do.
>
> Furthermore, the interpretation of data from a protocol like RC should be
> treated with caution for a very wide range of reef organisms listed as
> health indicators by RC. A good example is the use of humpheaded parrot fish
> and maori wrass as 'health indicators' in the Pacific. Both species have
> very large home ranges that probably extend to a kilometer or more of reef
> tract, so when very small belt transects are used (RC suggests 20m x 5m belt
> transect replicates), any observation can only be interpreted as the species
> being being present, and nothing can be said about their true abundance or
> density as the sampling is inappropriate with respect to the target species'
> normal home range. Most importantly, the absence of such species in RC
> survey records does not mean they are not present, just that the sampling
> regime is inadequate and did not cover the likelihood of them being
> observed.
>
> Another significant issue is the choice of sites for RC as to their
> 'representativeness' with respect to the aims of a project. RC predominantly
> utilises volunteers and diver/resort type people to establish monitoring
> sites and to gather data, but from my experience, the choice of sites are
> highly biased towards habitats and sites that are appealing to such
> volunteers such as high coral cover sites, drop offs, or sites where tourism
> operators can safely moor boats.
>
> In summary, drawing conclusions on the 'health' of large regional reef
> systems and the management of coastal fisheries using RC data is obviously
> wrong for any inappropriately sampled species groups and in most cases the
> RC protocol does not address the real issues of many countries.
>
> Don't get me wrong, I believe there is a use for RC but I think it is not in
> the areas that are becoming too frequently claimed by RC.
>
> Now, where is my crash helmet - I suspect I may need it.  Cheers Dave Fisk
>
>
>> Since 1997, Reef Check has carried out a global monitoring program of
> coral
>> reefs using a standardized method based on about 30 indicators
>> including living coral and recently killed coral, bleached and
>> diseased coral. Prior to 2005, Reef Check teams were trained and led
>> by Masters or PhD level scientists who were responsible for field
>> level Q & A. Since 2005, all
> Reef
>> Check data has been collected by individuals who have been through a
> formal
>> training program, tested and certified to collect data.  In fact about
>> 75% of the data has always been collected by research scientists
>> because Reef Check methods have been adopted by many countries and
>> research institutes as part of their core monitoring program.
>> Independent researchers such as
> John
>> Bruno et al., have used the database for local and regional meta analyses.
>> The database is available to any researcher. Our WRAS online coral
>> reef database has been taken offline to transition it from a GIS to a
>> Google Earth platform, and to merge it with our California NED online
>> rocky reef database. http://ned.reefcheck.org/ An analysis of the
>> 15-year coral reef dataset is due out in 2014. For a list of peer
>> reviewed and other
> technical
>> publications based on or related to Reef Check data please see:
>> http://reefcheck.org/about_RC_Reef/Publications.php
>>
>> Reef Check offers regular training programs throughout the world using
>> our local coordinators. If you think it is important to track coral
>> reefs during the next 20 years of global warming, please contact
>> rcinfo at reefcheck.orgto arrange a training or if you would like to help
>> coordinate surveys in a country where we do not have a coordinator.
>>
>> I am noticing two problems when researchers try to compare baselines
>> now with pre-1980s data:
>> 1. there is a shifting baseline in new cohorts of coral reef
>> researchers who have a hard time understanding what reefs actually
>> were like 40 years ago, but typically have not read the old
>> descriptive literature from pre-1970s such as Saville-Kent that
>> provide exceedingly detailed descriptions of
> what
>> the shallow reefs were like; 2. a problem with meta-analyses that do
>> not take into account the fact that in 2013, there are no "zero coral
> cover"
>> data from former reefs or zones of reefs that are no longer reefs
>> because they experienced 100% die off and so researchers no longer monitor
> them.
>> Some former reef zones that were populated by Acropora palmata can
>> still
> be
>> seen, as the dead skeletons are still in place, but those formerly
>> populated by the more fragile A.
>> cervicornis are often simply gone. This means that the current
>> regional
> and
>> global coral cover estimates are probably underestimating the decline
>> as researchers no longer include reefs that died and have not recovered.
>>
>> Gregor Hodgson, PhD
>> Executive Director
>> Reef Check Foundation
> _______________________________________________
> Coral-List mailing list
> Coral-List at coral.aoml.noaa.gov
> http://coral.aoml.noaa.gov/mailman/listinfo/coral-list
>
> _______________________________________________
> Coral-List mailing list
> Coral-List at coral.aoml.noaa.gov
> http://coral.aoml.noaa.gov/mailman/listinfo/coral-list