Jeff Gerbracht

Can Observation Skills of Citizen Scientists Be Estimated Using Species Accumulation Curves

Volunteers are increasingly being recruited into citizen science projects to collect observations for scientific studies. An additional goal of these projects is to engage and educate these volunteers. Thus, there are few barriers to participation resulting in volunteer observers with varying ability to complete the project’s tasks. To improve the quality of a citizen science project’s outcomes it would be useful to account for inter-observer variation, and to assess the rarely tested presumption that participating in a citizen science projects results in volunteers becoming better observers. Here we present a method for indexing observer variability based on the data routinely submitted by observers participating in the citizen science project eBird, a broad-scale monitoring project in which observers collect and submit lists of the bird species observed while birding. Our method for indexing observer variability uses species accumulation curves, lines that describe how the total number of species reported increase with increasing time spent in collecting observations. We find that differences in species accumulation curves among observers equates to higher rates of species accumulation, particularly for harder-to-identify species, and reveals increased species accumulation rates with continued participation. We suggest that these properties of our analysis provide a measure of observer skill, and that the potential to derive post-hoc data-derived measurements of participant ability should be more widely explored by analysts of data from citizen science projects. We see the potential for inferential results from analyses of citizen science data to be improved by accounting for observer skill.

Automated data verification in a large-scale citizen science project: A case study

Although citizen science projects can engage a very large number of volunteers to collect volumes of data, they are susceptible to issues with data quality. Our experience with eBird, which is a broad-scale citizen science project to collect bird observations, has shown that a massive effort by volunteer experts is needed to screen data, identify outliers and flag them in the database. The increasing volume of data being collected by eBird places a huge burden on these volunteer experts and other automated approaches to improve data quality are needed. In this work, we describe a case study in which we evaluate an automated data quality filter that improves data quality by identifying outliers and categorizing these outliers as either unusual valid observations or mis-identified (invalid) observations. This automated data filter involves a two-step process: first, a data-driven method detects outliers (ie. observations that are unusual for a given region and date). Next, we use a data quality model based on an observers predicted expertise to decide if an outlier should be flagged for review. We applied this automated data filter retrospectively to eBird data from Tompkins Co., NY and found that that this automated process significantly reduced the workload of reviewers by as much as 43% and identifies 52% more potentially invalid observations.

Best Practices for using Cloud Services for Digital Data Archive and Disaster Recovery

Managing digital data for long-term archival and disaster recovery is a key component of our collective responsibility in managing digital data and metadata. As more and more data are collected digitally and as the metadata for traditional museum collections becomes both digitized and more comprehensive, the need to ensure that these data are safe and accessible in the long term becomes essential. Unfortunately, disasters do occur and many irreplaceable datasets on biodiversity have been permanently lost. Maintaining a long-term archive and putting in place reliable disaster recovery processes can be prohibitively expensive, both in the cost of hardware and software as well as the costs of personnel to manage and maintain an archival system. Traditionally, storing digital data for the long term and ensuring the data are loss-less, safe and completely recoverable when a disaster occurs has been managed on-premises with a combination of on-site and off-site storage. This requires complex data workflows to ensure that all data are securely and redundantly stored in multiple highly dispersed locations to minimize the threat of data loss due to local or regional disasters. Files are often moved multiple times across operating systems and media types on their way to and from a deep archive, increasing the risk of file integrity issues. With the recent advent of an array of Cloud Services from organizations such as Amazon, Microsoft and Google to more focused offerings from Iron Mountain, Atempo and others, we have a number of options for long term archival of digital data. Deep archive solutions, storage where retrieval expected only in the case of a disaster, are ‡ ‡