Ken Aho

Model selection for ecologists: the worldviews of AIC and BIC

Ecologists frequently ask questions that are best addressed with a model comparison approach. Under this system, the merit of several models is considered without necessarily requiring that (1) models are nested, (2) one of the models is true, and (3) only current data be used. This is in marked contrast to the pragmatic blend of Neyman-Pearson and Fisherian significance testing conventionally emphasized in biometric texts (Christensen 2005), in which (1) just two hypotheses are under consideration, representing a pairwise comparison of models, (2) one of the models, H0, is assumed to be true, and (3) a single data set is used to quantify evidence concerning H0. As Murtaugh (2014) noted, null hypothesis testing can be extended to certain highly structured multi-model situations (nested with a clear sequence of tests), such as extra sums of squares approaches in general linear models, and drop in deviance tests in generalized linear models. This is especially true when there is the expectation that higher order interactions are not significant or nonexistent, and the testing of main effects does not depend on the order of the tests (as with completely balanced designs). There are, however, three scientific frameworks that are poorly handled by traditional hypothesis testing. First, in questions requiring model comparison and selection, the null hypothesis testing paradigm becomes strained. Candidate models may be non-nested, a wide number of plausible models may exist, and all of the models may be approximations to reality. In this context, we are not assessing which model is correct (since none are correct), but which model has the best predictive accuracy, in particular, which model is expected to fit future observations well. Extensive ecological examples can be found in Johnson and Omland (2004), Burnham and Anderson (2002), and Anderson (2008). Second, the null hypothesis testing paradigm is often inadequate for making inferences concerning the falsification or confirmation of scientific claims because it does not explicitly consider prior information. Scientists often do not consider a single data set to be adequate for research hypothesis rejection (Quinn and Keough 2002:35), particularly for complex hypotheses with a low degree of falsifiability (i.e., Popper 1959:266). Similarly, the support of hypotheses in the generation of scientific theories requires repeated corroboration (Ayala et al. 2008). Third, ecologists and other scientists are frequently concerned with the plausibility of existing or default models, what statistician would consider null hypotheses (e.g., the ideal free distribution, classic insular biogeography, mathematic models for species interactions, archetypes for community succession and assembly, etc.). However, null hypothesis testing is structured in such a way that the null hypothesis cannot be directly supported by evidence. Introductory statistical and biometric textbooks go to great lengths to make this conceptual point (e.g., DeVeaux et al. 2013:511, 618, Moore 2010:376, Devore and Peck 1997:300–303).

Pikas (Ochotona princeps : Lagomorpha) as allogenic engineers in an alpine ecosystem

Pikas (Ochotona princeps: Lagomorpha) build caches of vegetation (“haypiles”), which serve as a food source during winter in alpine and subalpine habitats. Haypiles appear to degrade over time and form patches of nutrient-rich soils in barren talus and scree areas. We sampled soils underneath and next to haypiles, and plants growing on and near haypiles in an alpine cirque in northwestern Wyoming, USA, to determine the effects of pika food caches on N, C, and C/N ratios in soils and plants. We found that (1) haypile soils had significantly higher carbon and nitrogen levels and lower C/N ratios than both adjacent soils and soils in the general study area, (2) two of three plant species tested (Polemonium viscosum and Oxyria digyna) had significantly higher levels of tissue percent N when growing on haypile soils, and (3) total standing plant biomass at the study site increased with soil percent N suggesting that vegetation was nitrogen limited. Pikas may therefore function as allogenic ecosystem engineers by modulating nutrient availability to plants.

A graphical framework for model selection criteria and significance tests: refutation, confirmation and ecology

Summary 1.In this paper we use a novel graphical heuristic to compare the way four methods: significance testing, two popular information-theoretic approaches (AIC and BIC), and Goods Bayes/Non-Bayes compromise (an underutilized hypothesis testing approach whose demarcation criterion adjusts for n) evaluate the merit of competing hypotheses, e.g., H0 and HA. 2.A primary goal of our work is to clarify the concept of strong consistency in model selection. Explicit considerations of this principle (including the strong consistency of BIC) are currently limited to technical derivations, inaccessible to most ecologists. We use our graphical framework to demonstrate, in simple terms, the strong consistency of both BIC and Goods compromise. 3.Our framework also locates the evaluated metrics (and ICs in general) along a conceptual continuum of hypothesis refutation/confirmation that considers n, parameter number, and effect size. Along this continuum, significance testing, and particularly AIC are refutative for H0, whereas Goods compromise, and particularly BIC are confirmatory for the true hypothesis. 4.Our work graphically demonstrates the well-known asymptotic bias of significance tests for HA, and the incorrectness of using statistically non-consistent methods for point hypothesis testing. To address these issues we recommend: 1) dedicated confirmatory methods with strong consistency like BIC for use in point hypothesis testing and confirmatory model selection; 2) significance tests for use in exploratory/refutative hypothesis testing, particularly when conjoined with rational approaches (e.g., Goods compromise, power analyses) to account for the effect of n on P-values; and 3) asymptotically efficient methods like AIC for exploratory model selection. This article is protected by copyright. All rights reserved.

Relative Abundance of and Composition within Fungal Orders Differ between Cheatgrass (Bromus tectorum) and Sagebrush (Artemisia tridentata)-Associated Soils

Nonnative Bromus tectorum (cheatgrass) is decimating sagebrush steppe, one of the largest ecosystems in the Western United States, and is causing regional-scale shifts in the predominant plant-fungal interactions. Sagebrush, a native perennial, hosts arbuscular mycorrhizal fungi (AMF), whereas cheatgrass, a winter annual, is a relatively poor host of AMF. This shift is likely intertwined with decreased carbon (C)-sequestration in cheatgrass-invaded soils and alterations in overall soil fungal community composition and structure, but the latter remain unresolved. We examined soil fungal communities using high throughput amplicon sequencing (ribosomal large subunit gene) in the 0–4 cm and 4–8 cm depth intervals of six cores from cheatgrass- and six cores from sagebrush-dominated soils. Sagebrush core surfaces (0–4 cm) contained higher nitrogen and total C than cheatgrass core surfaces; these differences mirrored the presence of glomalin related soil proteins (GRSP), which has been associated with AMF activity and increased C-sequestration. Fungal richness was not significantly affected by vegetation type, depth or an interaction of the two factors. However, the relative abundance of seven taxonomic orders was significantly affected by vegetation type or the interaction between vegetation type and depth. Teloschistales, Spizellomycetales, Pezizales and Cantharellales were more abundant in sagebrush libraries and contain mycorrhizal, lichenized and basal lineages of fungi. Only two orders (Coniochaetales and Sordariales), which contain numerous economically important pathogens and opportunistic saprotrophs, were more abundant in cheatgrass libraries. Pleosporales, Agaricales, Helotiales and Hypocreales were most abundant across all libraries, but the number of genera detected within these orders was as much as 29 times lower in cheatgrass relative to sagebrush libraries. These compositional differences between fungal communities associated with cheatgrass- and sagebrush-dominated soils warrant future research to examine soil fungal community composition across more sites and time points as well as in association with native grass species that also occupy cheatgrass- invaded ecosystems.

Psychoactive pharmaceuticals as environmental contaminants may disrupt highly inter-connected nodes in an Autism-associated protein-protein interaction network

BackgroundMost cases of idiopathic autism spectrum disorder (ASD) likely result from unknown environmental triggers in genetically susceptible individuals. These triggers may include maternal exposure of a fetus to minute concentrations of pharmaceuticals, such as carbamazepine (CBZ), venlafaxine (VNX) and fluoxetine (FLX). Unmetabolized pharmaceuticals reach drinking water through a variety of routes, including ineffectively treated sewage. Previous studies in our laboratory examined the extent to which gene sets were enriched in minnow brains treated with pharmaceuticals. Here, we tested the hypothesis that genes in fish brains and human cell cultures, significantly enriched by pharmaceuticals, would have distinct characteristics in an ASD-associated protein interaction network. We accomplished this by comparing these groups using 10 network indices.ResultsA network of 7212 proteins and 33,461 interactions was generated. We found that network characteristics for enriched gene sets for particular pharmaceuticals were distinct from each other, and were different from non-enriched ASD gene sets. In particular, genes in fish brains, enriched by CBZ and VNX 1) had higher network importance than that in the overall network, and those enriched by FLX, and 2) were distinct from FLX and non-enriched ASD genes in multivariate network space. Similarly, genes in human cell cultures enriched by pharmaceutical mixtures (at environmental concentrations) and valproate (at clinical dosages) had similar network signatures, and had greater network importance than genes in the overall ASD network.ConclusionsThe results indicate that important gene sets in the ASD network are particularly susceptible to perturbation by pharmaceuticals at environmental concentrations.

Confidence intervals for ratios of proportions: implications for selection ratios

Summary The selection ratio (proportional resource use divided by proportional availability) is often used by ecologists to measure the degree to which individuals and populations are selective in their food sources and habitats. Yet, confidence interval approaches for this metric are scarce and poorly evaluated. In this paper, we compare 13 methods that can be used to construct simultaneous confidence intervals for selection ratios. Seven of the methods are applicable when availabilities are unknown. These are bootstrapping and six methods adapted from relative risk (Katz-log, adjusted-log, Bailey, inverse hyperbolic sine, Koopman and Noether). The other six approaches are applicable when availabilities are known. These are bootstrapping, two existing methods for relative risk (Wald-adjusted, Noether-fixed) and three straightforward new methods (fixed-log, Agresti–Coull-adjusted and Bayes-beta). None of the 13 approaches have been previously evaluated in the context of selection ratios. In simulations with unknown availabilities, the Koopman method performed best, and the currently recommended Noether method performed worst. In simulations with fixed availabilities, the new Agresti–Coull-adjusted, fixed-log, Bayes-beta methods all outperformed the currently recommended Wald-adjusted method. In the context of real ecological data sets, the Noether and Wald-adjusted methods produced anomalous results that putatively would alter management decisions. We note that our findings, including those for new methods, are directly conferrable to relative risk, allowing extension of our work to the many branches of biology that rely on this measure. The poor performance of the Noether and Wald-adjusted methods is troubling because these are currently the most widely used procedures for calculating confidence intervals for type I resource-selection designs. Based on our findings, we recommend that the Noether method be replaced with the Koopman method, and the Wald-adjusted method be replaced with the Agresti–Coull-adjusted method.

Diversity and Ice Nucleation Activity of Microorganisms collected with a Small Unmanned Aircraft System (sUAS) in France and the United States

Many microbes relevant to crops, domestic animals, and humans are transported over long distances through the atmosphere. Some of these atmospheric microbes catalyze the freezing of water at higher temperatures and facilitate the onset of precipitation. We collected microbes from the lower atmosphere in France and the United States with a small unmanned aircraft system (sUAS). 55 sampling missions were conducted at two locations in France in 2014 (an airfield in Pujaut, and the top of Puy de Dôme), and three locations in the U.S. in 2015 (a farm in Blacksburg, Virginia, and a farm and a lake in Baton Rouge, Louisiana). The sUAS was a fixed-wing electric drone equipped with a remote-operated sampling device that was opened once the aircraft reached the desired sampling altitude (40–50 meters above ground level). Samples were collected on agar media (TSA, R4A, R2A, and CA) with and without the fungicide cycloheximide. Over 4,000 bacterial-like colonies were recovered across the 55 sUAS sampling missions. A positive relationship between sampling time and temperature and concentrations of culturable bacteria was observed for sUAS flights conducted in France, but not for sUAS flights conducted in Louisiana. A droplet freezing assay was used to screen nearly 2,000 colonies for ice nucleation activity, and 15 colonies were ice nucleation active at temperatures warmer than −8°C. Sequences from portions of 16S rDNA were used to identify 503 colonies from 54 flights to the level of genus. Assemblages of bacteria from sUAS flights in France (TSA) and sUAS flights in Louisiana (R4A) showed more similarity within locations than between locations. Bacteria collected with sUAS on TSA in France and Virginia were significantly different across all levels of classification tested (P < 0.001 for class, order, family, and genus). Principal Coordinates Analysis showed a strong association between the genera Curtobacterium, Pantoea, and Pseudomonas from sUAS flights in Virginia, and Agrococcus, Lysinibacillus, and Paenibacillus from sUAS flights in France. Future work aims to understand the potential origin of the atmospheric microbial assemblages collected with sUAS, and their association with mesoscale atmospheric processes.

Confidence intervals for a product of proportions: application to importance values

Importance values (proportional use × proportional availability of a resource) add a useful component to studies of resource selection. The merit of the metric is that it identifies commonly available resources that are nonetheless critical components in the ecology of an organism. Such resources would not be identified as being selected in comparisons of use relative to availability (e.g., use/availability). Importance values have received limited use in the ecological literature because they have been descriptive in nature and lack a framework for hypothesis testing. In this paper we present a simple asymptotic one- and two-tailed method for calculating confidence intervals for the product of proportions. The approach is easily extended to importance values. Results from simulations indicate that our method is effective across a wide range of sample sizes, but performs poorly when designs are unbalanced. As a result, we recommend rarifying availability data in matched-case designs for determining resour...

Antler and Body Size in Black-Tailed Deer: An Analysis of Cohort Effects

For long-lived species, environmental factors experienced early in life can have lasting effects persisting into adulthood. Large herbivores can be susceptible to cohort-wide declines in fitness as a result of decreases in forage availability, because of extrinsic factors, including extreme climate or high population densities. To examine effects of cohort-specific extrinsic factors on size of adults, we performed a retrospective analysis on harvest data of 450 male black-tailed deer (Odocoileus hemionus columbianus) over 19 years in central California, USA. We determined that population density of females had a more dominant effect than did precipitation on body size of males. Harvest of female deer resulted in increases in the overall size of males, even though a 6-year drought occurred during that treatment period. Body size was most influenced by female population density early in life, while antler size was highly affected by both weather early in life and the year directly before harvest. This study provides insights that improve our understanding of the role of cohort effects in body and antler size by cervids; and, in particular, that reduction in female population density can have a profound effect on the body and antler size of male deer.

Model Selection and Regression t-Statistics

ABSTRACT It is shown that dropping quantitative variables from a linear regression, based on t-statistics, is mathematically equivalent to dropping variables based on commonly used information criteria.