Simon Ferrier

Evaluating the predictive performance of habitat models developed using logistic regression

Abstract The use of statistical models to predict the likely occurrence or distribution of species is becoming an increasingly important tool in conservation planning and wildlife management. Evaluating the predictive performance of models using independent data is a vital step in model development. Such evaluation assists in determining the suitability of a model for specific applications, facilitates comparative assessment of competing models and modelling techniques, and identifies aspects of a model most in need of improvement. The predictive performance of habitat models developed using logistic regression needs to be evaluated in terms of two components: reliability or calibration (the agreement between predicted probabilities of occurrence and observed proportions of sites occupied), and discrimination capacity (the ability of a model to correctly distinguish between occupied and unoccupied sites). Lack of reliability can be attributed to two systematic sources, calibration bias and spread. Techniques are described for evaluating both of these sources of error. The discrimination capacity of logistic regression models is often measured by cross-classifying observations and predictions in a two-by-two table, and calculating indices of classification performance. However, this approach relies on the essentially arbitrary choice of a threshold probability to determine whether or not a site is predicted to be occupied. An alternative approach is described which measures discrimination capacity in terms of the area under a relative operating characteristic (ROC) curve relating relative proportions of correctly and incorrectly classified predictions over a wide and continuous range of threshold levels. Wider application of the techniques promoted in this paper could greatly improve understanding of the usefulness, and potential limitations, of habitat models developed for use in conservation planning and wildlife management.

Extended statistical approaches to modelling spatial pattern in biodiversity in northeast New South Wales. II. Community-level modelling

Statistical modelling of biological survey data in relation to remotely mapped environmental variables is a powerful technique for making more effective use of sparse data in regional conservation planning. Application of such modelling to planning in the northeast New South Wales (NSW) region of Australia represents one of the most extensive and longest running case studies of this approach anywhere in the world. Since the early 1980s, statistical modelling has been used to extrapolate distributions of over 2300 species of plants and animals, and a wide variety of higher-level communities and assemblages. These modelled distributions have played a pivotal role in a series of major land-use planning processes, culminating in extensive additions to the regions protected area system. This paper provides an overview of the analytical methodology used to model distributions of individual species in northeast NSW, including approaches to: (1) developing a basic integrated statistical and geographical information system (GIS) framework to facilitate automated fitting and extrapolation of species models; (2) extending this basic approach to incorporate consideration of spatial autocorrelation, land-cover mapping and expert knowledge; and (3) evaluating the performance of species modelling, both in terms of predictive accuracy and in terms of the effectiveness with which such models function as general surrogates for biodiversity.

An evaluation of alternative algorithms for fitting species distribution models using logistic regression

Logistic regression is being used increasingly to develop regional-scale predictive models of species distributions for use in regional conservation planning. These models are usually developed using automated stepwise procedures to select the explanatory variables to include in each model and to fit the functions relating each of these variables to the probability of species occurrence. Available procedures for fitting logistic regression models differ in terms of a number of factors, including the basic modelling technique employed (generalised linear or generalised additive modelling), the strategy used to select explanatory variables and to determine the complexity of fitted functions, and the approach used to correct for multiple testing. This study evaluates the effect that each of these factors has on the predictive accuracy of fitted models, using fauna and flora survey data from north-east New South Wales. The results suggest that predictive accuracy is maximised by employing variable selection procedures that stringently guard against the inclusion of extraneous variables in a model, such as forwards selection with a 5% significance level and removal of insignificant variables at each stage of the selection process. Models fitted using generalised additive modelling were more accurate than those derived using generalised linear modelling. The best approach to controlling the complexity of fitted models was less clear, as it tended to vary between the biological groups examined. Small reptile species were best modelled by complex relationships (3 or 4 df), and vascular plants and diurnal birds by simple relationships (1 or 2 df). Correction for multiple testing using the Bonferroni correction factor did not improve the accuracy of models.

Using abiotic data for conservation assessments over extensive regions: quantitative methods applied across New South Wales, Australia

New South Wales (NSW) can be regarded as one of the more “data-rich” parts of the world but its detailed biological data sets, like others elsewhere, are localised. These data are therefore not useable over large geographical areas for consistent reviews of established protected areas or future conservation priorities. In this sense, the constraints of data are similar to those in other parts of the world, including global biodiversity hotspots. We describe here the development of a new classification of landscapes at a scale of 1:250,000 across the whole 802,000 km2 of NSW. The classification is derived mainly from abiotic data and, in conjunction with new data on native vegetation cover, has allowed the first quantitative State-wide review of protected areas and future priorities at a scale approaching that of decisions about land use. We also describe methods for measuring biases in the coverage of reserves in relation to land use potential, mapping numerical conservation priorities across extensive areas, and producing quantitative profiles of priorities for the remaining native vegetation on private land relative to that on other tenures. The same or similar approaches to developing the landscape classification and analysing biases and priorities are feasible for many other jurisdictions or natural regions. We found that most of the 1486 landscapes in NSW are poorly reserved relative to an indicative conservation target of 15% of the total area of each (the baseline target in recent national planning for forest reserves). In the eastern 60% of the State, gaps in the reserve system are related to the concentration of reserves on land with high ruggedness and low potential for intensive land use. We measured the relative priority of landscapes to indicate the urgency of conservation action to prevent conservation targets being compromised (or further compromised) by clearing of native vegetation. Mapping of priorities shows large differences within and between natural regions and land tenures. More than 9% of private land is occupied by high-priority native vegetation and, across the whole State, about 85% of high-priority vegetation occurs on private land.

The practical value of modelling relative abundance of species for regional conservation planning: a case study

Abstract Statistical modelling of species presence/absence data in relation to mapped environmental predictors has been widely used to predict distributions of species for use in regional conservation planning. This paper evaluates the extent to which predictive mapping of habitat suitability might be refined by modelling relative abundance or density of a species instead of presence/absence. We use data collected at field survey sites in north-east New South Wales to develop models predicting the abundance of vascular plant and vertebrate fauna species as a function of regional-scale environmental variables. The predictive accuracy of these models is then evaluated using survey data collected at independent evaluation sites. A number of ‘direct’ abundance modelling techniques were evaluated including generalised linear and generalised additive Poisson regression, and zero-inflated negative binomial regression. We also evaluated the performance of predicted probability of occurrence generated by logistic regression modelling as an ‘indirect’ index of abundance. Both the direct and indirect modelling techniques generally failed to provide consistently reliable predictions of abundance. Reasonably accurate models were produced for only 12 of the 44 species evaluated. A further key finding was that, for all 12 of these species, predictions from direct abundance models performed no better as a relative index of abundance than predicted probabilities of occurrence generated by logistic regression modelling. Implications of these results for the use of predictive modelling in regional conservation planning are discussed.

Correction to 'Controlled comparison of species- and community-level models across novel climates and communities'.

[ Proc. R. Soc. B 283 , 20152817. (16 March 2016; Published online 9 March 2016) ([doi:10.1098/rspb.2015.2817][2])][2] One of the six climate variables used to fit the models was listed incorrectly in the Environmental variables section under Material and methods [[1][2]]. Mean yearly