Model performance analysis for landslide susceptibility in cold regions using accuracy rate and fluctuation characteristics

Abstract

Considering the increasing number of landslides due to permafrost degradation, this paper reports a performance evaluation of three classical landslide susceptibility models applied to cold regions. A landslide inventory was first constructed through a historical survey and image interpretation. Ten causative factors of landslides were then chosen based on the available data and the local environment. Multicollinearity diagnosis and factor effectiveness test were employed to perform a factor analysis. Subsequently, three evaluation models based on the frequency ratio (FR), logistic regression (LR), and artificial neural network (ANN) were established. These models were applied to obtain landslide susceptibility maps, which were then analyzed and compared. The model performance was evaluated in terms of the accuracy rate and fluctuation characteristics. The results showed no multicollinearity issue between the factors employed. The annual temperature difference and frozen depth are two indispensable factors when assessing landslide susceptibility in cold regions. A comparison between the susceptibility maps generated using the three models showed that the FR model-generated map is most in line with the principle of disaster zoning and has the highest degree of conformity with actual landslide points, followed by the maps generated using the LR and ANN models. An accuracy analysis showed that the ANN model yields the highest AUC value in the training and test states, 0.957 and 0.863, respectively; however, these values were not optimal given the fluctuation. Moreover, the fluctuation in the non-landslide data was greater than that in the landslide data. The fluctuation results revealed the drawback of the AUC value in the analysis of the model performance. In other words, the non-landslide error often covers up the landslide error. This study provides a scientific guidance for evaluating the model performance and for assessing landslide disasters in cold regions.