Multi-Level Ensemble Network for Scene Recognition

Abstract

Scene recognition is an important branch of computer vision and a common task for deep learning. As is known to all, different scenes are supported by different “key objects”. Therefore, the neural network used for the scene recognition task needs to extract the features of these key objects in the scene, sometimes even has to integrate the positional relation between objects to determine the class to which the scene belongs. Under some circumstances, key objects in the scenes are very small and the features of them become extremely inconspicuous or even disappear in the deep layers of the network. Such kind of phenomenon is called “small object-supported scenes”. In this paper, Multi-Level Ensemble Network (MLEN), a convolutional neural network, has been proposed, to improve the recognition accuracy of these “small object-supported scenes”. Features from multiple levels of the net are used to make separate predictions. Then ensemble learning is performed within the net to make the final prediction. Apart from all this, “Feature Transfer Path” is added and feature fusion methods are adopted to make full use of low-level and high-level features. Moreover, a class-weight loss function for the problem of non-uniform class distribution has been designed. This function can help further improve accuracy in most scene recognition datasets. The experiments involve the Urban Management Case (UMC) dataset collated from two smart urban management system databases by ourselves, and the Places-mini dataset, which is a subset of the well-known Places dataset [36]. The results show that our Multi-Level Ensemble Network achieves much higher accuracy than the state-of-the-art scene recognition networks on both datasets.