Class label autoencoder with structure refinement for zero-shot learning

Abstract

Abstract Existing zero-shot learning (ZSL) methods usually learn a projection function between a feature space and a semantic embedding space(text or attribute space) for recognizing unseen classes. However, the projection function is difficult to generalize the relationship description between the feature space and multi-semantic embedding spaces(all kinds of class names are embedded as different vectors in terms of the various semantic interpretation, for example attributes, word vectors, global vectors and hierarchical embedding), which have the diversity characteristic for detailing the different semantic information of the same class. To deal with this issue, we present a novel method to ZSL based on learning class label autoencoder with structure refinement(CLASR). CLASR can not only build a scalable framework for adapting to multi-semantic embedding spaces, but also utilize the encoder-decoder paradigm for constraining the bidirectional projection between the feature space and the class label space. Moreover, CLASR can focus on the fusion model between the relationship of feature classes (visual feature classes structure) and the relevance of the semantic classes (semantic classes structure) for improving zero-shot classification by structure refinement. The CLASR solution can provide both unseen class labels and the relation of the different classes structure(visual feature and semantic classes structure) that can encode the intrinsic structure of classes. Extensive experiments demonstrate the CLASR outperforms the state-of-art methods on four benchmark datasets, which are AwA, CUB, Dogs and ImNet-2.