Variational Information Bottleneck on Few Shot Model based on Weight Imprinting for Image Classification

Abstract

Few shot learning remains an open issue in computer vision. Among several recently proposed approaches, Weight Imprinting (WI) achieves superior performance on many challenging benchmarks. The performance of the imprinted weights heavily depends on the quality of the representations generated by the encoder. However, it is not known what characteristics are required for weight imprinting. The representations learned during the pre-training phase are optimized for classification accuracy for the pre-training base classes and not necessarily suitable for the downstream, imprinted tasks. In this paper, we investigate the effect of Variational Information Bottleneck on the few shot learning with weight imprinting. Variational Information Bottleneck strongly regularizes the representation by minimizing the mutual information between input data and representation while keeping the classification accuracy for pretraining task. We demonstrate that the encoder regularized by VIB achieves significantly better performance on few-shot learning tasks with imprinting. Furthermore, we comprehensively investigate the effect of combining VIB with other regularization methods including data augmentation and auxiliary data. We confirmed that with a proper auxiliary dataset, we can achieve even better accuracy on the downstream task.