Parametric q-Gaussian distributed stochastic neighbor embedding with Convolutional Neural Network

Abstract

For dimensionality reduction, t-distributed stochastic neighbor embedding (t-SNE) is famous. This technique represents the similarity between the pair of the samples in the high-dimensional space as Gaussian distribution. Then the similarity between the pair of the samples in the low-dimensional embedding space is also represented by using t-distribution to obtain the feature vectors in the embedding space from the high-dimensional data. The authors proposed q-Gaussian distributed stochastic neighbor embedding (q-SNE) as an extension of the t-SNE. The q-Gaussian distribution can express many distributions by setting hyperparameter q$= and includes the Gaussian distribution and the t-distribution as the special cases with hyperparameter q close to 1.0 and q = 2.0. However, these methods are applicable for a given data set and it is not possible to map new samples into the embedded space. To address this problem, the parametric t-SNE is proposed to construct the non-linear mapping by using a feed-forward neural network. In this paper, we propose a novel technique called parametric q-SNE with Convolutional Neural Network without pre-training. On MNIST, FashionMNIST, and COIL-20, the effectiveness of the parametric q -SNE is shown by using the visualization on 2-dimensional mapping, and the classification by using k nearest neighbors (k-NN) on the embedded space.