Cortical representation variability aligns with in-class variances and can help one-shot learning

Abstract

Learning invariance across a set of transformations is an important step in mapping high-dimensional inputs to a limited number of classes. After understanding the set of\\ invariances, can a new class be learned from one element? We propose a representation which can facilitate such learning: if the variability in representing individual elements across trials aligns with the variability among different elements in a class, then class boundaries learned from the variable representations of one element should be representative of the entire class. In this study, we test whether such a representation occurs in mouse visual systems. We use Neuropixels probes recording single unit activity in mice observing 200 repeats of natural movies taken from a set of 9 continuous clips. We observe that the trial-by-trial variability in the representation of individual frames is well aligned to the variability in representation of multiple frames from the same clip, but not well aligned to the variability among frames from different clips. Thus, the variable representations of images in the mouse cortex can be efficiently used to classify images into their clips. We compare these representations to those in artificial neural networks. We find that, when introducing noise in networks trained for classification (both feed-forward and recurrent networks), the variability in the representation of elements aligns with the in-class variance. The networks which best reproduce the in-vivo observed directions of variability were those trained on a hierarchical classification task. Taken together, these results point to a solution which the cortex can use for one-shot learning of a class: by using noise as a mechanism for generalization. This is a potential computational explanation for the high level of noise observed in the cortex.