Fabian H. Sinz

Trading convexity for scalability

Convex learning algorithms, such as Support Vector Machines (SVMs), are often seen as highly desirable because they offer strong practical properties and are amenable to theoretical analysis. However, in this work we show how non-convexity can provide scalability advantages over convexity. We show how concave-convex programming can be applied to produce (i) faster SVMs where training errors are no longer support vectors, and (ii) much faster Transductive SVMs.

Principles of connectivity among morphologically defined cell types in adult neocortex

A census of neocortical neurons Despite the importance of the brains neocortex, we still do not completely understand the diversity and functional connections of its cell types. Jiang et al. recorded, labeled, and classified over 1200 interneurons and more than 400 pyramidal neurons in the mature mouse visual cortex. Fifteen major classes of interneurons fell into three types: some connect to all neurons, some connect to other interneurons, and some form synapses with pyramidal neurons. Science, this issue p. 10.1126/science.aac9462 The connections between more than 10,000 pairs of individually classified neurons in the visual cortex of adult mice are mapped. INTRODUCTION The intricate microcircuitry of the cerebral cortex is thought to be a critical substrate from which arise the impressive capabilities of the mammalian brain. Until now, our knowledge of the stereotypical connectivity in neocortical microcircuits has been pieced together from individual studies of the connectivity between small numbers of neuronal cell types. Here, we provide unbiased, large-scale profiling of neuronal cell types and connections to reveal the essential building blocks of the cortex and the principles governing their assembly into cortical circuits. Using advanced techniques for tissue slicing, multiple simultaneous whole-cell recording, and morphological reconstruction, we are able to provide a comprehensive view of the connectivity between diverse types of neurons, particularly among types of γ-aminobutyric acid–releasing (GABAergic) interneurons, in the adult animal. RATIONALE We took advantage of a method for preparing high-quality slices of adult tissue and combined this technique with octuple simultaneous, whole-cell recordings followed by an improved staining method that allowed detailed recovery of axonal and dendritic arbor morphology. These data allowed us to perform a census of morphologically and electrophysiologically defined neuronal types (primarily GABAergic interneurons) in neocortical layers 1, 2/3, and 5 (L1, L23, and L5, respectively) and to observe their connectivity patterns in adult animals. RESULTS Our large-scale, comprehensive profiling of neocortical neurons differentiated 15 major types of interneurons, in addition to two lamina-defined types of pyramidal neurons (L23 and L5). Cortical interneurons comprise two types in L1 (eNGC and SBC-like), seven in L23 (L23MC, L23NGC, BTC, BPC, DBC, L23BC, and ChC), and six in L5 (L5MC, L5NGC, L5BC, SC, HEC, and DC) (see the figure). Each type has stereotypical electrophysiological properties and morphological features and can be differentiated from all others by cell type–specific axonal geometry and axonal projection patterns. Importantly, each type of neuron has its own characteristic input-output connectivity profile, connecting with other constituent neuronal types with varying degrees of specificity in postsynaptic targets, laminar location, and synaptic characteristics. Despite specific connection patterns for each cell type, we found that a small number of simple connectivity motifs are repeated across layers and cell types defining a canonical cortical microcircuit. CONCLUSION Our comprehensive profiling of neuronal cell types and connections in adult neocortex provides the most complete wiring diagram of neocortical microcircuits to date. Compared with current genetic labels for cell class, which paint the cortex in broad strokes, our analysis of morphological and electrophysiological properties revealed new cell classes and allowed us to derive a small number of simple connectivity rules that were repeated across layers and cell types. This detailed blueprint of cortical wiring should aid efforts to identify specific circuit abnormalities in animal models of brain disease and may eventually provide a path toward the development of comprehensive circuit-based, cell type–specific interventions. Connectivity among morphologically defined cell types in adult neocortex. (A) Simultaneous octuple whole-cell recording to study connectivity followed by morphological reconstruction. (B) Synaptic connectivity between morphologically distinct types of neurons, including pyramidal (P) neurons

Population code in mouse V1 facilitates readout of natural scenes through increased sparseness

Neural codes are believed to have adapted to the statistical properties of the natural environment. However, the principles that govern the organization of ensemble activity in the visual cortex during natural visual input are unknown. We recorded populations of up to 500 neurons in the mouse primary visual cortex and characterized the structure of their activity, comparing responses to natural movies with those to control stimuli. We found that higher order correlations in natural scenes induced a sparser code, in which information is encoded by reliable activation of a smaller set of neurons and can be read out more easily. This computationally advantageous encoding for natural scenes was state-dependent and apparent only in anesthetized and active awake animals, but not during quiet wakefulness. Our results argue for a functional benefit of sparsification that could be a general principle governing the structure of the population activity throughout cortical microcircuits.

Natural Image Coding in V1: How Much Use Is Orientation Selectivity?

Orientation selectivity is the most striking feature of simple cell coding in V1 that has been shown to emerge from the reduction of higher-order correlations in natural images in a large variety of statistical image models. The most parsimonious one among these models is linear Independent Component Analysis (ICA), whereas second-order decorrelation transformations such as Principal Component Analysis (PCA) do not yield oriented filters. Because of this finding, it has been suggested that the emergence of orientation selectivity may be explained by higher-order redundancy reduction. To assess the tenability of this hypothesis, it is an important empirical question how much more redundancy can be removed with ICA in comparison to PCA or other second-order decorrelation methods. Although some previous studies have concluded that the amount of higher-order correlation in natural images is generally insignificant, other studies reported an extra gain for ICA of more than 100%. A consistent conclusion about the role of higher-order correlations in natural images can be reached only by the development of reliable quantitative evaluation methods. Here, we present a very careful and comprehensive analysis using three evaluation criteria related to redundancy reduction: In addition to the multi-information and the average log-loss, we compute complete rate–distortion curves for ICA in comparison with PCA. Without exception, we find that the advantage of the ICA filters is small. At the same time, we show that a simple spherically symmetric distribution with only two parameters can fit the data significantly better than the probabilistic model underlying ICA. This finding suggests that, although the amount of higher-order correlation in natural images can in fact be significant, the feature of orientation selectivity does not yield a large contribution to redundancy reduction within the linear filter bank models of V1 simple cells.

Learning Depth from Stereo

We compare two approaches to the problem of estimating the depth of a point in space from observing its image position in two different cameras: 1. The classical photogrammetric approach explicitly models the two cameras and estimates their intrinsic and extrinsic parameters using a tedious calibration procedure; 2. A generic machine learning approach where the mapping from image to spatial coordinates is directly approximated by a Gaussian Process regression. Our results show that the generic learning approach, in addition to simplifying the procedure of calibration, can lead to higher depth accuracies than classical calibration although no specific domain knowledge is used.

Evaluating predictive uncertainty challenge

This Chapter presents the PASCAL Evaluating Predictive Uncertainty Challenge, introduces the contributed Chapters by the participants who obtained outstanding results, and provides a discussion with some lessons to be learnt. The Challenge was set up to evaluate the ability of Machine Learning algorithms to provide good “probabilistic predictions”, rather than just the usual “point predictions” with no measure of uncertainty, in regression and classification problems. Parti-cipants had to compete on a number of regression and classification tasks, and were evaluated by both traditional losses that only take into account point predictions and losses we proposed that evaluate the quality of the probabilistic predictions.

Lower bounds on the redundancy of natural images

The light intensities of natural images exhibit a high degree of redundancy. Knowing the exact amount of their statistical dependencies is important for biological vision as well as compression and coding applications but estimating the total amount of redundancy, the multi-information, is intrinsically hard. The common approach is to estimate the multi-information for patches of increasing sizes and divide by the number of pixels. Here, we show that the limiting value of this sequence--the multi-information rate--can be better estimated by using another limiting process based on measuring the mutual information between a pixel and a causal neighborhood of increasing size around it. Although in principle this method has been known for decades, its superiority for estimating the multi-information rate of natural images has not been fully exploited yet. Either method provides a lower bound on the multi-information rate, but the mutual information based sequence converges much faster to the multi-information rate than the conventional method does. Using this fact, we provide improved estimates of the multi-information rate of natural images and a better understanding of its underlying spatial structure.

Modelling Spikes with Mixtures of Factor Analysers

Identifying the action potentials of individual neurons from extracellular recordings, known as spike sorting, is a challenging problem. We consider the spike sorting problem using a generative model, mixtures of factor analysers, which concurrently performs clustering and feature extraction. The most important advantage of this method is that it quantifies the certainty with which the spikes are classified. This can be used as a means for evaluating the quality of clustering and therefore spike isolation. Using this method, nearly simultaneously occurring spikes can also be modelled which is a hard task for many of the spike sorting methods. Furthermore, modelling the data with a generative model allows us to generate simulated data.

DataJoint: Managing Big Scientific Data Using Matlab or Python

The rise of big data in modern research poses serious challenges for data management: Large and intricate datasets from diverse instrumentation must be precisely aligned, annotated, and processed in a variety of ways to extract new insights. While high levels of data integrity are expected, research teams have diverse backgrounds, are geographically dispersed, and rarely possess a primary interest in data science. Here we describe DataJoint, an open-source toolbox designed for manipulating and processing scientific data under the relational data model. Designed for scientists who need a flexible and expressive database language with few basic concepts and operations, DataJoint facilitates multiuser access, efficient queries, and distributed computing. With implementations in both MATLAB and Python, DataJoint is not limited to particular file formats, acquisition systems, or data modalities and can be quickly adapted to new experimental designs. DataJoint and related resources are available at http://datajoint.github.com.

Response to Comment on “Principles of connectivity among morphologically defined cell types in adult neocortex”

The critique of Barth et al. centers on three points: (i) the completeness of our study is overstated; (ii) the connectivity matrix we describe is biased by technical limitations of our brain-slicing and multipatching methods; and (iii) our cell classification scheme is arbitrary and we have simply renamed previously identified interneuron types. We address these criticisms in our Response.