Anirban Chakraborty

Consistent Re-identification in a Camera Network

Most existing person re-identification methods focus on finding similarities between persons between pairs of cameras (camera pairwise re-identification) without explicitly maintaining consistency of the results across the network. This may lead to infeasible associations when results from different camera pairs are combined. In this paper, we propose a network consistent re-identification (NCR) framework, which is formulated as an optimization problem that not only maintains consistency in re-identification results across the network, but also improves the camera pairwise re-identification performance between all the individual camera pairs. This can be solved as a binary integer programing problem, leading to a globally optimal solution. We also extend the proposed approach to the more general case where all persons may not be present in every camera. Using two benchmark datasets, we validate our approach and compare against state-of-the-art methods.

Adaptive Cell Segmentation and Tracking for Volumetric Confocal Microscopy Images of a Developing Plant Meristem

Automated segmentation and tracking of cells in actively developing tissues can provide high-throughput and quantitative spatiotemporal measurements of a range of cell behaviors; cell expansion and cell-division kinetics leading to a better understanding of the underlying dynamics of morphogenesis. Here, we have studied the problem of constructing cell lineages in time-lapse volumetric image stacks obtained using Confocal Laser Scanning Microscopy (CLSM). The novel contribution of the work lies in its ability to segment and track cells in densely packed tissue, the shoot apical meristem (SAM), through the use of a close-loop, adaptive segmentation, and tracking approach. The tracking output acts as an indicator of the quality of segmentation and, in turn, the segmentation can be improved to obtain better tracking results. We construct an optimization function that minimizes the segmentation error, which is, in turn, estimated from the tracking results. This adaptive approach significantly improves both tracking and segmentation when compared to an open loop framework in which segmentation and tracking modules operate separately.

Graph-based active learning of agglomeration (GALA): a Python library to segment 2D and 3D neuroimages

The aim in high-resolution connectomics is to reconstruct complete neuronal connectivity in a tissue. Currently, the only technology capable of resolving the smallest neuronal processes is electron microscopy (EM). Thus, a common approach to network reconstruction is to perform (error-prone) automatic segmentation of EM images, followed by manual proofreading by experts to fix errors. We have developed an algorithm and software library to not only improve the accuracy of the initial automatic segmentation, but also point out the image coordinates where it is likely to have made errors. Our software, called gala (graph-based active learning of agglomeration), improves the state of the art in agglomerative image segmentation. It is implemented in Python and makes extensive use of the scientific Python stack (numpy, scipy, networkx, scikit-learn, scikit-image, and others). We present here the software architecture of the gala library, and discuss several designs that we consider would be generally useful for other segmentation packages. We also discuss the current limitations of the gala library and how we intend to address them.

Network Consistent Data Association

Existing data association techniques mostly focus on matching pairs of data-point sets and then repeating this process along space-time to achieve long term correspondences. However, in many problems such as person re-identification, a set of data-points may be observed at multiple spatio-temporal locations and/or by multiple agents in a network and simply combining the local pairwise association results between sets of data-points often leads to inconsistencies over the global space-time horizons. In this paper, we propose a Novel Network Consistent Data Association (NCDA) framework formulated as an optimization problem that not only maintains consistency in association results across the network, but also improves the pairwise data association accuracies. The proposed NCDA can be solved as a binary integer program leading to a globally optimal solution and is capable of handling the challenging data-association scenario where the number of data-points varies across different sets of instances in the network. We also present an online implementation of NCDA method that can dynamically associate new observations to already observed data-points in an iterative fashion, while maintaining network consistency. We have tested both the batch and the online NCDA in two application areas - person re-identification and spatio-temporal cell tracking and observed consistent and highly accurate data association results in all the cases.

A context-aware delayed agglomeration framework for electron microscopy segmentation.

Electron Microscopy (EM) image (or volume) segmentation has become significantly important in recent years as an instrument for connectomics. This paper proposes a novel agglomerative framework for EM segmentation. In particular, given an over-segmented image or volume, we propose a novel framework for accurately clustering regions of the same neuron. Unlike existing agglomerative methods, the proposed context-aware algorithm divides superpixels (over-segmented regions) of different biological entities into different subsets and agglomerates them separately. In addition, this paper describes a “delayed” scheme for agglomerative clustering that postpones some of the merge decisions, pertaining to newly formed bodies, in order to generate a more confident boundary prediction. We report significant improvements attained by the proposed approach in segmentation accuracy over existing standard methods on 2D and 3D datasets.

Context aware spatio-temporal cell tracking in densely packed multilayer tissues

Modern live imaging technique enables us to observe the internal part of a tissue over time by generating serial optical images containing spatio-temporal slices of hundreds of tightly packed cells. Automated tracking of plant and animal cells from such time lapse live-imaging datasets of a developing multicellular tissue is required for quantitative, high throughput analysis of cell division, migration and cell growth. In this paper, we present a novel cell tracking method that exploits the tight spatial topology of neighboring cells in a multicellular field as contextual information and combines it with physical features of individual cells for generating reliable cell lineages. The 2D image slices of multicellular tissues are modeled as a conditional random field and pairwise cell to cell similarities are obtained by estimating marginal probability distributions through loopy belief propagation on this CRF. These similarity scores are further used in a spatio-temporal graph labeling problem to obtain the optimal and feasible set of correspondences between individual cell slices across the 4D image dataset. We present results on (3D+t) confocal image stacks of Arabidopsis shoot meristem and show that the method is capable of handling many visual analysis challenges associated with such cell tracking problems, viz. poor feature quality of individual cells, low SNR in parts of images, variable number of cells across slices and cell division detection.

Adaptive geometric tessellation for 3D reconstruction of anisotropically developing cells in multilayer tissues from sparse volumetric microscopy images.

The need for quantification of cell growth patterns in a multilayer, multi-cellular tissue necessitates the development of a 3D reconstruction technique that can estimate 3D shapes and sizes of individual cells from Confocal Microscopy (CLSM) image slices. However, the current methods of 3D reconstruction using CLSM imaging require large number of image slices per cell. But, in case of Live Cell Imaging of an actively developing tissue, large depth resolution is not feasible in order to avoid damage to cells from prolonged exposure to laser radiation. In the present work, we have proposed an anisotropic Voronoi tessellation based 3D reconstruction framework for a tightly packed multilayer tissue with extreme z-sparsity (2–4 slices/cell) and wide range of cell shapes and sizes. The proposed method, named as the ‘Adaptive Quadratic Voronoi Tessellation’ (AQVT), is capable of handling both the sparsity problem and the non-uniformity in cell shapes by estimating the tessellation parameters for each cell from the sparse data-points on its boundaries. We have tested the proposed 3D reconstruction method on time-lapse CLSM image stacks of the Arabidopsis Shoot Apical Meristem (SAM) and have shown that the AQVT based reconstruction method can correctly estimate the 3D shapes of a large number of SAM cells.

Context-Aware Activity Forecasting

In this paper, we investigate the problem of forecasting future activities in continuous videos. Ability to successfully forecast activities that are yet to be observed is a very important video understanding problem, and is starting to receive attention in the computer vision literature. We propose an activity forecasting strategy that models the simultaneous and/or sequential nature of human activities on a graph and combines that with the interrelationship between static scene cues and dynamic target trajectories, termed together as the ‘activity and scene context’. The forecasting problem is then posed as an inference problem on a MRF model defined on the graph. We perform experiments on the publicly available challenging VIRAT ground dataset and obtain high forecasting accuracy for most of the activities, as evidenced by the results.

Automated registration of live imaging stacks of Arabidopsis

For actively developing tissues, a computational platform capable of automatically registering, segmenting and tracking cells is very critical to obtaining high-throughput and quantitative measurements of a range of cell behaviors, and can lead to a better understanding of the underlying dynamics of morphogenesis. In this work, we present an automated landmark-based registration method to register shoot apical meristem of Arabidopsis Thaliana images obtained through the Confocal Laser Scanning Microscopy technique. The proposed landmark-based registration method uses local graph-based approach to automatically find corresponding landmark pairs. The registration algorithm combined with an existing tracking method is tested on multiple datasets and it significantly improves the accuracy of cell lineages and division statistics.

Cell Resolution 3D Reconstruction of Developing Multilayer Tissues from Sparsely Sampled Volumetric Microscopy Images

Understanding of the growth dynamics in developmental biology is often pursued through the analysis of cell sizes and shapes obtained from CLSM based imaging at cell resolution of multi-layer tissues. This necessitates the development of robust 3D reconstruction methods using such images. However, all of the current methods of 3D reconstruction using CLSM imaging require large number of cell slices. But in the case of live cell imaging, i.e., imaging a growing tissue, such high depth resolution is not feasible in order to avoid photodynamic damage to the growing cells from prolonged exposure to laser radiation. In this work, we have addressed the problem of 3D reconstruction at cell resolution of a developing multi-layer tissue in the plant meristem when the amount of data is as limited as two to four slices per cell. This introduces significant image analysis challenges in terms of sparsity of the data, low signal-to-noise ratio, and a wide range of shapes and sizes. Motivated by the physical structure of the cells, we propose to reconstruct a cell cluster as a packing of truncated ellipsoids representing the individual cells. We test the proposed computational method on time-lapse CLSM images of Shoot Apical Meristem (SAM) cells of model plant Arabidopsis Thaliana. We show that the 3D reconstruction can lead to 3D shape models of complete cell clusters, which is an essential first step towards obtaining growth statistics for individual cells.