Lopamudra Mukherjee

Half-integrality based algorithms for cosegmentation of images

We study the cosegmentation problem where the objective is to segment the same object (i.e., region) from a pair of images. The segmentation for each image can be cast using a partitioning/segmentation function with an additional constraint that seeks to make the histograms of the segmented regions (based on intensity and texture features) similar. Using Markov random field (MRF) energy terms for the simultaneous segmentation of the images together with histogram consistency requirements using the squared L2 (rather than L1) distance, after linearization and adjustments, yields an optimization model with some interesting combinatorial properties. We discuss these properties which are closely related to certain relaxation strategies recently introduced in computer vision. Finally, we show experimental results of the proposed approach.

Scale invariant cosegmentation for image groups

Our primary interest is in generalizing the problem of Cosegmentation to a large group of images, that is, concurrent segmentation of common foreground region(s) from multiple images. We further wish for our algorithm to offer scale invariance (foregrounds may have arbitrary sizes in different images) and the running time to increase (no more than) near linearly in the number of images in the set. What makes this setting particularly challenging is that even if we ignore the scale invariance desiderata, the Cosegmentation problem, as formalized in many recent papers (except [1]), is already hard to solve optimally in the two image case. A straightforward extension of such models to multiple images leads to loose relaxations; and unless we impose a distributional assumption on the appearance model, existing mechanisms for image-pair-wise measurement of foreground appearance variations lead to significantly large problem sizes (even for moderate number of images). This paper presents a surprisingly easy to implement algorithm which performs well, and satisfies all requirements listed above (scale invariance, low computational requirements, and viability for the multiple image setting). We present qualitative and technical analysis of the properties of this framework.

Gaze-enabled egocentric video summarization via constrained submodular maximization

With the proliferation of wearable cameras, the number of videos of users documenting their personal lives using such devices is rapidly increasing. Since such videos may span hours, there is an important need for mechanisms that represent the information content in a compact form (i.e., shorter videos which are more easily browsable/sharable). Motivated by these applications, this paper focuses on the problem of egocentric video summarization. Such videos are usually continuous with significant camera shake and other quality issues. Because of these reasons, there is growing consensus that direct application of standard video summarization tools to such data yields unsatisfactory performance. In this paper, we demonstrate that using gaze tracking information (such as fixation and saccade) significantly helps the summarization task. It allows meaningful comparison of different image frames and enables deriving personalized summaries (gaze provides a sense of the camera wearers intent). We formulate a summarization model which captures common-sense properties of a good summary, and show that it can be solved as a submodular function maximization with partition matroid constraints, opening the door to a rich body of work from combinatorial optimization. We evaluate our approach on a new gaze-enabled egocentric video dataset (over 15 hours), which will be a valuable standalone resource.

Cell type specific chromosome territory organization in the interphase nucleus of normal and cancer cells.

Numerous studies indicate that the genome of higher eukaryotes is organized into distinct chromosome territories and that the 3‐D arrangement of these territories may be closely connected to genomic function and the global regulation of gene expression. Despite this progress, the degree of non‐random arrangement remains unclear and no overall model has been proposed for chromosome territory associations. To address this issue, a re‐FISH approach was combined with computational analysis to analysis the pair‐wise associations for six pairs of human chromosomes (chr #1, 4, 11, 12, 16, 18) in the G0 state of normal human WI38 lung fibroblast and MCF10A epithelial breast cells. Similar levels of associations were found in WI38 and MCF10A for several of the chromosomes whereas others showed striking differences. A novel computational geometric approach, the generalized median graph (GMG), revealed a preferred probabilistic arrangement distinct for each cell line. Statistical analysis demonstrated that ∼50% of the associations depicted in the GMG models are present in each individual nucleus. A nearly twofold increase of chromosome 4/16 associations in a malignant breast cancer cell line (MCFCA1a) compared to the related normal epithelial cell line (MCF10A) further demonstrates cancer related changes in chromosome arrangements. Our findings of highly preferred chromosome association profiles that are cell type specific and undergo alterations in cancer cells, lead us to propose a probabilistic chromosome code whereby the 3‐D association profile of chromosomes contributes to the functional landscape of the cell nucleus, the global regulation of gene expression and the epigenetic state of chromatin. J. Cell. Physiol. 221: 130–138, 2009.

Ensemble clustering using semidefinite programming with applications

In this paper, we study the ensemble clustering problem, where the input is in the form of multiple clustering solutions. The goal of ensemble clustering algorithms is to aggregate the solutions into one solution that maximizes the agreement in the input ensemble. We obtain several new results for this problem. Specifically, we show that the notion of agreement under such circumstances can be better captured using a 2D string encoding rather than a voting strategy, which is common among existing approaches. Our optimization proceeds by first constructing a non-linear objective function which is then transformed into a 0-1 Semidefinite program (SDP) using novel convexification techniques. This model can be subsequently relaxed to a polynomial time solvable SDP. In addition to the theoretical contributions, our experimental results on standard machine learning and synthetic datasets show that this approach leads to improvements not only in terms of the proposed agreement measure but also the existing agreement measures based on voting strategies. In addition, we identify several new application scenarios for this problem. These include combining multiple image segmentations and generating tissue maps from multiple-channel Diffusion Tensor brain images to identify the underlying structure of the brain.

Identifying functional neighborhoods within the cell nucleus: Proximity analysis of early S-phase replicating chromatin domains to sites of transcription, RNA polymerase II, HP1γ, matrin 3 and SAF-A

Higher order chromatin organization in concert with epigenetic regulation is a key process that determines gene expression at the global level. The organization of dynamic chromatin domains and their associated protein factors is intertwined with nuclear function to create higher levels of functional zones within the cell nucleus. As a step towards elucidating the organization and dynamics of these functional zones, we have investigated the spatial proximities among a constellation of functionally related sites that are found within euchromatic regions of the cell nucleus including: HP1γ, nascent transcript sites (TS), active DNA replicating sites in early S‐phase (PCNA) and RNA polymerase II sites. We report close associations among these different sites with proximity values specific for each combination. Analysis of matrin 3 and SAF‐A sites demonstrates that these nuclear matrix proteins are highly proximal with the functionally related sites as well as to each other and display closely aligned and overlapping regions following application of the minimal spanning tree (MST) algorithm to visualize higher order network‐like patterns. Our findings suggest that multiple factors within the nuclear microenvironment collectively form higher order combinatorial arrays of function. We propose a model for the organization of these functional neighborhoods which takes into account the proximity values of the individual sites and their spatial organization within the nuclear architecture. J. Cell. Biochem. 105: 391–403, 2008.

Analyzing the subspace structure of related images: concurrent segmentation of image sets

We develop new algorithms to analyze and exploit the joint subspace structure of a set of related images to facilitate the process of concurrent segmentation of a large set of images. Most existing approaches for this problem are either limited to extracting a single similar object across the given image set or do not scale well to a large number of images containing multiple objects varying at different scales. One of the goals of this paper is to show that various desirable properties of such an algorithm (ability to handle multiple images with multiple objects showing arbitary scale variations) can be cast elegantly using simple constructs from linear algebra: this significantly extends the operating range of such methods. While intuitive, this formulation leads to a hard optimization problem where one must perform the image segmentation task together with appropriate constraints which enforce desired algebraic regularity (e.g., common subspace structure). We propose efficient iterative algorithms (with small computational requirements) whose key steps reduce to objective functions solvable by max-flow and/or nearly closed form identities. We study the qualitative, theoretical, and empirical properties of the method, and present results on benchmark datasets.

A probabilistic model for the arrangement of a subset of human chromosome territories in WI38 Human fibroblasts

There is growing evidence that chromosome territories have a probabilistic non‐random arrangement within the cell nucleus of mammalian cells. Other than their radial positioning, however, our knowledge of the degree and specificity of chromosome territory associations is predominantly limited to studies of pair‐wise associations. In this study we have investigated the association profiles of eight human chromosome pairs (numbers 1, 2, 3, 4, 6, 7, 8, 9) in the cell nuclei of G0‐arrested WI38 diploid lung fibroblasts. Associations between heterologous chromosome combinations ranged from 52% to 78% while the homologous chromosome pairs had much lower levels of association (3–25%). A geometric computational method termed the Generalized Median Graph enabled identification of the most probable arrangement of these eight chromosome pairs. Approximately 41% of the predicted associations are present in any given nucleus. The association levels of several chromosome pairs were very similar in a series of lung fibroblast cell lines but strikingly different in skin and colon derived fibroblast cells. We conclude that a large subset of human chromosomes has a preferred probabilistic arrangement in WI38 cells and that the resulting chromosomal associations show tissue origin specificity. J. Cell. Physiol. 221: 120–129, 2009.

Brachytherapy Seed Localization Using Geometric and Linear Programming Techniques

We propose an optimization algorithm to solve the brachytherapy seed localization problem in prostate brachytherapy. Our algorithm is based on novel geometric approaches to exploit the special structure of the problem and relies on a number of key observations which help us formulate the optimization problem as a minimization integer program (IP). Our IP model precisely defines the feasibility polyhedron for this problem using a polynomial number of half-spaces; the solution to its corresponding linear program is rounded to yield an integral solution to our task of determining correspondences between seeds in multiple projection images. The algorithm is efficient in theory as well as in practice and performs well on simulation data (~98% accuracy) and real X-ray images (~95% accuracy). We present in detail the underlying ideas and an extensive set of performance evaluations based on our implementation.

Spectral Clustering with a Convex Regularizer on Millions of Images

This paper focuses on efficient algorithms for single and multi-view spectral clustering with a convex regularization term for very large scale image datasets. In computer vision applications, multiple views denote distinct image-derived feature representations that inform the clustering. Separately, the regularization encodes high level advice such as tags or user interaction in identifying similar objects across examples. Depending on the specific task, schemes to exploit such information may lead to a smooth or non-smooth regularization function. We present stochastic gradient descent methods for optimizing spectral clustering objectives with such convex regularizers for datasets with up to a hundred million examples. We prove that under mild conditions the local convergence rate is \(O(1/\sqrt{T})\) where T is the number of iterations; further, our analysis shows that the convergence improves linearly by increasing the number of threads. We give extensive experimental results on a range of vision datasets demonstrating the algorithm’s empirical behavior.