Mehmet Emre Sargin

Audiovisual celebrity recognition in unconstrained web videos

The number of video clips available online is growing at a tremendous pace. Conventionally, user-supplied metadata text, such as the title of the video and a set of keywords, has been the only source of indexing information for user-uploaded videos. Automated extraction of video content for unconstrained and large scale video databases is a challenging and yet unsolved problem. In this paper, we present an audiovisual celebrity recognition system towards automatic tagging of unconstrained web videos. Prior work on audiovisual person recognition relied on the fact that the person in the video is speaking and the features extracted from audio and visual domain are associated with each other throughout the video. However, this assumption is not valid on unconstrained web videos. Proposed method finds the audiovisual mapping and hence improve upon the association assumption. Considering the scale of the application, all pieces of the system are trained automatically without any human supervision. We present the results on 26,000 videos and show the effectiveness of the method per-celebrity basis.

Tau isoform-specific modulation of kinesin-driven microtubule gliding rates and trajectories as determined with tau-stabilized microtubules.

We have utilized tau‐assembled and tau‐stabilized microtubules (MTs), in the absence of taxol, to investigate the effects of tau isoforms with three and four MT binding repeats upon kinesin‐driven MT gliding. MTs were assembled in the presence of either 3‐repeat tau (3R tau) or 4‐repeat tau (4R tau) at tau:tubulin dimer molar ratios that approximate those found in neurons. MTs assembled with 3R tau glided at 31.1 μm/min versus 25.8 μm/min for 4R tau, a statistically significant 17% difference. Importantly, the gliding rates for either isoform did not change over a fourfold range of tau concentrations. Further, tau‐assembled MTs underwent minimal dynamic instability behavior while gliding and moved with linear trajectories. In contrast, MTs assembled with taxol in the absence of tau displayed curved gliding trajectories. Interestingly, addition of 4R tau to taxol‐stabilized MTs restored linear gliding, while addition of 3R tau did not. The data are consistent with the ideas that (i) 3R and 4R tau‐assembled MTs possess at least some isoform‐specific features that impact upon kinesin translocation, (ii) tau‐assembled MTs possess different structural features than do taxol‐assembled MTs, and (iii) some features of tau‐assembled MTs can be masked by prior assembly by taxol. The differences in kinesin‐driven gliding between 3R and 4R tau suggest important features of tau function related to the normal shift in tau isoform composition that occurs during neural development as well as in neurodegeneration caused by altered expression ratios of otherwise normal tau isoforms.

Probabilistic occlusion boundary detection on spatio-temporal lattices

In this paper, we present an algorithm for occlusion boundary detection. The main contribution is a probabilistic detection framework defined on spatio-temporal lattices, which enables joint analysis of image frames. For this purpose, we introduce two complementary cost functions for creating the spatio-temporal lattice and for performing global inference of the occlusion boundaries, respectively. In addition, a novel combination of low-level occlusion features is discriminatively learnt in the detection framework. Simulations on the CMU Motion Dataset provide ample evidence that proposed algorithm outperforms the leading existing methods.

Combinatorial Tau Pseudophosphorylation MARKEDLY DIFFERENT REGULATORY EFFECTS ON MICROTUBULE ASSEMBLY AND DYNAMIC INSTABILITY THAN THE SUM OF THE INDIVIDUAL PARTS

Tau is a multiply phosphorylated protein that is essential for the development and maintenance of the nervous system. Errors in Tau action are associated with Alzheimer disease and related dementias. A huge literature has led to the widely held notion that aberrant Tau hyperphosphorylation is central to these disorders. Unfortunately, our mechanistic understanding of the functional effects of combinatorial Tau phosphorylation remains minimal. Here, we generated four singly pseudophosphorylated Tau proteins (at Thr231, Ser262, Ser396, and Ser404) and four doubly pseudophosphorylated Tau proteins using the same sites. Each Tau preparation was assayed for its abilities to promote microtubule assembly and to regulate microtubule dynamic instability in vitro. All four singly pseudophosphorylated Tau proteins exhibited loss-of-function effects. In marked contrast to the expectation that doubly pseudophosphorylated Tau would be less functional than either of its corresponding singly pseudophosphorylated forms, all of the doubly pseudophosphorylated Tau proteins possessed enhanced microtubule assembly activity and were more potent at regulating dynamic instability than their compromised singly pseudophosphorylated counterparts. Thus, the effects of multiple pseudophosphorylations were not simply the sum of the effects of the constituent single pseudophosphorylations; rather, they were generally opposite to the effects of singly pseudophosphorylated Tau. Further, despite being pseudophosphorylated at different sites, the four singly pseduophosphorylated Tau proteins often functioned similarly, as did the four doubly pseudophosphorylated proteins. These data lead us to reassess the conventional view of combinatorial phosphorylation in normal and pathological Tau action. They may also be relevant to the issue of combinatorial phosphorylation as a general regulatory mechanism.

Tracing Curvilinear Structures in Live Cell Images

Tracing of curvilinear structures is one of the fundamental tools in the quantitative analysis of biological images, for extracting information about structures such as blood vessels, neurons, microtubules, and similar entities. Due to the limitations in biological sample preparation and fluorescence imaging, typical images in live cell studies exhibit severe noise and considerable clutter. These images are manually analyzed through a laborious and approximate set of quantification tasks. In this paper, we describe a constrained optimization method for extracting curvilinear structures from live cell fluorescence images. We show that the proposed method is largely insensitive to frequent intersections, intensity variations along the curve, and generates successful traces within noisy regions. We demonstrate the results of our approach on live cell microtubule images.

Detecting highlights in sports videos: Cricket as a test case

In this paper, we propose a novel approach for detecting highlights in sports videos. The videos are temporally decomposed into a series of events based on an unsupervised event discovery and detection framework. The framework solely depends on easy-to-extract low-level visual features such as color histogram (CH) or histogram of oriented gradients (HOG), which can potentially be generalized to different sports. The unigram and bigram statistics of the detected events are then used to provide a compact representation of the video. The effectiveness of the proposed representation is demonstrated on cricket video classification: Highlight vs. Non-Highlight for individual video clips (7000 training and 7000 test instances). We achieve a low equal error rate of 12.1% using event statistics based on CH and HOG features.

Variable Length Open Contour Tracking Using a Deformable Trellis

This paper focuses on contour tracking, an important problem in computer vision, and specifically on open contours that often directly represent a curvilinear object. Compelling applications are found in the field of bioimage analysis where blood vessels, dendrites, and various other biological structures are tracked over time. General open contour tracking, and biological images in particular, pose major challenges including scene clutter with similar structures (e.g., in the cell), and time varying contour length due to natural growth and shortening phenomena, which have not been adequately answered by earlier approaches based on closed and fixed end-point contours. We propose a model-based estimation algorithm to track open contours of time-varying length, which is robust to neighborhood clutter with similar structures. The method employs a deformable trellis in conjunction with a probabilistic (hidden Markov) model to estimate contour position, deformation, growth and shortening. It generates a maximum a posteriori estimate given observations in the current frame and prior contour information from previous frames. Experimental results on synthetic and real-world data demonstrate the effectiveness and performance gains of the proposed algorithm.

Robust dynamical model for simultaneous registration and segmentation in a variational framework: A Bayesian approach

We introduce a dynamical model for simultaneous registration and segmentation in a variational framework for image sequences, where the dynamics is incorporated using a Bayesian formulation. A linear stochastic equation relating the tracked object (or a region of interest) is first derived under the assumption that the successive images in the sequence are related by a dense and possibly non-linear displacement field. This derivation allows for the use of a computationally efficient and recursive implementation of the Bayesian formulation in this framework. The contour of the tracked object returned by the dynamical model is not only close to the previously detected shape but is also consistent with the temporal statistics of the tracked object. The performance of the proposed approach is evaluated on real image sequences. It is shown that, with respect to a variety of error metrics such as F-measure, mean absolute deviation and Hausdorff distance, the proposed approach outperforms the state-of-the art approach without the dynamical model.

Deformable trellis: open contour tracking in bio-image sequences

This paper presents an open contour tracking method that employs an arc-emission hidden Markov model (HMM). The algorithm encodes the shape information of the structure in a spatially deformable trellis model that is iteratively modified to account for observations in subsequent frames. As the open contour is determined on the trellis of an HMM, a dynamic programming procedure reduces the computational complexity to linear in the length of the structure (or contour). The method was developed for tracking general curvilinear structures, and tested on subcellular image sequences, where microtubules grow, shrink and undergo lateral motion from frame to frame. Microtubule length changes are modeled by the addition of appropriate transient and absorbing states to the HMM. Our results provide experimental evidence for the proposed algorithms capability to track non-rigid curvilinear objects in challenging environments in terms of noise and clutter.

TRACING MICROTUBULES IN LIVE CELL IMAGES

Microtubule (MT) dynamics are traditionally analyzed from time lapse images by manual techniques that are laborious, approximate and often limited. Recently, computer vision techniques have been applied to the problem of automated tracking of MTs in live cell images. Aside of very low signal to noise ratios, live cell images of MTs exhibit severe clutter for accurate tracing of MT body. Moreover, intersecting and overlapping MT regions appear brighter due to additive fluorescence. In this paper, we present a MT body tracing algorithm that addresses the clutter without imposing directional constraints. We show that MT dynamics can be quantified with enhanced precision, and novel measurements that are beyond manual feasibility, can be obtained accurately. We demonstrate our results on actual images of MTs obtained by live cell fluorescence microscopy.