Kourosh Meshgi

An occlusion-aware particle filter tracker to handle complex and persistent occlusions

Enhanced particle filter tracker by latent occlusion flag to handle full occlusion.Handled persistent and/or complex occlusions in RGBD sequences.Developed data-driven occlusion mask to evaluate various parts of observation.Fused multiple feature from color and depth domains to gain occlusion robustness. Although appearance-based trackers have been greatly improved in the last decade, they still struggle with challenges that are not fully resolved. Of these challenges, occlusions, which can be long lasting and of a wide variety, are often ignored or only partly addressed due to the difficulty in their treatments. To address this problem, in this study, we propose an occlusion-aware particle filter framework that employs a probabilistic model with a latent variable representing an occlusion flag. The proposed framework prevents losing the target by prediction of emerging occlusions, updates the target template by shifting relevant information, expands the search area for an occluded target, and grants quick recovery of the target after occlusion. Furthermore, the algorithm employs multiple features from the color and depth domains to achieve robustness against illumination changes and clutter, so that the probabilistic framework accommodates the fusion of those features. This method was applied to the Princeton RGBD Tracking Dataset, and the performance of our method with different sets of features was compared with those of the state-of-the-art trackers. The results revealed that our method outperformed the existing RGB and RGBD trackers by successfully dealing with different types of occlusions.

Expanding histogram of colors with gridding to improve tracking accuracy

Using color information in object tracking is a prudent choice, but the vast variety of choices and difficulties of obtaining a desirable stable result, unnerves many scholars. Color histograms, as a compact and robust representation is the center of attention while it suffers from lack of spatial information about colors. Besides, comparison and updating such histograms in a meaningful and efficient manner is challenging. In this paper, we proposed the idea of gridding for color histogram, which grants specific statistical property to the histogram through a decomposition phase followed by a recombination stage. Additionally, a thorough comparison of the modern similarity functions and model update techniques in RGB colorspace is presented. This comparison reveals that our proposed method in combination with established similarity measures, enhances the tracking performance.

Active discriminative tracking using collective memory

Ever changing appearance of the targets in real-world scenarios mandates a discriminative tracker to update its classifier(s) on-the-fly, a process during which the model could be updated with irrelevant/noisy data, causing the tracker to drift away from the target over time. The updates should be frequent enough to reflect the latest changes in the targets appearance, whereas the tracker should keep the memory of previous templates to recover from occlusions or temporal variations in appearance of the target (aka the plasticity-stability dilemma). In this study, we proposed a committee of classifiers with different memory spans, to address the appearance changes with various durations. An active learning scheme selects the most disputed samples and queries their labels from a less-frequently updated long-term memory oracle. This combination of memory spans balances the plasticity-stability equilibrium as demonstrated by the experiments and provides a comparable performance to the state-of-the-art trackers with a relatively simple implementation.

Partial and synchronized captioning: A new tool to assist learners in developing second language listening skill

This paper introduces a novel captioning method, partial and synchronized captioning (PSC), as a tool for developing second language (L2) listening skills. Unlike conventional full captioning, which provides the full text and allows comprehension of the material merely by reading, PSC promotes listening to the speech by presenting a selected subset of words, where each word is synched to its corresponding speech signal. In this method, word-level synchronization is realized by an automatic speech recognition (ASR) system, dedicated to the desired corpora. This feature allows the learners to become familiar with the correspondences between words and their utterances. Partialization is done by automatically selecting words or phrases likely to hinder listening comprehension. In this work we presume that the incidence of infrequent or specific words and fast delivery of speech are major barriers to listening comprehension. The word selection criteria are thus based on three factors: speech rate, word frequency and specificity. The thresholds for these features are adjusted to the proficiency level of the learners. The selected words are presented to aid listening comprehension while the remaining words are masked in order to keep learners listening to the audio. PSC was evaluated against no-captioning and full-captioning conditions using TED videos. The results indicate that PSC leads to the same level of comprehension as the full-captioning method while presenting less than 30% of the transcript. Furthermore, compared with the other methods, PSC can serve as an effective medium for decreasing dependence on captions and preparing learners to listen without any assistance.

Data-Driven Probabilistic Occlusion Mask to Promote Visual Tracking

Occlusion, one of the biggest challenges of visual tracking, impedes many trackers by corrupting observations, decaying the template accuracy, or introducing distracting occluders to the tracker. In this study, we propose a technique to detect occlusions through learning the foreground probability distributions. In our approach, the target is divided into a grid cells and the likelihood of occlusion is determined for each cell in a data-driven fashion. We introduce an occlusion indicator for each of the cells. By learning corresponding distribution of this indicator for each cell, using a diverse set of videos and targets, we obtain a set of occlusion probability distributions which is universally applicable to any video or object. By assigning an occlusion likelihood to different cells of an observation (i.e., creating an occlusion mask), our proposed approach provides a confidence measure for different parts of input observations and can be coupled with many generic tracking methods. In this study, we adopt four particle filter-based trackers -- multi-cue PFT, IVT, L1T, and L1APG -- to test the effectiveness of our occlusion mask. Utilizing the proposed occlusion mask lowers the weight of the erroneous parts of observation, allows for a more robust template update, and mitigates distraction by occluders. The method was evaluated on challenging videos. The quantitative results highlighted the tracking accuracy improvement and demonstrated successful tracking under different occlusion scenarios.

Robust discriminative tracking via query-by-bagging

Adaptive tracking-by-detection is a popular approach to track arbitrary objects in various situations. Such approaches treat tracking as a classification task and constantly update the object model. The update procedure requires a set of labeled examples, where samples are collected from the last observation, and then labeled. However, these intermediate steps typically follow a set of heuristic rules for labeling and uninformed search in the sample space, which decrease the effectiveness of model update. In this study, we present a framework for adaptive tracking that utilizes active learning for effective sample selection and labeling them. The active sampler employs a committee of randomized-classifiers to select the most informative samples and query their label from an auxiliary detector with a long-term memory. The committee is then updated with the obtained labels. Experiments show that our algorithm outperforms state-of-the-art trackers on various benchmark videos.

Exploiting automatic speech recognition errors to enhance partial and synchronized caption for facilitating second language listening

Abstract This paper addresses the viability of using Automatic Speech Recognition (ASR) errors as the predictor of difficulties in speech segments, thereby exploiting them to improve Partial and Synchronized Caption (PSC), which we have proposed to train second language (L2) listening skill by encouraging listening over reading. The system uses ASR technology to make word-level text-to-speech synchronization and generates a partial caption. The baseline system determines difficult words based on three features: speech rate, word frequency and specificity. While it encompasses most of the difficult words, it does not cover a wide range of features that hinder L2 listening. Therefore, we propose the use of ASR systems as a model of L2 listeners and hypothesize that ASR errors can predict challenging speech segments for these learners. Among different cases of ASR errors, annotation results suggest the usefulness of four categories of homophones, minimal pairs, negatives, and breached boundaries for L2 listeners. A preliminary experiment with L2 learners focusing on these four categories of the ASR errors revealed that these cases highlight the problematic speech regions for L2 listeners. Based on the findings, the PSC system is enhanced to incorporate these kinds of useful ASR errors. An experiment with L2 learners demonstrated that the enhanced version of PSC is not only preferable, but also more helpful to facilitate the L2 listening process.