Alex Leykin

Robust Multi-Pedestrian Tracking in Thermal-Visible Surveillance Videos

In this paper we introduce a system to track pedestrians using a combined input from RGB and thermal cameras. Two major contributions are presented here. First is the novel model of the scene background where each pixel is represented as a multi-modal distribution with the changing number of modalities for both color and thermal input. We demonstrate how to eliminate the influence of shadows with this type of fusion. Second, based on our background model we introduce a pedestrian tracker designed as a particle filter. We further develop a number of informed reversible transformations to sample the model probability space in order to maximize our model posterior probability. The novelty of our tracking approach also comes from a way we formulate observation likelihoods to account for 3D locations of the bodies with respect to the camera and occlusions by other tracked human bodies as well as static objects. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

Pedestrian tracking by fusion of thermal-visible surveillance videos

In this paper we introduce a system to track pedestrians using a combined input from RGB and thermal cameras. Two major contributions are presented here. First is the novel probabilistic model of the scene background where each pixel is represented as a multi-modal distribution with the changing number of modalities for both color and thermal input. We demonstrate how to eliminate the influence of shadows with this type of fusion. Second, based on our background model we introduce a pedestrian tracker designed as a particle filter. We further develop a number of informed reversible transformations to sample the model probability space in order to maximize our model posterior probability. The novelty of our tracking approach also comes from a way we formulate observation likelihoods to account for 3D locations of the bodies with respect to the camera and occlusions by other tracked human bodies as well as static objects. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

An Examination of Social Influence on Shopper Behavior Using Video Tracking Data

This research investigates how the social elements of a retail store visit affect shoppers’ product interaction and purchase likelihood. The research uses a bivariate model of the shopping process, implemented in a hierarchical Bayes framework, which models the customer and contextual factors driving product touch and purchase simultaneously. A unique video tracking database captures each shoppers path and activities during the store visit. The findings reveal that interactive social influences (e.g., salesperson contact, shopper conversations) tend to slow the shopper down, encourage a longer store visit, and increase product interaction and purchase. When shoppers are part of a larger group, they are influenced more by discussions with companions and less by third parties. Stores with customers present encourage product interaction up to a point, beyond which the density of shoppers interferes with the shopping process. The effects of social influence vary by the salespersons demographic similarity to the shopper and the type of product category being shopped. Several behavioral cues signal when shoppers are in a potentially high need state and may be good sales prospects.

Estimating the photorealism of images: distinguishing paintings from photographs

Automatic classification of an image as a photograph of a real-scene or as a painting is potentially useful for image retrieval and Web site filtering applications. The main contribution of the paper is the proposition of several features derived from the color, edge, and gray-scale-texture information of the image that effectively discriminate paintings from photographs. For example, we found that paintings contain significantly more pure-color edges, and that certain gray-scale-texture measurements (mean and variance of Gabor filters) are larger for photographs. Using a large set of images (12000) collected from different Web sites, the proposed features exhibit very promising classification performance (over 90%). A comparative analysis of the automatic classification results and psychophysical data is reported, suggesting that the proposed automatic classifier estimates the perceptual photorealism of a given picture.

Thermal-Visible Video Fusion for Moving Target Tracking and Pedestrian Classification

The paper presents a fusion-tracker and pedestrian classifier for color and thermal cameras. The tracker builds a background model as a multi-modal distribution of colors and temperatures. It is constructed as a particle filter that makes a number of informed reversible transformations to sample the model probability space in order to maximize posterior probability of the scene model. Observation likelihoods of moving objects account their 3D locations with respect to the camera and occlusions by other tracked objects as well as static obstacles. After capturing the coordinates and dimensions of moving objects we apply a pedestrian classifier based on periodic gait analysis. To separate humans from other moving objects, such as cars, we detect, in human gait, a symmetrical double helical pattern, that can then be analyzed using the Frieze Group theory. The results of tracking on color and thermal sequences demonstrate that our algorithm is robust to illumination noise and performs well in the outdoor environments.

Distinguishing paintings from photographs

We addressed the problem of automatically differentiating photographs of real scenes from photographs of paintings. We found that photographs differ from paintings in their color, edge, and texture properties. Based on these features, we trained and tested a classifier on a database of 6000 paintings and 6000 photographs. Using single features results in ~70-80% correct discrimination performance, whereas a classifier using multiple features exceeds 90% correct discrimination.

Differences of edge properties in photographs and paintings

We compare the properties of intensity and color edges in photographs of real scenes and paintings. We demonstrate that paintings contain significantly more color-only edges, whereas the amount of intensity-only edges does not differ significantly between the two classes. In addition, color edge strength is significantly higher for paintings. The differences between paintings and photographs are more accentuated when high-resolution, losslessly compressed images are used. These distinguishing features can be used for the automatic differentiation between the two classes of images.

Real-time estimation of human attention field in LWIR and color surveillance videos

Knowing the visual attention field of a monitored subject is of great value for many applications including surveillance and marketing. This paper proposes first to track peoplepsilas bodies, and then estimates visual attention field for each human using head pose information. The proposed head pose technique aims at estimating the yaw angle only. The method is shown to operate on monocular color camera sequences and is further refined with the data from a thermal sensor. In typical monocular tracking sequences the resolution of the head is very low and parts of the head are occluded with the face often invisible to the camera. We propose a method of combining a skin color detector with the direction of motion in a probabilistic way. We show how head profile obtained from the thermal sequence can be used to further improve the result.

Identifying the Drivers of Shopper Attention, Engagement, and Purchase

Abstract To cope with the complexity of modern retail stores and personal time constraints, shoppers must be selective in processing information. During a typical shopping trip, they visit only a fraction of a store’s departments and categories, examine a small subset of the available products, and often make selections in just a few seconds. New research techniques can help marketers understand how customers allocate their attention and assess the impact of in-store factors on shopper behavior. This chapter summarizes studies using observational research, virtual reality simulations, and eye tracking to identify the drivers of shopper attention, product engagement, and purchase conversion. These include shopper goals; product assortment, package appearance, price, and merchandising; shelf space allocation, organization, and adjacencies; and salesperson interaction. The research reveals that small changes in a product’s appearance and presentation can have a powerful impact on consideration and choice.

Determining text readability over textured backgrounds in augmented reality systems

This paper concerns the application of pattern classification techniques to the domain of augmented reality. In many augmented reality applications, one of the ways in which information is presented to the user is to place a text label over the area of interest. However, if this information is placed over very busy and textured backgrounds, this can affect the readability of the text. The goal of this work was to identify methods of quantitatively describing conditions under which such text would be readable or unreadable. We used texture properties and other visual features to predict if a text placed on a particular background would be readable or not. Based on these features, a supervised classifier was built that was trained using data collected from human subjects judgement of text readability. Using a rather small training set of about 400 human evaluations over 50 heterogeneous textures the system is able to achieve a correct classification rate of over 85%.