Valery A. Petrushin

System, method and article of manufacture for an emotion detection system improving emotion recognition

A voice signal and an emotion associated therewith is provided. Then, the emotion associated with the voice signal is determined in an automated manner and subsequently stored. Next, a user determined emotion associated with the voice signal is determined by a user and received. The automatically determined emotion with the user determined emotion are then compared.

Multimedia Data Mining and Knowledge Discovery

into Multimedia Data Mining and Knowledge Discovery.- Multimedia Data Mining: An Overview.- Multimedia Data Exploration and Visualization.- A New Hierarchical Approach for Image Clustering.- Multiresolution Clustering of Time Series and Application to Images.- Mining Rare and Frequent Events in Multi-camera Surveillance Video.- Density-Based Data Analysis and Similarity Search.- Feature Selection for Classification of Variable Length Multiattribute Motions.- Multimedia Data Indexing and Retrieval.- FAST: Fast and Semantics-Tailored Image Retrieval.- New Image Retrieval Principle: Image Mining and Visual Ontology.- Visual Alphabets: Video Classification by End Users.- Multimedia Data Modeling and Evaluation.- Cognitively Motivated Novelty Detection in Video Data Streams.- Video Event Mining via Multimodal Content Analysis and Classification.- Exploiting Spatial Transformations for Identifying Mappings in Hierarchical Media Data.- A Novel Framework for Semantic Image Classification and Benchmark Via Salient Objects.- Extracting Semantics Through Dynamic Context.- Mining Image Content by Aligning Entropies with an Exemplar.- More Efficient Mining Over Heterogeneous Data Using Neural Expert Networks.- A Data Mining Approach to Expressive Music Performance Modeling.- Applications and Case Studies.- Supporting Virtual Workspace Design Through Media Mining and Reverse Engineering.- A Time-Constrained Sequential Pattern Mining for Extracting Semantic Events in Videos.- Multiple-Sensor People Localization in an Office Environment.- Multimedia Data Mining Framework for Banner Images.- Analyzing Users Behavior on a Video Database.- On SVD-Free Latent Semantic Indexing for Iris Recognition of Large Databases.- Mining Knowledge in Computer Tomography Image Databases.

Counting people using video cameras

The paper is devoted to the problem of estimating the number of people visible in a camera. It uses as features the ratio of foreground pixels in each cell of a rectangular grid. Using the above features and data mining techniques allowed reaching accuracy up to 85% for exact match and up to 95% for plus–minus one estimates for an indoor surveillance environment. Applying median filters to the sequence of estimation results increased the accuracy up to 91% for exact match. The architecture of a real-time people counting estimator is suggested. The results of analysis of experimental data are provided and discussed.

Multiple-Sensor Indoor Surveillance System

This paper describes a surveillance system that uses a network of sensors of different kind for localizing and tracking people in an office environment. The sensor network consists of video cameras, infrared tag readers, a fingerprint reader and a PTZ camera. The system implements a Bayesian framework that uses noisy, but redundant data from multiple sensor streams and incorporates it with the contextual and domain knowledge. The paper describes approaches to camera specification, dynamic background modeling, object modeling and probabilistic inference. The preliminary experimental results are presented and discussed.

Multiple-camera people localization in an indoor environment

With the rapid proliferation of video cameras in public places, the ability to identify and track people and other objects creates tremendous opportunities for business and security applications. This paper presents the Multiple Camera Indoor Surveillance project which is devoted to using multiple cameras, agent-based technology and knowledge-based techniques to identify and track people and summarize their activities. We also describe a people localization system, which identifies and localizes people in an indoor environment. The system uses low-level color features – a color histogram and average vertical color – for building people models and the Bayesian decision-making approach for people localization. The results of a pilot experiment that used 32 h of data (4 days × 8 h) showed the average recall and precision values of 68 and 59% respectively. Augmenting the system with domain knowledge, such as location of working places in cubicles, doors and passages, increased the average recall to 87% and precision to 73%.

Using virtual worlds for corporate training

We present virtual training worlds that are relatively low-cost distributed collaborative learning environments suitable for corporate training. A virtual training world allows a facilitator, experts and trainees communicating and acting in the virtual environment for practicing skills during collaborative problem solving. Using these environments is beneficial to both trainees and corporations. The design of a leadership training course is discussed in details.

Multiple sensor integration for indoor surveillance

Multiple Sensor Indoor Surveillance (MSIS) is a research project at Accenture Technology Labs aimed at exploring a variety of redundant sensors in a networked environment where each sensor is giving noisy information and the goal is to coherently reason about some aspect of the environment. We describe the objectives of the project, the problems it was designed to solve and some recent results. The environment includes 32 web cameras, an infrared badge ID system, a PTZ camera, and a fingerprint reader. We discuss two concrete problems that we have tackled in this project: (1) Visualizing events detected by 32 cameras during 24 hours, and (2) Localizing people using fusion of multiple streams of noisy sensory data with the contextual and domain knowledge that is provided by both the physical constraints imposed by the local environment and by the people that are involved in the surveillance tasks. We use Self-Organizing Maps to approach the first problem and suggest a Bayesian framework for the second one. The experimental data are provided and discussed.

eShopper modeling and simulation

The advent of e-commerce gives an opportunity to shift the paradigm of customer communication into a highly interactive mode. The new generation of commercial Web servers, such as the Blue Martinis server, combines the collection of data on a customer behavior with real-time processing and dynamic tailoring of a feedback page. The new opportunities for direct product marketing and cross selling are arriving. The key problem is what kind of information do we need to achieve these goals, or in other words, how do we model the customer? The paper is devoted to customer modeling and simulation. The focus is on modeling an individual customer. The model is based on the customers transaction data, click stream data, and demographics. The model includes the hierarchical profile of a customers preferences to different types of products and brands; consumption models for the different types of products; the current focus, trends, and stochastic models for time intervals between purchases; product affinity models; and some generalized features, such as purchasing power, sensitivity to advertising, price sensitivity, etc. This type of model is used for predicting the date of the next visit, overall spending, and spending for different types of products and brands. For some type of stores (for example, a supermarket) and stable customers, it is possible to forecast the shopping lists rather accurately. The forecasting techniques are discussed. The forecasting results can be used for on- line direct marketing, customer retention, and inventory management. The customer model can also be used as a generative model for simulating the customers purchasing behavior in different situations and for estimating customers features.

The Community of Multimedia Agents project

Challenges in multimedia analysis are calling for the sharing of research efforts, while in practice collaboration is hindered by technical and proprietary issues. The Community of Multimedia Agents project (COMMA) attempts to solve this problem by creating an open environment for developing, testing, and prototyping multimedia content analysis and annotation methods. Each method is represented as an agent (an executable module) that can communicate with the other agents based on descriptors and description schemes in the coming MPEG-7 standard. This allows multimedia-processing agents developed by different organizations to operate and collaborate with each other, regardless of their programming languages and internal architecture. The researchers can compare the performance of agents and combine them to build more powerful and robust system prototypes. It can also serve as a learning environment for researchers and students to acquire and test cutting edge multimedia analysis algorithms. Through sharing of media agents, the Community can increase efficiency of research while protecting the intellectual property of the inventors.

The Community of Multimedia Agents

Multimedia data mining requires the ability to automatically analyze and understand the content. The Community of Multimedia Agents project is devoted to creating a community of researchers and students who are interested in developing multimedia annotation algorithms. It provides an open environment for developing, testing, learning and prototyping multimedia content analysis and annotation methods. It serves as a medium for researchers to contribute and share their achievements while protecting their proprietary techniques. Each method is represented as an agent that can communicate with the other agents registered in the environment using templates that are based on the descriptors and description schemes in the MPEG-7 standard. Using the standard allows agents that are developed by different organizations to operate and communicate with each other seamlessly regardless of their programming languages and internal architecture. A development environment is provided to facilitate the construction of media analysis methods. The tool contains a workbench, which allows the user integrating agents to build more sophisticated systems, and a blackboard browser, which visualizes the processing results. It enables researchers to compare the performance of different agents and combine them to build a rapid prototype of more powerful and robust system. The Community can also serve as a learning environment for researchers and students to acquire and exchange of cutting edge multimedia analysis algorithms.