Koray Balci

A Multimodal 3D Healthcare Communication System

We present a system that integrates gesture recognition and 3D talking head technologies for a patient communication application at a hospital or healthcare setting for supporting patients treated in bed. As a multimodal user interface, we get the input from patients using hand gestures and provide feedback by using a 3D talking avatar.

Automatic analysis and identification of verbal aggression and abusive behaviors for online social games

Display Omitted We propose a system to automatically evaluate player complaints in a social game.Our database contains 100,000 players, 1000 complaints and 240 abusive players.We experiment with several pieces of player information and their combinations.Our system can correctly identify abusive players with up to 85% precision.We can isolate and identify severe abuse cases with a higher confidence. Online multiplayer games create new social platforms, with their own etiquette, social rules of conduct and ways of expression. What counts as aggressive and abusing behavior may change depending on the platform, but most online gaming companies need to deal with aggressive and abusive players explicitly. This usually is tied to a reporting mechanism where the offended player reports an offense. In this paper, we develop tools for validating whether a verbal aggression offense report refers to a real offense or not, in the context of a very popular online social game, called Okey. Our approach relies on the analysis of player behavior and characteristics of offending players. In the proposed system, chat records and other social activities in the game are taken into account, as well as player history. This methodology is sufficiently generic, and it can be applied to similar gaming platforms, thus describing a useful tool for game companies. We report our results on data collected over a six months period, involving 100,000 users and 800,000 game records, and illustrate the viability of such analysis, while providing insights on the factors associated with verbal aggression and abusive behavior for social games.

An audio-driven dancing avatar

We present a framework for training and synthesis of an audio-driven dancing avatar. The avatar is trained for a given musical genre using the multicamera video recordings of a dance performance. The video is analyzed to capture the time-varying posture of the dancer’s body whereas the musical audio signal is processed to extract the beat information. We consider two different marker-based schemes for the motion capture problem. The first scheme uses 3D joint positions to represent the body motion whereas the second uses joint angles. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In the synthesis phase, an audio signal of unknown musical type is first classified, within a time interval, into one of the genres that have been learnt in the analysis phase, based on mel frequency cepstral coefficients (MFCC). The motion parameters of the corresponding dance figures are then synthesized via the trained HMM structures in synchrony with the audio signal based on the estimated tempo information. Finally, the generated motion parameters, either the joint angles or the 3D joint positions of the body, are animated along with the musical audio using two different animation tools that we have developed. Experimental results demonstrate the effectiveness of the proposed framework.

Clustering poses of motion capture data using limb centroids

With increasing popularity of using motion capture hardware for motion synthesis, studies that exploit large motion databases for faster indexing and retrieval become more important. Here, an automated procedure to analyze motion databases and cluster poses according to various feature vectors is presented. We propose the use of limb centroids as an alternative feature vector. Limb centroids have a smaller size compared to standard alternatives, thus time complexity is reduced while satisfactory clusters are achieved when used with our iterative clustering procedure. In addition, along with two different metrics to evaluate formed clusters, we also propose pose clouds for visualization and perceptual analysis.

Generating motion graphs from clusters of individual poses

In this paper, we propose a methodology to generate novel motion clips from existing motion capture data. We present an automated procedure that performs clustering on the data in order to generate a robust and powerful motion graph. We discard the temporal connection between frames during clustering phase, which lets us spot hubs that motions tend to visit more often and use these poses as nodes of our motion graph. This approach greatly reduces computational times and complexity of the final graph structure while maintaining the representational power.

Automatic Classification of Player Complaints in Social Games

Artificial intelligence and machine learning techniques are not only useful for creating plausible behaviors for interactive game elements, but also for the analysis of the players to provide a better gaming environment. In this paper, we propose a novel framework for automatic classification of player complaints in a social gaming platform. We use features that describe both parties of the complaint (namely, the accuser and the suspect), as well as interaction features of the game itself. The proposed classification approach, based on gradient boosting machines, is tested on the COPA Database of 100 000 unique users and 800 000 individual games. We advance the state of the art in this challenging problem.

Authorship recognition in a multiparty chat scenario

Users of online social networks often use multiple identities. This paper investigates the possibility of identifying a user from his or her chat behavior in such a setting. We have collected a large corpus of multiparty chat records in Turkish, obtained from a multiplayer game database. The most active 978 users are selected according to their participation in game chat sessions. This corpus is used in a biometric identification experiment where we seek each user among a gallery of users. Character matrices for each player are used as features, and re-centered local profiles and cosine similarity measure are preferred as identification methods. We systematically assess the effect of text normalization on identification. We report comparative results, the best of which reach around 75% rank-1 accuracy for a gallery size of 978.

Automatic Assessment of Dimensional Affective Content in Turkish Multi-party Chat Messages

This study presents a model for affective text analysis of online multi-party chat records in Turkish language. Online chats have challenges like non-standard word usage, grammatical irregularities, abbreviation usage, and spelling mistakes. We propose several pre-processing steps to deal with these. We adapt an affective word dictionary from English to Turkish, and by expanding it, obtain 15,222 words with annotations for valence, arousal, and dominance. We also employ a list of abbreviations, emoticons, interjections, modifiers (intensifiers and diminishers), and other linguistic indicators to capture the overall affective state at the sentence level. Lastly, we recruit and train annotators to obtain affective ground truth, and assess the accuracy of the proposed rule-based approach on a multi-party chat database collected from an online gaming environment.

Analysis and synthesis of multiview audio-visual dance figures

This paper presents a framework for audio-driven human body motion analysis and synthesis. The video is analyzed to capture the time-varying posture of the dancerpsilas body whereas the musical audio signal is processed to extract the beat information. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In synthesis, given an audio signal of a learned musical type, the motion parameters of the corresponding dance figures are synthesized via the trained HMM structures in synchrony with the input audio signal based on the estimated tempo information. Finally, the generated motion parameters are animated along with the musical audio using a graphical animation tool. Experimental results demonstrate the effectiveness of the proposed framework.

Sign Language Motion Tracking and Generating 3D Motion Pieces Using 2D Features

In this study we designed a system that can track 3D motion and generate new pieces of motion, using a stereo camera setup. A dataset from some expressions of Turkish sign language is built using markers. In order to find the center of the markers and track their coordinates the correspondence problem had to be solved. Results of newly proposed marker detection and tracking algorithms are compared. Finally, following the scale normalization, the trajectories are modelled using Hidden Markov Models (HMM). Using these models, new signs are generated.