Hao Shen

Dimensionality Reduction in HRTF by Using Multiway Array Analysis

In a human centered robotic system, it is important to provide the robotic platform with multimodal human-like sensing, e.g. haptic, vision and audition, in order to improve interactions between the human and the robot. Recently, Head Related Transfer Functions (HRTFs) based techniques have become a promising methodology for robotic binaural hearing, which is a most prominent concept in human robot communication. In complex and dynamical applications, due to its high dimensionality, it is inefficient to utilize the originial HRTFs. To cope with this difficulty, Principle Component Analysis (PCA) has been successfully used to reduce the dimensionality of HRTF datasets. However, it requires in general a vectorization process of the original dataset, which is a three-way array, and consequently might cause loss of structure information of the dataset. In this paper we apply two multi-way array analysis methods, namely the Generalized Low Rank Approximations of Matrices (GLRAM) and the Tensor Singular Value Decomposition (Tensor-SVD), to dimensionality reductions in HRTF based applications. Our experimental results indicate that an optimized GLRAM outperforms significantly the PCA and performs nearly as well as Tensor-SVD with less computational complexity.

Geometric algorithms for the non-whitened one-unit linear Independent Component Analysis problem

In this paper, we study the problem of one-unit linear Independent Component Analysis (ICA) without whitening. The FastICA algorithm is arguably the most popular algorithm for solving the whitened one-unit linear ICA problem. Although a modified FastICA has been already proposed to solve the non-whitened one-unit linear ICA problem, there is unfortunately no known analysis regarding its effectiveness and efficiency. In this work, the non-whitened FastICA algorithm is revisited and analyzed in the framework of geometric optimization algorithms. In this paper, a conjugate gradient (CG) algorithm for the non-whitened one-unit linear ICA problem is developed as well. Local convergence properties of both algorithms are discussed. Finally, local convergence performance of the algorithms is investigated by several numerical experiments.

Learning to walk with prior knowledge

In this work a novel approach to Transfer Learning for the use in Deep Reinforcement Learning is introduced. The agent is realized as an actor-critic framework, namely the Deep Deterministic Policy Gradient algorithm. The Q-function and the policy are represented as deep feed-forward networks, that are trained by minimizing the mean squared Bellman error and by maximizing the expected reward, respectively. For Transfer Learning, the actor is modified with a new regularization term, called the knowledge regularizer. It allows to include prior knowledge in from of an existing policy in the learning process. The knowledge regularizer shifts the current weight vector during the gradient descent step towards a region of the weight space, that is centered around the existing policy. Because neural networks are universal and smooth function approximators, the weights of the existing policy and the new ones have to lie close to each other in the weight space. Solving a task therefore benefits from the prior knowledge, when it is used to manipulate the gradient given by the critic. We could experimentally verify, that the knowledge regularizer results in a higher performance achieved by the agent and in a reduction of the learning time. Furthermore, the knowledge regularizer can be used as a replacement for labeled training data, which renders it especially useful for physical applications.

Accelerated gradient temporal difference learning algorithms

In this paper we study Temporal Difference (TD) Learning with linear value function approximation. The classic TD algorithm is known to be unstable with linear function approximation and off-policy learning. Recently developed Gradient TD (GTD) algorithms have addressed this problem successfully. Despite their prominent properties of good scalability and convergence to correct solutions, they inherit the potential weakness of slow convergence as they are a stochastic gradient descent algorithm. Accelerated stochastic gradient descent algorithms have been developed to speed up convergence, while still keeping computational complexity low. In this work, we develop an accelerated stochastic gradient descent method for minimizing the Mean Squared Projected Bellman Error (MSPBE), and derive a bound for the Lipschitz constant of the gradient of the MSPBE, which plays a critical role in our proposed accelerated GTD algorithms. Our comprehensive numerical experiments demonstrate promising performance in solving the policy evaluation problem, in comparison to the GTD]algorithm family. In particular, accelerated TDC surpasses state-of-the-art algorithms.

HRTF-based localization and separation of multiple sound sources

The human auditory system excels at pinpointing and distinguishing multiple sound sources in noisy and reverberant environments. Mobile robotic platforms implement such capabilities with varying success, classically solving localization and separation independently. This paper presents an algorithm utilizing Head-Related Transfer Function (HRTF) based localization to aid the task of separation. HRTFs for robotic binaural hearing represent the digital emulation of a humans innate direction-dependent filtering for solving the localization problem in a compact and robust manner. The overall result of the presented algorithm for robotic binaural hearing is an HRTF-based localization and separation system, capable of dynamically and intelligently processing simultaneously active sound sources.

Formation control using GQ(λ) reinforcement learning

Formation control is an important subtask for autonomous robots. From flying drones to swarm robotics, many applications need their agents to control their group behavior. Especially when moving autonomously in humanrobot teams, motion and formation control of a group of agents is a critical and challenging task. In this work, we propose a method of applying the GQ(λ) reinforcement learning algorithm to a leader-follower formation control scenario on the e-puck robot platform. In order to allow control via classical reinforcement learning, we present how we modeled a formation control problem as a Markov decision making process. This allows us to use the Greedy-GQ(λ) algorithm for learning a leader-follower control law. The applicability and performance of this control approach is investigated in simulation as well as on real robots. In both experiments, the followers are able to move behind the leader. Additionally, the algorithm improves the smoothness of the followers path online, which is beneficial in the context of human-robot interaction.

Texture Retrieval Using Scattering Coefficients and Probability Product Kernels

In this paper we introduce a content based image retrieval system that leverages the benefits of the scattering transform as a means of feature extraction. To measure similarity between feature vectors, we adapt a probability product kernel and derive an approximate version which can be implemented efficiently. The proposed approach achieves a retrieval performance superior to comparable filterbank transform systems.