Hung Quoc Ngo

Incremental learning using partial feedback for gesture-based human-swarm interaction

In this paper we consider a human-swarm interaction scenario based on hand gestures. We study how the swarm can incrementally learn hand gestures through the interaction with a human instructor providing training gestures and correction feedback. The main contribution of the paper is a novel incremental machine learning approach that makes the robot swarm learn and recognize the gestures in a distributed and decentralized fashion using binary (i.e., yes/no) feedback. It exploits cooperative information exchange and swarms intrinsic parallelism and redundancy. We perform extensive tests using real gesture images, showing that good classification accuracies are obtained even with rather few training samples and relatively small swarms. We also show the good scalability of the approach and its relatively low requirements in terms of communication overhead.

Learning skills from play: Artificial curiosity on a Katana robot arm

Artificial curiosity tries to maximize learning progress. We apply this concept to a physical system. Our Katana robot arm curiously plays with wooden blocks, using vision, reaching, and grasping. It is intrinsically motivated to explore its world. As a by-product, it learns how to place blocks stably, and how to stack blocks.

A comprehensive middleware architecture for context-aware ubiquitous computing systems

Ubiquitous computing is viewed as a computing paradigm where minimal user intervention is necessitated emphasizing detection of environmental conditions and user behaviors in order to maximize user experience. Context-awareness plays vital role in achieving such user-centered ubiquity. In this paper, we describe the desired characteristics of a middleware for context-aware ubiquitous computing. Four key issues are addressed: unified sensing framework, formal modeling and representation of the real world, pluggable reasoning engines for high-level contexts, and response to the real world. Our implementation experience indicates that a comprehensive approach throughout the system layers results in a flexible and reusable middleware framework.

MEPA: A New Protocol for Energy-Efficient, Distributed Clustering in Wireless Sensor Networks

Clustering is an effective approach to hierarchically organizing network topology for efficient data aggregation in wireless sensor networks. Distributed protocols with simple local computations to accomplish a desired global goal, offer a good prospect for achieving energy efficiency. This paper presents MEPA - an energy-efficient distributed clustering protocol using simple and local message-passing rules. Our proposed clustering protocol combines both node residual energy and network topology features to recursively elect a near-optimal set of cluster heads. Simulation results show that MEPA can produce a set of cluster heads with compelling characteristics, and effectively prolong the network lifetime.

Approximate planning for bayesian hierarchical reinforcement learning

In this paper, we propose to use hierarchical action decomposition to make Bayesian model-based reinforcement learning more efficient and feasible for larger problems. We formulate Bayesian hierarchical reinforcement learning as a partially observable semi-Markov decision process (POSMDP). The main POSMDP task is partitioned into a hierarchy of POSMDP subtasks. Each subtask might consist of only primitive actions or hierarchically call other subtasks’ policies, since the policies of lower-level subtasks are considered as macro actions in higher-level subtasks. A solution for this hierarchical action decomposition is to solve lower-level subtasks first, then higher-level ones. Because each formulated POSMDP has a continuous state space, we sample from a prior belief to build an approximate model for them, then solve by using a recently introduced Monte Carlo Value Iteration with Macro-Actions solver. We name this method Monte Carlo Bayesian Hierarchical Reinforcement Learning. Simulation results show that our algorithm exploiting the action hierarchy performs significantly better than that of flat Bayesian reinforcement learning in terms of both reward, and especially solving time, in at least one order of magnitude.

Research issues in the development of context-aware middleware architectures

Context-aware middleware encompasses uniform abstractions and reliable services for common operations, supports for most of the tasks involved in dealing with context, and thus simplifying the development of context-aware applications. In this paper, we address some key issues of a middleware for context-aware ubiquitous computing, ranging from design considerations of a unified sensing framework, formal modeling and representation of the real world, pluggable reasoning engines for high-level contexts, and context delivery-runtime service composition mechanisms. Our implementation experience indicates that a comprehensive approach throughout the system layers results in a flexible and reusable middleware architecture.

Efficient Interactive Multiclass Learning from Binary Feedback

We introduce a novel algorithm called upper confidence-weighted learning (UCWL) for online multiclass learning from binary feedback (e.g., feedback that indicates whether the prediction was right or wrong). UCWL combines the upper confidence bound (UCB) framework with the soft confidence-weighted (SCW) online learning scheme. In UCB, each instance is classified using both score and uncertainty. For a given instance in the sequence, the algorithm might guess its class label primarily to reduce the class uncertainty. This is a form of informed exploration, which enables the performance to improve with lower sample complexity compared to the case without exploration. Combining UCB with SCW leads to the ability to deal well with noisy and nonseparable data, and state-of-the-art performance is achieved without increasing the computational cost. A potential application setting is human-robot interaction (HRI), where the robot is learning to classify some set of inputs while the human teaches it by providing only binary feedback—or sometimes even the wrong answer entirely. Experimental results in the HRI setting and with two benchmark datasets from other settings show that UCWL outperforms other state-of-the-art algorithms in the online binary feedback setting—and surprisingly even sometimes outperforms state-of-the-art algorithms that get full feedback (e.g., the true class label), whereas UCWL gets only binary feedback on the same data sequence.

Wisdom of the swarm for cooperative decision-making in human-swarm interaction

Human-swarm interaction (HSI) is a developing field of research in which the problem of gesture-based control has been attracting an increasing attention, being at the same time a natural form of interaction and an effective way to point and select individual or groups of robots in the swarm. Gesture-based interaction usually requires vision-based recognition and classification of the gesture from the swarm. At this aim, existing methods for cooperative sensing and recognition make use of distributed consensus algorithms, which include for instance averaging and frequency counting. In this work we present a distributed consensus protocol that allows robot swarms to learn efficiently gestures from online interactions with a human teacher. The protocol also facilitates the integration of different consensus algorithms. Experiments have been performed in emulation using on real data acquired by a swarm of robots. The results indicate that effectively exploiting the collective decision-making of the swarm is a viable way to rapidly achieve good learning performance.

Lifetime optimized hierarchical architecture for correlated data gathering in wireless sensor networks

In-network aggregation is essential for correlated data gathering in wireless sensor networks which are resource-constraint in terms of energy, computation and storage. In this paper, we consider the problem of building a minimum cost hierarchical architecture for correlated data gathering with in-network aggregation, which is formulated as a min-sum optimization problem. To solve the problem, we first develop a minimum-cost distributed algorithm which involves only simple message-passing rules. The algorithm is then tuned to be energy-aware so that high-energy sensor nodes are preferably selected to become cluster heads (CHs), which act as encoding and relaying nodes for the raw sensing data from their corresponding one-hop member nodes. After the cluster formation phase, joint-entropy coding technique with explicit communication (specifically, foreign coding) is applied at every CH to remove possible data redundancy (due to the spatial data correlation) for in-network aggregation. Simulations show that the network lifetime can be significantly extended using our minimum cost cluster-based approach.

A message-passing approach to min-cost distributed clustering in wireless sensor networks

Clustering is an effective approach to hierarchically organizing network topology for efficient data aggregation in wireless sensor networks (WSNs). In this paper, we present a new approach to energy-efficient, distributed clustering in WSNs using the recent modeling and computational methodology of factor graphs and message-passings. We first formulate the sensor clustering as an optimization problem that minimizes the total data transmission cost weighted by node residual energy. We then derive simplified, localized, min-sum recursive message-passing rules which can elect a near-optimal set of cluster heads. We show through simulations that the proposed algorithm quickly achieves a good approximation of the minimum cost found by a centralized algorithm, and effectively prolongs the network lifetime compared to a popular sensor clustering algorithm.