Paulo Martins Engel

Exploration method using harmonic functions

Abstract Harmonic functions provide optimal potential maps for robot navigation in a previously explored static environment. Here we investigate the performance of an algorithm for exploration based on partial updates of a harmonic potential in an occupancy grid. We consider that while the robot moves it carries along an activation window whose size is of the order of the sensor’s range. The activation window recruits grid points to participate in the potential calculation. By using simulations and experiments with the Nomad 200 robot we investigate the algorithm performance in respect to parameters such as the frequency of updates and the numerical method used to calculate the harmonic potential.

Dealing with non-stationary environments using context detection

In this paper we introduce RL-CD, a method for solving reinforcement learning problems in non-stationary environments. The method is based on a mechanism for creating, updating and selecting one among several partial models of the environment. The partial models are incrementally built according to the systems capability of making predictions regarding a given sequence of observations. We propose, formalize and show the efficiency of this method both in a simple non-stationary environment and in a noisy scenario. We show that RL-CD performs better than two standard reinforcement learning algorithms and that it has advantages over methods specifically designed to cope with non-stationarity. Finally, we present known limitations of the method and future works.

Exploratory Navigation Based on Dynamical Boundary Value Problems

The paper presents a general framework for concurrent navigation and exploration of unknown environments based on discrete potential fields that guide the robot motion. These potentials are obtained from a class of partial differential equation (PDE) problems called boundary value problems (BVP). The boundaries are generated from sensor readings and therefore they change as the robot moves. This framework corresponds to an extension of our previous work (Prestes, E., Idiart, M. A. P., Engel, P. and Trevisan, M.: Exploration technique using potential fields calculated from relaxation methods, in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2001, p. 2012; Prestes, E., Engel, P. M., Trevisan, M. and Idiart, M. A.: Exploration method using harmonic functions, Robot. Auton. Syst.40(1) (2002), 25–42). Here, we propose that a careful choice of the PDE and the boundary conditions can produce efficient exploratory behaviors in sparse and dense environments. Furthermore, we show how to extend the exploratory behavior to produce new ones by changing dynamically the boundary function (the value of the potential at the boundaries) as the exploration takes course. Our framework is validated through a series of experiments with a real robot in office environments.

Autonomous Learning Architecture for Environmental Mapping

Here we propose an architecture for an autonomous mobile agent that explores while mapping a two-dimensional environment. The map is a discretized model for the localization of obstacles, on top of which a harmonic potential field is computed. The potential field serves as a fundamental link between the modeled (discrete) space and the real (continuous) space where the agent operates. It indicates safe paths towards non-explored regions. Harmonic functions were originally used as global path planners in mobile robotics. In this paper, we extend its functionality to environment exploration. We demonstrate our idea through experimental results obtained using a Nomad 200 robot platform.

Mining Maximal Generalized Frequent Geographic Patterns with Knowledge Constraints

In frequent geographic pattern mining a large amount of patterns is well known a priori. This paper presents a novel approach for mining frequent geographic patterns without associations that are previously known as non- interesting. Geographic dependences are eliminated during the frequent set generation using prior knowledge. After the dependence elimination maximal generalized frequent sets are computed to remove redundant frequent sets. Experimental results show a significant reduction of both the number of frequent sets and the computational time for mining maximal frequent geographic patterns.

Mining frequent geographic patterns with knowledge constraints

The large amount of patterns generated by frequent pattern mining algorithms has been extensively addressed in the last few years. In geographic pattern mining, besides the large amount of patterns, many are well known geographic domain associations. Existing algorithms do not warrant the elimination of all well known geographic dependences since no prior knowledge is used for this purpose. This paper presents a two step method for mining frequent geographic patterns without associations that are previously known as non-interesting. In the first step the input space is reduced as much as possible. This is as far as we know still the most efficient method to reduce frequent patterns. In the second step, all remaining geographic dependences that can only be eliminated during the frequent set generation are removed in an efficient way. Experiments show an elimination of more than 50% of the total number of frequent patterns, and which are exactly the less interesting.

An incremental probabilistic neural network for regression and reinforcement learning tasks

This paper presents a new probabilistic neural network model, called IPNN (for Incremental Probabilistic Neural Network), which is able to learn continuously probability distributions from data flows. The proposed model is inspired by the Spechts general regression neural network, but have several improvements which makes it more suitable to be used on-line in and robotic tasks. Moreover, IPNN is able to automatically define the network structure in an incremental way, with new units added whenever necessary to represent new training data. The performed experiments shows that IPNN is very useful in regression and reinforcement learning tasks.

Rough Clustering: An Alternative to Find Meaningful Clusters by Using the Reducts from a Dataset

Rough Sets Theory has been applied to build classifiers by exploring symbolic relations in data. Indiscernibility relations combined with the concept notion, and the application of set operations, lead to knowledge discovery in an elegant and intuitive way. In this paper we argue that the indiscernibility relation has a strong appeal to be applied in clustering since itself is a sort of natural clustering in the n-dimensional space of attributes. We explore this fact to build a clustering scheme that discovers straight structures for clusters in the sub-dimensional space of the attributes. As the usual clustering process is a kind of search for concepts, the scheme here proposed provides a better description of such clusters allowing the analyst to figure out what cluster has meaning to be considered as a concept. The basic idea is to find reducts in a set of objects and apply them to any clustering procedure able to cope with discrete data. We apply the approach to a toy example of animal taxonomy in order to show its functionality.

Improving reinforcement learning with context detection

In this paper we propose a method for solving reinforcement learning problems in non-stationary environments. The basic idea is to create and simultaneously update multiple partial models of the environment dynamics. The learning mechanism is based on the detection of context changes, that is, on the detection of significant changes in the dynamics of the environment. Based on this motivation, we propose, formalize and show the efficiency of a method for detecting the current context and the associated model of prediction, as well as a method for updating each of the incrementally built models.

Exploration technique using potential fields calculated from relaxation methods

The use of relaxation methods for calculation of harmonic potentials has proved to be a powerful technique for path planning in a known environment. We show that this idea can be successfully extended to exploration of unknown environments. The potential is calculated in a partial version of the map, represented on an occupancy grid, and it indicates safe paths towards the unexplored regions. We demonstrate that a complete relaxation of the potential is not necessary to accomplish smooth performances. Furthermore, we discuss the effect of different relaxation methods in the calculation of harmonic potential.