M. Isabel Ribeiro

An analysis of reinforcement learning with function approximation

We address the problem of computing the optimal Q-function in Markov decision problems with infinite state-space. We analyze the convergence properties of several variations of Q-learning when combined with function approximation, extending the analysis of TD-learning in (Tsitsiklis & Van Roy, 1996a) to stochastic control settings. We identify conditions under which such approximate methods converge with probability 1. We conclude with a brief discussion on the general applicability of our results and compare them with several related works.

3D environment modelling using laser range sensing

This paper describes a technique for constructing a geometric model of an unknown environment based on data acquired by a Laser Range Finder on board of a mobile robot. The geometric model would be most useful both for navigation and verification purposes. The paper presents all the steps needed for the description of the environment, including the range image acquisition and processing, 3D surface reconstruction and the problem of merging multiple images in order to obtain a complete model.

Q-learning with linear function approximation

In this paper, we analyze the convergence of Q-learning with linear function approximation. We identify a set of conditions that implies the convergence of this method with probability 1, when a fixed learning policy is used. We discuss the differences and similarities between our results and those obtained in several related works. We also discuss the applicability of this method when a changing policy is used. Finally, we describe the applicability of this approximate method in partially observable scenarios.

RAPOSA: Semi-Autonomous Robot for Rescue Operations

This work describes a semi-autonomous robot for rescue operations, nicknamed RAPOSA (FOX in English). The robot was designed and built to operate in outdoor environments hostile to the human presence, such as debris resulting from the collapse of built structures, and is targeted to the tele-operated detection of potential survivors using a set of specific sensors whose information is transmitted to a remote human operator. RAPOSAs mechanical structure is composed of a main body and a front body, whose locomotion is supported on tracked wheels, allowing motion even when the robot is upside down. The front body has variable tilting capabilities, providing means to overcome edges higher than the robot main body (e.g., when climbing a stair) and is also useful to grab the lower ground when only the main body has ground contact. This front body has one thermal camera and two web cameras installed. Additional sensors include gas, temperature and humidity sensors, Web cams, light diodes, microphone and loudspeaker. The robot uses wireless communications, with an option for tethered operation. The tether carries both power and communications, with an access point on its end, and can also be used to suspend the robot inside a deep hole. Docking and undocking the robot to the tether is accomplished remotely by the operator with the help of a camera located inside the robot, and represents the most innovative feature of RAPOSA

A search and rescue robot with tele‐operated tether docking system

Purpose – To describe a robot designed and built to operate in outdoor environments hostile to the human presence, such as debris resulting from the collapse of built structures, and targeted to the tele‐operated detection of potential survivors using a set of specific sensors whose information is transmitted to a remote human operator.Design/methodology/approach – RAPOSAs mechanical structure is composed of a main body and a front body, whose locomotion is supported on tracked wheels, allowing motion even when the robot is upside down. The front body has variable tilting capabilities, providing means to overcome edges higher than the robot main body (e.g. when climbing a stair) and is also useful to grab the lower ground when only the main body has ground contact. This front body has one thermal camera and two webcameras installed. Additional sensors include gas, temperature and humidity sensors, web cams, light diodes, microphone and loudspeaker. The robot uses wireless communications, with an option f...

Trajectory optimization for autonomous mobile robots in ITER

The Cask and Plug Remote Handling System (CPRHS) is one of the remote handling systems that will operate in the International Thermonuclear Experimental Reactor (ITER), transporting heavy and highly activated in-vessel components between the Tokamak Building and the Hot Cell Building, the two main buildings of the nuclear facility. The CPRHS has similar dimensions as an autobus, maximum weight of 100 tons, with kinematics of a rhombic-like vehicle (two drivable and steerable wheels) and has to move in cluttered environments. Two main approaches for trajectory optimization were developed and implemented aiming at providing smooth paths that maximize the clearance to obstacles taking into account the flexibility of rhombic-like vehicles: line guidance (same path for both wheels) and free roaming (different paths for each wheel). The line guidance approach includes maneuvers when necessary and the ability of maximizing the common parts of different paths used in the most of the nominal operations. Free roaming is mainly used when line guidance is not possible, namely in rescue operations. Both approaches were implemented in a standalone application that receives 2D CAD models of the buildings and returns the best trajectories, including a report of the most risky points of collision and the swept volume of the vehicle along the missions. This paper also presents the main results of these approaches applied in the models of the real scenarios, crucial to proceed with the construction of the Tokamak Building. Conclusions and future work are presented and discussed. Path planning for mobile robots in cluttered environments with high safety constrains as in nuclear facilities like ITER.Line guidance and free roaming path planning methodologies for vehicles with rhombic-like kinematics.Integration of multiple maneuvers in path planning.Methodology that maximizes the common parts of different trajectories.

A Nested-Loop Architecture for Mobile Robot Navigation

This paper describes a navigation architecture for mobile robots, structured as a set of nested control loops whose depth is related to their knowledge of the environment and the ability to drive the actuator, and involving as well competing behaviors that will ultimately generate the robot motion. The architecture has been successfully used on a mobile platform to support three-dimensional environment reconstruction tasks. The architecture may be classified as belonging to the hybrid type but made up of hybrid elements as well, allowing virtually any level of input awareness and ranging from high-level task planning to direct motion commands issued by external user or applications. A monitorized data path ensures the construction of the most adequate and safe motion, as well as an unlimited set of behaviors depending on the already known and perceived environment. Added concepts of path recovering and assisted navigation fulfill the demands for the three-dimensional acquisition scheme involved. Some comparison with existing architectures is carried out throughout the text. The versatility and robustness of the architecture are supported by extensive results.

Path optimization for rhombic-like vehicles: An approach based on rigid body dynamics

This paper proposes a new path optimization method to improve feasibility and optimality of rough collision-free paths provided by global planners. Inspired on the dynamics of rigid bodies, this method redefines the elastic bands framework to explicitly handle vehicle geometric constraints and explore the high maneuvering ability of rhombic vehicles. The proposed method was tested as a post optimization technique on global path solutions planned for a large transporter that will operate in the International Thermonuclear Experimental Reactor (ITER). Presented results show the proficiency of this method on handling feasible and reliable paths in cluttered scenarios such as those in ITER.

An Autonomous Sensor for 3D Reconstruction

We describe an automated approach to the reconstruction of 3D interiors from laser range data and digital images. This is achieved using a scanning laser rangefinder and digital camera that are mounted on an autonomous mobile platform known as the AEST. The objective is to reproduce complete interiors that are accurate enough for surveying, virtual studio and Augmented Reality applications. The AEST selects and navigates to a series of capture points to progressively reconstruct a 3D textured model to the required degree of accuracy. Navigation and structural information is used to register the data from each new capture point relative to the partial model. The user interface is a web browser with a radio link to the AEST. Results can be viewed in a VRML window as they are obtained. The AEST has been developed in EU-ACTS project RESOLV.

Hybrid control of semi-autonomous robots

This paper presents a hybrid architecture for robot control supported on basic concepts from the geometry of Hilbert spaces and nonsmooth calculus. The architecture develops in two classic layers. A supervising layer chooses which motion strategies to apply from a set contained in an execution layer. The paper focuses mainly in the lower execution layer, motivated by the control of semi-autonomous robots. A set of experiments on the control of unicycle robots is presented.