Daniel Meziat

Optimal motion planning by reinforcement learning in autonomous mobile vehicles

The aim of this work has been the implementation and testing in real conditions of a new algorithm based on the cell-mapping techniques and reinforcement learning methods to obtain the optimal motion planning of a vehicle considering kinematics, dynamics and obstacle constraints. The algorithm is an extension of the control adjoining cell mapping technique for learning the dynamics of the vehicle instead of using its analytical state equations. It uses a transformation of cell-to-cell mapping in order to reduce the time spent during the learning stage. Real experimental results are reported to show the satisfactory performance of the algorithm.

Accurate on-line avatar control with collision anticipation

Interactive control of a virtual character through full body movement has a wide range of applications. However, there is a need for systems that accurately reproduce the motion of a performer while accounting for surrounding obstacles. We propose an approach based on a Prioritized Inverse Kinematics constraint solver. Several markers are placed on the users body. A set of kinematic constraints make the virtual character track these markers. At the same time, we monitor the instantaneous displacements of a set of geometric primitives, called observers, attached to different parts of the virtual character. When an observer enters the influence area of an obstacle, its motion is damped by means of automatically created preventive constraints. The IK solver satisfies both maker and preventive constraints simultaneously, yielding postures of the virtual character that remain close to those of the user, while avoiding collisions with the virtual environment. Our performance measurements show the maturity of the IK technology for real-time full-body interactions.

Reaching volumes generated by means of octal trees and Cartesian constraints

We present a system to analyze the reachability of the human body. The inverse kinematics technique is employed to find which regions of space are reachable using a certain reach strategy. This information is stored in a data structure called volume approximation tree (VATree). This tree has proven itself to be an appropriate data structure for two reasons: it provides an efficient representation of the reachable volumes and it reduces the number of inverse kinematics simulations necessary for its construction. Once the VATrees are constructed for the different reach strategies, that information can be used to determine in real time which strategy is most suitable for a given reach task.

Visualizing human fatigue at joint level with the half-joint pair concept

We present a model to predict and represent human fatigue in a 3D interactive system. A fatigue model has been developed for the fatigue assessment of several joints of the human body within the static case hypothesis. The model incorporates normalized torques, joint strength and maximum holding time as parameters. Fatigue evolution is predicted taking into account how these variables evolve over time. The fatigue model is embedded within an Inverse Kinematics engine that tries to achieve user-defined goals. During the animation, the predicted fatigue level is given to the graphical system in order to visualize it around its associated joint. The current fatigue value is exploited by the fatigue model to perform a new iteration towards the goal. The traditional joint model is broken down into two half-joints that better represents the anatomic organization of motion production through two independent muscle groups. Based on this organization, we can calculate and visualize independent fatigue variables for each antagonist muscle group. This type of visualization gives an intuitive and clear feedback. Each half-joint maintains its own fatigue model and variable. The two fatigue variables are represented by means of dynamic semicircles. Visual guides as semicircle’s length and gradual color indicates fatigue evolution along time.

An electronic device that suits space research requirements for ion detection

An electronics system for low-energy cosmic ion detection has been designed and built. This instrument was designed to he shipped onboard a satellite with a mission to detect solar energetic particles (SEP) and to study the anomalous component of cosmic rays. The electronics of the instrument were verified by four functional tests. In the first of these tests, its response to well-known electronic pulses, analogous to those produced in the detectors, was checked. During the second test, the coincidence-anticoincidence performances have been verified. The third test consisted of the calibration of each detector and its corresponding electronics chain with alpha particles from a /sup 241/Am source. All these preliminary tests were carried out in an Alcala University laboratory. The final test was done with nuclear reaction fragments at the VICKSI (Van de Graaff Isochron Cyctotron Kombination fur Schwere Ionen) heavy ion accelerator facility of the Hahn Meitner Institute in Berlin. The trails proved to be successful, with satisfactory results when compared with those achieved by standard nuclear instrumentation.

Progressive Cartesian Inequality Constraints for the Inverse Kinematic Control of Articulated Chains.

We propose an Inverse Kinematic Control architecture capable of handling tasks that are expressed in terms of inequality constraints in the Cartesian space. These inequality constraints are progressive in the sense that their influence manifests itself in a zone of finite thickness by damping the progression toward the strict limit of the constraint. We show how to enforce this family of constraints in a two stage process with our prioritized IK sheme. Various examples highlight the potential of this approach for managing complex articulated chains in cluttered environments where obstacles are modelled with this type of constraints.

Detector system for low-energy cosmic ions study

Abstract A low-energy cosmic ion detector system composed of a telescope and its amplification electronics has been designed and constructed. The detector system is able to detect ions from hydrogen to iron in the energy range of 1–50 MeV/nucleon. The amplification electronics has been designed using space components so that its weight, dimensions and power consumption would be small enough to allow the telescope to be used for cosmic ion detection in space aboard a satellite. The system was calibrated in a heavy ion accelerator, and the results show good charge and mass discrimination for the registered ions as well as a good response from the amplification electronics.

Optimal control for Wheeled Mobile Vehicles based on Cell Mapping techniques

The aim of this paper has been to integrate the kinematics and dynamic spaces in only one mathematical model, suitable to employ the cell-mapping technique in order to obtain the minimum-time solution to the optimal motion planning of a wheeled mobile vehicle. Through transformation of the cell-to-cell transitions the time spent in the knowledge of the vehicle dynamics and environment has been notably reduced. Four state variables have been considered: the velocity of the vehicle, X and Y Cartesian coordinates and the orientation of the vehicle. Also, two different control actions can act on the vehicle: the traction torque used for speeding up/braking the vehicle and the steering angle. The results show the applicability of the proposed algorithm in environments with presence of obstacles.

Optimal Control Applied to Wheeled Mobile Vehicles

The goal of the work described in this paper is to develop a particular optimal control technique based on a Cell-Mapping technique in combination with the Q-learning reinforcement learning method to control wheeled mobile vehicles. This approach manages 4 state variables due to a dynamic model is performed instead of a kinematics model which can be done with less variables. This new solution can be applied to non-linear continuous systems where reinforcement learning methods have multiple constraints. Emphasis is given to the new combination of techniques, which applied to optimal control problems produce satisfactory results. The proposed algorithm is very robust to any change involved in the vehicle parameters because the vehicle model is estimated in real time from received experience.

Bringing the human arm reachable space to a virtual environment for its analysis

Human reaching in a 3D environment is an interesting matter of research due to its application to workplace or vehicle-interior design. We introduce a 3D environment where a virtual human performs reaching tasks over 3D objects in the world. This environment also provides tools to generate and visualize reachable volumes. Reachable spaces are approximated using adjacent box-shaped voxels. We define several strategies in order to model different types of reaching, and we employ our system to construct and analyze reachable spaces for these strategies. In general, different strategies do have reachable spaces that share a common region of intersection. Therefore, goals exists that can be reached using two or more strategies. For those goals, a high-level layer is responsible for selecting the most appropriate given a certain reaching task. As a practical application, this paper presents a comparison of two usual strategies to model standing and seated reaching. The generated reach spaces show that, for each of them, a strategy is clearly more adequate than the other.