Pierre Payeur

Intelligent haptic sensor system for robotic manipulation

The paper discusses an intelligent sensor system developed for the haptic-control of the robotic manipulation of 3D objects. Based on a 16 /spl times/ 16 array of force sensing resistor (FSR) elements, the sensor system is able to provide an estimate of the objects surface orientation along with a fine description of features located within the sensing area.

Probabilistic octree modeling of a 3D dynamic environment

Probabilistic occupancy grids have proved to be very useful for workspace modeling in 2D environments. Due to the expansion of computational load, this approach was not tractable for mapping a 3D environment in real applications. In this paper, the original occupancy grid scheme is revisited and a generic closed-form function is introduced to avoid numerical computation of probabilities for a range sensor with Gaussian error distribution. Occupancy probabilities are computed and stored in a multiresolution octree for improved performance and compactness. Occupancy models are built in local reference frames and linked to a global reference frame through uncertain spatial relationships that can be updated dynamically. This scheme is used for building a 3D map in a telerobotic maintenance application of electric power lines where perturbations may cause motion of object assembly.

Calibration of a network of Kinect sensors for robotic inspection over a large workspace

This paper presents an approach for calibrating a network of Kinect devices used to guide robotic arms with rapidly acquired 3D models. The method takes advantage of the rapid 3D measurement technology embedded in the Kinect sensor and provides registration accuracy within the range of the depth measurements accuracy provided by this technology. The internal calibration of the sensor in between the color and depth measurement is also presented. The resulting system is developed to inspect large objects, such as vehicles, positioned within an enlarged field of view created by the network of RGB-D sensors.

Dexterous Robotic Manipulation of Deformable Objects with Multi-Sensory Feedback - a Review

Designing autonomous robotic systems able to manipulate deformable objects without human intervention constitutes a challenging area of research. The complexity of interactions between a robot manipulator and a deformable object originates from a wide range of deformation characteristics that have an impact on varying degrees of freedom. Such sophisticated interaction can only take place with the assistance of intelligent multisensory systems that combine vision data with force and tactile measurements. Hence, several issues must be considered at the robotic and sensory levels to develop genuine dexterous robotic manipulators for deformable objects. This chapter presents a thorough examination of the modern concepts developed by the robotic community related to deformable objects grasping and manipulation. Since the convention widely adopted in the literature is often to extend algorithms originally proposed for rigid objects, a comprehensive coverage on the new trends on rigid objects manipulation is initially proposed. State-of-the-art techniques on robotic interaction with deformable objects are then examined and discussed. The chapter proposes a critical evaluation of the manipulation algorithms, the instrumentation systems adopted and the examination of end-effector technologies, including dexterous robotic hands. The motivation for this review is to provide an extensive appreciation of state-of-the-art solutions to help researchers and developers determine the best possible options when designing autonomous robotic systems to interact with deformable objects. Typically in a robotic setup, when robot manipulators are programmed to perform their tasks, they must have a complete knowledge about the exact structure of the manipulated object (shape, surface texture, rigidity) and about its location in the environment (pose). For some of these tasks, the manipulator becomes in contact with the object. Hence, interaction forces and moments are developed and consequently these interaction forces and moments, as well as the position of the end-effector, must be controlled, which leads to the concept of “force controlled manipulation” (Natale, 2003). There are different control strategies used in 28

Structured Light Stereoscopic Imaging with Dynamic Pseudo-random Patterns

Structured light stereoscopic imaging offers an efficient and affordable solution to 3D modeling of objects. The majority of structured light patterns that have been proposed either provide a limited resolution or are sensitive to the inherent texture on the surface of the object. This paper proposes an innovative imaging strategy that accomplishes 3D reconstruction of objects using a combination of spatial-neighboring and time-multiplexing structured light patterns encoded with uniquely defined pseudo-random color codes. The approach is extended with the concept of dynamic patterns that adaptively increases the reconstruction resolution. Original techniques are introduced to recover and validate pseudo-random codes from stereoscopic images, and to consistently map color and texture over the reconstructed surface map. Experimental results demonstrate the potential of the solution to create reconstructions with various densities of points and prove the robustness of the approach on objects with different surface properties.

Dense Stereo Range Sensing with Marching Pseudo-Random Patterns

As an extension to classical structured lighting techniques, the use of bi-dimensional pseudo-random color codes is explored to perform range sensing with variable density from a stereo calibrated rig and a projector. Pseudo-random codes are used to create artificial textures on a scene which are extracted and grouped in a confidence map to ensure reliable feature matching between pairs of images taken from two cameras. Depth estimation is performed on corresponding points with progressive refinement as the pseudo-random pattern projection is marched over the scene to increase the density of matched features, and achieve dense 3D reconstruction. The potential of bi-dimensional pseudo-random color patterns for structured lighting is demonstrated in terms of patterns computation, ease of extraction, matching confidence level, as well as density of depth estimation for 3D reconstruction.

Preliminary results of severity of illness measures of rheumatoid arthritis using infrared imaging

For the first phase of a large project, we used an infrared imaging camera (thermograph) to obtain accurate measurements of body temperature in joints of twelve human normal subjects (control group) and for thirteen patients who had been diagnosed with rheumatoid arthritis (RA) by a rheumatologist. The ultimate goal is to create a low cost effective method to diagnose early synovitis. Temperature measurements of hands were analyzed with first order statistics. Results show significant temperature differences between control subjects and patients for every joint and hand portion measured. Future work will complete the analysis of knees, elbows, ankles, combine infrared (IR) imaging and intra-optical (IO) imaging, and incorporate feature extraction and classification approaches to stratify patients into severity of illness prior to, and after receiving treatment.

Trajectory prediction for moving objects using artificial neural networks

A method to predict the trajectory of moving objects in a robotic environment in real-time is proposed and evaluated. The position, velocity, and acceleration of the object are estimated by several neural networks using the six most recent measurements of the object coordinates as inputs. The architecture of the neural nets and the training algorithm are presented and discussed. Simulation results obtained for both 2D and 3D cases are presented to illustrate the performance of the prediction algorithm. Real-time implementation of the neural networks is considered. Finally, the potential of the proposed trajectory prediction method in various applications is discussed. >

Soft Object Deformation Monitoring and Learning for Model-Based Robotic Hand Manipulation

This paper discusses the design and implementation of a framework that automatically extracts and monitors the shape deformations of soft objects from a video sequence and maps them with force measurements with the goal of providing the necessary information to the controller of a robotic hand to ensure safe model-based deformable object manipulation. Measurements corresponding to the interaction force at the level of the fingertips and to the position of the fingertips of a three-finger robotic hand are associated with the contours of a deformed object tracked in a series of images using neural-network approaches. The resulting model captures the behavior of the object and is able to predict its behavior for previously unseen interactions without any assumption on the objects material. The availability of such models can contribute to the improvement of a robotic hand controller, therefore allowing more accurate and stable grasp while providing more elaborate manipulation capabilities for deformable objects. Experiments performed for different objects, made of various materials, reveal that the method accurately captures and predicts the objects shape deformation while the object is submitted to external forces applied by the robot fingers. The proposed method is also fast and insensitive to severe contour deformations, as well as to smooth changes in lighting, contrast, and background.

Evaluation of growing neural gas networks for selective 3D scanning

This paper addresses the issue of intelligent sensing for advanced robotic applications and is a continuation of our research in the area of innovative approaches for automatic selection of regions of observation for fixed and mobile sensors to collect only relevant measurements without human guidance. The growing neural gas network solution proposed here for adaptively selecting regions of interest for further sampling from a cloud of sparsely collected 3D measurements provides several advantages over the previously proposed neural gas solution in terms of user intervention, size of resulting scan and training time. Experimental results and comparative analysis are presented in the context of selective vision sampling.