Malik Mallem

Robust camera pose estimation using 2d fiducials tracking for real-time augmented reality systems

Augmented reality (AR) deals with the problem of dynamically and accurately align virtual objects with the real world. Among the used methods, vision-based techniques have advantages for AR applications, their registration can be very accurate, and there is no delay between the motion of real and virtual scenes. However, the downfall of these approaches is their high computational cost and lack of robustness. To address these shortcomings we propose a robust camera pose estimation method based on tracking calibrated fiducials in a known 3D environment, the camera location is dynamically computed by the Orthogonal Iteration Algorithm. Experimental results show the robustness and the effectiveness of our approach in the context of real-time AR tracking.

Active virtual guides as an apparatus for augmented reality based telemanipulation system on the Internet

In many current teleoperation architectures, remote tasks are indirectly performed by a human operator (HO) by means of a virtual environment consisting in a virtual or symbolic representation of the remote site. In order to achieve virtual tasks, the interaction of the HO and the virtual representation is monitored. Monitoring results are subsequently translated into a sequence of instructions sent to the remote robot for actual execution. This paper focuses on different strategies designed to allow a user-friendly operator interaction with the virtual representation in order to achieve complex remote tasks via the Internet. The use of active virtual guides to assist the HO in performing simple or complex tasks with enhanced performance (speed, precision and safety) is also discussed. Techniques such as virtual reality (VR), augmented reality (AR) combined with Internet-based programming facilities are investigated as part of the proposed teleoperation system named ARITI (acronym for Augmented Reality Interface for Telerobotic applications via Internet).

A performance study for camera pose estimation using visual marker based tracking

Vision-based tracking systems are widely used for augmented reality (AR) applications. Their registration can be very accurate and there is no delay between real and virtual scene. However, vision-based tracking often suffers from limited range, errors, heavy processing time and present erroneous behavior due to numerical instability. To address these shortcomings, robust method are required to overcome these problems. In this paper, we survey classic vision-based pose computations and present a method that offers increased robustness and accuracy in the context of real-time AR tracking. In this work, we aim to determine the performance of four pose estimation methods in term of errors and execution time. We developed a hybrid approach that mixes an iterative method based on the extended Kalman filter (EKF) and an analytical method with direct resolution of pose parameters computation. The direct method initializes the pose parameters of the EKF algorithm which performs an optimization of these parameters thereafter. An evaluation of the pose estimation methods was obtained using a series of tests and an experimental protocol. The analysis of results shows that our hybrid algorithm improves stability, convergence and accuracy of the pose parameters.

FOLLOW-ME: a new 3D interaction technique based on virtual guides and granularity of interaction

The implementation of an interaction between a user and a Virtual Environment (VE) in Virtual Reality (VR) may use various techniques. However, in some cases, the interaction must be very precise and comfortable for the user. In this paper, we introduce a new selection and manipulation technique called FOLLOW-ME. The model associated to this technique splits the Virtual Environment into three zones in which a specific interaction model is used: a free manipulation zone, a scaled manipulation zone and a precise manipulation zone. Each one of the three zones is characterized by a specific interaction granularity which defines the properties of the interaction in the concerned zone. This splitting is created in order to have both precision near the object to reach or to manipulate (scaled and precise manipulation zones) and to maintain a realistic and free interaction in the VE (free manipulation zone). In the precise manipulation zone, we use the concept of virtual guides in order to assist user. In this paper, we exhibit the formalization of the model associated to the FOLLOW-ME technique. Then, we both compare our technique with classical interaction techniques in the case of a task which consists in trying to reach an object in VE and give some insights about the conditions of usefulness of virtual guides in a selection task. Some preliminary experimental results are presented and discussed.

A Robust Circular Fiducial Detection Technique and Real-Time 3D Camera Tracking

In this paper a new marker-based approach is presented for 3D camera pose tracking in indoor Augmented Reality (AR). We propose to combine a circular fiducials detection technique with a particle filter to incrementally compute the camera 3D pose parameters. In order to deal with partial occlusions, we have implemented an efficient method for fitting ellipse to scattered data. So even incomplete data will always return an ellipse corresponding to the visible part of the fiducial image. The other advantage of our approach comparing to the related camera pose estimation works is its capacity to naturally discard outliers which occur because of image noises. Results from real data in an augmented reality setup are presented, demonstrating the efficiency and robustness of the proposed method.

Computer-assisted visual perception in teleoperated robotics

Due to the complexity of teleoperation tasks, human operators figure in the teleoperator perception-decisioncontrol loop. The operator needs an interactive system to handle the huge flow of data between himself and the teleoperator. The scene represented by the robot and its environment is viewed by one or more cameras. However, the video image may be degraded in extreme environments (underwater, space, etc.) or simply inadequate (2-D image). In this paper we describe the visual perception aids based of the scene, and more specifically how these are generated by the method we put forward. The system developed at the LRE superimposes a 3-D synthetic image onto the video picture, and animates the scene in real-time on the basis of sensor information feedback. The graphic image can be generated from models, if the objects are known, otherwise interactively, with the cooperation of the operator if the objects are completely unknown. Experiments show that these graphic aids improve the operators performance in task execution.

HYBRID CAMERA POSE ESTIMATION COMBINING SQUARE FIDUCIALS LOCALIZATION TECHNIQUE AND ORTHOGONAL ITERATION ALGORITHM

Camera pose estimation from video images is a fundamental problem in machine vision and Augmented Reality (AR) systems. Most developed solutions are either linear for both n points and n lines, or iterative depending on nonlinear optimization of some geometric constraints. In this paper, we first survey several existing methods and compare their performances in an AR context. Then, we present a new linear algorithm which is based on square fiducials localization technique to give a closed-form solution to the pose estimation problem, free of any initialization. We also propose an hybrid technique which combines an iterative method, in fact the orthogonal iteration (OI) algorithm, with our own closed form solution. An evaluation of the methods has shown that this hybrid pose estimation technique is accurate and robust. Numerical experiments from real data are given comparing the performances of our hybrid method with several iterative techniques, and demonstrating the efficiency of our approach.

A new formal model of collaboration by multi-agent systems

This paper presents an approach to modelling collaboration. This approach is based on multi-agent system (MAS) and computer supported collaborative work (CSCW) research areas. In fact, a formal model of multi-agent system intended to design collaborative teleoperation architectures is proposed. This proposed model is based on an approach combining astutely two research areas: MAS and CSCW. In the first area, a formal model of the MAS (Ferbers approach) is used to give us features of the agent. The second area provides us a useful approach of collaboration based on the communication, coordination and production theories. Finally the MAS is implemented using a JADE (Java agent development framework) and some evaluations taking into account execution time and collaboration rate are presented and discussed.

Virtual and augmented reality for cultural computing and heritage: a case study of virtual exploration of underwater archaeological sites (preprint)

The paper presents different issues dealing with both the preservation of cultural heritage using virtual reality (VR) and augmented reality (AR) technologies in a cultural context. While the VR/AR technologies are mentioned, the attention is paid to the 3D visualization, and 3D interaction modalities illustrated through three different demonstrators: the VR demonstrators (immersive and semi-immersive) and the AR demonstrator including tangible user interfaces. To show the benefits of the VR and AR technologies for studying and preserving cultural heritage, we investigated the visualisation and interaction with reconstructed underwater archaeological sites. The base idea behind using VR and AR techniques is to offer archaeologists and general public new insights on the reconstructed archaeological sites allowing archaeologists to study directly from within the virtual site and allowing the general public to immersively explore a realistic reconstruction of the sites. Both activities are based on the same VR engine, but drastically differ in the way they present information and exploit interaction modalities. The visualisation and interaction techniques developed through these demonstrators are the results of the ongoing dialogue between the archaeological requirements and the technological solutions developed.

On the hybrid aid-localization for outdoor augmented reality applications

In mobile outdoor augmented reality applications, accurate localization is critical to register virtual augmentations over a real scene. Vision-based approaches provide accurate localization estimates but are still too sensitive to outdoor conditions (brightness changes, occlusions, etc.). This drawback can be overcome by adding other types of sensors. In this work, we combine a GPS and an inertial sensor with a camera to provide accurate localization. We will present the calibration process and we will discuss how to quantify the 3D localization accuracy. Experimental results on real data are presented.