Mark Fiala

ARTag, a fiducial marker system using digital techniques

Fiducial marker systems consist of patterns that are mounted in the environment and automatically detected in digital camera images using an accompanying detection algorithm. They are useful for augmented reality (AR), robot navigation, and general applications where the relative pose between a camera and object is required. Important parameters for such marker systems is their false detection rate (false positive rate), their inter-marker confusion rate, minimal detection size (in pixels) and immunity to lighting variation. ARTag is a marker system that uses digital coding theory to get a very low false positive and inter-marker confusion rate with a small required marker size, employing an edge linking method to give robust lighting variation immunity. ARTag markers are bi-tonal planar patterns containing a unique ID number encoded with robust digital techniques of checksums and forward error correction (FEC). This proposed new system, ARTag has very low and numerically quantifiable error rates, does not require a grey scale threshold as does other marker systems, and can encode up to 2002 different unique IDs with no need to store patterns. Experimental results are shown validating this system.

Comparing ARTag and ARToolkit Plus fiducial marker systems

Fiducial marker systems are systems of unique patterns and computer vision algorithms that help solve the correspondence problem, automatically finding features in different camera images that belong to the same object point in the world. Fiducial marker systems consist of patterns that are mounted in the environment and automatically detected in digital images using an accompanying detection algorithm, useful for augmented reality (AR), robot navigation, 3D modeling, and other applications. This paper compares the two recently developed systems ARTag and ARToolkit Plus on their reliability, detection rates, and immunity to lighting and occlusion. Processing in fiducial systems are defined as two stages, unique feature detection and verification/identification. The systems are compared considering these stages, experimental results are shown.

Self-identifying patterns for plane-based camera calibration

Determining camera calibration parameters is a time-consuming task despite the availability of calibration algorithms and software. A set of correspondences between points on the calibration target and the camera image(s) must be found, usually a manual or manually guided process. Most calibration tools assume that the correspondences are already found. We present a system which allows a camera to be calibrated merely by passing it in front of a panel of self-identifying patterns. This calibration scheme uses an array of fiducial markers which are detected with a high degree of confidence, each detected marker provides one or four correspondence points. Experiments were performed calibrating several cameras in a short period of time with no manual intervention. This marker-based calibration system was compared to one using the OpenCV chessboard grid finder which also finds correspondences automatically. We show how our new marker-based system more robustly finds the calibration pattern and how it provides more accurate intrinsic camera parameters.

Automatic Projector Calibration Using Self-Identifying Patterns

Calibrating multiple monitor or projector display elements to provide a composite image can be a time-consuming task if performed manually. Ideally the user would like to roughly aim a number of projectors at a surface, define the desired display corners, and have some automatic method to align the display. A digital camera and computer vision can be used to calibrate the projectors with the assistance of self-identifying patterns. To account for distortion effects and to equalize brightness, it is desirable to know the mapping of many points within each projector image. A small set of images can be projected from each display element if a self-identifying pattern is used. An array of ARTag markers are used as a self-identifying pattern which is displayed in turn by each of the display monitors or projectors and recognized in the camera image. In this way an ad-hoc arrangement of projectors can be calibrated in seconds. Experimental results are shown validating this architecture.

Pano-presence for teleoperation

Telepresence and teleoperation are immersive viewing and control by a user from a remote location. Usual implementations use a standard narrow field of view (FOV) camera and a communications link, and a head mounted display (HMD). Teleoperating a robotic vehicle or surveying a scene with such system and a computer monitor is difficult for human operators due to the narrow FOV of standard cameras, the unintuitive interface for directing the camera, and the loss of directional sense. For this reason these systems often use a head mounted display (HMD) instead of a monitor, however this introduces the HMD pose latency problem of latency and slow update due to the mechanical motion of the camera and the communications link. Even with good equipment, the experience is disorienting and slow. This paper proposes a pano-presence architecture for telepresence for applications such as teleoperation based on panoramic cameras, a communications link, and an HMD. The panoramic camera, capable of capturing light from all azimuth directions, provides a panorama which is be transported over the communications link to a panorama frame buffer for viewing in the HMD screen(s). The panorama viewing rate is decoupled from the communications latency so the user can look around freely without experiencing HMD pose latency problem, the delay in the HMDs image alignment with the head position. A panorama frame format of an image cube is chosen since it can be viewed at full frame rate with the acceleration in consumer graphics cards. Two prototype systems, one telepresence and one teleoperation, using this architecture are described.

Vision guided control of multiple robots

A vision based system is presented for controlling multiple robot platforms in real time using imagery from a top view video camera. This article is a system paper that demonstrates the advantages of computer vision for robot control. A control loop using vision feedback allows creation of a robust working system while greatly reducing the complexity of the rest of the system. Planar marker pattern marker detection is an example of computer vision that is robust enough with today¿s technology for industry, and is employed for locating the robots with passive vision. Image processing in software obviates the need for complex systems for odometry, sonar, etc normally found in robotics. The control of two robotic platforms is achieved with the simplest possible hardware, with computer vision as the only feedback in the system. A system is shown and described where a user can choose desired locations for the robots, which are guided around obstacles to their destinations using ARToolkit marker patterns detected in imagery from a stationary digital video camera.

Automatic alignment and graph map building of panoramas

Panoramic cameras can capture a 360/spl deg/ view from a point providing new capabilities for multimedia, tele-presence and robotic applications. For example, virtual walk-throughs of an environment can be created from a sequence of panoramic images, where perspective views are created according to a users position and view direction. For this and other applications, the panoramic images need to be aligned to one another and a topological or metric map created. An automatic method to achieve this would remove a lot of tedious preparations for multimedia systems and enable robotic positioning systems. This paper presents three methods to address these problems; finding the relative orientation between panoramas, using the essential matrix is created to determine the relative rotation and translation direction, and an image search based algorithm to detect when the camera path crosses over itself for creating a topological map. The SIFT feature detector is used to find correspondences between panoramic images. Experimental results are shown for determining the rotation and cross-overs.

Magic Mirror System with Hand-held and Wearable Augmentations

A magic mirror paradigm is an augmented reality (AR) system where a camera and display device act as a mirror where one can see a reflection of oneself and virtual objects together. Fiducial markers mounted on a number of hand held and wearable objects allow them to be recognized by computer vision, different virtual objects can be rendered relative to the objects depending on the chosen theme. The experience can be enjoyed by many onlookers without special equipment, unlike other AR experiences such as with HMDs or tablet PCs. A series of theoretical and practical problems were overcome to produce a working system suitable for educational and entertainment for the public

The SQUASH 1000 tangible user interface system

A hand-held object whose pose can be determined automatically is useful for augmented reality (AR) and other applications needing human-computer interaction. Some existing such input devices use active (ex. LED lighting) or passive markers to be recognized in a video image by computer vision. Markers are typically mounted onto flat objects which are not ergonomic, or can only have limited number of sides due to the small library and inter-marker confusion rate of the marker system used. A system is presented based on the ARTag fiducial marker system where objects of arbitrary shape can be covered with many small markers, the pose of the object is recovered automatically and can be used as an input device. Other tangible user interface systems require specialized hardware, whereas this approach needs only a printer, a video camera or Webcam, and a large garden vegetable. The utility of this system both for measuring pose and for 3D modeling is shown.

Linear markers for robot navigation with panoramic vision

A vision based navigation system is presented for determining a mobile robots position and orientation using panoramic imagery. An omni-directional image sensor mounted on the robot is useful in obtaining a 360 field of view, permitting navigational markers from all sides to be viewed simultaneously. A robust marker-based system is presented using vertically positioned linear markers as landmarks. The markers consist of linearly encoded digital patterns, similar to a barcode but distinguishable with less pixels. A set of patterns are orthogonal from one another and are readily recognized with any continous section visible. With a vertically posed panoramic image sensor, these vertically mounted linear markers appear along radial lines. The panoramic image is pre-processed according to edge directions to find candidate regions which are spatially sampled into digital symbols. This extracted binary sequence is examined to determine if it belongs in the marker pattern set. This system is shown to be robust even with the low resolution of a panoramic sensor with 800x800 active pixels. Experiments are shown with synthetic imagery and with three real prototype systems.i