James Fung

Computer vision signal processing on graphics processing units

This paper shows speedups attained by using computer graphics hardware for implementation of computer vision algorithms by efficiently mapping mathematical operations of computer vision onto modem computer graphics architecture. As an example computer vision algorithm, we implement a real-time projective camera motion tracking routine on modern, GeForce FX class GPUs (graphics processing units). Algorithms are implemented using OpenGL and the nVIDIA Cg fragment shaders. Trade-offs between computer vision requirements and GPU resources are discussed. Algorithm implementation is examined closely, and hardware bottlenecks are addressed to examine the performance of GPU architecture for computer vision. It is shown that significant speedups can be achieved, while leaving the CPU free for other signal processing tasks. Applications of our work include wearable, computer mediated reality systems that use both computer vision and computer graphics, and require realtime processing with low-latency and high throughput provided by modem GPUs.

Using multiple graphics cards as a general purpose parallel computer: applications to computer vision

Pattern recognition and computer vision tasks are computationally intensive, repetitive, and often exceed the capabilities of the CPU, leaving little time for higher level tasks. We present a novel computer architecture which uses multiple commodity computer graphics devices to perform pattern recognition and computer vision tasks many times faster than the CPU. This is a parallel computing architecture that is quickly and easily constructed from the readily available hardware. It is based on parallel processing done on multiple graphics processing units (GPUs). An eigenspace image recognition approach is implemented on this parallel graphics architecture. This paper discusses methods of mapping computer vision algorithms to run efficiently on multiple graphics devices to maximally utilize the underlying graphics hardware. The additional memory and memory bandwidth provided by the graphics hardware provided for significant speedup of the eigenspace approach. We show that graphics devices parallelize well and provide significant speedup over a CPU implementation, providing an immediately constructible low cost architecture well suited for pattern recognition and computer vision.

EyeTap devices for augmented, deliberately diminished, or otherwise altered visual perception of rigid planar patches of real-world scenes

Diminished reality is as important as augmented reality, and both are possible with a device called the Reality Mediator. Over the past two decades, we have designed, built, worn, and tested many different embodiments of this device in the context of wearable computing. Incorporated into the Reality Mediator is an EyeTap system, which is a device that quantifies and resynthesizes light that would otherwise pass through one or both lenses of the eye(s) of a wearer. The functional principles of EyeTap devices are discussed, in detail. The EyeTap diverts into a spatial measurement system at least a portion of light that would otherwise pass through the center of projection of at least one lens of an eye of a wearer. The Reality Mediator has at least one mode of operation in which it reconstructs these rays of light, under the control of a wearable computer system. The computer system then uses new results in algebraic projective geometry and comparametric equations to perform head tracking, as well as to track motion of rigid planar patches present in the scene. We describe how our tracking algorithm allows an EyeTap to alter the light from a particular portion of the scene to give rise to a computer-controlled, selectively mediated reality. An important difference between mediated reality and augmented reality includes the ability to not just augment but also deliberately diminish or otherwise alter the visual perception of reality. For example, diminished reality allows additional information to be inserted without causing the user to experience information overload. Our tracking algorithm also takes into account the effects of automatic gain control, by performing motion estimation in both spatial as well as tonal motion coordinates.

Optimal preventive replacement, lot sizing and inspection policy for a deteriorating production system

Considers an economic manufacturing quantity (EMQ) model with an unreliable production facility and a production process subject to random deterioration. Notes that the shift of the process to the “out‐of‐control” state, which may result in producing defective items, is recognized only through inspections; and that the production unit can be replaced preventively or overhauled after finishing a certain number of production runs. Proposes that the objective is to determine the lot size, inspection interval and a preventive replacement time minimizing the expected average cost per unit of time. Obtains the formula for the expected average cost for a generally distributed time to failure. Presents computational results and studies the joint effect of process deterioration and machine breakdowns on the optimal policy.

Exploring design through wearable computing art(ifacts)

Usability is taken into account in design, however analysis of underlying technological values (such as trust, privacy, security) might become overlooked. In this paper, we illustrate how performance art can be used to elicit information about device design and usage. Wearable computing devices or art(ifacts) were used to spark behavior and debate. It was found that the degree of acceptability of the design was related to the perceived control the wearer had over the device. We suggest that what is learned from performance art can be incorporated into future design.

A parallel mediated reality platform

Realtime image processing provides a general framework for robust mediated reality problems. This paper presents a realtime mediated reality system that is built upon realtime image processing algorithms. It has been shown that the graphics processing unit (GPU) is capable of efficiently performing image processing tasks. The system presented uses a parallel GPU architecture for image processing that enables realtime mediated reality. Our implementation has many benefits; the graphics hardware has high throughput and low latency; the GPUs are not prone to jitter. Additionally, the CPU is kept available for user applications. The system is easily constructed, consisting of readily available commodity hardware.

An inspection model with generally distributed restoration and repair times

Abstract In classical inspection models, it is assumed that the inspection time and the restoration time are either zero or fixed, and the production facility never breaks down. However, in real production, the production system is subject to random failures and the repair and restoration times are usually random. In this paper, we will study the effect of the exponential failure time and generally distributed restoration and repair time on the optimal inspection interval of a production unit subject to deterioration. Treating the process as a semi-regenerative process (SRP) and analyzing the SRP by Markov renewal theory, the formula for the long-run expected average cost per unit time and formulae for the steady-state probabilities of the SRP are obtained in an explicit form. The optimal inspection interval is obtained by minimizing the average cost function.

Design of quadratic filters based on the D norm for seismic deconvolution

In this paper, quadratic filters are designed for seismic deconvolution. These nonlinear filters can be easily obtained from optimizing the D norm for the Minimum Entropy Deconvolution (MED) algorithm. Due to the symmetric structure of the quadratic filter, the actual number of filter coefficients is reduced. In finding the filter coefficients, a system of linear equations is obtained. Recently, efficient methods to realise these quadratic filters have been reported. Simulation results show that low order quadratic filters are usually sufficient and that the filtered output indicates the seismic events even when the seismic wavelet is nonminimum phase. When the data are contaminated with noise, quadratic filters can still perform reasonably well. In the case of a complex reflectivity series, nonlinear filtering is superior to linear filtering. Examples of processing synthetic seismic data by linear and nonlinear filtering are presented.