Jonathan Cook

Accurate camera calibration for off-line, video-based augmented reality

Camera tracking is a fundamental requirement for video-based augmented reality applications. The ability to accurately calculate the intrinsic and extrinsic camera parameters for each frame of a video sequence is essential if synthetic objects are to be integrated into the image data in a believable way. In this paper, we present an accurate and reliable approach to camera calibration for off-line video-based augmented reality applications. We first describe an improved feature tracking algorithm, based on the widely used Kanade-Lucas-Tomasi tracker. Estimates of inter-frame camera motion are used to guide tracking, greatly reducing the number of incorrectly tracked features. We then present a robust hierarchical scheme that merges sub-sequences together to form a complete projective reconstruction. Finally, we describe how RANSAC-based random sampling can be applied to the problem of self-calibration, allowing for more reliable upgrades to metric geometry. Results of applying our calibration algorithms are given for both synthetic and real data.

Rapid shadow generation in real-world lighting environments

We propose a new algorithm that uses consumer-level graphics hardware to render shadows cast by synthetic objects and a real lighting environment. This has immediate benefit for interactive Augmented Reality applications, where synthetic objects must be accurately merged with real images. We show how soft shadows cast by direct and indirect illumination sources may be generated and composited into a background image at interactive rates. We describe how the sources of light (and hence shadow) affecting each point in an image can be efficiently encoded using a hierarchical shaft-based subdivision of line-space. This subdivision is then used to determine the sources of light that are occluded by synthetic objects, and we show how the contributions from these sources may be removed from a background image using facilities available on modern graphics hardware. A trade-off may be made at run-time between shadow accuracy and rendering cost, converging towards a result that is subjectively similar to that obtained using ray-tracing based differential rendering algorithms. Examples of the proposed technique are given for a variety of different lighting environments, and the visual fidelity of images generated by our algorithm is compared to both real photographs and synthetic images generated using non-real-time techniques.

Interactive reconstruction of virtual environments from video sequences

Abstract There are many real-world applications of Virtual Reality requiring the construction of complex and accurate three-dimensional models that represent real environments. In this paper, we describe a rapid and robust semi-automatic system that allows such environments to be quickly and easily built from video sequences captured with standard consumer-level digital cameras. The system combines an automatic camera calibration algorithm with an interactive model-building phase, followed by automatic extraction and synthesis of surface textures from frames of the video sequence. The capabilities of the system are illustrated using a variety of example reconstructions.

DEVA3: architecture for a large-scale distributed virtual reality system

In this paper we present work undertaken by the Advanced Interfaces Group at the University of Manchester on the design and development of a system to support large numbers of geographically distributed users in complex, large-scale virtual environments (VEs).We shown how the problem of synchronisation in the face of network limitations is being addressed by the Deva system through the exploitation of subjectivity. Further, we present a model for flexibly describing object behaviours in the VEs.Applications of the system in use are described.

Gnu/Maverik: A Microkernel for Large-Scale Virtual Environments

This paper describes a publicly available virtual reality (VR) system, GNU/MAVERIK, which forms one component of a complete VR operating system. We give an overview of the architecture of MAVERIK, and show how it is designed to use application data in an intelligent way, via a simple, yet powerful, callback mechanism that supports an object-oriented framework of classes, objects, and methods. Examples are given to illustrate different uses of the system and typical performance levels.

Virtual reality for large-scale industrial applications

In this paper we present work undertaken by the Advanced Interfaces Group at the University of Manchester into the use of virtual reality (VR) for large-scale industrial applications, the benefits this can bring, and the problems of applying VR to large problems. We show how these problems are being addressed by the MAVERIK system under development within the group. MAVERIK is a VR system which can be tightly integrated into an existing application such that it can exploit that applications data structures, algorithms and semantics.

ICARUS: interactive reconstruction from uncalibrated image sequences

The ICARUS system is a suite of software packages, developed at the University of Manchester, that allows geometric models to be quickly and easily reconstructed from image and video sequences captured with uncalibrated digital cameras. The system combines automatic and semi-automatic camera calibration algorithms with an easy-to-use interactive model-building phase (Figure 1). Surface textures are automatically extracted from images and mapped onto the reconstructed models.

A critical approach to the use of isolated liver cells for the study of metabolic events

Although isolated cell preparations from adipose tissue had been introduced in 1964, it was not until after the publication of the paper by Berry & Friend (1969) that the use of isolated liver cells became widespread. It is true that there are earlier reports of liver cell experiments (well reviewed by Bhargava, 1968) but the problems of low yield and loss of metabolic function had proved major headaches before the advent of the collagenase (EC 3.4.24.3)-based technique. The basic principles of cell isolation still remain unchanged although several, relatively minor, modifications have been introduced (Berry, 1974; Krebs et al. 1974; Elliott et al. 1976; Seglen, 1976; Elliott, 1979). Such has been the impact of this technique that symposia are now devoted to studies with liver cells, or, for those who prefer Greek, hepatocytes (Tager et al. 1976; Harris & Cornell, 1983). There seems little doubt that the isolated cell technique is, in principle, preferable to that based on the incubation of liver slices with collagenase (Sturdee et af. 1983). Slice techniques giving cells with reasonable viability have been elaborated (Fry et af. 1976; Ash & Pogson, 1977; Bellemann et al. 1977) but it has proved possible to use perfusion even with large animal material (Lomax et al. 1983) and such inherently difficult preparations as human biopsy samples (Reese & Byard, 1981; Guguen-Guillouzo et al. 1982, 1983; Byard et al. 1983). Over the years there has been more general agreement as to which factors are important and which not in the perfusion-based procedure (see Pogson et al. 1983). What is not in doubt, however, is that it is essential that the viability of cell preparations be checked, if possible on each occasion. Viability is not easily defined, if only for the simple reason that any isolated cell must lack functions that are a consequence of cell-cell contact. A number of criteria have been used, and some of these are more critical than others. Measurement of dye exclusion is convenient (Berry & Friend, 1969balthough not as easy as it at first appears (Seglen, 1976)-but gives over-optimistic estimates of the ‘goodness’ of any cell populations. Microscopic appearance is also somewhat misleading. Assay of enzyme content and comparison with that of whole liver may give erroneous estimates because of the number of cells which ‘leak’ small solutes but not macromolecules. Perhaps the best tests of viability are based on metabolic performance, particularly in respect of the pathway or process to be studied. Rates of gluconeogenesis and protein synthesis correlate well with other findings but are

Minimising latency and maintaining consistency in distributed virtual prototyping

This paper describes a computer aided design tool for mechanical engineering applications, combining component assembly simulation, the modelling of rigid and flexible bodies and haptic interaction in a multi-user distributed virtual environment. It presents the research challenges encountered, and an architecture designed to address these.

Real-time photo-realistic augmented reality for interior design

The aim of the ARIS (Augmented Reality Image Synthesis) project is the real-time photo-realistic augmentation of a photographic image with synthetic computer generated objects. The driving application for this development is interior design where a user wishes to interactively view and manipulate new items of furniture in the context of the room for which they are intended, checking for fit and aesthetics. The techniques that are being developed could easily be applied to many other scenarios.