Istvan Andorko

Hardware implementation of a real-time 3D video acquisition system

An implementation of a real-time 3D video acquisition system on a single FPGA is presented. Our approach is based on the use of stereo image sensors and can display real-time 3D video when combined with shutter glasses. The 3D effect is generated based on the parallax of the stereo sensors. An overview of the system architecture is given, including some details of the FPGA implementation of actual image processing pipeline. This architecture provides the basis for a low-cost, personal 3D imaging appliance.

Proof-of-concept and evaluation of a dual function visible/NIR camera for iris authentication in smartphones

As a robust method of person authentication, iris biometrics is making its way in to consumer devices such as smartphones. Current iris image acquisition devices typically work under controlled environment and constrained acquisition conditions. In this paper, the adaption of iris biometrics for unconstrained, hand-held devices such as smartphones is investigated. A prototype device is presented with full system description. This device is equipped with a single image sensor with both visible and NIR sensing capabilities. The device is analysed in terms of its optical properties and iris imaging capabilities. Preliminary results indicate that there are challenges to achieve a reliable recognition performance from the images captured using this device. Current system acquires images with marginal optical quality and spatial resolution in an unconstrained acquisition scenario for iris recognition. Nevertheless, the analyses presented in this paper indicate a similar camera module with improved optics and sensor could combine iris biometrics with conventional front camera functions such as video call and the capture of selfie images.

FPGA based stereo imaging system with applications in computer gaming

A real-time stereo imaging system is described. It incorporates dual image acquisition chains on a single FPGA device and is able to provide real-time synchronized video output from twin CMOS imaging sensors. The platform is designed to provide hardware support for future implementations of advanced face analysis algorithms. In the context of computer gaming applications this system can provide real-time capture of a game players facial actions and expressions serving as an enabling technology for next generation gaming. Potential applications include (i) generating 3D real-time responsive game avatars and (ii) employing real-time face data for next-generation game UI.

Enhanced real-time face models from stereo imaging for gaming applications

Techniques for improved 2D active appearance face models are presented. When these are applied to stereoscopic image pairs we show that sufficient information on image depth is obtained to generate an approximate 3D face model. Two techniques are investigated, the first based on 2D+3D AAMs and the second using methods based on thin plate splines. The resulting 3D models can offer a practical real-time face model which is suitable for a range of applications in computer gaming. Due to the compact nature of AAMs these are also very suitable for use in embedded devices such as gaming peripherals!!

A dual image processing pipeline camera with CE applications

The implementation of a dual image processing pipeline camera on a state-of-art FPGA system is described. This camera can simultaneously acquire dual images of a scene and process, analyze and merge both images to enable a range of realtime image enhancement algorithms to be explored. Several examples of image enhancement algorithms implemented on the prototype camera are described.

Proposal of a universal test scene for depth map evaluation

Nowadays, a large number of depth map generation methods use additional devices, for example Infra-Red (IR) sensors, Time of Flight (ToF) cameras, etc. One of the disadvantages of these methods is, that they cannot use test images like the ones from the Middlebury database [1] to test their accuracy, for obvious reasons. We are planning to propose a universal test scene, which can be re-created by any researcher by providing a selection of universally accessible objects, scene layout measurements and test environment conditions such as light intensity and temperature.

Depth map generation and depth layer separation for information customization in computer gaming

This paper presents a system where online game players can track the reactions and body movements of others by using only a small bandwidth. A depth map generation and depth layer separation system is presented which allows the game manufacturer to send customized information to different parties. The stereo image acquisition and depth map generation system is implemented on a single FPGA device. The depth layer separation is implemented in Matlab and is currently in a test phase.

Hierarchical real-time depth map generation algorithm

A depth map generation algorithm suitable for real-time hardware implementation is proposed. Currently, there are algorithms capable of providing high quality depth maps, but due to their complexity, they cannot be used in real-time applications. The aim of the proposed algorithm is to provide depth maps at a suitable quality for dedicated embedded applications, in the same time meeting the key requirement of matching the frame rate of the image acquisition devices.