Prashan Premaratne

Hand gesture tracking and recognition system using Lucas-Kanade algorithms for control of consumer electronics

Abstract Dynamic hand gesture tracking and recognition system can simplify the way humans interact with computers and many other non-critical consumer electronic equipments. This system is based on the well-known “Wave Controller” technology developed at the University of Wollongong [1] , [2] , [3] and certainly a step forward in video gaming and consumer electronics control interfaces. Currently, computer interfacing mainly involves keyboard, mouse, joystick or gaming wheels and occasionally voice recognition for user input. These modes of interaction have constrained the artistic ability of many users, as they are required to respond to the computer through pressing buttons or moving other apparatus. Voice recognition is seen as unreliable and impractical in areas where more than one user is present. All these drawbacks can be tackled by using a reliable hand gesture tracking and recognition system based on both Lucas–Kanade and Moment Invariants approaches. This will facilitate interaction between users and computers and other consumer electronic equipments in real time. This will further enhance the user experience as users are no longer have any physical connection to the equipment being controlled. In this research, we have compared our proposed moment invariant based algorithm with template based and Fourier descriptor based methods to highlight the advantages and limitations of the proposed system.

Human Computer Interaction Using Hand Gestures

Hand gesture is a very natural form of human interaction and can be used effectively in human computer interaction (HCI). This project involves the design and implementation of a HCI using a small hand-worn wireless module with a 3-axis accelerometer as the motion sensor. The small stand-alone unit contains an accelerometer and a wireless Zigbee transceiver with microcontroller. To minimize intrusiveness to the user, the module is designed to be small (3cm by 4 cm). A time-delay neural network algorithm is developed to analyze the time series data from the 3-axis accelerometer. Power consumption is reduced by the non-continuous transmission of data and the use of low-power components, efficient algorithm and sleep mode between sampling for the wireless module. A home control interface is designed so that the user can control home appliances by moving through menus. The results demonstrate the feasibility of controlling home appliances using hand gestures and would present an opportunity for a section of the aging population and disabled people to lead a more independent life.

Intelligent Computing Theories and Application

In this paper, we proposed an algorithm based on variable neighborhood search (VNS) for the capacitated m-Ring-Star problem. This problem has several real applications in communications networks, rapid transit system planning and optical fiber networks. The problem consists in design m rings or cycles that begins of a central depot and visits a set of customers and transition or steiner nodes. While the nodes don’t belong to a ring these must be allocated or assign to a customer or steiner node that belongs to a ring. The number of customers allocated or visited in each ring must not exceed the maximum capacity. The goal is to minimize the visiting and allocation cost. For solving the problem, we propose a VNS approach based on random perturbation for escaping from the local optimal solutions. Our method reached the optimal solution in a reasonable amount of time in a set of instances from the literature.

Dynamic hand gesture recognition system using moment invariants

We have developed a dynamic hand gesture recognition system that can simplify the way humans interact with computers and many other non-critical consumer electronic equipment. The proposed system is based on the well-known “Wave Controller” technology developed at the University of Wollongong [1–3] and will revolutionize video gaming and consumer electronics control interfaces. Currently, computer interfacing mainly involves keyboard, mouse, joystick or gaming wheels and occasionally voice recognition for user input. These modes of interaction have restrained the artistic ability of many users, as they are required to respond to the computer through pressing buttons or moving other apparatus. Voice recognition is seen as unreliable and impractical in areas where more than one user is present. All these drawbacks can be tackled by using a reliable hand gesture recognition system that facilitates interaction between users and computers and other consumer electronic equipment in real time. This will further enhance the user experience as users no longer have any physical connection to the equipment being controlled. This system can also be extended to a sign language system for the benefit of the disabled including those with speech disabilities.

2D Barcodes as Watermarks in Image Authentication

2D barcodes are increasingly used as tags in every type of goods for unique identification. Compared with the 1D barcodes, 2D barcodes not only can carry more data but also can withstand errors in subsequent scans. This property has significant parallels to watermarking logos as such watermarks will withstand multiple manipulations that are common with image transactions. We use a 2D Barcode as the watermark as this has error correction capabilities and show that this can be used to insert data imperceptibly into the host image. One Barcode is inserted into the low frequency component of the image and a second Barcode watermark is embedded into low pass component of any wavelet decomposition at a specific level only known to the author. This improves the resistance of the watermarking scheme to attack. Our experimental results indicate that these invisible watermarks can carry significant information and are robust to many image manipulations.

Image matching using moment invariants

Abstract Matching images using Mean Squared Error (MSE) and Peak Signal to Noise (PSNR) ratios does not well conform to the Human Visual System (HVS). When matching two images, HVS operates both globally and locally when it identifies features of a scenery and this process is not matched adequately by PSNR or MSE. A low MSE or very high PSNR may not necessarily mean that images are similar. Similarly, when images are similar as HVS would identify, the corresponding MSE may not be very low and PSNR may not be very high. However, quite recently, a new measure has been proposed to circumvent the drawbacks of PSNR or MSE. This measure known as Structural Similarity Measure (SSIM) has received acclaim due to its ability to produce results on a par with Human Visual System. However, experimental results indicate that noise and blur seriously degrade the performance of the SSIM metric. Furthermore, despite SSIM׳s popularity, it does not provide adequate insight into how it handles ‘structural similarity’ of images. We propose a new structural similarity measure based on approximation level of a given discrete Wavelet decomposition that evaluates moment invariants to capture the structural similarity with superior results over SSIM.

Hand gesture recognition: An overview

Hand gesture recognition has been applied to many fields in recent years, especially in man-machine interaction (MMI) area, which is regarded as a more natural and flexible input. In this paper, an overview of hand gesture recognition research up to date is presented, which includes common stages of hand gesture recognition, common methods and technique of each stage, the state of the recent research and summaries of some successful hand gesture recognition models.

Australian sign language recognition using moment invariants

Human Computer Interaction is geared towards seamless human machine integration without the need for LCDs, Keyboards or Gloves. Systems have already been developed to react to limited hand gestures especially in gaming and in consumer electronics control. Yet, it is a monumental task in bridging the well-developed sign languages in different parts of the world with a machine to interpret the meaning. One reason is the sheer extent of the vocabulary used in sign language and the sequence of gestures needed to communicate different words and phrases. Auslan the Australian Sign Language is comprised of numbers, finger spelling for words used in common practice and a medical dictionary. There are 7415 words listed in Auslan website. This research article tries to implement recognition of numerals using a computer using the static hand gesture recognition system developed for consumer electronics control at the University of Wollongong in Australia. The experimental results indicate that the numbers, zero to nine can be accurately recognized with occasional errors in few gestures. The system can be further enhanced to include larger numerals using a dynamic gesture recognition system.

Intelligent Computing Theories

This article integrated rule expression capacity of fuzzy logic inference with self-learning ability of the neural network, proposed to build Takagi-Sugeno fuzzy neural network’s quantitative identification of mixed gas by combining T-S fuzzy neural network with neural network. The results indicated that this system has generalization, learning, mapping capabilities. It can better realize quantitative identification of mixed gas. This system will provide method for intelligent identification of mixed gas.

Thinking head: Towards human centred robotics

Thinking Head project is a multidisciplinary approach to building intelligent agents for human machine interaction. The Thinking Head Framework evolved out of the Thinking Head Project and it facilitates loose coupling between various components and forms the central nerve system in a multimodal perception-action system. The paper presents the overall architecture, components and the attention system. The paper then concludes with a preliminary behavioral experiment that studies the intelligibility of the audiovisual speech output produced by the Embodied Conversational Agent (ECA) that is part of the system. These results provide the baseline for future evaluations of the system as the project progresses through multiple evaluate and refine cycles.