Pekka Sangi

Vision-based motion estimation for interaction with mobile devices

This paper introduces a novel interaction technique for handheld mobile devices which enables the user interface to be controlled by the motion of the users hand. A feature-based approach is proposed for global motion estimation that exploits gradient measures for both feature selection and feature motion uncertainty analysis. A voting-based scheme is presented for outlier removal. A Kalman filter is applied for smoothing motion trajectories. A fixed-point implementation of the method was developed due to the lack of floating-point hardware. Experiments testify the effectiveness of the approach on a camera-enabled mobile phone.

Document Image Mosaicing with Mobile Phones

This paper presents a novel user interaction concept for document image scanning with mobile phones. A high resolution mosaic image is constructed in two main stages. Firstly, online camera motion estimation is applied to the phone to assist the user to capture small image patches of the document page. Automatic image stitching process with the help of estimated device motion is carried out to reconstruct the full view of the document. Experiments on document images captured and processed with mosaicing software clearly show the feasibility of the approach.

A Vision-Based Approach for Controlling User Interfaces of Mobile Devices

We introduce a novel user interface solution for mobile devices which enables the display to be controlled by the motion of the user’s hand. A feature-based approach is proposed for dominant global motion estimation that exploits gradient measures for both feature selection and motion uncertainty analysis. We also present a voting-based scheme for outlier removal. A Kalman filter is utilized for smoothing motion trajectories. A fixed-point implementation of the method was made on a mobile device platform that sets computational restrictions for the algorithms used. Experiments with synthetic and real image sequences show the effectiveness of the method and demonstrate the practicality of the approach in a smartphone.

Face Tracking for Spatially Aware Mobile User Interfaces

This paper introduces a new face tracking approach for controlling user interfaces in hand-held mobile devices. The proposed method detects the face and the eyes of the user by employing a method based on local texture features and boosting. An extended Kalman filter combines local motion features extracted from the face region and the detected eye positions to estimate the 3-D position and orientation of the camera with respect to the face. The camera position is used as an input for the spatially aware user interface. Experimental results on real image sequences captured with a camera-equipped mobile phone validate the feasibility of the method.

Adaptive motion-based gesture recognition interface for mobile phones

In this paper, we introduce a new vision based interaction technique for mobile phones. The user operates the interface by simply moving a finger in front of a camera. During these movements the finger is tracked using a method that embeds the Kalman filter and ExpectationMaximization (EM) algorithms. Finger movements are interpreted as gestures using Hidden Markov Models (HMMs). This involves first creating a generic model of the gesture and then utilizing unsupervised Maximum a Posteriori (MAP) adaptation to improve the recognition rate for a specific user. Experiments conducted on a recognition task involving simple control commands clearly demonstrate the performance of our approach.

Motion analysis using frame differences with spatial gradient measures

The paper considers making inferences about the underlying true 2-D motion when only evaluations of a local block-based cost function, the mean of absolute or squared differences, for a set of motion candidates are available. Considering bounds for these criteria, it is shown that simple local image gradient measures provide useful information for interpreting the criterion values. Based on analysis, a thresholding scheme for the criteria is proposed. Using a Gaussian approximation for the thresholding result, estimates of local motions and related uncertainties can be obtained.

Camera-Based Motion Recognition for Mobile Interaction

Multiple built-in cameras and the small size of mobile phones are underexploited assets for creating novel applications that are ideal for pocket size devices, butmay notmakemuch sense with laptops. In this paper we present two vision-basedmethods for the control of mobile user interfaces based on motion tracking and recognition. In the first case the motion is extracted by estimating the movement of the device held in the user’s hand. In the second it is produced from tracking the motion of the user’s finger in front of the device. In both alternatives sequences of motion are classified using Hidden Markov Models. The results of the classification are filtered using a likelihood ratio and the velocity entropy to reject possibly incorrect sequences. Our hypothesis here is that incorrect measurements are characterised by a higher entropy value for their velocity histogram denotingmore random movements by the user. We also show that using the same filtering criteria we can control unsupervised Maximum A Posteriori adaptation. Experiments conducted on a recognition task involving simple control gestures formobile phones clearly demonstrate the potential usage of our approaches and may provide for ingredients for new user interface designs.

Selection of the Lagrange multiplier for block-based motion estimation criteria

In hybrid video coding, motion vectors used for motion compensation constitute an important set of decisions. Cost functions for block motion estimation that take the smoothness of the resulting motion vector field into account, in addition to the motion compensated prediction error, have been proposed. Computationally simple derivatives of sum of absolute differences and sum of squared differences-based criteria are studied in this paper. Cost functions are based on Lagrangian rate-distortion formulation, and the basic question is how the Lagrangian multiplier involved should be selected. Assumptions behind these cost functions are discussed, and a new method is derived for determining the multiplier. Comparisons with other strategies are made with experiments. The results show that the selection of the multiplier is not critical.

Motion-Based Handwriting Recognition for Mobile Interaction

This paper presents a new interaction technique for camera-enabled mobile devices. The handheld device can be used for writing just by moving the device. In our method, interframe dominant motion is estimated from images, and the discrete cosine transform is used for computing discriminating features from motion trajectories. The k-nearest neighbor rule is applied for classification. A realtime implementation of the method was developed for a mobile phone. In experiments, recognition rates ranging from 92 % to 98 % were achieved, which testifies to the practicality of our approach

Camera motion estimation from non-stationary scenes using EM-based motion segmentation

An algorithm for recovering 3-D camera motion from sequences of images is proposed. The algorithm has four stages. In the first stage, the motion vector field is segmented using an EM-based method. The resulting segments are compared and the coherent regions are merged in the second stage. The candidates for the background regions are determined and finally used for 3-D motion estimation in the last two stages. Unlike most of the other methods, this approach tolerates also non-rigid motion in the scene. The experiments performed show that in some cases more information or reasoning is needed for selecting plausible motion parameters from several hypotheses.