Kian Ming Lim

Self-taught learning of a deep invariant representation for visual tracking via temporal slowness principle

Visual representation is crucial for visual tracking methodÂ?s performances. Conventionally, visual representations adopted in visual tracking rely on hand-crafted computer vision descriptors. These descriptors were developed generically without considering tracking-specific information. In this paper, we propose to learn complex-valued invariant representations from tracked sequential image patches, via strong temporal slowness constraint and stacked convolutional autoencoders. The deep slow local representations are learned offline on unlabeled data and transferred to the observational model of our proposed tracker. The proposed observational model retains old training samples to alleviate drift, and collect negative samples which are coherent with targetÂ?s motion pattern for better discriminative tracking. With the learned representation and online training samples, a logistic regression classifier is adopted to distinguish target from background, and retrained online to adapt to appearance changes. Subsequently, the observational model is integrated into a particle filter framework to perform visual tracking. Experimental results on various challenging benchmark sequences demonstrate that the proposed tracker performs favorably against several state-of-the-art trackers. HighlightsTemporal slowness principle is exploited for learning tracking representation.Learned invariant representation is decomposed into amplitude and phase features.Higher-level features are learned by stacking autoencoders convolutionally.A novel observational model to counter drift and collect relevant samples online.Tracking experiments show our method is superior to state-of-the-art trackers.

A feature covariance matrix with serial particle filter for isolated sign language recognition

A fusion of median and mode filtering for better background model.A serial particle filter that can better detect and track the object of interest.A novel covariance matrix feature for isolated sign language representation. As is widely recognized, sign language recognition is a very challenging visual recognition problem. In this paper, we propose a feature covariance matrix based serial particle filter for isolated sign language recognition. At the preprocessing stage, the fusion of the median and mode filters is employed to extract the foreground and thereby enhances hand detection. We propose to serially track the hands of the signer, as opposed to tracking both hands at the same time, to reduce the misdirection of target objects. Subsequently, the region around the tracked hands is extracted to generate the feature covariance matrix as a compact representation of the tracked hand gesture, and thereby reduce the dimensionality of the features. In addition, the proposed feature covariance matrix is able to adapt to new signs due to its ability to integrate multiple correlated features in a natural way, without any retraining process. The experimental results show that the hand trajectories as obtained through the proposed serial hand tracking are closer to the ground truth. The sign gesture recognition based on the proposed methods yields a 87.33% recognition rate for the American Sign Language. The proposed hand tracking and feature extraction methodology is an important milestone in the development of expert systems designed for sign language recognition, such as automated sign language translation systems.

Block-based histogram of optical flow for isolated sign language recognition

A normalized histogram of optical flow as a hand representation of the sign language.Block-based histogram provides spatial information and local translation invariant.Block-based histogram of optical flow enables sign language length invariance. In this paper, we propose a block-based histogram of optical flow (BHOF) to generate hand representation in sign language recognition. Optical flow of the sign language video is computed in a region centered around the location of the detected hand position. The hand patches of optical flow are segmented into M spatial blocks, where each block is a cuboid of a segment of a frame across the entire sign gesture video. The histogram of each block is then computed and normalized by its sum. The feature vector of all blocks are then concatenated as the BHOF sign gesture representation. The proposed method provides a compact scale-invariant representation of the sign language. Furthermore, block-based histogram encodes spatial information and provides local translation invariance in the extracted optical flow. Additionally, the proposed BHOF also introduces sign language length invariancy into its representation, and thereby, produce promising recognition rate in signer independent problems.

A Four Dukkha State-space Model for Hand Tracking

Abstract In this paper, we propose a hand tracking method which was inspired by the notion of the four dukkha: birth, aging, sickness and death (BASD) in Buddhism. Based on this philosophy, we formalize the hand tracking problem in the BASD framework, and apply it to hand track hand gestures in isolated sign language videos. The proposed BASD method is a novel nature-inspired computational intelligence method which is able to handle complex real-world tracking problem. The proposed BASD framework operates in a manner similar to a standard state-space model, but maintains multiple hypotheses and integrates hypothesis update and propagation mechanisms that resemble the effect of BASD. The survival of the hypothesis relies upon the strength, aging and sickness of existing hypotheses, and new hypotheses are birthed by the fittest pairs of parent hypotheses. These properties resolve the sample impoverishment problem of the particle filter. The estimated hand trajectories show promising results for the American sign language.

Interactive android-based indoor parking lot vehicle locator using QR-code

QR code has been applied in many ways from marketing products, locating promotional items on shelves, finding stores and etc. In this study, we report on an android based application development aimed to provide navigation services to locate parked vehicles in an indoor parking space of shopping malls. We utilise the motion sensor, bar code scanner function and camera function built in smartphones. This application is able to show the route from user current location to his parked vehicle based on an indoor map of the parking area stored in a database. In addition, it is also able to automatically detect users current movement based on steps calculation. A field test was conducted in a shopping mall indoor parking space to evaluate the performance of the application. In general, the application has shown promising results.

Flash it read it: A Chinese character flashcard recognizer

This paper proposes a Chinese character flashcard recognizer for kids learning purposes. Firstly, the system will capture an image of the flashcard shown to the webcam. Secondly, a preprocessing procedure based on thresholding is performed to remove noisy edges and convert the color image into binary image. Subsequently, a Principal Component Analysis based feature extraction is conducted to encode the image into a compact representation. Lastly, character recognition is performed using Euclidean distance. The kids are able to learn Chinese characters with their Pinyin and meaning using the system. From the experiments, promising recognition results are achieved.

Finger spelling recognition using neural network

Finger spelling is a way of communication by expressing words using hand signs in order to ensure deaf and dumb community can communicate with others effectively. Therefore, a system that can understand finger spelling is needed. As a result of that, this work is conducted to primarily develop a tutoring system for finger spelling. To develop a robust real-time finger spelling tutoring system, it is necessary to ensure the accuracy of the finger spelling recognition. Even though there are existing solutions available for a decade, but most of them are just focusing on improving accuracy rate without implementing their solutions as a complete tutoring system for finger spelling. Consequently, it inspires this research project to develop a tutoring system for finger spelling. Microsoft Kinect sensor is used to acquire color images and depth images of the finger spells. Depth images are used to perform segmentation on the color images. After that, the segmented images are used as input and pass into a two hidden layers backpropagation neural network for classification.