Dehui Kong

Similarity Assessment Model for Chinese Sign Language Videos

This paper proposes a model for measuring similarity between videos which content is Chinese Sign Language (CSL), vision and sign language semantic are considered for the model. Vision component of the model is distance based on Volume Local Binary Patterns (VLBP), which is robust for motion and illumination. Semantic component of the model computes semantic distance based on definition of sign language semantic, which is defined as hand shape, location, orientation and movements. While quantizing the sign language semantic, contour is used to measure shape and orientation; trajectory is used for measuring location and movement. Experiment results show that proposed assessment model is effective and assessing result given by the model is close to subjective scoring.

Image super-resolution based on multi-space sparse representation

Sparse representation provides a new method of generating a super-resolution image from a single low resolution input image. An over-complete base for sparse representation is an essential part of such methods. However discovering the over-complete base with efficient representation from a large amount of image patches is a difficult problem. We make efforts in sparse representation and its implementation to solve the problem. In the representation, image patches are decomposed into two structure and texture components represented by the over-complete bases of their own spaces so that their high-level features can be captured by the bases. In the implementation, a prior knowledge about low resolution images generation is combined to the typical base construction for high construction quality. Finally a super-resolution construction based on multi-space sparse representation is proposed. Experiment results demonstrate that the proposed method significantly improve the PSNR and visual quality of reconstructed high-resolution image.

Synthesis of sign language co-articulation based on key frames

Co-articulation is a language phenomenon. In sign language (SL), it takes the form of impact among adjacent signs, which results in variations of signs from their standard configurations. Standard configuration is the appearance of a sign when it appears singly, without context. Without co-articulation, SL animation based on virtual character will be a simple concatenation of signs. The movement of virtual character will be mechanical, lacking fluency and realism. This paper presents a key frame based SL co-articulation animation scheme aiming at the three most important elements of co-articulation, i.e. hand shape, hand position and SL speed. To generate co-articulation, motion data of signs which appear sequentially is parsed to identify hand shapes and positions included in these signs. Then, co-articulation will be achieved through some modification to the motion data according to the interaction between adjacent hand shapes and adjacent hand positions. SL Speed acts as an adjusting parameter which dynamically impacts co-articulation. Different expression speed will lead to different degree of co-articulation.

Chinese Sign Language animation generation considering context

Sign language (SL) is a kind of natural language for the deaf. Chinese Sign Language (CSL) synthesis aims to translate text into virtual human animation, which makes information and service accessible to the deaf. Generally, sign language animation based on key frames is realized by concatenating sign words captured independently. That means a sign language word has the same pattern in diverse context, which is different from realistic sign language expression. This paper studies the effect of context on manual gesture and non-manual gesture, and presents a method for generating stylized manual gesture and non-manual gesture according to the context. Experimental results show that synthesized sign language animation considering context based on the proposed method is more accurate and intelligible than that irrespective of context.

Synthesis of Chinese Sign Language Prosody Based on Head

The movement of the head not only helps to express sign language vividly, but also has specific meaning in sign language. In the effect of context, for example, emphasize, and the different personalities, the range and speed of the movement of head will be a great change. On the analysis of the characteristic of Chinese sign language words, this article establishes a set of rules of Sign language behavior and also proposes a parametric synthesis method of changing the range of head movement which is based on thought of motion deviation mapping. And then applies it to the emphasizing the prosody model of Chinese sign language synthesis system. Experimental results show that this method will realize any emphasis prosodic expression of head movement and raise the intelligibility and sense of reality of the sign language effectively.

Making smooth transitions based on a multi-dimensional transition database for joining Chinese sign-language videos

This paper proposes a new method to generate smooth transition frames for serializing Chinese sign-language video clips. Each transition frame is composed of images of arms, hands, head and torso. Of them, the hand images are computed based on 3D hand models, and images of other parts are selected from a database. Experimental results show the method provided by this paper can improve the joined video’s visual presentation.

High-Resolution Light Field Capture With Coded Aperture

Acquiring light field with larger angular resolution and higher spatial resolution in low cost is the goal of light field capture. Combining or modifying traditional optical cameras is a usual method for designing light field capture equipment, among which most models should deliberate trade-off between angular and spatial resolution, but augmenting coded aperture avoids this consideration by multiplexing information from different views. On the basis of coded aperture, this paper suggests an improved light field camera model that has double measurements and one mask. The two compressive measurements are respectively realized by a coded aperture and a random convolution CMOS imager, the latter is used as imaging sensor of the camera. The single mask design permits high light efficiency, which enables the sampling images to have high clarity. The double measurement design keeps more correlation information, which is conductive to enhancing the reconstructed light field. The higher clarity and more correlation of samplings mean higher quality of rebuilt light field, which also means higher resolution under condition of a lower PSNR requirement for rebuilt light field. Experimental results have verified advantage of the proposed design: compared with the representative mask-based light field camera models, the proposed model has the highest reconstruction quality and a higher light efficiency.

Fast mode dependent directional transform via butterfly-style transform and integer lifting steps

Mode dependent directional transform (MDDT) can improve the coding efficiency of H.264/AVC but it also brings high computation complexity. In this paper we present a new design for implementing fast MDDT transform through integer lifting steps. We first approximate the optimal MDDT by a proper transform matrix that can be implemented with butterfly-style operation. We further factorize the butterfly-style transform into a series of integer lifting steps to eliminate the need of multiplications. Experimental results show that the proposed fast MDDT can significantly reduce the computation complexity while introducing negligible loss in the coding efficiency. Due to the merit of integer lifting steps, the proposed fast MDDT is reversible and can be implemented on hardware very easily.

A Novel Multiple-Description Image Coding on Wavelet

Multiple description coding (MDC) is a way of trading off coding gain with robustness to channel errors. This paper presents a novel and efficient two-description image coding scheme in wavelet domain. Here, the two-layered MDC method is adopted based on the feature of wavelet coefficients. Firstly, the low-frequency wavelet coefficients are partitioned into two new correlated groups as two basic descriptions with pairwise correlating transform (PCT). Secondly, we can get two enhanced descriptions from the high-frequency wavelet subbands of the highest-level decomposition by adaptive down-sampling. Finally, four template operators are adopted to detect the texture direction of high-frequency subbands, and we use the Wavelet Domain Interpolation for tree Reconstruction (WDIR) to conceal coefficients loss in Wavelet-Trees with the texture direction when one enhanced description lost during transmission. Experimental results demonstrate that the proposed MDC scheme can achieve good coding performance.

Sparse Pose Regression via Componentwise Clustering Feature Point Representation

We propose two-dimensional pose estimation from a single range image of the human body, using sparse regression with a componentwise clustering feature point representation (CCFPR) model. CCFPR includes primary feature points and secondary feature points. The primary feature points consist of the torso center and five extremal points of human body, and further serve to classify all body pixels as the points of six body components. The secondary feature points are given by the cluster centers of each of the five components other than the torso, using K-means cluster. The human pose is obtained by learning a sparse projection matrix, which maps CCFPR to the skeleton points of human body, based on the assumption that each skeleton point be represented by a combination of a few feature points of associated body components. Experimental results on both virtual data and real data show that, under the sparse regression model with a suitably selected cluster number, CCFPR outperforms the random decision forest approach and prediction results of Kinect sensor v2 .