Mariusz Oszust

Recognition of signed expressions observed by Kinect Sensor

In this paper we present an approach to recognition of signed expressions based on visual sequences obtained with Kinect sensor. Two variants of time series representing the expressions are considered: the first based on skeletal images of the body, and the second describing shape and position of hands extracted as skin coloured regions. Time series characterising isolated Polish sign language words are examined using three clustering algorithms and popular clustering quality indices which reveal natural gesture data division and indicate gesture samples difficult in further recognition. Ten-fold cross-validation recognition tests for the k-nearest neighbour classifier with dynamic time warping technique are shown. Recognition rate obtained with the skeletal image based features were improved from 89% to 95% by changing gesture representation from time series to a vector containing pairwise distances between gesture samples. The approach with skin colour based features involving utilisation of depth information of each pixel obtained by Kinect yielded 98% recognition rate.

Polish sign language words recognition with Kinect

The paper considers Polish sign language (PSL) words recognition with sensor Kinect. The nearest neighbour classifier with dynamic time warping technique was examined. The classifier was using two sets of features, the first produced by Kinect (in a form of 3D positions of most important joints of observed person body - a skeletal image or a skeleton) and the second describing hands, which were tracked as skin colour regions in the images acquired by Kinect. Obtained feature vectors representing PSL words are clustered in order to discover natural data divisions among them. This step is helpful in revealing possible problems which could occur during recognition and provides additional information about processed data. Results of ten-fold cross-validation tests for the classifier with two feature sets are given. The classifier with data provided by Kinect yielded promising recognition accuracy (89.33%) but in order to obtain better results (98.33%) adding new features responsible for hand shape description should be considered.

Decision Fusion for Image Quality Assessment using an Optimization Approach

The proliferation of electronic means of communication entails distortion of visual information carried by processed images. Therefore, automatic evaluation of image perceptual quality in a way that is consistent with human perception is important. In this letter, an approach to full-reference image quality assessment (IQA) is proposed. The perceptual quality of the image is evaluated using an aggregated decision of several IQA measures. An optimization problem of designing a decision fusion of 18 IQA measures is defined and solved using a genetic algorithm. Obtained fusion strategies are evaluated on widely used large image benchmarks and compared with 32 state-of-the-art IQA approaches. Results of comparison reveal that the proposed approach outperforms other competing techniques.

Full-Reference Image Quality Assessment with Linear Combination of Genetically Selected Quality Measures.

Information carried by an image can be distorted due to different image processing steps introduced by different electronic means of storage and communication. Therefore, development of algorithms which can automatically assess a quality of the image in a way that is consistent with human evaluation is important. In this paper, an approach to image quality assessment (IQA) is proposed in which the quality of a given image is evaluated jointly by several IQA approaches. At first, in order to obtain such joint models, an optimisation problem of IQA measures aggregation is defined, where a weighted sum of their outputs, i.e., objective scores, is used as the aggregation operator. Then, the weight of each measure is considered as a decision variable in a problem of minimisation of root mean square error between obtained objective scores and subjective scores. Subjective scores reflect ground-truth and involve evaluation of images by human observers. The optimisation problem is solved using a genetic algorithm, which also selects suitable measures used in aggregation. Obtained multimeasures are evaluated on four largest widely used image benchmarks and compared against state-of-the-art full-reference IQA approaches. Results of comparison reveal that the proposed approach outperforms other competing measures.

Hand Gesture Recognition Using Time-of-Flight Camera and Viewpoint Feature Histogram

Time-of-flight (ToF) cameras acquire 3D information about observed scenes. They are increasingly used for hand gesture recognition. This paper is also related to this problem. In contrast to other works which try to segment the hands we propose using point cloud processing and the Viewpoint Feature Histogram (VFH) as the global descriptor of the scene. To empower the distinctiveness of the descriptor a modification is proposed which consists in dividing the work space into smaller cells and calculating the VFH for each of them. The method is applied to five sample static gestures which are relatively difficult to recognise because hands are not the objects nearest the camera and/or touch each other, touch the head or appear in the background of the face. Results of ten-fold cross validation that justify the proposed approach are given.

Recognition of Hand Gestures Observed by Depth Cameras

We focus on gesture recognition based on 3D information in the form of a point cloud of the observed scene. A descriptor of the scene is built on the basis of a Viewpoint Feature Histogram (VFH). To increase the distinctiveness of the descriptor the scene is divided into smaller 3D cells and VFH is calculated for each of them. A verification of the method on publicly available Polish and American sign language datasets containing dynamic gestures as well as hand postures acquired by a time-of-flight (ToF) camera or Kinect is presented. Results of cross-validation test are given. Hand postures are recognized using a nearest neighbour classifier with city-block distance. For dynamic gestures two types of classifiers are applied: (i) the nearest neighbour technique with dynamic time warping and (ii) hidden Markov models. The results confirm the usefulness of our approach.

Using Computer Graphics, Vision and Gesture Recognition Tools for Building Interactive Systems Supporting Therapy of Children

The paper presents a prototype of a system which can be used as a therapeutic and educational tool for children with developmental problems. Natural body movements and gestures are used in the system to interact with virtual objects displayed on the screen. Nowadays such systems can be built with the use of widely available free software tools for both graphical and vision applications. Such tools are also shortly presented in the paper.

Image quality assessment with lasso regression and pairwise score differences

The reception of multimedia applications often depends on the quality of processed and displayed visual content. This is the main reason for the development of automatic image quality assessment (IQA) techniques which try to mimic properties of human visual system and produce objective scores for evaluated images. Most of them require a training step in which subjective scores, obtained in tests with human subjects, are used for parameters tuning. In this paper, it is shown that pairwise score differences (PSD) can be successfully used for training a full-reference hybrid IQA measure based on the least absolute shrinkage and selection operator (lasso) regression. The results of extensive experimental evaluation on four largest IQA benchmarks show that the proposed IQA technique is statistically better than its version trained using raw scores, and both approaches are statistically better than state-of-the-art full-reference IQA measures. They are also better than other hybrid approaches. In the paper, the evaluation protocol is extended with tests using PSD.

Towards binary robust fast features using the comparison of pixel blocks

Binary descriptors have become popular in many vision-based applications, as a fast and efficient replacement of floating point, heavy counterparts. They achieve a short computation time and low memory footprint due to many simplifications. Consequently, their robustness against a variety of image transformations is lowered, since they rely on pairwise pixel intensity comparisons. This observation has led to the emergence of techniques performing tests on intensities of predefined pixel regions. These approaches, despite a visible improvement in the quality of the obtained results, suffer from a long computation time, and their patch partitioning strategies produce long binary strings requiring the use of salient bit detection techniques. In this paper, a novel binary descriptor is proposed to address these shortcomings. The approach selects image patches around a keypoint, divides them into a small number of pixel blocks and performs binary tests on gradients which are determined for the blocks. The size of each patch depends on the keypoints scale. The robustness and distinctiveness of the descriptor are evaluated according to five demanding image benchmarks. The experimental results show that the proposed approach is faster to compute, produces a short binary string and offers a better performance than state-of-the-art binary and floating point descriptors.

No-Reference Image Quality Assessment Using Image Statistics and Robust Feature Descriptors

The aim of no-reference image quality assessment (NR-IQA) techniques is to measure the perceptual quality of an image without access to the reference image. In this letter, a novel NR-IQA measure is introduced in which quality-aware statistics are used as perceptual features for the quality prediction. In the method, the distorted image is converted to grayscale and filtered using gradient operators. Then, the speeded-up robust feature (SURF) technique is employed to detect and describe keypoints in obtained images. The SURF interest point detection method is affected by distortions in the filtered image. Therefore, it can be used to reflect the decreased attention of the human visual system caused by image distortions. In the method, statistics are calculated for processed images and their SURF descriptors. Finally, they are mapped into subjective opinion scores using a support vector regression technique. The experimental evaluation conducted on four demanding large benchmark datasets, which contain images corrupted by single and multiple distortions, demonstrates that the proposed technique outperforms the state-of-the-art NR measures.