Dimitrios S. Alexiadis

Estimation of Motions in Color Image Sequences Using Hypercomplex Fourier Transforms

Although the motion estimation problem has been extensively studied, most of the proposed estimation approaches deal mainly with monochrome videos. The most usual way to apply them also in color image sequences is to process each color channel separately. A different, more sophisticated approach is to process the color channels in a ldquoholisticrdquo manner using quaternions, as proposed by Ell and Sangwine. In this paper, we extend standard spatiotemporal Fourier-based approaches to handle color image sequences, using the hypercomplex Fourier transform. We show that translational motions are manifested as energy concentration along planes in the hypercomplex 3D Fourier domain and we describe a methodology to estimate the motions, based on this property. Furthermore, we compare the three-channels-separately approach with our approach and we show that the computational effort can be reduced by a factor of 1/3, using the hypercomplex Fourier transform. Also, we propose a simple, accompanying method to extract the moving objects in the hypercomplex Fourier domain. Our experimental results on synthetic and natural images verify our arguments throughout the paper.

Quaternionic Signal Processing Techniques for Automatic Evaluation of Dance Performances From MoCap Data

In this paper, the problem of automatic dance performance evaluation from human Motion Capture (MoCap) data is addressed. A novel framework is presented, using data captured by Kinect-based human skeleton tracking, where the evaluation of users performance is achieved against a gold-standard performance of a teacher. The framework addresses several technical challenges, including global and local temporal synchronization, spatial alignment and comparison of two “dance motion signals.” Towards the solution of these technical challenges, a set of appropriate quaternionic vector-signal processing methodologies is proposed, where the 4D (spatiotemporal) human motion data are represented as sequences of pure quaternions. Such a quaternionic representation offers several advantages, including the facts that joint angles and rotations are inherently encoded in the phase of quaternions and the three coordinates variables ( X,Y,Z) are treated jointly, with their intra-correlations being taken into account. Based on the theory of quaternions, a number of advantageous algorithms are formulated. Initially, global temporal synchronization of dance MoCap data is achieved by the use of quaternionic cross-correlations, which are invariant to rigid spatial transformations between the users. Secondly, a quaternions-based algorithm is proposed for the fast spatial alignment of dance MoCap data. Thirdly, the MoCap data can be temporally synchronized in a local fashion, using Dynamic Time Warping techniques adapted to the specific problem. Finally, a set of quaternionic correlation-based measures (scores) are proposed for evaluating and ranking the performance of a dancer. These quaternions-based scores are invariant to rigid transformations, as proved and demonstrated. A total score metric, through a weighted combination of three different metrics is proposed, where the weights are optimized using Particle Swarm Optimization (PSO). The presented experimental results using the Huawei/3DLife/EMC2 dataset are promising and verify the effectiveness of the proposed methods.

Toward Real-Time and Efficient Compression of Human Time-Varying Meshes

In this paper, a novel skeleton-based approach to human time-varying mesh (H-TVM) compression is presented. The topic of TVM compression is new and has many challenges, such as handling the lack of obvious mapping of vertices across frames and handling the variable connectivity across frames, while maintaining efficiency, which are the most important ones. Very few works exist in the literature, while not all of the challenges have been addressed yet. In addition, developing an efficient and real-time solution, handling the above, obviously is a difficult task. We attempt to address the H-TVM compression problem inspired from video coding using different types of frames and trying to efficiently remove inter-frame geometric redundancy utilizing the recent advances in human skeleton tracking. The overall approach focuses on compression efficiency, low distortion, and low computation time enabling for real-time transmission of H-TVMs. It efficiently compresses geometry and vertex attributes of TVMs. In addition, this paper is the first to provide an efficient method for connectivity coding of TVMs, by introducing a modification to the state-of-the-art MPEG-4 TFAN algorithm. Experiments are conducted in the MPEG-3DGC TVM database. The method outperforms the state-of-the-art standardized static mesh coder MPEG-4 TFAN at low bit-rates, while remaining competent at high bit-rates. It gives a practical proof of concept that in the combined problem of geometry, connectivity, and vertex attribute coding of TVMs, efficient inter-frame redundancy removal is possible, establishing ground for further improvements. Finally, this paper proposes a method for motion-based coding of H-TVMs that can further enhance the overall experience when H-TVM compression is used in a tele-immersion scenario.

Estimation of Multiple Accelerated Motions Using Chirp-Fourier Transform and Clustering

Motion estimation in the spatiotemporal domain has been extensively studied and many methodologies have been proposed, which, however, cannot handle both time-varying and multiple motions. Extending previously published ideas, we present an efficient method for estimating multiple, linearly time-varying motions. It is shown that the estimation of accelerated motions is equivalent to the parameter estimation of superpositioned chirp signals. From this viewpoint, one can exploit established signal processing tools such as the chirp-Fourier transform. It is shown that accelerated motion results in energy concentration along planes in the 4-D space: spatial frequencies-temporal frequency-chirp rate. Using fuzzy c-planes clustering, we estimate the plane/motion parameters. The effectiveness of our method is verified on both synthetic as well as real sequences and its advantages are highlighted

An Integrated Platform for Live 3D Human Reconstruction and Motion Capturing

The latest developments in 3D capturing, processing, and rendering provide means to unlock novel 3D application pathways. The main elements of an integrated platform, which target tele-immersion and future 3D applications, are described in this paper, addressing the tasks of real-time capturing, robust 3D human shape/appearance reconstruction, and skeleton-based motion tracking. More specifically, initially, the details of a multiple RGB-depth (RGB-D) capturing system are given, along with a novel sensors’ calibration method. A robust, fast reconstruction method from multiple RGB-D streams is then proposed, based on an enhanced variation of the volumetric Fourier transform-based method, parallelized on the Graphics Processing Unit, and accompanied with an appropriate texture-mapping algorithm. On top of that, given the lack of relevant objective evaluation methods, a novel framework is proposed for the quantitative evaluation of real-time 3D reconstruction systems. Finally, a generic, multiple depth stream-based method for accurate real-time human skeleton tracking is proposed. Detailed experimental results with multi-Kinect2 data sets verify the validity of our arguments and the effectiveness of the proposed system and methodologies.

Fast and smooth 3D reconstruction using multiple RGB-Depth sensors

In this paper, the problem of real-time, full 3D reconstruction of foreground moving objects, an important task for Tele-Immersion applications, is addressed. More specifically, the proposed reconstruction method receives input from multiple consumer RGB-Depth cameras. A fast and efficient method to calibrate the sensors in initially described. More importantly, an efficient method to smoothly fuse the captured raw point sets is then presented, followed by a volumetric method to produce watertight and manifold meshes. Given the implementation details, the proposed method can operate at high frame rates. The experimental results, with respect to reconstruction quality and rates, verify the effectiveness of the proposed methodology.

Three-Dimensional Nondestructive “Sampling” of Art Objects Using Acoustic Microscopy and Time–Frequency Analysis

The microsampling destructions, which are caused by the sampling procedures of analytical spectroscopic methods, are, in most cases, not permitted to art objects, which are extremely valuable, rare, and fragile. Consequently, the development of nondestructive analysis techniques becomes a necessity. In this paper, we present a technique and method for the nondestructive identification of the stratigraphic structure of the paint layers of art objects. Using acoustic microscopy, in combination with time-frequency representations, the continuous or discrete wavelet transform, or the Hilbert-Huang transform, the depth profile of the stratigraphy is determined.

Estimation of Multiple, Time-Varying Motions Using Time-Frequency Representations and Moving-Objects Segmentation

We extend existing spatiotemporal approaches to handle time-varying motions estimation of multiple objects. It is shown that multiple, time-varying motions estimation is equivalent to the instantaneous frequency estimation of superpoliteness FM sinusoids. Therefore, we apply established signal processing tools, such as time-frequency representations to show that for each time instant, the energy is concentrated along planes in the 3-D space: spatial frequencies - instantaneous frequency. Using fuzzy C-planes, we estimate indirectly the instantaneous velocities. Furthermore, adapting existing approaches to our problem, we attain the identification of the moving objects. The experimental results verify the effectiveness of our methodology.

Diffuse reflectance spectroscopic mapping imaging applied to art objects materials determination from 200 up to 5000 nm

The microsampling destructions caused by the sampling of analytical spectroscopic methods are generally not permitted to art objects. Consequently, the development of nondestructive analysis techniques is a necessity. In this work we present a set of signal processing and artificial intelligence techniques which support the operation of a novel device developed for the nondestructive identification of art objects. The proposed device combines ultraviolet, visible, near infrared, and midinfrared spectroscopy in diffuse reflectance mode to identify the materials that exist in each paint layer of an artwork.

A case study for tele-immersion communication applications: From 3D capturing to rendering

The primary objective of this paper is to present and analyze key aspects related to next-generation tele-immersion applications, studying the end-to-end chain from 3D capturing of remote users to rendering. The key modules for 3D reconstruction of moving humans and their mesh compression, are presented and discussed. The chain performance is evaluated in terms of frame-rates, delay, and visual quality.