Paisarn Muneesawang

An Approach to Ballet Dance Training through MS Kinect and Visualization in a CAVE Virtual Reality Environment

This article proposes a novel framework for the real-time capture, assessment, and visualization of ballet dance movements as performed by a student in an instructional, virtual reality (VR) setting. The acquisition of human movement data is facilitated by skeletal joint tracking captured using the popular Microsoft (MS) Kinect camera system, while instruction and performance evaluation are provided in the form of 3D visualizations and feedback through a CAVE virtual environment, in which the student is fully immersed. The proposed framework is based on the unsupervised parsing of ballet dance movement into a structured posture space using the spherical self-organizing map (SSOM). A unique feature descriptor is proposed to more appropriately reflect the subtleties of ballet dance movements, which are represented as gesture trajectories through posture space on the SSOM. This recognition subsystem is used to identify the category of movement the student is attempting when prompted (by a virtual instructor) to perform a particular dance sequence. The dance sequence is then segmented and cross-referenced against a library of gestural components performed by the teacher. This facilitates alignment and score-based assessment of individual movements within the context of the dance sequence. An immersive interface enables the student to review his or her performance from a number of vantage points, each providing a unique perspective and spatial context suggestive of how the student might make improvements in training. An evaluation of the recognition and virtual feedback systems is presented.

Automatic relevance feedback for video retrieval

This paper presents an automatic relevance feedback method for improving retrieval accuracy in video database. We first demonstrate a representation based on a template-frequency model (TFM) that allows the full use of the temporal dimension. We then integrate the TFM with a self-training neural network structure to adaptively capture different degrees of visual importance in a video sequence. Forward and backward signal propagation is the key in this automatic relevance feedback method in order to enhance retrieval accuracy.

A New Learning Algorithm for the Fusion of Adaptive Audio---Visual Features for the Retrieval and Classification of Movie Clips

This paper presents a new learning algorithm for audiovisual fusion and demonstrates its application to video classification for film database. The proposed system utilized perceptual features for content characterization of movie clips. These features are extracted from different modalities and fused through a machine learning process. More specifically, in order to capture the spatio-temporal information, an adaptive video indexing is adopted to extract visual feature, and the statistical model based on Laplacian mixture are utilized to extract audio feature. These features are fused at the late fusion stage and input to a support vector machine (SVM) to learn semantic concepts from a given video database. Based on our experimental results, the proposed system implementing the SVM-based fusion technique achieves high classification accuracy when applied to a large volume database containing Hollywood movies.

Size measurement of nanoparticle assembly using multilevel segmented TEM images

Multilevel image segmentation is demonstrated as a rapid and accurate method of quantitative analysis for nanoparticle assembly in TEM images. The procedure incorporating K-means clustering algorithm and watershed transform is tested on transmission electron microscope (TEM) images of FePt-based nanoparticles whose diameters are less than 5 nm. By solving the nanoparticle segmentation and separation problems, this unsupervised method is useful not only in the nonoverlapping case but also for agglomerated nanoparticles. Furthermore, the method exhibits scale invariance based on comparable results from images of different magnifications.

An Advanced Vision System for the Automatic Inspection of Corrosions on Pole Tips in Hard Disk Drives

Pole tips of hard disk drives are made of FeCo film, which is prone to corrosion due to the corrosive nature of the environment and wide range of pH experienced during the production. A machine vision system is utilized for the automatic auditing of corroded pole tips. This comprises three steps: 1) extraction of the top shield region; 2) fusion of extracted features; and 3) decision making. This model is capable of detecting corrosion in the critical part of the pole tips. The experiments show that the proposed system detects corrosion with high accuracy, whereas the overall processing time meets industrial requirements.

iARM - an interactive video retrieval system

This work presents the iARM system for content-based video retrieval in an interactive framework. This system explores a new model-based video indexing technique to improve the effectiveness of relevance feedback and make interactive video retrieval a user-friendly environment. The system emphasizes the accuracy in modeling spatio-temporal information in a video clip, so that relevance feedback analysis needs only a few cycles and a few training samples, which greatly reduce the search time for video transmissions over the network. We investigate the resilience of the system, and apply the interactive content-based retrieval method to an automatically indexed database of 20 hours of video.

Image retrieval with embedded sub-class information using Gaussian mixture models

This paper describes content-based image retrieval techniques within the relevance feedback framework. The Gaussian mixture model (GMM) is used to characterize sub-class information to increase retrieval accuracy and reduce number of interactions during a query session. The implementation of GMM is based on the radial basis function using a new learning algorithm that can cope with small training samples in the relevance feedback cycle. The proposed retrieval system is successfully applied to image databases of very large sizes, and experimental results show that the proposed system competes favorably with the other recently proposed interactive systems.

Application of Image Processing to Determine Size Distribution of Magnetic Nanoparticles

Digital image processing has increasingly been implemented in nanostructural analysis and would be an ideal tool to characterize the morphology and position of self-assembled magnetic nanoparticles for high density recording. In this work, magnetic nanoparticles were synthesized by the modified polyol process using Fe(acac)3 and Pt(acac)2 as starting materials. Transmission electron microscope (TEM) images of as-synthesized products were inspected using an image processing procedure. Grayscale images (800 × 800 pixels, 72 dot per inch) were converted to binary images by using Otsu’s thresholding. Each particle was then detected by using the closing algorithm with disk structuring elements of 2 pixels, the Canny edge detection, and edge linking algorithm. Their centroid, diameter and area were subsequently evaluated. The degree of polydispersity of magnetic nanoparticles can then be compared using the size distribution from this image processing procedure.

Audio visual cues for video indexing and retrieval

This paper studies content-based video retrieval using the combination of audio and visual features. The visual feature is extracted by an adaptive video indexing technique that places a strong emphasis on accurate characterization of spatio-temporal information within video clips. Audio feature is extracted by a statistical time-frequency analysis method that applies Laplacian mixture models to wavelet coefficients. The proposed joint audio-visual retrieval framework is highly flexible and scalable, and can be effectively applied to various types of video databases.

Multimedia Database Retrieval

This chapter presents machine learning methods for adaptive image retrieval. In a retrieval session, a nonlinear kernel is applied to measure image relevancy. Various new learning procedures are covered and applied specifically for adaptive image retrieval applications. These include the adaptive radial basis function (RBF) network, short term learning with the gradient-decent method, and the fuzzy RBF network. These methods constitute the likelihood estimation corresponding to visual content in a short-term relevance feedback (STRF). The STRF component can be further incorporated in a fusion module with contextual information in long-term relevance feedback (LTRF) using the Bayesian framework. This substantially increases retrieval accuracy.