Meijun Sun

A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos

Abstract Although research on detection of saliency and visual attention has been active over recent years, most of the existing work focuses on still image rather than video based saliency. In this paper, a deep learning based hybrid spatiotemporal saliency feature extraction framework is proposed for saliency detection from video footages. The deep learning model is used for the extraction of high-level features from raw video data, and they are then integrated with other high-level features. The deep learning network has been found extremely effective for extracting hidden features than that of conventional handcrafted methodology. The effectiveness for using hybrid high-level features for saliency detection in video is demonstrated in this work. Rather than using only one static image, the proposed deep learning model take several consecutive frames as input and both the spatial and temporal characteristics are considered when computing saliency maps. The efficacy of the proposed hybrid feature framework is evaluated by five databases with human gaze complex scenes. Experimental results show that the proposed model outperforms five other state-of-the-art video saliency detection approaches. In addition, the proposed framework is found useful for other video content based applications such as video highlights. As a result, a large movie clip dataset together with labeled video highlights is generated.

Effective Denoising and Classification of Hyperspectral Images Using Curvelet Transform and Singular Spectrum Analysis

Hyperspectral imaging (HSI) classification has become a popular research topic in recent years, and effective feature extraction is an important step before the classification task. Traditionally, spectral feature extraction techniques are applied to the HSI data cube directly. This paper presents a novel algorithm for HSI feature extraction by exploiting the curvelet-transformed domain via a relatively new spectral feature processing technique-singular spectrum analysis (SSA). Although the wavelet transform has been widely applied for HSI data analysis, the curvelet transform is employed in this paper since it is able to separate image geometric details and background noise effectively. Using the support vector machine classifier, experimental results have shown that features extracted by SSA on curvelet coefficients have better performance in terms of classification accuracy over features extracted on wavelet coefficients. Since the proposed approach mainly relies on SSA for feature extraction on the spectral dimension, it actually belongs to the spectral feature extraction category. Therefore, the proposed method has also been compared with some state-of-the-art spectral feature extraction techniques to show its efficacy. In addition, it has been proven that the proposed method is able to remove the undesirable artifacts introduced during the data acquisition process. By adding an extra spatial postprocessing step to the classified map achieved using the proposed approach, we have shown that the classification performance is comparable with several recent spectral-spatial classification methods.

How to predict the sugariness and hardness of melons: A near-infrared hyperspectral imaging method

Hyperspectral imaging (HSI) in the near-infrared (NIR) region (900-1700nm) was used for non-intrusive quality measurements (of sweetness and texture) in melons. First, HSI data from melon samples were acquired to extract the spectral signatures. The corresponding sample sweetness and hardness values were recorded using traditional intrusive methods. Partial least squares regression (PLSR), principal component analysis (PCA), support vector machine (SVM), and artificial neural network (ANN) models were created to predict melon sweetness and hardness values from the hyperspectral data. Experimental results for the three types of melons show that PLSR produces the most accurate results. To reduce the high dimensionality of the hyperspectral data, the weighted regression coefficients of the resulting PLSR models were used to identify the most important wavelengths. On the basis of these wavelengths, each image pixel was used to visualize the sweetness and hardness in all the portions of each sample.

Monte Carlo Convex Hull Model for classification of traditional Chinese paintings

While artists demonstrate their individual styles through paintings and drawings, how to describe such artistic styles well selected visual features towards computerized analysis of the arts remains to be a challenging research problem. In this paper, we propose an integrated feature-based artistic descriptor with Monte Carlo Convex Hull (MCCH) feature selection model and support vector machine (SVM) for characterizing the traditional Chinese paintings and validate its effectiveness via automated classification of Chinese paintings authored by well-known Chinese artists. The integrated artistic style descriptor essentially contains a number of visual features including a novel feature of painting composition and object feature, each of which describes one element of the artistic style. In order to ensure an integrated discriminating power and certain level of adaptability to the variety of artistic styles among different artists, we introduce a novel feature selection method to process the correlations and the synergy across all elements inside the integrated feature and hence complete the proposed style-based descriptor design. Experiments on classification of Chinese paintings via a parallel MCCH model illustrate that the proposed descriptor outperforms the existing representative technique in terms of precision and recall rates.

Effective compression of hyperspectral imagery using an improved 3D DCT approach for land-cover analysis in remote-sensing applications

Although hyperspectral imagery (HSI), which has been applied in a wide range of applications, suffers from very large volumes of data, its uncompressed representation is still preferred to avoid compression loss for accurate data analysis. In this paper, we focus on quality-assured lossy compression of HSI, where the accuracy of analysis from decoded data is taken as a key criterion to assess the efficacy of coding. An improved 3D discrete cosine transform-based approach is proposed, where a support vector machine (SVM) is applied to optimally determine the weighting of inter-band correlation within the quantization matrix. In addition to the conventional quantitative metrics signal-to-noise ratio and structural similarity for performance assessment, the classification accuracy on decoded data from the SVM is adopted for quality-assured evaluation, where the set partitioning in hierarchical trees (SPIHT) method with 3D discrete wavelet transform is used for benchmarking. Results on four publically available HSI data sets have indicated that our approach outperforms SPIHT in both subjective (qualitative) and objective (quantitative) assessments for land-cover analysis in remote-sensing applications. Moreover, our approach is more efficient and generates much reduced degradation for subsequent data classification, hence providing a more efficient and quality-assured solution in effective compression of HSI.

Brushstroke based sparse hybrid convolutional neural networks for author classification of Chinese ink-wash paintings

A novel stroke based sparse hybrid convolutional neural networks (CNNs) method is proposed for author classification of Chinese ink-wash paintings (IWPs). As Chinese IWPs usually have many authors in several art styles, this differs from real images or western paintings and has led to a big challenge. In our work, we classify Chinese IWPs of different artists by analyzing a set of automatically extracted brushstrokes. A sparse hybrid CNNs in a deep-learning framework is then proposed to extract brushstroke features to replace the commonly used handcrafted ones such as edge, color, intensity and texture. Using 120 IWPs from six famous artists, promising results have been shown in successfully classifying authors in comparison to two other state-of-the-art approaches.

What's wrong with the murals at the Mogao Grottoes: a near-infrared hyperspectral imaging method

Although a significant amount of work has been performed to preserve the ancient murals in the Mogao Grottoes by Dunhuang Cultural Research, non-contact methods need to be developed to effectively evaluate the degree of flaking of the murals. In this study, we propose to evaluate the flaking by automatically analyzing hyperspectral images that were scanned at the site. Murals with various degrees of flaking were scanned in the 126th cave using a near-infrared (NIR) hyperspectral camera with a spectral range of approximately 900 to 1700 nm. The regions of interest (ROIs) of the murals were manually labeled and grouped into four levels: normal, slight, moderate, and severe. The average spectral data from each ROI and its group label were used to train our classification model. To predict the degree of flaking, we adopted four algorithms: deep belief networks (DBNs), partial least squares regression (PLSR), principal component analysis with a support vector machine (PCA + SVM) and principal component analysis with an artificial neural network (PCA + ANN). The experimental results show the effectiveness of our method. In particular, better results are obtained using DBNs when the training data contain a significant amount of striping noise.

Effective Classification of Microcalcification Clusters Using Improved Support Vector Machine with Optimised Decision Making

Classification of micro calcification clusters is very essential for early detection of breast cancer from mammograms. In this paper, an improved support vector machine (SVM) scheme is proposed, where optimized decision making is introduced for effective and more accurate data classification. Experimental results on the well-known DDSM database have shown that the proposed method can significantly increase the performance in terms of F1 and Az measurements for the successful classification of clustered micro calcifications.

System for Hyperspectral Data Analysis, Visualization and Fresco Deterioration Detection

In this paper we proposed a scalable interactive system for fresco deterioration detection by hyper-spectral image data analysis. The system integrates data mining and visualization algorithm and process the hyper-spectral big data from fresco efficiently and conveniently. Firstly, a Geospatial Data Abstraction Library (GDAL) is adapted which provides data reading, image preview and cropping functions, Secondly, the Principal Components Analysis (PCA) algorithm is employed for dimension reduction and compression, Then, the partial least squares (PLS) algorithm is used for training the fresco deterioration detection model. Finally, the predicted results are visualized by using popular visualization method. Experimental results show that the proposed hyper-spectral data analysis system is effectively and efficiently for fresco deterioration detection.

GPU Based 3D Chinese Painting Animation

In this paper, we presents our research on the technology of 3D Chinese painting animation which is made based on GPU through close observation and quantitative analysis to Chinese painting. Firstly, we adopt outline rendering and inside rendering algorithms to render the 3D model. Secondly, a single stroke rendering method is used to draw the subjects in the far. All the methods we introduced here are implemented on GPU. The results show that our works has a great practical effect on making Chinese painting styled 3D animation.