Dingsheng Wan

A novel approach to hyperspectral band selection based on spectral shape similarity analysis and fast branch and bound search

With the development of hyperspectral remote sensing technology, the spectral resolution of the hyperspectral image data becomes denser, which results in large number of bands, high correlation between neighboring bands, and high data redundancy. It is necessary to reduce these bands before further analysis, such as land cover classification and target detection. Aiming at the classification task, this paper proposes an effective band selection method from the novel perspective of spectral shape similarity analysis with key points extraction and thus retains physical information of hyperspectral remote sensing images. The proposed approach takes all the bands of hyperspectral remote sensing images as time series. Firstly, spectral clustering is utilized to cluster all the training samples, which produces the prototypical spectral curves of each cluster. Then a set of initial candidate bands are obtained based on the extraction of key points from the processed hyperspectral curves, which preserve discriminative information and narrow down the candidate band subset for the following search procedure. Finally, filtering contiguous bands according to conditional mutual information and branch and bound search are further performed sequentially to gain the optimal band combination. To verify the effectiveness of the integrated band selection method put forward in this paper, classification employing the Support Vector Machine (SVM) classifier is performed on the selected spectral bands. The experimental results on two publicly available benchmark data sets demonstrate that the presented approach can select those bands with discriminative information, usually about 10 out of 200 original bands. Compared with previous studies, the newly proposed method is competitive with far fewer bands selected and a lower computational complexity, while the classification accuracy remains comparable.

Multivariate Time Series Similarity Searching

Multivariate time series (MTS) datasets are very common in various financial, multimedia, and hydrological fields. In this paper, a dimension-combination method is proposed to search similar sequences for MTS. Firstly, the similarity of single-dimension series is calculated; then the overall similarity of the MTS is obtained by synthesizing each of the single-dimension similarity based on weighted BORDA voting method. The dimension-combination method could use the existing similarity searching method. Several experiments, which used the classification accuracy as a measure, were performed on six datasets from the UCI KDD Archive to validate the method. The results show the advantage of the approach compared to the traditional similarity measures, such as Euclidean distance (ED), cynamic time warping (DTW), point distribution (PD), PCA similarity factor (SPCA), and extended Frobenius norm (Eros), for MTS datasets in some ways. Our experiments also demonstrate that no measure can fit all datasets, and the proposed measure is a choice for similarity searches.

Time Series Outlier Detection Based on Sliding Window Prediction

In order to detect outliers in hydrological time series data for improving data quality and decision-making quality related to design, operation, and management of water resources, this research develops a time series outlier detection method for hydrologic data that can be used to identify data that deviate from historical patterns. The method first built a forecasting model on the history data and then used it to predict future values. Anomalies are assumed to take place if the observed values fall outside a given prediction confidence interval (PCI), which can be calculated by the predicted value and confidence coefficient. The use of PCI as threshold is mainly on the fact that it considers the uncertainty in the data series parameters in the forecasting model to address the suitable threshold selection problem. The method performs fast, incremental evaluation of data as it becomes available, scales to large quantities of data, and requires no preclassification of anomalies. Experiments with different hydrologic real-world time series showed that the proposed methods are fast and correctly identify abnormal data and can be used for hydrologic time series analysis.

Hydrological Time Series Anomaly Mining Based on Symbolization and Distance Measure

Large amount of hydrological data set is a kind of big data, which has much hidden and potentially useful knowledge. It is necessary to extract these knowledge from hydrological data set, which can provide more valuable hydrological information and be useful for future hydrological forecasting. Data mining based on time series is widely used currently. There are some techniques based on time series to extract anomaly. However, most of these techniques cannot suit big unstable data such as hydrological big data set. Some important problems are high fitting error after dimension reduction and low accuracy of mining results. In this work we propose a new idea to solve the problem of hydrological anomaly mining based on time series. The idea combines time series symbolization with distance measure. It proposes Feature Points Symbolic Aggregate Approximation (FP SAX) to improve the selection of feature points, and then measures the distance of strings by Symbol Distance based Dynamic Time Warping (SD DTW). Finally, the distance which we have got are sorted. A set of dedicated experiments are performed to validate our approach. The experimental data set is based on the water level data set obtained from Xiaomeikou gauge station in the Taihu Lake from 1956 to 2005. The results of experiments show that our approach has lower fitting error and higher accuracy.

Spectral similarity-preserving hyperspectral band selection

Due to the high spectral resolution, hyperspectral images (HSI) have been widely used in land cover classification and material identification. Band selection is one of the necessary preprocessings to reduce the data volume and the redundancy therein for the subsequent analysis. Aiming at speeding up the search-based band selection process, this letter proposes a new technique from the perspective of spectral curve shape similarity. Through a newly defined measure for band subset discriminativeness (BSD), class-specific important bands (IBs) are retained which can preserve the spectral similarity of the samples from the same class and narrow down candidate band subset for the following search procedure. Then optimal search is performed in the aggregated band subset from all classes. Experiments on the Indian Pine benchmark data set have proved the efficiency and effectiveness of the proposed method.

A new algorithm for water information extraction from high resolution remote sensing imagery

Current algorithms that utilize water index to extract water information from high resolution remote sensing image are inadequate in that it is difficult to determine the optimal thresholds, the result of water boundary is not satisfactory and prone to error. We propose a new algorithm which combines image segmentation algorithm based on MRF model with Normalized Difference Water Index (NDWI) for extracting water information. We represent the pixels in image as random variables in a MRF model and introduce a hybrid feature space in the energy function on these variables, then minimize the energy function by iterative graph cut scheme to find the optimal water boundary. Water boundary is refined according to water index and color features of extracted main water body. The experiments on ShiLianghe Reservoir show that our approach can achieve a better accuracy and the boundary is precise, especially at the reservoirs that surrounding environments are complicated.

Hyperspectral band selection from the spectral similarity perspective

This paper proposes a new technique for hyperspectral band selection from the spectral similarity perspective. Through a newly defined measure for band subset discriminativeness, class-specific important bands are retained which can preserve the spectral similarity of the samples from the same class and narrow down candidate band subset for the following search procedure. Then optimal search is performed in the aggregated band subset from all classes. Experiments on the Indian Pine benchmark data set have proved the efficiency and effectiveness of the proposed method.

Multi-measure Similarity Searching for Time Series

In this paper, we evaluate some techniques for the time series similarity searching. Many distance measures have been proposed as alternatives to the Euclidean distance in the similarity searching. To verify the assumption that the combination of various similarity measures may produce more accurate similarity searching results, we propose an multi-measure algorithm to combine several measures based on weighted BORDA voting method. The proposed method is validated by the analysis results of the flood data obtained from Wangjiaba in the Huaihe basin of China.

A Combined Data Mining Method and Its Application in Water Quality Trends Analysis

A combined method with moving average smoothing method, reverse test method and linear regression model is presented to analyze water quality trends in this chapter. The method is applied to the trends analysis for Yangtze-Taihu Water Diversion, the conclusions are verified by the practice and application of Wuxi Hydrology and Water Resources Monitoring Council, and the method is proved effective.