Daniel Mateos-García

On the evolutionary optimization of k-NN by label-dependent feature weighting

Different approaches of feature weighting and k-value selection to improve the nearest neighbour technique can be found in the literature. In this work, we show an evolutionary approach called k-Label Dependent Evolutionary Distance Weighting (kLDEDW) which calculates a set of local weights depending on each class besides an optimal k value. Thus, we attempt to carry out two improvements simultaneously: we locally transform the feature space to improve the accuracy of the k-nearest-neighbour rule whilst we search for the best value for k from the training data. Rigorous statistical tests demonstrate that our approach improves the general k-nearest-neighbour rule and several approaches based on local weighting.

An evolutionary-weighted majority voting and support vector machines applied to contextual classification of LiDAR and imagery data fusion

Data classification is a critical step to convert remotely sensed data into thematic information. Environmental researchers have recently maximized the synergy between passive sensors and LiDAR (Light Detection and Ranging) for land cover classification by means of machine learning. Although object-based paradigm is frequently used to classify high resolution imagery, it often requires a high level of expertise and time effort. Contextual classification may lead to similar results with a decrease in time and costs for non-expert users. This work shows a novel contextual classifier based on a Support Vector Machine (SVM) and an Evolutionary Majority Voting (SVM-EMV) to develop thematic maps from LiDAR and imagery data. Subsequently, the performance of SVM-EMV is compared to that achieved by a pixel-based SVM as well as to a contextual classified based on SVM and MRF. The classifiers were tested over three different areas of Spain with well differentiated environmental characteristics. Results show that SVM-EMV statistically outperforms the rest (SVM, SVM-MRF) for the three datasets obtaining a 77%, 91% and 92% of global accuracy for Trabada, Huelva and Alto Tajo, respectively.

A comparative study between two regression methods on LiDAR data: a case study

Airborne LiDAR (Light Detection and Ranging) has become an excellent tool for accurately assessing vegetation characteristics in forest environments. Previous studies showed empirical relationships between LiDAR and field-measured biophysical variables. Multiple linear regression (MLR) with stepwise feature selection is the most common method for building estimation models. Although this technique has provided very interesting results, many other data mining techniques may be applied. The overall goal of this study is to compare different methodologies for assessing biomass fractions at stand level using airborne Li-DAR data in forest settings. In order to choose the best methodology, a comparison between two different feature selection techniques (stepwise selection vs. genetic-based selection) is presented. In addition, classical MLR is also compared with regression trees (M5P). The results when each methodology is applied to estimate stand biomass fractions from an area of northern Spain show that genetically-selected M5P obtains the best results.

EVOR-STACK: A label-dependent evolutive stacking on remote sensing data fusion

Land use and land covers (LULC) maps are remote sensing products that are used to classify areas into different landscapes. Data fusion for remote sensing is becoming an important tool to improve classical approaches. In addition, artificial intelligence techniques such as machine learning or evolutive computation are often applied to improve the final LULC classification. In this paper, a hybrid artificial intelligence method based on an ensemble of multiple classifiers to improve LULC map accuracy is shown. The method works in two processing levels: first, an evolutionary algorithm (EA) for label-dependent feature weighting transforms the feature space by assigning different weights to every attribute depending on the class. Then a statistical raster from LIDAR and image data fusion is built following a pixel-oriented and feature-based strategy that uses a support vector machine (SVM) and a weighted k-NN restricted stacking. A classical SVM, the original restricted stacking (R-STACK) and the current improved method (EVOR-STACK) are compared. The results show that the evolutive approach obtains the best results in the context of the real data from a riparian area in southern Spain.

A SVM and k-NN restricted stacking to improve land use and land cover classification

Land use and land cover (LULC) maps are remote sensing products that are used to classify areas into different landscapes The newest techniques have been applied to improve the final LULC classification and most of them are based on SVM classifiers In this paper, a new method based on a multiple classifiers ensemble to improve LULC map accuracy is shown The method builds a statistical raster from LIDAR and image fusion data following a pixel-oriented strategy Then, the pixels from a training area are used to build a SVM and k-NN restricted stacking taking into account the special characteristics of spatial data A comparison between a SVM and the restricted stacking is carried out The results of the tests show that our approach improves the results in the context of the real data from a riparian area of Huelva (Spain).

An evolutionary voting for k-nearest neighbours

We optimize the voting system for the k nearest neighbours.We use evolutionary computation.We study the influence of the closeness of neighbours on the search process.The results are statistically validated. This work presents an evolutionary approach to modify the voting system of the k-nearest neighbours (kNN) rule we called EvoNN. Our approach results in a real-valued vector which provides the optimal relative contribution of the k-nearest neighbours. We compare two possible versions of our algorithm. One of them (EvoNN1) introduces a constraint on the resulted real-valued vector where the greater value is assigned to the nearest neighbour. The second version (EvoNN2) does not include any particular constraint on the order of the weights. We compare both versions with classical kNN and 4 other weighted variants of the kNN on 48 datasets of the UCI repository. Results show that EvoNN1 outperforms EvoNN2 and statistically obtains better results than the rest of the compared methods.

Label dependent evolutionary feature weighting for remote sensing data

Nearest neighbour (NN) is a very common classifier used to develop important remote sensing products like land use and land cover (LULC) maps Evolutive computation has often been used to obtain feature weighting in order to improve the results of the NN In this paper, a new algorithm based on evolutionary computation which has been called Label Dependent Feature Weighting (LDFW) is proposed The LDFW method transforms the feature space assigning different weights to every feature depending on each class This multilevel feature weighting algorithm is tested on remote sensing data from fusion of sensors (LIDAR and orthophotography) The results show an improvement on the NN and resemble the results obtained with a neural network which is the best classifier for the study area.

Improving the k-Nearest Neighbour Rule by an Evolutionary Voting Approach

This work presents an evolutionary approach to modify the voting system of the k-Nearest Neighbours (kNN). The main novelty of this article lies on the optimization process of voting regardless of the distance of every neighbour. The calculated real-valued vector through the evolutionary process can be seen as the relative contribution of every neighbour to select the label of an unclassified example. We have tested our approach on 30 datasets of the UCI repository and results have been compared with those obtained from other 6 variants of the kNN predictor, resulting in a realistic improvement statistically supported.

On the evolutionary weighting of neighbours and features in the k-nearest neighbour rule

Abstract This paper presents an evolutionary method for modifying the behaviour of the k-Nearest-Neighbour classifier (kNN) called Simultaneous Weighting of Attributes and Neighbours (SWAN). Unlike other weighting methods, SWAN presents the ability of adjusting the contribution of the neighbours and the significance of the features of the data. The optimization process focuses on the search of two real-valued vectors. One of them represents the votes of neighbours, and the other one represents the weight of each feature. The synergy between the two sets of weights found in the optimization process helps to improve significantly, the classification accuracy. The results on 35 datasets from the UCI repository suggest that SWAN statistically outperforms the other weighted kNN methods.

A Preliminary Study of the Suitability of Deep Learning to Improve LiDAR-Derived Biomass Estimation

Light Detection and Ranging (LiDAR) is a remote sensor able to extract three-dimensional information about forest structure. Biophysical models have taken advantage of the use of LiDAR-derived information to improve their accuracy. Multiple Linear Regression (MLR) is the most common method in the literature regarding biomass estimation to define the relation between the set of field measurements and the statistics extracted from a LiDAR flight. Unfortunately, there exist open issues regarding the generalization of models from one area to another due to the lack of knowledge about noise distribution, relationship between statistical features and risk of overfitting. Autoencoders (a type of deep neural network) has been applied to improve the results of machine learning techniques in recent times by undoing possible data corruption process and improving feature selection. This paper presents a preliminary comparison between the use of MLR with and without preprocessing by autoencoders on real LiDAR data from two areas in the province of Lugo (Galizia, Spain). The results show that autoencoders statistically increased the quality of MLR estimations by around 15–30%.