Hasan Badem

A new efficient training strategy for deep neural networks by hybridization of artificial bee colony and limited–memory BFGS optimization algorithms

Abstract Working up with deep learning techniques requires profound understanding of the mechanisms underlying the optimization of the internal parameters of complex structures. The major factor limiting this understanding is that there exist only a few optimization methods such as gradient descent and Limited–memory Broyden–Fletcher–Goldfarb–Shannon (L-BFGS) to find the best local minima of the problem space for these complex structures such as deep neural network (DNN). Therefore, in this paper, we represent a new training approach named hybrid artificial bee colony based training strategy (HABCbTS) to tune the parameters of a DNN structure, which includes one or more autoencoder layers cascaded to a softmax classification layer. In this strategy, a derivative-free optimization algorithm “ABC” is combined with a derivative-based algorithm “L-BFGS” to construct “HABC”, which is used in the HABCbTS. Detailed simulation results supported by statistical analysis show that the proposed training strategy results in better classification performance compared to the DNN classifier trained with the L-BFGS, ABC and modified ABC. The obtained classification results are also compared with the state-of-the-art classifiers, including MLP, SVM, KNN, DT and NB on 15 data sets with different dimensions and sizes.

Classification of human activity by using a Stacked Autoencoder

This paper investigates the application of a deep neural network architecture that consists of stackted autoencoder with two autoencoders and a softmax layer for the purpose of human activity classification. Th performance of the proposed architecture is tested on a commonly used data set known as Human Activity Recognition Using Smartphones. It is observed that the proposed method yields better classification results than the representative state-of-the-art methods provided that the parameters of the deep network are suitably optimized.

Performance improvement of deep neural network classifiers by a simple training strategy

Improving the classification performance of Deep Neural Networks (DNN) is of primary interest in many different areas of science and technology involving the use of DNN classifiers. In this study, we present a simple training strategy to improve the classification performance of a DNN. In order to attain our goal, we propose to divide the internal parameter space of the DNN into partitions and optimize these partitions individually. We apply our proposed strategy with the popular L-BFGS optimization algorithm even though it can be applied with any optimization algorithm. We evaluate the performance improvement obtained by using our proposed method by testing it on a number of well-known classification benchmark data sets and by performing statistical analysis procedures on classification results. The DNN classifier trained with the proposed strategy is also compared with the state-of-the-art classifiers to demonstrate its effectiveness. Our classification experiments show that the proposed method significantly enhances the training process of the DNN classifier and yields considerable improvements in the accuracy of the classification results.

Deep neural network classifier for hand movement prediction

The prediction of the hand movement based on Electromyography (EMG) signals has been extensively studied over the past three decades. However, recent EGM applications pose an emerging need of efficient classification of EMG signals. Toward this goal, we propose a deep neural network (DNN) classifier in this study to classify 6 different hand movement from EMG signals. DNN classifier has ability to extract new features from raw data and reduce the dimension of the data set. Our experimental results based on human subjects demonstrate that DNN classifier can efficiently classify the EMG signals to accurately distinguish different hand movement.

Deep neural network based diagnosis system for melanoma skin cancer

Melanoma is a serious cancer that causes many people to lose their lives. This disease can be diagnosed by a dermatologist as a result of interpretation of the dermoscopy images by the ABCD rule. In this study, a deep neural network (DNN) is used as a new method for diagnosis of melanoma skin cancer. This method is compared with the-state-art-methods in literature. According to the obtained results, DNN was more successful than the comparative methods.

A new hybrid optimization method combining artificial bee colony and limited-memory BFGS algorithms for efficient numerical optimization

Abstract In this paper, a new optimization method, which is developed especially for optimization of functions with a large number of local minima, is presented. The proposed method is a hybrid optimization algorithm which employs the artificial bee colony (ABC) and limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithms for combining their powerful features. The most prominent feature of the proposed method over other methods is that it provides accurate results and valuable convergence speeds, as well as easy implementation at the same time. Extensive simulation results supported by detailed statistical analyses show that the proposed method can be used for efficient optimization of functions including well-known benchmark functions and CEC2016 competition functions.

Detecting Direction of Pepper Stem by Using CUDA-Based Accelerated Hybrid Intuitionistic Fuzzy Edge Detection and ANN

In recent years, computer vision systems have been used in almost every field of industry. In this study, image processing algorithm has been developed by using CUDA (GPU) which is 79 times faster than CPU. We had used this accelerated algorithm in destemming process of pepper. 65 percent of total national production of pepper is produced in our cities, Kahramanmaras and Gaziantep in Turkey. Firstly, hybrid intuitionistic fuzzy algorithm edge detection has been used for preprocessing of original image and Otsu method has been used for determining automatic threshold in this algorithm. Then the multilayer perceptron artificial neural network has been used for the classification of patterns in processed images. Result of ANN test for detection direction of pepper has shown high accuracy performance in CPU-based implementation and in GPU-based implementation.

The effect of autoencoders over reducing the dimensionality of a dermatology data set

The effect of using autoencoders for dimensionality reduction of a medical data set is investigated. A stack of two autoencoders has been trained for popular benchmark medical data set for dermatological disease diagnosis. The improvement of the presented approach has been visualized by the Principal Component Analysis method. Results shows that the use of a autoencoders significantly improves the accuracy of dermatological disease diagnosis.

Accelerated intuitionistic fuzzy edge detection algorithm by using CUDA

Intuitionistic fuzzy edge detection (IFED) algorithm has been used in the signification or characterization of images. IFED algorithm has been designed by the experts and the algorithm provides to aim to minimize errors of them. To be applicable in parallel of IFED is pave the way for accelerating of algorithm by performing in the graphics card. In this study, IFED algorithm was tested by transferring different size images to graphics cards which has different computing capacity via Compute Unified Device Architecture (CUDA) programming environment which is manufactured by NVIDIA. Parallel model of the algorithm adapted to CUDA platform, compared to serial application running on processor, and has seen that shortened runtime at least 67 times, most 641 times.

Accelerating Discrete Haar Wavelet Transform on GPU cluster

The Discrete Haar Wavelet Transform has a wide range of applications from signal processing to video and image processing. Data-intensive structure and easy of implementation make Discrete Haar Wavelet Transform convenient to distribute fundamental operations to multi-CPU and multi-GPU systems. In this paper, the wavelet transform was ported in a compute-efficient way to CPU cluster and programmable GPU cluster by utilizing MPI and CUDA respectively. Experimental studies conducted as part of the parallelization strategies for two-dimensional Discrete Haar Wavelet Transform show that the total running time required to process all rows and columns of an image with different size is significantly decreased on the GPU cluster when compared to the its counterparts on a single CPU, single GPU and CPU cluster. Besides the speedup of the GPU based transform, preliminary analysis also showed that the size of the image is an important parameter on the scalability of the GPU cluster.