Jyh-Horng Jeng

Preliminary Study on Wilcoxon Learning Machines

As is well known in statistics, the resulting linear regressors by using the rank-based Wilcoxon approach to linear regression problems are usually robust against (or insensitive to) outliers. This motivates us to introduce in this paper the Wilcoxon approach to the area of machine learning. Specifically, we investigate four new learning machines, namely Wilcoxon neural network (WNN), Wilcoxon generalized radial basis function network (WGRBFN), Wilcoxon fuzzy neural network (WFNN), and kernel-based Wilcoxon regressor (KWR). These provide alternative learning machines when faced with general nonlinear learning problems. Simple weights updating rules based on gradient descent will be derived. Some numerical examples will be provided to compare the robustness against outliers for various learning machines. Simulation results show that the Wilcoxon learning machines proposed in this paper have good robustness against outliers. We firmly believe that the Wilcoxon approach will provide a promising methodology for many machine learning problems.

Study on Huber Fractal Image Compression

In this paper, a new similarity measure for fractal image compression (FIC) is introduced. In the proposed Huber fractal image compression (HFIC), the linear Huber regression technique from robust statistics is embedded into the encoding procedure of the fractal image compression. When the original image is corrupted by noises, we argue that the fractal image compression scheme should be insensitive to those noises presented in the corrupted image. This leads to a new concept of robust fractal image compression. The proposed HFIC is one of our attempts toward the design of robust fractal image compression. The main disadvantage of HFIC is the high computational cost. To overcome this drawback, particle swarm optimization (PSO) technique is utilized to reduce the searching time. Simulation results show that the proposed HFIC is robust against outliers in the image. Also, the PSO method can effectively reduce the encoding time while retaining the quality of the retrieved image.

DCT based simple classification scheme for fractal image compression

In this paper, a fast fractal encoding algorithm using simple classification scheme is proposed. During the encoding process, the range blocks and domain blocks are classified first. Then, each range block is limited to search in the corresponding domain class to find the best match. Since the searching space is reduced, the encoding speed is improved. Three classes of image blocks are defined, which are smooth class, diagonal/sub-diagonal edge class and horizontal/vertical edge class. The classification operation is performed using only the lowest horizontal and vertical DCT coefficients of the given block. Thus the classification scheme is simple and computationally efficient. Moreover, since the classification mechanism is designed according to the edge properties and the intrinsic idea of fractal coding, the quality of the decoded image can be preserved. The thresholds for the classifier are also adaptively determined from the range pool so as to reduce the overhead and to guarantee a stable speedup ratio of 3. Simulation results show that the stable speedup ratio of the proposed algorithm can be achieved and is independent of images while the quality of the decoded image is almost the same as that of the full search method.

Schema genetic algorithm for fractal image compression

In this paper, fractal image compression using schema genetic algorithm (SGA) is proposed. Utilizing the self-similarity property of a natural image, the partitioned iterated function system (PIFS) will be found to encode an image through genetic algorithm (GA) method. In SGA, the genetic operators are adapted according to the schema theorem in the evolutionary process performed on the range blocks. Such a method can speed up the encoder and also preserve the image quality. Simulations show that the encoding time of our method is over 100 times faster than that of the full search method, while the retrieved image quality is still acceptable. The proposed method is also compared to another GA method proposed by Vences and Rudomin. Simulations also show that our method is superior to their method in both the speedup ratio and retrieved quality. Finally, a comparison of the proposed SGA to the traditional GA is presented to demonstrate that when the schema theorem is embedded, the performance of GA has significant improvement.

Active contour model via multi-population particle swarm optimization

In this paper, the multi-population particle swarm optimization (PSO) is utilized to enhance the concavity searching capability for the control points of active contour model (ACM). In the traditional methods for ACM, each control point searches its new position in a small nearby window. Consequently, the boundary concavities cannot be searched accurately. Some improvements have been made in the past to enlarge the searching space, yet they are still time-consuming. To overcome these drawbacks, a multi-population particle swarm optimization technique is adopted in this paper to reduce the search time but in a larger searching window. In the proposed scheme, to each control point in the contour there is a corresponding swarm of particles with the best swarm particle as the new control point. The proposed optimizer not only inherits the spirit of the original PSO in each swarm but also shares information of the surrounding swarms. Experimental results demonstrate that the proposed method can improve the search of object concavities without extra computation time.

Fractal image compression using visual-based particle swarm optimization

Fractal image compression is promising both theoretically and practically. The encoding speed of the traditional full search method is a key factor rendering the fractal image compression unsuitable for real-time applications. In this paper, particle swarm optimization (PSO) method by utilizing the visual information of the edge property is proposed, which can speedup the encoder and preserve the image quality. Instead of the full search, a direction map is built according to the edge-type of image blocks, which directs the particles in the swarm to regions consisting of candidates of higher similarity. Therefore, the searching space is reduced and the speedup can be achieved. Also, since the strategy is performed according to the edge property, better visual effect can be preserved. Experimental results show that the visual-based particle swarm optimization speeds up the encoder 125 times faster with only 0.89dB decay of image quality in comparison to the full search method.

Three-parameter sequential minimal optimization for support vector machines

The well-known sequential minimal optimization (SMO) algorithm is the most commonly used algorithm for numerical solutions of the support vector learning problems. At each iteration in the traditional SMO algorithm, also called 2PSMO algorithm in this paper, it jointly optimizes only two chosen parameters. The two parameters are selected either heuristically or randomly, whilst the optimization with respect to the two chosen parameters is performed analytically. The 2PSMO algorithm is naturally generalized to the three-parameter sequential minimal optimization (3PSMO) algorithm in this paper. At each iteration of this new algorithm, it jointly optimizes three chosen parameters. As in 2PSMO algorithm, the three parameters are selected either heuristically or randomly, whilst the optimization with respect to the three chosen parameters is performed analytically. Consequently, the main difference between these two algorithms is that the optimization is performed at each iteration of the 2PSMO algorithm on a line segment, whilst that of the 3PSMO algorithm on a two-dimensional region consisting of infinitely many line segments. This implies that the maximum can be attained more efficiently by 3PSMO algorithm. Main updating formulae of both algorithms for each support vector learning problem are presented. To assess the efficiency of the 3PSMO algorithm compared with the 2PSMO algorithm, 14 benchmark datasets, 7 for classification and 7 for regression, will be tested and numerical performances are compared. Simulation results demonstrate that the 3PSMO outperforms the 2PSMO algorithm significantly in both executing time and computation complexity.

On maximum likelihood fuzzy neural networks

In this paper, M-estimators, where M stands for maximum likelihood, used in robust regression theory for linear parametric regression problems will be generalized to nonparametric maximum likelihood fuzzy neural networks (MFNNs) for nonlinear regression problems. Emphasis is put particularly on the robustness against outliers. This provides alternative learning machines when faced with general nonlinear learning problems. Simple weight updating rules based on gradient descent and iteratively reweighted least squares (IRLS) will be derived. Some numerical examples will be provided to compare the robustness against outliers for usual fuzzy neural networks (FNNs) and the proposed MFNNs. Simulation results show that the MFNNs proposed in this paper have good robustness against outliers.

ROBUST TEMPLATE DECOMPOSITION WITH RESTRICTED WEIGHTS FOR CELLULAR NEURAL NETWORKS IMPLEMENTING AN ARBITRARY BOOLEAN FUNCTION

In this paper, a general problem of the robust template decomposition with restricted weights for cellular neural networks (CNNs) implementing an arbitrary Boolean function is investigated. First, the geometric margin of a linear classifier with respect to a training data set is used to define the robustness of an uncoupled CNN implementing a linearly separable Boolean function. Second, maximal margin classifiers, i.e. robust CNNs, for such Boolean functions can be designed via support vector machines (SVMs). Third, some general properties of robust CNNs with or without restricted weights are discussed. Moreover, all robust CNNs with restricted weights are characterized. Finally, for an arbitrarily given Boolean function, we propose an algorithm, which is the generalized version of the well-known CFC algorithm, to find a sequence of robust uncoupled CNNs implementing the given Boolean function. Several illustrative examples demonstrate the efficiency of the proposed method. It will be seen that, in general, the tradeoff between the complexity regarding the number of terms in the decomposition and the guaranteed robustness regarding the geometric margins of the resulting CNNs must be made in the robust template decomposition with restricted weights.

A robust watermarking and image authentication technique

In the past several years, there has been an explosive growth in digital imaging technology and applications. Since digital images and video are now widely distributed via the Internet and various public channels, there is an urgent need for copyright protection against unauthorized data reproduction. Digital watermarking is an effective and popular technique for discouraging illegal copying and distribution of copyrighted digital image information. Furthermore, it can also provide an alternative solution for image authentication. A robust watermarking and image authentication technique is proposed. The proposed scheme includes two parts. The first is a robust watermarking scheme performed in the frequency domain, which can be used to prove the ownership. The second is a standard signature process, which can be used to prove the integrity of the image. The input of the signature process is the edge properties extracted from the image. The signature can be correctly verified when the image is incidentally damaged such as lossy compression.