Ray Bing Chen

Optimizing Latin hypercube designs by particle swarm

Latin hypercube designs (LHDs) are widely used in many applications. As the number of design points or factors becomes large, the total number of LHDs grows exponentially. The large number of feasible designs makes the search for optimal LHDs a difficult discrete optimization problem. To tackle this problem, we propose a new population-based algorithm named LaPSO that is adapted from the standard particle swarm optimization (PSO) and customized for LHD. Moreover, we accelerate LaPSO via a graphic processing unit (GPU). According to extensive comparisons, the proposed LaPSO is more stable than existing approaches and is capable of improving known results.

TVICA-Time varying independent component analysis and its application to financial data

A new method of ICA, TVICA, is proposed. Compared to the conventional ICA, the TVICA method allows the mixing matrix to be time dependent. Estimation is conducted under local homogeneity that assumes at any particular time point, there exists an interval over which the mixing matrix can be well approximated as constant. A sequential log likelihood-ratio testing procedure is used to automatically identify such local intervals. Numerical analysis demonstrates that TVICA provides good performance in homogeneous situations and does improve accuracy in nonstationary settings with possible structural change. In real data analysis with application to risk management, the TVICA confirms a superior performance when compared to several alternatives, including ICA, PCA and DCC-based models.

Stochastic matching pursuit for Bayesian variable selection

This article proposes a stochastic version of the matching pursuit algorithm for Bayesian variable selection in linear regression. In the Bayesian formulation, the prior distribution of each regression coefficient is assumed to be a mixture of a point mass at 0 and a normal distribution with zero mean and a large variance. The proposed stochastic matching pursuit algorithm is designed for sampling from the posterior distribution of the coefficients for the purpose of variable selection. The proposed algorithm can be considered a modification of the componentwise Gibbs sampler. In the componentwise Gibbs sampler, the variables are visited by a random or a systematic scan. In the stochastic matching pursuit algorithm, the variables that better align with the current residual vector are given higher probabilities of being visited. The proposed algorithm combines the efficiency of the matching pursuit algorithm and the Bayesian formulation with well defined prior distributions on coefficients. Several simulated examples of small n and large p are used to illustrate the algorithm. These examples show that the algorithm is efficient for screening and selecting variables.

Minimax optimal designs via particle swarm optimization methods

Particle swarm optimization (PSO) techniques are widely used in applied fields to solve challenging optimization problems but they do not seem to have made an impact in mainstream statistical applications hitherto. PSO methods are popular because they are easy to implement and use, and seem increasingly capable of solving complicated problems without requiring any assumption on the objective function to be optimized. We modify PSO techniques to find minimax optimal designs, which have been notoriously challenging to find to date even for linear models, and show that the PSO methods can readily generate a variety of minimax optimal designs in a novel and interesting way, including adapting the algorithm to generate standardized maximin optimal designs.

A Modified Particle Swarm Optimization Technique for Finding Optimal Designs for Mixture Models

Particle Swarm Optimization (PSO) is a meta-heuristic algorithm that has been shown to be successful in solving a wide variety of real and complicated optimization problems in engineering and computer science. This paper introduces a projection based PSO technique, named ProjPSO, to efficiently find different types of optimal designs, or nearly optimal designs, for mixture models with and without constraints on the components, and also for related models, like the log contrast models. We also compare the modified PSO performance with Fedorovs algorithm, a popular algorithm used to generate optimal designs, Cocktail algorithm, and the recent algorithm proposed by [1].

Using animal instincts to design efficient biomedical studies via particle swarm optimization

Particle swarm optimization (PSO) is an increasingly popular metaheuristic algorithm for solving complex optimization problems. Its popularity is due to its repeated successes in finding an optimum or a near optimal solution for problems in many applied disciplines. The algorithm makes no assumption of the function to be optimized and for biomedical experiments like those presented here, PSO typically finds the optimal solutions in a few seconds of CPU time on a garden-variety laptop. We apply PSO to find various types of optimal designs for several problems in the biological sciences and compare PSO performance relative to the differential evolution algorithm, another popular metaheuristic algorithm in the engineering literature.

Optimizing Two-Level Supersaturated Designs Using Swarm Intelligence Techniques

Supersaturated designs (SSDs) are often used to reduce the number of experimental runs in screening experiments with a large number of factors. As more factors are used in the study, the search for an optimal SSD becomes increasingly challenging because of the large number of feasible selection of factor level settings. This article tackles this discrete optimization problem via an algorithm based on swarm intelligence. Using the commonly used E(s2) criterion as an illustrative example, we propose an algorithm to find E(s2)-optimal SSDs by showing that they attain the theoretical lower bounds found in previous literature. We show that our algorithm consistently produces SSDs that are at least as efficient as those from the traditional CP exchange method in terms of computational effort, frequency of finding the E(s2)-optimal SSD, and also has good potential for finding D3-, D4-, and D5-optimal SSDs. Supplementary materials for this article are available online.

Discrete particle swarm optimization for constructing uniform design on irregular regions

Central composite discrepancy (CCD) has been proposed to measure the uniformity of a design over irregular experimental region. However, how CCD-based optimal uniform designs can be efficiently computed remains a challenge. Focusing on this issues, we proposed a particle swarm optimization-based algorithm to efficiently find optimal uniform designs with respect to the CCD criterion. Parallel computation techniques based on state-of-the-art graphic processing unit (GPU) are employed to accelerate the computations. Several two- to five-dimensional benchmark problems are used to illustrate the advantages of the proposed algorithms. By solving a real application in data center thermal management, we further demonstrate that the proposed algorithm can be extended to incorporate desirable space-filling properties, such as the non-collapsing property.

Tuning Block Size for QR Factorization on CPU-GPU Hybrid Systems

In CPU-GPU hybrid systems, the QR factorization in MAGMA results in CPU idle due to the fixed block size. To improve the computational efficiency of MAGMA QR factorization, we propose a variable block size auto-tuning scheme on CPU-GPU hybrid systems. First, we fit the CPU and GPU costs in MAGMA QR factorization via two independent regression models as CPU and GPU performance models. Next, we propose a block size optimization scheme to tune the block size adaptively and therefore to minimize a cost objective function. The cost objective function is designed to balance the workloads between CPU and GPU based on the performance models. Finally, several numerical results demonstrate the performance gains due to the novel QR factorization algorithm.

Bayesian Sparse Group Selection

This article proposes a Bayesian approach for the sparse group selection problem in the regression model. In this problem, the variables are partitioned into different groups. It is assumed that only a small number of groups are active for explaining the response variable, and it is further assumed that within each active group only a small number of variables are active. We adopt a Bayesian hierarchical formulation, where each candidate group is associated with a binary variable indicating whether the group is active or not. Within each group, each candidate variable is also associated with a binary indicator, too. Thus, the sparse group selection problem can be solved by sampling from the posterior distribution of the two layers of indicator variables. We adopt a group-wise Gibbs sampler for posterior sampling. We demonstrate the proposed method by simulation studies as well as real examples. The simulation results show that the proposed method performs better than the sparse group Lasso in terms of selecting the active groups as well as identifying the active variables within the selected groups. Supplementary materials for this article are available online.