Genetha Anne Gray

Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization

APPSPACK is software for solving unconstrained and bound-constrained optimization problems. It implements an asynchronous parallel pattern search method that has been specifically designed for problems characterized by expensive function evaluations. Using APPSPACK to solve optimization problems has several advantages: No derivative information is needed; the procedure for evaluating the objective function can be executed via a separate program or script; the code can be run serially or in parallel, regardless of whether the function evaluation itself is parallel; and the software is freely available. We describe the underlying algorithm, data structures, and features of APPSPACK version 4.0, as well as how to use and customize the software.

Bayesian Guided Pattern Search for Robust Local Optimization

Optimization for complex systems in engineering often involves the use of expensive computer simulation. By combining statistical emulation using treed Gaussian processes with pattern search optimization, we are able to perform robust local optimization more efficiently and effectively than when using either method alone. Our approach is based on the augmentation of local search patterns with location sets generated through improvement prediction over the input space. We further develop a computational framework for asynchronous parallel implementation of the optimization algorithm. We demonstrate our methods on two standard test problems and our motivating example of calibrating a circuit device simulator.

Optimal bundling of transmembrane helices using sparse distance constraints

We present a two‐step approach to modeling the transmembrane spanning helical bundles of integral membrane proteins using only sparse distance constraints, such as those derived from chemical cross‐linking, dipolar EPR and FRET experiments. In Step 1, using an algorithm, we developed, the conformational space of membrane protein folds matching a set of distance constraints is explored to provide initial structures for local conformational searches. In Step 2, these structures refined against a custom penalty function that incorporates both measures derived from statistical analysis of solved membrane protein structures and distance constraints obtained from experiments. We begin by describing the statistical analysis of the solved membrane protein structures from which the theoretical portion of the penalty function was derived. We then describe the penalty function, and, using a set of six test cases, demonstrate that it is capable of distinguishing helical bundles that are close to the native bundle from those that are far from the native bundle. Finally, using a set of only 27 distance constraints extracted from the literature, we show that our method successfully recovers the structure of dark‐adapted rhodopsin to within 3.2 Å of the crystal structure.

Modeling an Augmented Lagrangian for Blackbox Constrained Optimization

Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth. To narrow that gap, we propose a combination of response surface modeling, expected improvement, and the augmented Lagrangian numerical optimization framework. This hybrid approach allows the statistical model to think globally and the augmented Lagrangian to act locally. We focus on problems where the constraints are the primary bottleneck, requiring expensive simulation to evaluate and substantial modeling effort to map out. In that context, our hybridization presents a simple yet effective solution that allows existing objective-oriented statistical approaches, like those based on Gaussian process surrogates and expected improvement heuristics, to be applied to the constrained setting with minor modification. This work is motivated by a challenging, real-data benchmark problem from hydrology where, even with a simple linear objective function, learning a nontrivial valid region complicates the search for a global minimum. Supplementary materials for this article are available online.

Variationally constrained numerical solution of electrical impedance tomography

We propose a novel, variational inversion methodology for the electrical impedance tomography (EIT) problem, where we seek electrical conductivity σ inside a bounded, simply connected domain � ,g iven si multaneous measurements of electric currents I and corresponding potentials V at the boundary. Explicitly, we make use of natural, variational constraints on the space of admissible functions σ ,t oobtain efficient reconstruction methods which make best use of the data. We give a detailed analysis of the variational constraints; we propose a variety of reconstruction algorithms for the static problem in a simple continuum model. We discuss their advantages and disadvantages and we assess the performance of our algorithms through numerical simulations and comparisons with other, well established, numerical reconstruction methods. (Some figures in this article are in colour only in the electronic version)

Optimizing an Empirical Scoring Function for Transmembrane Protein Structure Determination

We examine the problem of transmembrane protein structure determination. Like many questions that arise in biological research, this problem cannot be addressed generally by traditional laboratory experimentation alone. Instead, an approach that integrates experiment and computation is required. We formulate the transmembrane protein structure determination problem as a bound-constrained optimization problem using a special empirical scoring function, called Bundler, as the objective function. In this paper, we describe the optimization problem and its mathematical properties, and we examine results obtained using two different derivative-free optimization algorithms.

Hybrid optimization schemes for simulation-based problems

The inclusion of computer simulations in the study and design of complex engineering systems has created a need for efficient approaches to simulation-based optimization. For example, in water resources management problems, optimization problems regularly consist of objective functions and constraints that rely on output from a PDE-based simulator. Various assumptions can be made to simplify either the objective function or the physical system so that gradient-based methods apply, however the incorporation of realistic objection functions can be accomplished given the availability of derivative-free optimization methods. A wide variety of derivative-free methods exist and each method has both advantages and disadvantages. Therefore, to address such problems, we propose a hybrid approach, which allows the combining of beneficial elements of multiple methods in order to more efficiently search the design space. Specifically, in this paper, we illustrate the capabilities of two novel algorithms; one which hybridizes pattern search optimization with Gaussian Process emulation and the other which hybridizes pattern search and a genetic algorithm. We describe the hybrid methods and give some numerical results for a hydrological application which illustrate that the hybrids find an optimal solution under conditions for which traditional optimal search methods fail.

Selection of a Representative Sample

Sometimes a larger dataset needs to be reduced to just a few points, and it is desirable that these points be representative of the whole dataset. If the future uses of these points are not fully specified in advance, standard decision-theoretic approaches will not work. We present here methodology for choosing a small representative sample based on a mixture modeling approach.

Disparate data fusion for protein phosphorylation prediction

New challenges in knowledge extraction include interpreting and classifying data sets while simultaneously considering related information to confirm results or identify false positives. We discuss a data fusion algorithmic framework targeted at this problem. It includes separate base classifiers for each data type and a fusion method for combining the individual classifiers. The fusion method is an extension of current ensemble classification techniques and has the advantage of allowing data to remain in heterogeneous databases. In this paper, we focus on the applicability of such a framework to the protein phosphorylation prediction problem.

Traditional and Hybrid Derivative-Free Optimization Approaches for Black Box Functions

Picking a suitable optimization solver for any optimization problem is quite challenging and has been the subject of many studies and much debate. This is due in part to each solver having its own inherent strengths and weaknesses. For example, one approach may be global but have slow local convergence properties, while another may have fast local convergence but is unable to globally search the entire feasible region. In order to take advantage of the benefits of more than one solver and to overcome any shortcomings, two or more methods may be combined, forming a hybrid. Hybrid optimization is a popular approach in the combinatorial optimization community, where metaheuristics (such as genetic algorithms, tabu search, ant colony, variable neighborhood search, etc.) are combined to improve robustness and blend the distinct strengths of different approaches. More recently, metaheuristics have been combined with deterministic methods to form hybrids that simultaneously perform global and local searches. In this Chapter, we will examine the hybridization of derivative-free methods to address black box, simulation-based optimization problems. In these applications, the optimization is guided solely by function values (i.e. not by derivative information), and the function values require the output of a computational model. Specifically, we will focus on improving derivative-free sampling methods through hybridization.We will review derivative-free optimization methods, discuss possible hybrids, describe intelligent hybrid approaches that properly utilize both methods, and give an examples of the successful application of hybrid optimization to a problem from the hydrological sciences.