Andreas Alexander Albrecht

Two applications of the LSA machine

We present two applications of a learning algorithm that combines logarithmic simulated annealing with the perceptron algorithm. The implementation of the learning algorithm is called LSA machine and has been successfully applied already to the classification of liver tissue from CT images. We investigate the performance of the LSA machine on two sets of numerical data: The Wisconsin breast cancer diagnosis (WBCD) database and microarray data published by Golub et al. (1999). The WBCD data consist of 683 samples with 9 input values that are divided into 444 benign cases (positive examples) and 239 malignant cases (negative examples). The LSA machine has been trained on 50% and 75% of the entire sample set, and the test has been performed on the remaining samples. In both cases, we obtain a correct classification close to 99% which is comparable to the best results published on WBCD data. The training set of the microarray data consists of I I samples of acute myeloid leukemia (AML) and 27 samples of acute lymphoblastic leukemia (ALL), each of them with 7129 input values (gene-expression data). For the test, 14 AML samples and 20 ALL samples are used. We obtain a single classification error (which is a ALL test sample) on seven genes only, which improves on the results published by Golub et al. (1999) by using the model of self-organising maps. Our result is competitive to the best results published for support vector machines.

An Epicurean learning approach to gene-expression data classification

We investigate the use of perceptrons for classification of microarray data where we use two datasets that were published in [Nat. Med. 7 (6) (2001) 673] and [Science 286 (1999) 531]. The classification problem studied by Khan et al. is related to the diagnosis of small round blue cell tumours (SRBCT) of childhood which are difficult to classify both clinically and via routine histology. Golub et al. study acute myeloid leukemia (AML) and acute lymphoblastic leukemia (ALL). We used a simulated annealing-based method in learning a system of perceptrons, each obtained by resampling of the training set. Our results are comparable to those of Khan et al. and Golub et al., indicating that there is a role for perceptrons in the classification of tumours based on gene-expression data. We also show that it is critical to perform feature selection in this type of models, i.e. we propose a method for identifying genes that might be significant for the particular tumour types. For SRBCTs, zero error on test data has been obtained for only 13 out of 2308 genes; for the ALL/AML problem, we have zero error for 9 out of 7129 genes that are used for the classification procedure. Furthermore, we provide evidence that Epicurean-style learning and simulated annealing-based search are both essential for obtaining the best classification results.

Stochastic protein folding simulation in the three-dimensional HP-model

We present results from three-dimensional protein folding simulations in the HP-model on ten benchmark problems. The simulations are executed by a simulated annealing-based algorithm with a time-dependent cooling schedule. The neighbourhood relation is determined by the pull-move set. The results provide experimental evidence that the maximum depth D of local minima of the underlying energy landscape can be upper bounded by D<n(2/3). The local search procedure employs the stopping criterion (m/delta)(D/gamma), where m is an estimation of the average number of neighbouring conformations, gamma relates to the mean of non-zero differences of the objective function for neighbouring conformations, and 1-delta is the confidence that a minimum conformation has been found. The bound complies with the results obtained for the ten benchmark problems.

Stochastic local search for the FEATURE SET problem, with applications to microarray data

We prove a (m/δ)O(κ) · na time bound for finding minimum solutions Smin of Feature Set problems, where n is the total size of a given Feature Set problem, κ ⩽ ∣Smin∣, m equals the number of non-target features, a is a (relatively small) constant, and 1 − δ is the confidence that the solution is of minimum length. In terms of parameterized complexity of NP-complete problems, our time bound differs from an FPT-type bound by the factor mO(κ) for fixed δ. The algorithm is applied to a prominent microarray dataset: The classification of gene-expression data related to acute myeloid leukaemia (AML) and acute lymphoblastic leukaemia (ALL). From the set of potentially significant features calculated by the algorithm we can identify three genes (D88422, M92287, L09209) that produce zero errors on the test set by using a simple, straightforward evaluation procedure (performing the test on the single gene M84526 produces only one error).

Genetic local search for multicast routing with pre-processing by logarithmic simulated annealing

Over the past few years, several local search algorithms have been proposed for various problems related to multicast routing in the off-line mode. We describe a population-based search algorithm for cost minimisation of multicast routing. The algorithm utilises the partially mixed crossover operation (PMX) under the elitist model: for each element of the current population, the local search is based upon the results of a landscape analysis that is executed only once in a pre-processing step; the best solution found so far is always part of the population. The aim of the landscape analysis is to estimate the depth of the deepest local minima in the landscape generated by the routing tasks and the objective function. The analysis employs simulated annealing with a logarithmic cooling schedule (logarithmic simulated annealing-LSA). The local search then performs alternating sequences of descending and ascending steps for each individual of the population, where the length of a sequence with uniform direction is controlled by the estimated value of the maximum depth of local minima. We present results from computational experiments on three different routing tasks, and we provide experimental evidence that our genetic local search procedure that combines LSA and PMX performs better than algorithms using either LSA or PMX only.

A Firefly-Inspired Method for Protein Structure Prediction in Lattice Models

We introduce a Firefly-inspired algorithmic approach for protein structure prediction over two different lattice models in three-dimensional space. In particular, we consider three-dimensional cubic and three-dimensional face-centred-cubic (FCC) lattices. The underlying energy models are the Hydrophobic-Polar (H-P) model, the Miyazawa–Jernigan (M-J) model and a related matrix model. The implementation of our approach is tested on ten H-P benchmark problems of a length of 48 and ten M-J benchmark problems of a length ranging from 48 until 61. The key complexity parameter we investigate is the total number of objective function evaluations required to achieve the optimum energy values for the H-P model or competitive results in comparison to published values for the M-J model. For H-P instances and cubic lattices, where data for comparison are available, we obtain an average speed-up over eight instances of 2.1, leaving out two extreme values (otherwise, 8.8). For six M-J instances, data for comparison are available for cubic lattices and runs with a population size of 100, where, a priori, the minimum free energy is a termination criterion. The average speed-up over four instances is 1.2 (leaving out two extreme values, otherwise 1.1), which is achieved for a population size of only eight instances. The present study is a test case with initial results for ad hoc parameter settings, with the aim of justifying future research on larger instances within lattice model settings, eventually leading to the ultimate goal of implementations for off-lattice models.

Research article: Population-based local search for protein folding simulation in the MJ energy model and cubic lattices

We present experimental results on benchmark problems in 3D cubic lattice structures with the Miyazawa-Jernigan energy function for two local search procedures that utilise the pull-move set: (i) population-based local search (PLS) that traverses the energy landscape with greedy steps towards (potential) local minima followed by upward steps up to a certain level of the objective function; (ii) simulated annealing with a logarithmic cooling schedule (LSA). The parameter settings for PLS are derived from short LSA-runs executed in pre-processing and the procedure utilises tabu lists generated for each member of the population. In terms of the total number of energy function evaluations both methods perform equally well, however, PLS has the potential of being parallelised with an expected speed-up in the region of the population size. Furthermore, both methods require a significant smaller number of function evaluations when compared to Monte Carlo simulations with kink-jump moves.

Stochastic protein folding simulation in the d-dimensional HP-model

We present results from two- and three-dimensional protein folding simulations in the HP-model on selected benchmark problems. The importance of the HP-model for investigating general complexity issues of protein folding has been recently demonstrated by Fu & Wang (LNCS 3142:630-644, 2004) in proving an exp(O(n1-1/d ċ ln n)) time bound for d-dimensional protein folding simulation of sequences of length n. The time bound is close to the approximation of real folding times of exp(λ ċ n2/3 ± Χ ċ n1/2/2)ns by Finkelstein & Badretdinov (FOLD DES 2:115-121, 1997), where λ and Χ are constants close to unity. We utilise a stochastic local search procedure that is based on logarithmic simulated annealing. We obtain that after (m/δ)a.D Markov chain transitions the probability to be in a minimum energy conformation is at least 1 - δ, where m ≤ b(d) ċ n is the maximum neighbourhood size for a small integer b(d), a is a small constant, and D is the maximum value of the minimum escape height from local minima of the underlying energy landscape. We note that the time bound is sequence-specific, and we conjecture D < n1 -1/d as a worst case upper bound. We analyse D

Markowitz-type heuristics for computing Jacobian matrices efficiently

We consider the problem of accumulating the Jacobian matrix of a nonlinear vector function by using a minimal number of arithmetic operations. Two new Markowitz-type heuristics are proposed for vertex elimination in linearized computational graphs, and their superiority over existing approaches is shown by several tests. Similar ideas are applied to derive new heuristics for edge elimination techniques. The well known superiority of edge over vertex elimination can be observed only partially for the heuristics discussed in this paper. Nevertheless, significant improvements can be achieved by the new heuristics both in terms of the quality of the results and their robustness with respect to different tiebreaking criteria.

A Stopping Criterion for Logarithmic Simulated Annealing

We perform a convergence analysis of simulated annealing-based search for the special case of logarithmic cooling schedules. Emphasis is put on the impact of structural parameters of the underlying configuration space on the number of transitions k≥L that is sufficient to achieve a certain probability (confidence 1−δ) of being in an optimum configuration. Since such a lower bound L of the transition number depends on some constants that are difficult to calculate, we evaluate a much simplified version L≪L of the lower bound for the problem of finding short conjunctions representing a ``positive Boolean vector and rejecting a set of ``negative Boolean vectors. The evaluation is based on computational experiments where the frequency of occurrences of configurations is calculated for simulated annealing-based search that terminates after L transitions. The experiments produce a good correspondence between frequencies of minimum configurations and the required confidence 1−δ, i.e., our study provides empirical evidence that the relation of basic parameters in the lower bound L, if calculated for small constants assigned to the parameters and thus resulting in L, can be used as a termination criterion in simulated annealing-based search.