Rolf Wanka

Local divergence of Markov chains and the analysis of iterative load-balancing schemes

We develop a general technique for the quantitative analysis of iterative distributed load balancing schemes. We illustrate the technique by studying two simple, intuitively appealing models that are prevalent in the literature: the diffusive paradigm, and periodic balancing circuits (or the dimension exchange paradigm). It is well known that such load balancing schemes can be roughly modeled by Markov chains, but also that this approximation can be quite inaccurate. Our main contribution is an effective way of characterizing the deviation between the actual loads and the distribution generated by a related Markov chain, in terms of a natural quantity which we call the local divergence. We apply this technique to obtain bounds on the number of rounds required to achieve coarse balancing in general networks, cycles and meshes in these models. For balancing circuits, we also present bounds for the stronger requirement of perfect balancing, or counting.

Particle Swarm Optimization in High-Dimensional Bounded Search Spaces

When applying particle swarm optimization (PSO) to real world optimization problems, often boundary constraints have to be taken into account. In this paper, we show that the bound handling mechanism essentially influences the swarm behavior, especially in high-dimensional search spaces. In our theoretical analysis, we prove that all particles are initialized very close to the boundary with overwhelming probability, and that the global guide is expected to leave the search space in every forth dimension. Afterwards, we investigate the initialization process when optimizing the sphere function, a widely used benchmark, in more detail in order to provide a first step towards explaining previously observed phenomena. Moreover, we present a broad experimental study of commonly applied bound handling mechanisms on a variety of benchmark functions which is useful for choosing an appropriate strategy in real world applications. Finally, we derive some guidelines for the practical application of the PSO algorithm in high-dimensional bounded search spaces

Theoretical Analysis of Initial Particle Swarm Behavior

In this paper, particle trajectories of PSO algorithms in the first iteration are studied. We will prove that many particles leave the search space at the beginning of the optimization process when solving problems with boundary constraints in high-dimensional search spaces. Three different velocity initialization strategies will be investigated, but even initializing velocities to zero cannot prevent this particle swarm explosion. The theoretical analysis gives valuable insight into PSO in high-dimensional bounded spaces, and highlights the importance of bound handling for PSO: As many particles leave the search space in the beginning, bound handling strongly influences particle swarm behavior. Experimental investigations confirm the theoretical results.

Strongly Adaptive Token Distribution

The token distribution (TD) problem, an abstract static variant of load balancing, is defined as follows: letM be a (parallel processor) network with setP of processors. Initially, each processorP ∈P has a certain amountl(P) of tokens. The goal of a TD algorithm, run onM, is to distribute the tokens evenly among the processors. In this paper we introduce the notion of strongly adaptive TD algorithms, i.e., algorithms whose running times come close to the best possible runtime, the off-line complexity of the TD problem, for each individual initial token distributionl. Until now, only weakly adaptive algorithms have been considered, where the running time is measured in terms of the maximum initial load max{l(P)∥P ∈P}.We design an almost optimal, strongly adaptive algorithm on mesh-connected networks of arbitrary dimension extended by a single 1-bit bus. This result shows that an on-line TD algorithm can come close to the optimal (off-line) bound for each individual initial load. Furthermore, we exactly characterize the off-line complexity of arbitrary initial token distributions on arbitrary networks. As an intermediate result, we design almost optimal weakly adaptive algorithms for TD on mesh-connected networks of arbitrary dimension.

Strongly adaptive token distribution

The token distribution (TD) problem, an abstract static variant of load balancing, is defined as follows: letM be a (parallel processor) network with setP of processors. Initially, each processorP ∈P has a certain amountl(P) of tokens. The goal of a TD algorithm, run onM, is to distribute the tokens evenly among the processors. In this paper we introduce the notion of strongly adaptive TD algorithms, i.e., algorithms whose running times come close to the best possible runtime, the off-line complexity of the TD problem, for each individual initial token distributionl. Until now, only weakly adaptive algorithms have been considered, where the running time is measured in terms of the maximum initial load max{l(P)∥P ∈P}.We design an almost optimal, strongly adaptive algorithm on mesh-connected networks of arbitrary dimension extended by a single 1-bit bus. This result shows that an on-line TD algorithm can come close to the optimal (off-line) bound for each individual initial load. Furthermore, we exactly characterize the off-line complexity of arbitrary initial token distributions on arbitrary networks. As an intermediate result, we design almost optimal weakly adaptive algorithms for TD on mesh-connected networks of arbitrary dimension.

Particle swarm optimization almost surely finds local optima

Particle swarm optimization (PSO) is a popular nature-inspired meta-heuristic for solving continuous optimization problems. Although this technique is widely used, up to now only some partial aspects of the method have been formally investigated. In particular, while it is well-studied how to let the swarm converge to a single point in the search space, no general theoretical statements about this point or on the best position any particle has found have been known. For a very general class of objective functions, we provide for the first time results about the quality of the solution found. We show that a slightly adapted PSO almost surely finds a local optimum by investigating the newly defined potential of the swarm. The potential drops when the swarm approaches the point of convergence, but increases if the swarm remains close to a point that is not a local optimum, meaning that the swarm charges potential and continues its movement.

3-SAT on CUDA: Towards a massively parallel SAT solver

This work presents the design and implementation of a massively parallel 3-SAT solver, specifically targeting random problem instances. Our approach is deterministic and features very little communication overhead and basically no load-balancing cost at all. In the context of most current parallel SAT solvers running only on a handful of cores, we implemented our solver on Nvidias CUDA platform, utilizing more than 200 parallel streaming processors, and employing several millions of threads to work through single problem instances. As most common sequential solver techniques had to be discarded, our approach is additionally supported by a new set of global heuristics, designed specifically to be easily exploited by the underlying thread parallelism.

Fast and feasible periodic sorting networks of constant depth

A periodic comparator network has depth (or period) k, if for every t>k, the compare-exchange operations performed at step t are executed between exactly the same registers as at step t-k. We introduce a general method that converts an arbitrary comparator network that sorts n items in time T(n) and that has layout area A into a periodic sorting network of depth 5 that sorts /spl Theta/(n/spl middot/T(n)) items in time O(T(n)/spl middot/log n) and has layout area O(A/spl middot/T(n)). This scheme applied to the AKS network yields a depth 5 periodic comparator network that sorts in time O(log/sup 2/ n). More practical networks with runtime O(log/sup 3/ n) can be obtained from Batchers networks. Developing the techniques for the main result, we improve some previous results: Let us fix a d/spl isin/N. Then we can construct a network of depth 3 based on a d-dimensional mesh sorting n items in time O(n/sup 1/d//spl middot/log/sup O(d/) n).<<ETX>>

Fairness in academic course timetabling

We consider the problem of creating fair course timetables in the setting of a university. The central idea is that undesirable arrangements in the course timetable, i.e., violations of soft constraints, should be distributed in a fair way among the stakeholders. We propose and discuss in detail two fair versions of the popular curriculum-based course timetabling (CB-CTT) problem, the MMF-CB-CTT problem and the JFI-CB-CTT problem, which are based on max–min fairness (MMF) and Jain’s fairness index (JFI), respectively. For solving the MMF-CB-CTT problem, we present and experimentally evaluate an optimization algorithm based on simulated annealing. We introduce three different energy difference measures and evaluate their impact on the overall algorithm performance. The proposed algorithm improves the fairness on 20 out of 32 standard instances compared to the known best timetables. The JFI-CB-CTT problem formulation focuses on the trade-off between fairness and the aggregated soft constraint violations. Here, our experimental evaluation shows that the known best solutions to 32 CB-CTT standard instances are quite fair with respect to JFI. Our experiments show that the fairness can often be improved at the cost of only a small increase in the overall amount of penalty.

Particle Swarm Optimization with Velocity Adaptation

Particle swarm optimization (PSO) algorithms have gained increasing interest for dealing with continuous optimization problems in recent years. Often such problems involve boundary constraints. In this case, one has to cope with the situation that particles may leave the feasible search space. To deal with such situations different bound handling methods have been proposed in the literature and it has been observed that the success of PSO algorithms depends on a large degree on the used bound handling method. In this paper, we propose an alternative approach to cope with bounded search spaces. The idea is to introduce a velocity adaptation mechanism into PSO algorithms that is similar to step size adaptation used in evolution strategies. Using this approach we show that the bound handling method becomes less important for PSO algorithms and that using velocity adaptation leads to better results for a wide range of benchmark functions.