Una-May O'Reilly

Meta optimization: improving compiler heuristics with machine learning

Compiler writers have crafted many heuristics over the years to approximately solve NP-hard problems efficiently. Finding a heuristic that performs well on a broad range of applications is a tedious and difficult process. This paper introduces Meta Optimization, a methodology for automatically fine-tuning compiler heuristics. Meta Optimization uses machine-learning techniques to automatically search the space of compiler heuristics. Our techniques reduce compiler design complexity by relieving compiler writers of the tedium of heuristic tuning. Our machine-learning system uses an evolutionary algorithm to automatically find effective compiler heuristics. We present promising experimental results. In one mode of operation Meta Optimization creates application-specific heuristics which often result in impressive speedups. For hyperblock formation, one optimization we present in this paper, we obtain an average speedup of 23% (up to 73%) for the applications in our suite. Furthermore, by evolving a compilers heuristic over several benchmarks, we can create effective, general-purpose heuristics. The best general-purpose heuristic our system found for hyperblock formation improved performance by an average of 25% on our training set, and 9% on a completely unrelated test set. We demonstrate the efficacy of our techniques on three different optimizations in this paper: hyperblock formation, register allocation, and data prefetching.

Genetic programming needs better benchmarks

Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks.

OpenTuner: an extensible framework for program autotuning

Program autotuning has been shown to achieve better or more portable performance in a number of domains. However, autotuners themselves are rarely portable between projects, for a number of reasons: using a domain-informed search space representation is critical to achieving good results; search spaces can be intractably large and require advanced machine learning techniques; and the landscape of search spaces can vary greatly between different problems, sometimes requiring domain specific search techniques to explore efficiently. This paper introduces OpenTuner, a new open source framework for building domain-specific multi-objective program autotuners. OpenTuner supports fully-customizable configuration representations, an extensible technique representation to allow for domain-specific techniques, and an easy to use interface for communicating with the program to be autotuned. A key capability inside OpenTuner is the use of ensembles of disparate search techniques simultaneously; techniques that perform well will dynamically be allocated a larger proportion of tests. We demonstrate the efficacy and generality of OpenTuner by building autotuners for 7 distinct projects and 16 total benchmarks, showing speedups over prior techniques of these projects of up to 2.8χ with little programmer effort.

SENSING AND MANIPULATING BUILT-FOR-HUMAN ENVIRONMENTS

We report on a dynamically balancing robot with a dexterous arm designed to operate in built-for-human environments. Our initial target task was for the robot to navigate, identify doors, open them, and proceed through them.

Better GP benchmarks: community survey results and proposals

We present the results of a community survey regarding genetic programming benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a “blacklist” of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.

Evolutionary Approaches To Minimizing Network Coding Resources

We wish to minimize the resources used for network coding while achieving the desired throughput in a multicast scenario. We employ evolutionary approaches, based on a genetic algorithm, that avoid the computational complexity that makes the problem NP-hard. Our experiments show great improvements over the sub-optimal solutions of prior methods. Our new algorithms improve over our previously proposed algorithm in three ways. First, whereas the previous algorithm can be applied only to acyclic networks, our new method works also with networks with cycles. Second, we enrich the set of components used in the genetic algorithm, which improves the performance. Third, we develop a novel distributed framework. Combining distributed random network coding with our distributed optimization yields a network coding protocol where the resources used for coding are optimized in the setup phase by running our evolutionary algorithm at each node of the network. We demonstrate the effectiveness of our approach by carrying out simulations on a number of different sets of network topologies.

Program Search with a Hierarchical Variable Lenght Representation: Genetic Programming, Simulated Annealing and Hill Climbing

This paper emphasizes the general value of a hierarchical variable length representation for program induction by demonstrating that different search strategies and operators complementary to them can be used to obtain solutions. It presents a comparison of Genetic Programming (GP) with Simulated Annealing (SA) and Stochastic Iterated Hill Climbing (SIHC). All three search algorithms employ the hierarchical variable length representation for programs brought into recent prominence with the GP paradigm [K-92]. We experiment with three GP crossover operators and a new hierarchical variable length mutation operator developed for use in SA and SIHC. The results do not favor any one search technique which bears out the observation that a search strategy should be chosen in view of the landscape determined by fitness function and representation.

Hybridized crossover-based search techniques for program discovery

Addresses the problem of program discovery as defined by genetic programming. By combining a hierarchical crossover operator with two traditional single-point search algorithms (simulated annealing and stochastic iterated hill climbing), we have solved some problems by processing fewer candidate solutions and with a greater probability of success than genetic programming. We have also enhanced genetic programming by hybridizing it with the simple idea of hill climbing from a few individuals, at a fixed interval of generations.

Behavioral programming: a broader and more detailed take on semantic GP

In evolutionary computation, the fitness of a candidate solution conveys sparse feedback. Yet in many cases, candidate solutions can potentially yield more information. In genetic programming (GP), one can easily examine program behavior on particular fitness cases or at intermediate execution states. However, how to exploit it to effectively guide the search remains unclear. In this study we apply machine learning algorithms to features describing the intermediate behavior of the executed program. We then drive the standard evolutionary search with additional objectives reflecting this intermediate behavior. The machine learning functions independent of task-specific knowledge and discovers potentially useful components of solutions (subprograms), which we preserve in an archive and use as building blocks when composing new candidate solutions. In an experimental assessment on a suite of benchmarks, the proposed approach proves more capable of finding optimal and/or well-performing solutions than control methods.

Integrating generative growth and evolutionary computation for form exploration

We present an algorithmic growth process that is an extension of Lindenmayer’s Map L-systems. This growth process relies upon a set of rewrite rules, a map axiom and a geometric interpreter which is integrated with a 3D simulated environment. The outcome of the growth process is a digital surface in 3D space which has “grown” within and in response to its environment. We have developed a complementary evolutionary algorithm that is able to take over the task of generating the rewrite rules set for a growth process. Using a quantitative multi-objective fitness function that evaluates a variety of surface properties, the integrated system (evolutionary algorithm and growth process) can explore and generate diverse and interesting surfaces with a resemblance of organic form. The algorithms have been implemented to create a design tool for architects called Genr8.