Emanuel Kitzelmann

Inductive Programming: A Survey of Program Synthesis Techniques

Inductive programming (IP)—the use of inductive reasoning methods for programming, algorithm design, and software development—is a currently emerging research field. A major subfield is inductive program synthesis, the (semi-)automatic construction of programs from exemplary behavior. Inductive program synthesis is not a unified research field until today but scattered over several different established research fields such as machine learning, inductive logic programming, genetic programming, and functional programming. This impedes an exchange of theory and techniques and, as a consequence, a progress of inductive programming. In this paper we survey theoretical results and methods of inductive program synthesis that have been developed in different research fields until today.

Inductive programming meets the real world

Inductive programming can liberate users from performing tedious and repetitive tasks.

Inductive rule learning on the knowledge level

We present an application of the analytical inductive programming system Igor to learning sets of recursive rules from positive experience. We propose that this approach can be used within cognitive architectures to model regularity detection and generalization learning. Induced recursive rule sets represent the knowledge which can produce systematic and productive behavior in complex situations - that is, control knowledge for chaining actions in different, but structural similar situations. We argue, that an analytical approach which is governed by regularity detection in example experience is more plausible than generate-and-test approaches. After introducing analytical inductive programming with Igor we will give a variety of example applications from different problem solving domains. Furthermore, we demonstrate that the same generalization mechanism can be applied to rule acquisition for reasoning and natural language processing.

Analytical Inductive Functional Programming

We describe a new method to induce functional programs from small sets of non-recursive equations representing a subset of their input-output behaviour. Classical attempts to construct functional Lisp programs from input/output-examples are analytical , i.e., a Lisp program belonging to a strongly restricted program class is algorithmically derived from examples. More recent approaches enumerate candidate programs and only test them against the examples until a program which correctly computes the examples is found. Theoretically, large program classes can be induced generate-and-test based, yet this approach suffers from combinatorial explosion. We propose a combination of search and analytical techniques. The method described in this paper is search based in order to avoid strong a-priori restrictions as imposed by the classical analytical approach. Yet candidate programs are computed based on analytical techniques from the examples instead of being generated independently from the examples. A prototypical implementation shows first that programs are inducible which are not in scope of classical purely analytical techniques and second that the induction times are shorter than in recent generate-and-test based methods.

Analysis and Evaluation of Inductive Programming Systems in a Higher-Order Framework

In this paper we present a comparison of several inductive programming (IP) systems. IP addresses the problem of learning (recursive) programs from incomplete specifications, such as input/output examples. First, we introduce conditional higher-order term rewriting as a common framework for inductive program synthesis. Then we characterise the ILP system Golem and the inductive functional system MagicHaskeller within this framework. In consequence, we propose the inductive functional system Igor II as a powerful and efficient approach to IP. Performance of all systems on a representative set of sample problems is evaluated and shows the strength of Igor II.

Approaches and Applications of Inductive Programming

This report documents the program and the outcomes of Dagstuhl Seminar 13502 “Approaches and Applications of Inductive Programming”. After a short introduction to inductive programming research, an overview of the talks and the outcomes of discussion groups is given.

I/O guided detection of list catamorphisms: towards problem specific use of program templates in IP

Inductive programming (IP), usually defined as a search in a space of candidate programs, is an inherent exponentially complex problem. To constrain the search space, program templates have ever been one of the first choices. In previous approaches to incorporate program schemes, either an (often very well) informed expert user has to provide a template in advance, or templates are used simply on suspicion, regardless whether they are target-aiming or not. Instead of rather fit the data to the template, we present an approach to fit a template to the data. We propose to utilise universal properties of higher-order functions to detect the appropriateness of a certain template in the input/output examples. We use this technique to introduce catamorphisms on lists in our IP system Igor2.

Cost Optimality and Predictability of Parallel Programming with Skeletons

Skeletons are reusable, parameterized components with well-defined semantics and pre-packaged efficient parallel implementation. This paper develops a new, provably cost-optimal implementation of the DS (double-scan) skeleton for the divide-and-conquer paradigm. Our implementation is based on a novel data structure called plist (pointed list); implementation’s performance is estimated using an analytical model. We demonstrate the use of the DS skeleton for parallelizing a tridiagonal system solver and report experimental results for its MPI implementation on a Cray T3E and a Linux cluster: they confirm the performance improvement achieved by the cost-optimal implementation and demonstrate its good predictability by our performance model.

Applying Inductive Program Synthesis to Induction of Number Series A Case Study with IGOR2

Induction of number series is a typical task included in intelligence tests. It measures the ability to detect regular patterns and to generalize over them, which is assumed to be crucial for general intelligence. There are some computational approaches to solve number problems. Besides special-purpose algorithms, applicability of general purpose learning algorithms to number series prediction was shown for E-generalization and artificial neural networks (ANN). We present the applicability of the analytical inductive programming system Igor2 to number series problems. An empirical comparison of Igor2 shows that Igor2 has comparable performance on the test series used to evaluate the ANN and the E-generalization approach. Based on findings of a cognitive analysis of number series problems by Holzman et al. (1982, 1983) we conducted a detailed case study, presenting Igor2 with a set of number series problems where the complexity was varied over different dimensions identified as sources of cognitive complexity by Holzman. Our results show that performance times of Igor2 correspond to the cognitive findings for most dimensions.

Porting IgorII from Maude to Haskell

This paper describes our efforts and solutions in porting our IP system Igor 2 from the termrewriting language Maude to Haskell. We describe how, for our purpose necessary features of the homoiconic language Maude, especially the treatment of code as data and vice versa, can be simulated in Haskell using a stateful monad transformer which makes type and class information available. With our new implementation we are now able to use higher-order context during our synthesis and extract information from type classes useable as background knowledge. Keeping our new implementation as close as possible to our old, we could keep all features of our system.