Robert Zembowicz

Automated Discovery of Empirical Equations from Data

We describe a machine discovery system for automated finding regularities in numerical data. It can detect a broad range of empirical equations useful in different sciences, and can be easily expanded by addition of new variable transformations. Our system treats experimental error and evaluation of equations in a systematic and statistically sound manner in contradistinction to systems such as BACON, ABACUS, which include error-related parameters, but disregard problems of error analysis and propagation, leading to paradoxical results. Our system propagates error to the transformed variables and assigns error to parameters in equations. Furthermore, it uses errors in weighted least squares fitting, in the evaluation of equations, including their acceptance, rejection and ranking, and uses parameter error to eliminate spurious parameters. In the last part of our paper we analyse the evaluation of equation finding systems. We introduce two convergence tests and we analyze the performance of our system on those tests.

Theories that Combine Many Equivalence and Subset Relations

Knowledge comes in different species, such as equations, contingency tables, taxonomies, rules, and concepts. We show how starting from contingency tables, simple tests can distinguish various special forms of 2-D knowledge. Our experience in data mining with the application of the 49er system shows that the exploration of even a modestly sized database frequently leads to large numbers of regularities. The problem that we address in this paper comes from the recurring observation of users who show serious confusion when faced with thousands of regularities discovered in a database. As a response, 49er uses tests which classify regularities into different categories and applies automated methods which combine large numbers of regularities in each category into concise, useful forms of taxonomies, inclusion graphs and other multi-dimensional theories. We focus on detection of 2-D regularities which can be represented by equivalence and implication relation, and we show how taxonomies and subset graphs can capture large numbers of regularities in those categories. We illustrate the presented algorithms by applications on two databases.

Mining patterns at each scale in massive data

An important but neglected aspect of data analysis is discovering phenomena at different scale in the same data. Scale plays the role analogous to error. It can be used to focus data exploration on differences that exceed the given scale (error) and to disregard those smaller. We introduce a discovery mechanism that applies to bi-variate patterns, in particular to time series. It combines search for maxima and minima with search for regularities in the form of equations. If it cannot find a regularity for all data, it uses other discovered patterns to divide data into subsets, and explores recursively each subset. Detected patterns are subtracted from data and the search continues in the residuals. Our mechanism does not skip patterns at any scale. Applied at many scales and to many data sets, it seems explosive, but it terminates surprisingly fast because of data reduction and the requirements of pattern stability and significance. We show application of our method on a time series of a half million datapoints. Our example shows that even simple data can reveal many surprising phenomena, and our method leads to fine conclusions about the environment in which they have been gathered.

Recognition of Functional Dependencies in Data

Discovery of regularities in data involves search in many spaces, for instance in the space of functional expressions. If data do not fit any solution in a particular space, much time could be saved if that space was not searched at all. A test which determines the existence of a solution in a particular space, if available, can prevent unneeded search. We discuss a functionality test, which distinguishes data satisfying the functional dependence definition. The test is general and computationally simple. It permits error in data, limited number of outliers, and background noise. We show, how our functionality test works in database exploration within the 49er system as a trigger for the computationally expensive search in the space of equations. Results of tests show the savings coming from application of the test. Finally, we discuss how the functionality test can be used to recognize multifunctions.

The first phase of real-world discovery: determining repeatability and error of experiments**The work described in this paper was supported by Office of Naval Research under the grants No. N00014–88-K-0226 and N00014–90-J-1603

Using un-repeatable data and forgetting about measurement error are two cardinal sins in empirical sciences. A machine discovery system must be able to handle both before attempting serious discoveries. We describe an application of the discovery system FAHRENHEIT in a science laboratory, focused on the preparatory stage of the empirical discovery process, i.e. the investigation of repeatability and the measurement of error. To cope with real-world empirical discovery, FAHRENHEIT has been reorganized as a distributed multi-process system and a robotic component including external manipulators and measuring instruments. We present the application of FAHRENHEIT to an experiment in which the system discovers repeatability conditions and error in the context of dispensing liquids in a chemistry laboratory. We then present the theory of the process. Many quantitative discovery systems distinguish between dependent and independent control variables. We argue that to handle repeatability and error, independent variables should be divided further into two categories: theory formation variables and experiment refinement variables. The former have been used by BACON, FAHRENHEIT and other empirical discovery systems to re-discover scientific laws. The latter are used to determine the repeatability and error, prior to the system discovery of the main theory using theory formation variables.